text:
Extremely Rare Heckler & Koch HK41 Rifle 7.62 in NATO Up for Auction
With fewer than 1,300 rifles ever produced, a 1966 Heckler & Koch HK41 is up for grabs courtesy of Milestone Premier Antiques & Modern Firearms Auction.
The Heckler & Koch HK41 is rare enough that you might not have even heard of it. It's a semi-auto civilian variant of the more well-known Heckler & Koch G3. Fewer than 1,300 HK41s saw the light of day with less than 400 ever making their way to the U.S. But now, you can grab one.
HK Incursion: Heckler & Koch's Full-Size Out-the-Front Automatic Knife
An HK41 from 1966 is one of the739 lots chosen for the upcoming Milestone Premier Antiques & Modern Firearms Auction on June 27. The gun is actually already accepting bids at LiveAuctioneers.com. Milestone estimates the gun between $8,000 to $10,000. The current bid as of writing this is $4,100.
"The HK41 was intended to be a legal-to-own semi-automatic rifle for use by West German military reservists of the Bundeswehr [unified armed forces of Germany], hence its German designation as a paramilitary rifle," Milestone's firearms expert Dave Bushing said.
Heckler & Koch HK41 Up for Grabs
Serial number on the auction gun is 10B707. It features an 18-inch barrel and is chambered in 7.62 NATO/.308 WIN. It was produced under contract for Santa Fe and imported by Golden State Arms Corporation of Pasadena, Calif. Here's part of the official writeup from Milestone:
This early example has post front and four position rear drum sights, standard model, serial number, and "5/66" date code markings on the left side of the magazine well. This is one of the earliest models produced and imported. It features a "pushpin" receiver and a paddle magazine release, as well as a few modifications to the bolt and trigger control group to prevent conversion of the rifle in compliance with U.S. law. Parkerized and matte black finish with polymer fore-end and pistol grip. It is currently equipped with a single-position collapsible stock. It is technically designated as the HK41A3, but also comes with the solid butt stock and buffer spring.
The auction gun comes with an original H&K 66 dated waffle magazine. It also includes an original leather sling. Milestone says the bore is excellent and the action is flawless. Overall, the finish is approximately 99 percent with a few very minor handling marks on the top of the receiver.
So, there you have it. If you happen to have some cash to burn, head over to LiveAuctioneers.com and take a crack at it. The HK41 would certainly improve anyone's collection.
VIDEO: Gun Drummer Covers Lamb of God's 'Walk With Me in Hell'
In a second video for Ballistic, we had our pal Gun Drummer cover the...
by Ballistic Staff	/ Jun 9, 2020
Aimpoint Duty RDS: a New Red Dot Sight for Law...
The 5 Best MMA Fighters of All Time to Never...


text:
Woman injured in Nelson Street shooting discharged from hospital
Carnival GLAM Hub makes glamour effortless for masqueraders
2 held, 12 kilos of cocaine seized during Toco bust
Venezuelan housekeeper tied up, robbed in San Fernando
Arson suspected in Couva fire
Prince Day celebration brings fans to Rock Hall of Fame
Loop News Created : 8 June 2017 Entertainment
Prince hasn't been forgotten by fans in Cleveland, home to the Rock and Roll Hall of Fame.
Cleveland.com reports that around 400 people clad in purple showed up to participate Wednesday in a "Prince Day" celebration at the downtown museum. Fans showed off their best dance moves and rocked out during an air guitar contest to songs including "When Doves Cry."
The celebration came on Prince's date of birth. Organisers were careful to not refer to it as a birthday out of respect for the artist's faith as a Jehovah's Witness, whose practitioners don't celebrate birthdays.
Kathryn Metz, manager of community and family programs for the Rock Hall, says Prince is a cultural icon who was important to everyone.
Iman remembers late husband David Bowie on 25th anniversary
Rebel Wilson sues magazine publisher over 'nasty' articles
Jordan Peele's 'Get Out' follow-up set for 2019 release
PM Harris: Team Unity working for the peole of St Kitts & Nevis

redpajama_set_name: RedPajamaCommonCrawl


We had a lovely, short vacation to Montréal this past week. A city that feels European and is less than six hours from our front door ~Oui! I'm pretty sure we walked every main thoroughfare and rue in Old Port, Le Village, Plateau Mont-Royal, St-Denis, Mile End, Chinatown and more. There is indeed a great and easy metro system, but you can't see things if you can't see things, so leisurely walking allowed us to take in the most of this stylish city. I should have many more pictures, but am pleased to share with you these few that give me wonderful ideas for color, line and detail. I see suggestions for cut-outs, color use, lip lines (tops of pots) and there is surely a flower vessel in those amazing tea cakes pour sûr!
I couldn't think of a subject for a new post, so posed the question (to myself), "What have you been thinking about?" I'm in a making cycle, bisqued last weekend, glazed all week for upcoming deadlines and commissions leading up to another firing, and have some new, big ideas, but what I'm really thinking about is my garden.
I have discovered that I garden the same way I make pots: research and preparation balanced with a healthy dose of impatience and stubbornness. I wound up having little interest in planning, mostly choosing and placing plants together based on color, contrast and shape. As an artist, I seem to be drawn to unusual colors (loving deep purple and bronzed–leaf plants, for example, next to the chartreuse and limes), odd shapes (alliums and lupines) and rich textures. Gardening is the quintessential 3D design experience for me.
One of my goals with the garden—other than mental relaxation—was to attract butterflies and birds. I become giddy (yes, giddy) as new blooms bring big, clumsy bumblebees, darting ruby-throated hummingbirds, waves of monarchs, lazy tiger swallowtails and lone hummingbird moths (the insect I had never seen before having this garden). Because the garden is so close to the house, lining the entire front porch, I can stand at the door in the morning when I brush my teeth, or sit out there in the evening with a glass of wine in hand, surveying the scene.
All images courtesy of my garden.


text:
Suspect - Presunto colpevole (Suspect) è un thriller legale del 1987, diretto da Peter Yates.

Trama
L'avvocato Kathleen Riley è chiamata a difendere d'ufficio Carl Wayne Anderson, un reduce della guerra del Vietnam accusato di aver ucciso una donna, Elizabeth Quinn, il cui cadavere è stato ritrovato sulla sponda del fiume Potomac, con la gola tagliata da sinistra verso destra (taglio destrorso). I suoi effetti personali, la sua borsa, il suo portamonete, contenente la modesta cifra di nove dollari, vengono rinvenuti nel nascondiglio di Anderson dalla polizia di Washington D.C. assieme a un coltello, possibile arma del delitto. Pochi giorni prima, un giudice della Corte Suprema, per la quale Quinn lavorava come segretaria, era stato rinvenuto suicida per arma da fuoco.

Anderson viene trattato con durezza dopo il suo arresto, e ferito dai poliziotti, ma Riley capisce ben presto che Anderson non è un minorato mentale, ma solo un sordomuto con shock post-traumatico dovuto agli orrori della guerra. Comunica con lui scrivendo su fogli di carta, trattandolo con gentilezza. Anderson aveva contratto la meningite che l'aveva reso sordo, e il mutismo era derivato dalle turbe mentali.

Durante l'esame della giuria, Riley incontra un giurato particolarmente acuto, Eddie Sanger, un ottimo osservatore, che difatti lavora come mediatore e "portaborse" al Congresso. Sanger spera di essere scartato per poter tornare al suo lavoro, ma viene accolto in giuria. Nel frattempo il giudice Helms chiede a un collega di poter supervisionare il caso, sostenendo che il giudizio sarebbe stato breve e lui sarebbe stato così libero di potersi dedicare alla sua auspicata nomina alla Corte Suprema. Helms non dà un attimo di tregua a Riley, che pur tenendo in aula comportamenti poco ligi al regolamento, tenta in ogni modo di dimostrare l'innocenza del suo cliente, come il fargli afferrare al volo un oggetto, cosa che Anderson fa con la mano sinistra, dimostrando di essere mancino, e che quindi non poteva aver inflitto un taglio destrorso.

Sanger prende in simpatia l'avvocato Riley e la causa di Anderson, e tenta in ogni modo di aiutarla, fino a sconfinare nell'illegalità, avendo continui colloqui, telefonate e perfino effusioni con lei. Sospettando, ma non potendo dimostrare nulla, il giudice Helms riunisce la giuria in un albergo, in modo che non abbia rapporti con l'esterno. Avvocato e giurato battono diverse piste, una delle quali è l'auto di Quinn, ancora nel parcheggio, nella cui autoradio Riley trova una cassetta con una dichiarazione del giudice suicidatosi il quale ammette di essere stato corrotto anni addietro in un importante processo a un personaggio politico. Il procuratore in quel processo era proprio il giudice Helms.

Eddie Sanger ricorda poi che la targa dell'auto di Helms fa parte della nota del parcheggio dove era ancora rimasta l'auto di Elizabeth Quinn. Comprende subito che il giudice Helms è coinvolto e vuole avvisare Kathleen Riley, e per uscire dall'albergo fa scattare l'allarme antincendio. Eddie salva Kathleen da un aggressore, che si scoprirà essere il giudice Helms, il quale viene ferito alla mano destra. Durante la sessione successiva Riley convoca il procuratore Gray e come testimone il giudice Helms, palesemente contro ogni regola. Durante una stringente e incalzante accusa, Riley dimostra che Helms ha ucciso Elizabeth Rose Quinn perché in possesso dell'audiocassetta che ne rivelava la compromissione. Il procuratore Gray assiste e sarà suo compito prendere provvedimenti. Il giudice Helms china il capo di fronte alle accuse ammettendo implicitamente la sua colpa.

Finito il processo Riley viene lodata dai colleghi per la sua audacia e - non più vincolata dalle regole legali - può dimostrare a Sanger il suo interesse.

Collegamenti esterni

Film thriller
Film giudiziari
Film ambientati a Washington
Film sulla guerra del Vietnam

redpajama_set_name: RedPajamaWikipedia


text:
\section{Introduction}

We have recently designed and implemented a language for the specification of console I/O programs \cite{TFPIE19-paper}\cite{flops20-paper} that allows the formulation of desired I/O behavior and probabilistic testing of actual programs, written in Haskell, against the specified behavior.
We built this language to bring our testing capabilities of exercises on Haskell I/O more in line with how one can test exercises on pure programs using, for example, QuickCheck~\cite{claessen2000quickcheck}.
The testing capabilities gained this way are used in the e-learning system \cite{rahn2008leipzig,DBLP:conf/abp/Waldmann17} accompanying our course on programming paradigms.

Instead of only automatically checking exercise submissions against desired behavior, we also want to automatically generate the tasks themselves.
Automatic task generation, among other things, gives students more material to practice with, without the need for an instructor to write all of this material by hand.
Because our specification language is designed to describe the I/O behavior of programs, we will (only) end up with automatic task generation for tasks on Haskell I/O in this work.

Hand-written tasks are usually very heavy in terms of verbal descriptions of what a student has to do in order to solve the task.
This is a major problem when trying to generate tasks automatically, since natural language generation is not exactly easy.
Because of this, many task generation systems rely on templates that define a fixed (verbal) framing of a task containing parameters that can be filled in in order to get the finial task.
Choosing different parameters to fill into these holes, either randomly or by drawing from a fixed set of possibilities, then results in different tasks.
Whether such a template, and a fixed verbal framing, is easy to write depends on the domain for which tasks should be generated.
For example, it is easy to do so for many different math tasks (``Solve for $x$.'' etc.), but for programming tasks finding a fixed and general verbal framing is not as easy.
Essentially we need a way to describe varying requirements in a precise but non verbal way.
To achieve such a description our approach uses different artifacts, like actual program code, derived from the requirements underlying the task.
As an example, take the following tasks:
\begin{flushleft}
\texttt{Give the output of the following program for input 7.}
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{6}{@{}>{\hspre}l<{\hspost}@{}}%
\column{8}{@{}>{\hspre}l<{\hspost}@{}}%
\column{10}{@{}>{\hspre}l<{\hspost}@{}}%
\column{14}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{main}\mathrel{=}\mathbf{do}{}\<[E]%
\\
\>[B]{}\Varid{v}\leftarrow \Varid{readLn}{}\<[E]%
\\
\>[B]{}\mathbf{let}\;{}\<[6]%
\>[6]{}\Varid{loop}\;\mathrm{1}{}\<[14]%
\>[14]{}\mathrel{=}\Varid{print}\;\mathrm{1}{}\<[E]%
\\
\>[6]{}\Varid{loop}\;\Varid{x}{}\<[14]%
\>[14]{}\mathrel{=}\mathbf{do}{}\<[E]%
\\
\>[6]{}\hsindent{2}{}\<[8]%
\>[8]{}\Varid{print}\;\Varid{x}{}\<[E]%
\\
\>[6]{}\hsindent{2}{}\<[8]%
\>[8]{}\mathbf{if}\;\Varid{even}\;\Varid{x}{}\<[E]%
\\
\>[8]{}\hsindent{2}{}\<[10]%
\>[10]{}\mathbf{then}\;\Varid{loop}\;(\Varid{x}\mathbin{\Varid{`div`}}\mathrm{2}){}\<[E]%
\\
\>[8]{}\hsindent{2}{}\<[10]%
\>[10]{}\mathbf{else}\;\Varid{loop}\;(\Varid{x}\mathbin{+}\mathrm{1}){}\<[E]%
\\
\>[B]{}\Varid{loop}\;\Varid{v}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
\end{flushleft}

Here the verbal description will not change, no matter what we give as the program text.
As we will see in section~\ref{sec:examples}, there is a wide variety of possibilities to express tasks in this artifact driven style: from simple decision tasks over program comprehension to full blown programming exercises.

We present a newly developed EDSL, in Haskell, for describing such task templates.
Our approach is built  on top of the existing implementation of the specification language from our previous work.
We start from a specification, hand-written or generated randomly, and use the EDSL to derive different artifacts, like example runs and sample solutions, that we can use for building task templates.
Now a task basically consists of giving students one of these artifacts and ask for the other.
Additionally we also have the testing capabilities of our implementation of the specification language at our disposal.
Therefore we can also automatically check solution candidates for the generated tasks for correctness.

We will not go into the technical details of how we create these artifacts in the implementation.
Instead this presentation focuses on the framework's versatility in expressing interesting task ideas and generating variations.

We will start by looking at how to encode task templates with Haskell types in a domain independent way.
Next we give a short overview of the previously introduced specification language.
Using these two components together, we show how to design a diverse range of exercise tasks on Haskell-I/O.
\section{Describing parameterized tasks}

First we need to define how we encode a task template.
In its simplest form it is a function from some concrete values to an instance of the task.
That is, the argument of the function stands for the parameterized parts of the template.

We represent a task instance as a pair of a textual description and a requirement to what constitutes a correct solution.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\mathbf{type}\;\Conid{Description}\mathrel{=}\Conid{String}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}\mathbf{data}\;\Conid{TaskInstance}\;\Varid{s}\mathrel{=}\Conid{TaskInstance}{}\<[E]%
\\
\>[B]{}\hsindent{5}{}\<[5]%
\>[5]{}\{\mskip1.5mu \Varid{question}\mathbin{::}\Conid{Description}{}\<[E]%
\\
\>[B]{}\hsindent{5}{}\<[5]%
\>[5]{},\Varid{requires}\mathbin{::}\Conid{Require}\;\Varid{s}{}\<[E]%
\\
\>[B]{}\hsindent{5}{}\<[5]%
\>[5]{}\mskip1.5mu\}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks

A \ensuremath{\Conid{TaskInstance}} is polymorphic in the type of solutions it expects.
\ensuremath{\Conid{Require}\;\Varid{s}} then takes a value of type \ensuremath{\Varid{s}} and decides whether that value is a correct solution.
Instead of simply returning a value of type \ensuremath{\Conid{Bool}} we use QuickCheck's \ensuremath{\Conid{Property}} type for reasons that will become clear later. %
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\mathbf{newtype}\;\Conid{Require}\;\Varid{s}\mathrel{=}\Conid{Require}\;\{\mskip1.5mu \Varid{check}\mathbin{::}\Varid{s}\to \Conid{Property}\mskip1.5mu\}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks

Now we can define a parameterized task as a simple function.
The basic setup for tasks is not yet specific to Haskell I/O, so we as an example define a very simple task for adding two numbers:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{taskAdd}\mathbin{::}\Conid{Int}\to \Conid{Int}\to \Conid{TaskInstance}\;\Conid{Int}{}\<[E]%
\\
\>[B]{}\Varid{taskAdd}\;\Varid{x}\;\Varid{y}\mathrel{=}\Conid{TaskInstance}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\{\mskip1.5mu \Varid{question}\mathrel{=}\text{\ttfamily \char34 Give~the~sum~of~\char34}\mathbin{+\!\!\!+} \Varid{show}\;\Varid{x}\mathbin{+\!\!\!+} \text{\ttfamily \char34 ~and~\char34}\mathbin{+\!\!\!+} \Varid{show}\;\Varid{y}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{},\Varid{requires}\mathrel{=}\Varid{exactAnswer}\;(\Varid{x}\mathbin{+}\Varid{y}){}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\mskip1.5mu\}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{exactAnswer}\mathbin{::}(\Conid{Eq}\;\Varid{a},\Conid{Show}\;\Varid{a})\Rightarrow \Varid{a}\to \Conid{Require}\;\Varid{a}{}\<[E]%
\\
\>[B]{}\Varid{exactAnswer}\;\Varid{x}\mathrel{=}\Conid{Require}\mathbin{\$}\lambda \Varid{s}\to \Varid{s}\mathbin{===}\Varid{x}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
We define the requirement for an exact answer in terms of QuickCheck's \ensuremath{(\mathbin{===})} operator that produces an informative feedback message in case the test fails.
For example, giving the wrong solution to an instance of the above tasks might result in the following error when testing the requirement:
\begin{tabbing}\ttfamily
~\char42{}\char42{}\char42{}~Failed\char33{}~Falsified~\char40{}after~1~test\char41{}\char58{}\\
\ttfamily ~4~\char47{}\char61{}~5
\end{tabbing}
We can therefore leverage QuickCheck to provide students with feedback on their solution.

Note that since we use actual Haskell functions to describe parameterized tasks we can in principle have different instances based on what parameters we choose to fill the holes in the task description with.
We can take this idea one step further and not only choose \ensuremath{\Conid{TaskInstance}}'s based on the parameters, but add random variations to the instances themselves.
With this extension we get the following datatype for templates:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\mathbf{data}\;\Conid{TaskTemplate}\;\Varid{p}\;\Varid{s}\mathrel{=}\Conid{TaskTemplate}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\{\mskip1.5mu \Varid{parameter}\mathbin{::}\Conid{Gen}\;\Varid{p}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{},\Varid{inst}\mathbin{::}\Varid{p}\to \Conid{Gen}\;(\Conid{TaskInstance}\;\Varid{s}){}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\mskip1.5mu\}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
Here \ensuremath{\Conid{Gen}} is QuickCheck's generator monad type, representing the randomized variations in the parameters and task instance.
Every template for a task consists of a generator for parameters and a parameterized generator for an actual instance.
To instantiate a template we first generate a parameter value and then the instance body:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{generateTaskInstance}\mathbin{::}\Conid{TaskTemplate}\;\Varid{p}\;\Varid{s}\to \Conid{IO}\;(\Conid{TaskInstance}\;\Varid{s}){}\<[E]%
\\
\>[B]{}\Varid{generateTaskInstance}\;\Varid{task}\mathrel{=}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{generate}\mathbin{\$}\mathbf{do}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}\Varid{p}\leftarrow \Varid{parameter}\;\Varid{task}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}\Varid{inst}\;\Varid{task}\;\Varid{p}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks

Additionally, given a way to retrieve a solution candidate from somewhere, we can generate a \ensuremath{\Conid{TaskInstance}} and check the solution candidate against the requirements like this:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{runTaskIO}\mathbin{::}\Conid{TaskTemplate}\;\Varid{p}\;\Varid{s}\to \Conid{IO}\;\Varid{s}\to \Conid{IO}\;(){}\<[E]%
\\
\>[B]{}\Varid{runTaskIO}\;\Varid{task}\;\Varid{getAnswer}\mathrel{=}\mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Conid{TaskInstance}\;\Varid{q}\;\Varid{req}\leftarrow \Varid{generateTaskInstance}\;\Varid{task}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{putStrLn}\;\Varid{q}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{s}\leftarrow \Varid{getAnswer}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{quickCheck}\mathbin{\$}\Varid{check}\;\Varid{req}\;\Varid{s}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks

For convenience we define some combinators for defining templates and instances that make these definitions more directly readable.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{forFixed}\mathbin{::}\Varid{p}\to (\Varid{p}\to \Conid{Gen}\;(\Conid{TaskInstance}\;\Varid{s}))\to \Conid{TaskTemplate}\;\Varid{p}\;\Varid{s}{}\<[E]%
\\
\>[B]{}\Varid{forFixed}\;\Varid{p}\mathrel{=}\Conid{TaskTemplate}\;(\Varid{pure}\;\Varid{p}){}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{forUnknown}\mathbin{::}\Conid{Gen}\;\Varid{p}\to (\Varid{p}\to \Conid{Gen}\;(\Conid{TaskInstance}\;\Varid{s}))\to \Conid{TaskTemplate}\;\Varid{p}\;\Varid{s}{}\<[E]%
\\
\>[B]{}\Varid{forUnknown}\mathrel{=}\Conid{TaskTemplate}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{solveWith}\mathbin{::}\Conid{Description}\to \Conid{Require}\;\Varid{s}\to \Conid{TaskInstance}\;\Varid{s}{}\<[E]%
\\
\>[B]{}\Varid{solveWith}\;\Varid{d}\;\Varid{r}\mathrel{=}\Conid{TaskInstance}\;\Varid{d}\;\Varid{r}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks

\section{The specification language}
Before diving into the description of I/O tasks, we give a short summary of the specification language, introduced in our previous work~\cite{TFPIE19-paper}, as it forms the basis of the presented approach.
Specification expressed in the language describe program behavior in terms of the traces, i.e, sequences of read and written values, a program should produce.
By checking for different (carefully randomized) inputs wether a program produces traces satisfying the specification, we can automatically test programs against specifications.
The specifications themselves let us state task requirements in a precise way and the ability to automatically test programs against a specified behavior enables automatic grading of exercise tasks.

We will not present the language in the formal way of \cite{TFPIE19-paper}, but instead use the Haskell embedding, since this embedding is what we are going to use.
Specifications are built from three atomic specifications, a branching and a loop construct and sequential composition.
The atomic specifications are:
\begin{itemize}
\item The empty specification, representing a program with no I/O behavior.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{nop}\mathbin{::}\Conid{Specification}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks

\item Reading in a value drawn from a set of possible values.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{readInput}\mathbin{::}\Conid{Varname}\to \Conid{ValueSet}\to \Conid{Specification}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
Here \ensuremath{\Conid{ValueSet}} is a type describing which values we expect to be read in with the respective input action.
We will use the predefined values \ensuremath{\Varid{ints}\mathbin{::}\Conid{ValueSet}} and \ensuremath{\Varid{nats}\mathbin{::}\Conid{ValueSet}} for integers and natural numbers.

It is important to note that variables, in the specification language, always store all values previously read into them.
That is, each variable is associated with a historic list of values.
There are two ways to access a variable in a term:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}getAll\; \mathbin{::}\Conid{Varname}\to \Conid{Term}\;[\mskip1.5mu \Varid{a}\mskip1.5mu]{}\<[E]%
\\
\>[B]{}\Varid{getCurrent}\mathbin{::}\Conid{Varname}\to \Conid{Term}\;\Varid{a}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
We either access the complete list of all read values into a variable or we access only the current or most recent value, i.e., the last element of the stored list.

\item Printing the result of evaluating some term on the current variable environment, potentially decorated with some extra textual pattern
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{writeOutput}\mathbin{::}[\mskip1.5mu \Conid{OutputPattern}\mskip1.5mu]\to [\mskip1.5mu \Conid{Term}\;\Varid{a}\mskip1.5mu]\to \Conid{Specification}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
The list parameters here stand for different output possibilities, thereby introducing a form of nondeterminism.
\ensuremath{\Conid{OutputPattern}} describes what kind of output we expect from a  program.
These patterns can reference the values of \ensuremath{\Conid{Term}}s in the second parameter of \ensuremath{\Varid{writeOutput}}.
We will not go into detail on these types.
Their usage in the examples should be intuitive.
However, we rely on being able to not only evaluate terms but also inspect the syntactic structure of terms.
Therefore, the implementation provides functions to convert a \ensuremath{\Conid{Term}} into an abstract syntax tree.
Essentially, a value of type \ensuremath{\Conid{Term}\;\Varid{a}} is a pair of a value of type \ensuremath{\Varid{a}} and it's syntactic representation.
\end{itemize}

Sequential composition of \ensuremath{\Conid{Specification}}s forms a \ensuremath{\Conid{Monoid}}, which gives us \ensuremath{(\mathbin{<>})\mathbin{::}\Conid{Specification}\to \Conid{Specification}\to \Conid{Specification}} for combining smaller \ensuremath{\Conid{Specification}}s into larger ones.

In addition to the atomic specifications we have constructs for conditions and looping.
To allow for conditional behavior, we use:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{branch}\mathbin{::}\Conid{Term}\;\Conid{Bool}\to \Conid{Specification}\to \Conid{Specification}\to \Conid{Specification}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
Depending on the Boolean value the condition evaluates to, either the left or right specification is required to hold.
Here the left branch corresponds to \ensuremath{\Conid{False}} and the right to \ensuremath{\Conid{True}}, due to the way the operator is notated in the original definition.

To repeat a certain (sub)specification, we use:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{tillExit}\mathbin{::}\Conid{Specification}\to \Conid{Specification}{}\<[E]%
\\
\>[B]{}\Varid{exit}\mathbin{::}\Conid{Specification}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
The specification \ensuremath{\Varid{tillExit}\;\Varid{s}}, for some \ensuremath{\Varid{s}\mathbin{::}\Conid{Specification}}, results in repeatedly requiring the behavior specified by \ensuremath{\Varid{s}} until somewhere in \ensuremath{\Varid{s}} we hit an occurrence of the loop end marker \ensuremath{\Varid{exit}}, which leads to immediate termination of the loop.
Note that there can be more than one such marker inside \ensuremath{\Varid{s}}.

As an example, the following specification describes the behavior of reading in a natural number followed by that many additional integers and then printing the sum of these integers:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{6}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{example}\mathbin{::}\Conid{Specification}{}\<[E]%
\\
\>[B]{}\Varid{example}\mathrel{=}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{readInput}\;\text{\ttfamily \char34 n\char34}\;\Varid{nats}\mathbin{<>}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{tillExit}\;({}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}\Varid{branch}\;(\Varid{length}\;(getAll\; \;\text{\ttfamily \char34 x\char34})\mathrel{==}\Varid{getCurrent}\;\text{\ttfamily \char34 n\char34})\;{}\<[E]%
\\
\>[5]{}\hsindent{1}{}\<[6]%
\>[6]{}(\Varid{readInput}\;\text{\ttfamily \char34 x\char34}\;\Varid{ints})\;{}\<[E]%
\\
\>[5]{}\hsindent{1}{}\<[6]%
\>[6]{}\Varid{exit}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{})\mathbin{<>}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{writeOutput}\;[\mskip1.5mu \Varid{var}\;\mathrm{0}\mskip1.5mu]\;[\mskip1.5mu \Varid{sum}\mathbin{\$}getAll\; \;\text{\ttfamily \char34 x\char34}\mskip1.5mu]{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
\label{ref:example}
Note that \ensuremath{\Varid{length}}, \ensuremath{\Varid{sum}} and \ensuremath{\mathrel{==}} are not the functions of the same names from the Haskell prelude.
Instead they are syntactically inspectable versions, over values of type \ensuremath{\Conid{Term}}, with equivalent behavior.
This inspection is needed to built printable programs from specifications for the use in tasks descriptions.

\section{Tasks on Haskell I/O}
\label{sec:examples}
The testing approach of the specification framework relies on programs being represented in an inspectable version of the standard Haskell \ensuremath{\Conid{IO}} monad \cite{TFPIE19-paper,swierstra2007}.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\mathbf{type}\;\Conid{Program}\mathrel{=}\mathit{IO}_\mathit{rep}\;(){}\<[E]%
\ColumnHook
\end{hscode}\resethooks

Together with a \ensuremath{\Conid{Specification}} we can construct a \ensuremath{\Conid{Property}} that checks whether the program satisfies the behavior defined by the specification.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{fulfills}\mathbin{::}\Conid{Program}\to \Conid{Specification}\to \Conid{Property}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
Using \ensuremath{\Varid{fulfills}}, which is provided by the implementation of \cite{TFPIE19-paper}, we can construct our first I/O specific requirement for a correct task solution.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{behavior}\mathbin{::}\Conid{Specification}\to \Conid{Require}\;\Conid{Program}{}\<[E]%
\\
\>[B]{}\Varid{behavior}\;\Varid{s}\mathrel{=}\Conid{Require}\;(\mathbin{`\Varid{fulfills}`}\Varid{s}){}\<[E]%
\ColumnHook
\end{hscode}\resethooks
We can also build a requirement to check a given interaction trace, i.e. the sequence of values read and written by the program, against some specification, using an implementation of the \ensuremath{\Varid{accept}} function from \cite{TFPIE19-paper}.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{sampleTrace}\mathbin{::}\Conid{Specification}\to \Conid{Require}\;\Conid{Trace}{}\<[E]%
\\
\>[B]{}\Varid{sampleTrace}\;\Varid{s}\mathrel{=}\Conid{Require}\mathbin{\$}\lambda \Varid{t}\to \Varid{property}\mathbin{\$}\Varid{accept}\;\Varid{s}\;\Varid{t}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
We will define additional requirements as we discuss the various example tasks.

In addition to requirements we also need something to build descriptions from.
Our system offers essentially two options to produce such artifacts from specifications: program code and example traces.
In both cases the system provides parameterized generators to build the artifacts.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{haskellProgram}\mathbin{::}\Conid{Specification}\to \Conid{Gen}\;\Conid{Description}{}\<[E]%
\\
\>[B]{}\Varid{pythonProgram}\mathbin{::}\Conid{Specification}\to \Conid{Gen}\;\Conid{Description}{}\<[E]%
\\
\>[B]{}\Varid{exampleTraces}\mathbin{::}\Conid{Specification}\to \Conid{Int}\to \Conid{Gen}\;[\mskip1.5mu \Conid{Trace}\mskip1.5mu]{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
These generators will generate programs that fulfill the given specification or a number of example traces that a program with the specified behavior could produce.
For program code we mainly use Haskell code in our tasks, but it is also useful to have access to code in other languages and paradigms.
For example, we sometimes use Python code, since that is the language our students know from a previous course, to highlight how I/O functions differently in Haskell compared to an imperative language.

We will not discuss the technical details of the code generation, as our focus is on showcasing the different types of tasks that are expressible in the system.
There is, however, one aspect of this generation that is important for our presentation:
Even though we are going to treat the generation of code as a primitive operation provided by the framework, in the actual implementation the generation is adjustable in different ways.
Most importantly we can generate code in different programming languages, by defining rendering functions from an internal representation of programs to the desired target language.
With this mechanism we can also generate partial programs by omitting certain parts of the internal representation, either deterministically or randomized.
Such partial programs are useful in a lot of different ways, as we will see later.

With all of these tools we can now define a diverse range of task templates.
We will mainly use specifications, i.e. behavior, as parameters for our tasks and therefore assume the existence of some suitable generators.
A basic sketch of how to write such a generator can be found in \cite{flops20-paper}.
For our task templates we assume the existente of these two generators.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{randomSpecification}\mathbin{::}\Conid{Gen}\;\Conid{Specification}{}\<[E]%
\\
\>[B]{}\Varid{similarSpecifications}\mathbin{::}\Conid{Gen}\;(\Conid{Specification},\Conid{Specification}){}\<[E]%
\ColumnHook
\end{hscode}\resethooks
The first generator simply generates a random, but meaningful, specification.
The second generator generates a pair of specifications that are closely related but have different behavior.

The source code of the implementation and all examples from this paper, can be found at \url{https://github.com/fmidue/IOTasks}.%
\footnote{
The repository also contains instructions explaining how to, for the given examples, generate and inspect task instances that have been omitted from this presentation.
}
\subsection{Simple program comprehension tasks}

To get started exploring the task design space for, we can build a task around the simple question ``What does this program do?''.
An instance of such a task would be:
\begin{flushleft}
\texttt{Give the interaction trace of the following program for input(s) 1, 9.}
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{9}{@{}>{\hspre}l<{\hspost}@{}}%
\column{20}{@{}>{\hspre}l<{\hspost}@{}}%
\column{22}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{prog}\mathbin{::}\Conid{IO}\;(){}\<[E]%
\\
\>[B]{}\Varid{prog}\mathrel{=}\mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{n}\leftarrow \Varid{readLn}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\mathbf{let}\;\Varid{loop}\;\Varid{s}\;\Varid{l}\mathrel{=}{}\<[E]%
\\
\>[3]{}\hsindent{6}{}\<[9]%
\>[9]{}\mathbf{if}\;\Varid{l}\mathrel{==}\Varid{n}\;{}\<[20]%
\>[20]{}\mathbf{then}\;\Varid{print}\;\Varid{s}{}\<[E]%
\\
\>[20]{}\mathbf{else}\;\mathbf{do}{}\<[E]%
\\
\>[20]{}\hsindent{2}{}\<[22]%
\>[22]{}\Varid{v}\leftarrow \Varid{readLn}{}\<[E]%
\\
\>[20]{}\hsindent{2}{}\<[22]%
\>[22]{}\Varid{loop}\;(\Varid{s}\mathbin{+}\Varid{v})\;(\Varid{l}\mathbin{+}\mathrm{1}){}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{loop}\;\mathrm{0}\;\mathrm{0}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
\end{flushleft}
The template expression for such tasks looks like this:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{6}{@{}>{\hspre}l<{\hspost}@{}}%
\column{9}{@{}>{\hspre}l<{\hspost}@{}}%
\column{11}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{trace1}\mathbin{::}\Conid{TaskTemplate}\;\Conid{Specification}\;\Conid{Trace}{}\<[E]%
\\
\>[B]{}\Varid{trace1}\mathrel{=}\Varid{forUnknown}\;\Varid{randomSpecification}\mathbin{\$}\lambda \Varid{s}\to \mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{prog}\leftarrow \Varid{haskellProgram}\;\Varid{s}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\mathord{\sim}[\mskip1.5mu \Varid{t}\mskip1.5mu]\leftarrow \Varid{exampleTraces}\;\Varid{s}\;\mathrm{1}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\mathbf{let}\;\Varid{is}\mathrel{=}\Varid{inputs}\;\Varid{t}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\mathbin{\$}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}({}\<[9]%
\>[9]{}\text{\ttfamily \char34 Give~the~interaction~trace~of~the~following~program\char34}{}\<[E]%
\\
\>[9]{}\hsindent{2}{}\<[11]%
\>[11]{}\mathbin{+\!\!\!+} \text{\ttfamily \char34 for~input(s)~\char34}\mathbin{+\!\!\!+} \Varid{show}\;\Varid{is}\mathbin{+\!\!\!+} \text{\ttfamily \char34 !\char34}{}\<[E]%
\\
\>[5]{}\hsindent{1}{}\<[6]%
\>[6]{}\mathbin{\$\$}\Varid{prog}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{})\mathbin{`\Varid{solveWith}`}\Varid{exactAnswer}\;\Varid{t}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}(\mathbin{\$\$})\mathbin{::}\Conid{Description}\to \Conid{Description}\to \Conid{Description}{}\<[E]%
\\
\>[B]{}\Varid{s}\mathbin{\$\$}\Varid{t}\mathrel{=}\Varid{s}\mathbin{+\!\!\!+} \text{\ttfamily \char34 \char92 n\char34}\mathbin{+\!\!\!+} \Varid{t}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
An even more basic variation of this task would be to not give an input sequence at all and simply ask for any possible interaction trace the program can produce.
For a more involved task we can generate two similar looking programs with different behavior and let students find an input sequence for which the given programs exhibit different I/O behavior.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{6}{@{}>{\hspre}l<{\hspost}@{}}%
\column{9}{@{}>{\hspre}l<{\hspost}@{}}%
\column{11}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\mathbf{type}\;\Conid{Input}\mathrel{=}\Conid{String}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{trace2}\mathbin{::}\Conid{TaskTemplate}\;(\Conid{Specification},\Conid{Specification})\;[\mskip1.5mu \Conid{Input}\mskip1.5mu]{}\<[E]%
\\
\>[B]{}\Varid{trace2}\mathrel{=}\Varid{forUnknown}\;\Varid{similarSpecifications}\mathbin{\$}\lambda (\Varid{spec1},\Varid{spec2})\to \mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{p1}\leftarrow \Varid{haskellProgram}\;\Varid{spec1}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{p2}\leftarrow \Varid{haskellProgram}\;\Varid{spec2}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\mathbin{\$}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}({}\<[9]%
\>[9]{}\text{\ttfamily \char34 Give~a~sequence~of~input~values~for~which~the~two\char34}{}\<[E]%
\\
\>[9]{}\hsindent{2}{}\<[11]%
\>[11]{}\mathbin{+\!\!\!+} \text{\ttfamily \char34 programs~below~behave~differently!\char34}{}\<[E]%
\\
\>[5]{}\hsindent{1}{}\<[6]%
\>[6]{}\mathbin{\$\$}\Varid{p1}\mathbin{\$\$}\text{\ttfamily \char34 ---\char34}\mathbin{\$\$}\Varid{p2}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{})\mathbin{`\Varid{solveWith}`}\Varid{triggeringDifference}\;\Varid{spec1}\;\Varid{spec2}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{triggeringDifference}\mathbin{::}\Conid{Specification}\to \Conid{Specification}\to \Conid{Require}\;[\mskip1.5mu \Conid{Input}\mskip1.5mu]{}\<[E]%
\\
\>[B]{}\Varid{triggeringDifference}\;\Varid{s1}\;\Varid{s2}\mathrel{=}\Conid{Require}\mathbin{\$}\lambda \Varid{is}\to {}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}((\mathrel{=\hspace{-1ex}/\hspace{-1ex}=})\mathbin{`\Varid{on}`}(\Varid{runProgram}\;\Varid{is}\mathbin{\circ}\Varid{buildComputation}))\;\Varid{s1}\;\Varid{s2}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
In order to check the requirement for this tasks, we here make use of the ability of the specification framework to derive an executable program from a specification via the \ensuremath{\Varid{buildComputation}} function.
Details on this can be found in~\cite{flops20-paper}.

\subsection{Program writing tasks}

To explore other kinds of task, we can reverse the question underlying the example above.
The reverse direction of ``What does this program do?'' is ``Do this with a program!'', where ``this'' would refer to a given description of behavior.
Producing such descriptions in a precise and automatic way is exactly what the artifact primitives are for.
These primitives are the central ingredients that enable us to describe the diverse set of tasks we will see in this section.

A direct reversal of ``What is the behavior of this program for input X?'' is the, arguably, not very interesting task ``Write a program that does Y for input X!''.
But generalizing from a single input to a larger set of ``program runs'', i.e. traces, we immediately get a more compelling template:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{7}{@{}>{\hspre}l<{\hspost}@{}}%
\column{10}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{prog1}\mathbin{::}\Conid{TaskTemplate}\;\Conid{Specification}\;\Conid{Program}{}\<[E]%
\\
\>[B]{}\Varid{prog1}\mathrel{=}\Varid{forUnknown}\;\Varid{randomSpecification}\mathbin{\$}\lambda \Varid{s}\to \mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{ts}\leftarrow \Varid{exampleTraces}\;\Varid{s}\;\mathrm{5}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\mathbin{\$}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}({}\<[10]%
\>[10]{}\text{\ttfamily \char34 Write~a~program~capable~of~these~interactions:\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\text{\ttfamily \char34 (?~represent~inputs,~!~represent~outputs)\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\Varid{unlines}\;(\Varid{map}\;\Varid{show}\;\Varid{ts}){}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{})\mathbin{`\Varid{solveWith}`}\Varid{producingTraces}\;\Varid{ts}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{producingTraces}\mathbin{::}[\mskip1.5mu \Conid{Trace}\mskip1.5mu]\to \Conid{Require}\;\Conid{Program}{}\<[E]%
\\
\>[B]{}\Varid{producingTraces}\;\Varid{ts}\mathrel{=}\Conid{Require}\mathbin{\$}\lambda \Varid{p}\to {}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{property}\mathbin{\$}\Varid{all}\;(\lambda \Varid{t}\to \Varid{runProgram}\;(\Varid{inputs}\;\Varid{t})\;\Varid{p}\mathrel{==}\Varid{t})\;\Varid{ts}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
The requirement here checks that the solution produces exactly the given traces for the respective inputs.
An instance of this task could look like this:
\begin{tabbing}\ttfamily
~Write~a~program~capable~of~these~interactions\char58{}\\
\ttfamily ~\char40{}\char63{}~represent~inputs\char44{}~\char33{}~represent~outputs\char41{}\\
\ttfamily ~\char63{}0~\char33{}0~stop\\
\ttfamily ~\char63{}1~\char63{}\char45{}3~\char33{}\char45{}3~stop\\
\ttfamily ~\char63{}2~\char63{}1~\char63{}5~\char33{}6~stop\\
\ttfamily ~\char63{}2~\char63{}10~\char63{}10~\char33{}20~stop\\
\ttfamily ~\char63{}2~\char63{}\char45{}3~\char63{}\char45{}2~\char33{}\char45{}5~stop
\end{tabbing}

With such a task, however, we can no longer expect a solution to have the exact same behavior as our underlying specification.
The description essentially gives a list of unit tests a correct solution has to fulfill.
The fact that there is an underlying specification guarantees that abstraction is possible, i.e, there is at least on program that solves the task without simply hard-coding the given interactions.
Simply writing such a program that only works on the given inputs and crashes on all others, however, would still be a correct solution with regard to the task requirement.
If we do not want students to write such solutions, we can either give a very large number of program runs, making it unpractical to explicitly implement each case, or we add some restrictions ruling out such solutions.
One possibility is to provide parts of the solution as a skeleton to complete.
This can either be a general skeleton with a specific style, or an actual part of a solution.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{7}{@{}>{\hspre}l<{\hspost}@{}}%
\column{10}{@{}>{\hspre}l<{\hspost}@{}}%
\column{11}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{prog2}\mathbin{::}\Conid{TaskTemplate}\;\Conid{Specification}\;\Conid{Program}{}\<[E]%
\\
\>[B]{}\Varid{prog2}\mathrel{=}\Varid{forFixed}\;\Varid{example}\mathbin{\$}\lambda \Varid{s}\to \mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{ts}\leftarrow \Varid{exampleTraces}\;\Varid{s}\;\mathrm{5}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\mathbin{\$}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}({}\<[10]%
\>[10]{}\text{\ttfamily \char34 Complete~the~given~skeleton~into~a~program~capable\char34}{}\<[E]%
\\
\>[10]{}\hsindent{1}{}\<[11]%
\>[11]{}\mathbin{+\!\!\!+} \text{\ttfamily \char34 of~these~interactions:\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\Varid{unlines}\;(\Varid{map}\;\Varid{show}\;\Varid{ts}){}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\text{\ttfamily \char34 ---\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\text{\ttfamily \char34 main~::~IO~()\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\text{\ttfamily \char34 main~=~do\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\text{\ttfamily \char34 ~~n~<-~readLn\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\text{\ttfamily \char34 ~~let~loop~s~m~=~undefined\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\text{\ttfamily \char34 ~~loop~0~0\char34}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{})\mathbin{`\Varid{solveWith}`}\Varid{producingTraces}\;\Varid{ts}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
Note that adherence to the skeleton is unchecked here.
In a general setting we would add an additional requirement to the template, but we currently already pose hand-written versions of such tasks and our current automatic grading setup for Haskell tasks already supports such checks~\cite{rahn2008leipzig,DBLP:conf/abp/Waldmann17}.
For example, we already give students programs such as the one above (together with a verbal task description), where they have to replace all occurrences of \ensuremath{\mathit{undefined}} by something appropriate.
The system then makes sure that only occurrences of \ensuremath{\mathit{undefined}} are replaced.

Additionally, we here use a fixed specification, \ensuremath{\Varid{example}} from page~\pageref{ref:example}, instead of a random one.
Using a random specification, without any further restrictions, potentially results in an unsolvable task since the skeleton can have the wrong structure for the underlying behavior.

Instead of using a fixed skeleton to guide a solution into a certain direction, we can also create the skeleton from a generated program.
As described above, internally program generation goes through a rendering step in which the final appearance of the code is determined.
We can create partial programs during this step, for example, by replacing certain parts of the program with holes.
This way we can create a skeleton that can always be complete to a valid solution.
Since we do not concern ourselves with the implementation details of this rendering, we simply assume that we have an implementation of the following generator available:
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{haskellWithHoles}\mathbin{::}\Conid{Specification}\to \Conid{Gen}\;\Conid{Description}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
Besides producing program skeletons, such partial programs are useful for other tasks as well.
For example, we can test students abilities to produce syntactically or well typed programs by letting them complete a partial program.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{7}{@{}>{\hspre}l<{\hspost}@{}}%
\column{10}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\mathbf{type}\;\Conid{Code}\mathrel{=}\Conid{String}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{prog3}\mathbin{::}\Conid{TaskTemplate}\;\Conid{Specification}\;\Conid{Code}{}\<[E]%
\\
\>[B]{}\Varid{prog3}\mathrel{=}\Varid{forUnknown}\;\Varid{randomSpecification}\mathbin{\$}\lambda \Varid{s}\to \mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{prog}\leftarrow \Varid{haskellWithHoles}\;\Varid{s}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\mathbin{\$}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}({}\<[10]%
\>[10]{}\text{\ttfamily \char34 Complete~the~following~template~into~a~syntactically\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{+\!\!\!+} \text{\ttfamily \char34 correct~program\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\text{\ttfamily \char34 (replace~the~???~with~calls~to~readLn~and~print)\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\Varid{prog}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{})\mathbin{`\Varid{solveWith}`}\Varid{compilingProgram}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{compilingProgram}\mathbin{::}\Conid{Require}\;\Conid{Code}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
The way the requirement is implemented, depends on the setting in which such a task is used.
In the context of our current e-learning setup every submission is already automatically check for compiler errors when submitted.
That means, we can implement the requirement with a property that always succeeds, as no non-compiling solution will ever trigger the actual test.
Also note that, in contrast to previous tasks, we here specify a requirement on the textual representation of a program instead of the ``program-value'' itself.

Another area where inspection of the program text is useful are requirements that enforce a solution to use, or not use, certain function or idioms.
For example, we can pose a task to rewrite some program that uses a list to accumulate values, and then does some computation expressible as a fold on this list, into a version not using any list, but instead directly carrying out the computation without first accumulating values.
Of course such a rewrite should preserve the original program's behavior.
Therefore, the task has requirements on both the actual program as well as its textual representation.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{7}{@{}>{\hspre}l<{\hspost}@{}}%
\column{9}{@{}>{\hspre}l<{\hspost}@{}}%
\column{11}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{prog4}\mathbin{::}\Conid{TaskTemplate}\;\Conid{Specification}\;(\Conid{Program},\Conid{Code}){}\<[E]%
\\
\>[B]{}\Varid{prog4}\mathrel{=}\Varid{forFixed}\;\Varid{example}\mathbin{\$}\lambda \Varid{spec}\to \mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{p}\leftarrow \Varid{haskellFoldProgram}\;\Varid{spec}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\mathbin{\$}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}({}\<[9]%
\>[9]{}\text{\ttfamily \char34 Re-write~the~given~program~s.t.~it~does~not~contain\char34}{}\<[E]%
\\
\>[9]{}\hsindent{2}{}\<[11]%
\>[11]{}\mathbin{+\!\!\!+} \text{\ttfamily \char34 any~accumulation~list.\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\Varid{p}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{})\mathbin{`\Varid{solveWith}`}(\Varid{behavior}\;\Varid{spec}\mathbin{/\char92 }\Varid{noLists}){}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{noLists}\mathbin{::}\Conid{Require}\;\Conid{Code}{}\<[E]%
\\
\>[B]{}\Varid{noLists}\mathrel{=}\Conid{Require}\mathbin{\$}\lambda \Varid{code}\to \Varid{property}\mathbin{\$}\neg \mathbin{\$}\text{\ttfamily \char34 ++\char34}\mathbin{`\Varid{isInfixOf}`}\Varid{code}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}(\mathbin{/\char92 })\mathbin{::}\Conid{Require}\;\Varid{a}\to \Conid{Require}\;\Varid{b}\to \Conid{Require}\;(\Varid{a},\Varid{b}){}\<[E]%
\\
\>[B]{}(\mathbin{/\char92 })\mathrel{=}divided\lambda \Varid{footnotemark}{}\<[E]%
\\[\blanklineskip]%
\>[B]{}\Varid{haskellFoldProgram}\mathbin{::}\Conid{Specification}\to \Conid{Gen}\;\Conid{Description}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
Once again, the details on how \ensuremath{\Varid{haskellFoldProgram}} is implemented internally are outside the scope of this presentation.
\footnotetext{\ensuremath{\Conid{Require}\;\Varid{s}} is a \ensuremath{\Conid{Divisible}}, \ensuremath{\Conid{Contravariant}} functor in s. \ensuremath{(\mathbin{/\char92 })} therefore simply is \ensuremath{\Varid{divided}} from \ensuremath{\Conid{\Conid{Data}.\Conid{Functor}.\Conid{Contravariant}.Divisible}} (see \url{https://hackage.haskell.org/package/contravariant-1.5.2/docs/Data-Functor-Contravariant-Divisible.html}).}
Almost none of the above tasks lets students start writing a program on a blank page.
Always some part of a solution is already given.
To a certain degree this is unavoidable in our approach to task generation, since we need a (non-verbal) description of the task's requirements.
But the amount of information about a solution present in this description can be varied.
For a free-form style programming task we can, as an example, give the behavior to be implemented in a different programming language, ideally using a different paradigm.
In our case, we use Python code for this purpose.
Our students learn Python as their first language and are therefore familiar with it.
This gives us the opportunity to precisely state what behavior a correct solution has without giving away too much information.
Mainly because I/O programs in Haskell usually have a different structure compared to imperative languages (e.g. recursive functions vs. explicit loops).
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{7}{@{}>{\hspre}l<{\hspost}@{}}%
\column{10}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{prog5}\mathbin{::}\Conid{TaskTemplate}\;\Conid{Specification}\;\Conid{Program}{}\<[E]%
\\
\>[B]{}\Varid{prog5}\mathrel{=}\Varid{forUnknown}\;\Varid{randomSpecification}\mathbin{\$}\lambda \Varid{s}\to \mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{prog}\leftarrow \Varid{pythonProgram}\;\Varid{s}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\mathbin{\$}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}({}\<[10]%
\>[10]{}\text{\ttfamily \char34 Re-implement~the~following~Python~program~in~Haskell:\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\Varid{prog}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{})\mathbin{`\Varid{solveWith}`}\Varid{behavior}\;\Varid{s}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
From an instance of this task one can gather almost no information on what a Haskell solution could look like:
\begin{tabbing}\ttfamily
~Re\char45{}implement~the~following~Python~program~in~Haskell\char58{}\\
\ttfamily ~n~\char61{}~int\char40{}input\char40{}\char41{}\char41{}\\
\ttfamily ~x~\char61{}~\char91{}\char93{}\\
\ttfamily ~while~len\char40{}x\char41{}~\char33{}\char61{}~n~\char58{}\\
\ttfamily ~~~v~\char61{}~int\char40{}input\char40{}\char41{}\char41{}\\
\ttfamily ~~~x~\char43{}\char61{}~\char91{}v\char93{}\\
\ttfamily ~print\char40{}sum\char40{}x\char41{}\char41{}
\end{tabbing}

\subsection{Decision tasks}

The examples above always ask students to provide either a program or some trace(s).
In most cases the respective other artifact was provided as the requirement description.
That is, for a given program a trace needs to be given or vice versa.
It might seem that these two options are the only possible ways to pose tasks in the presented framework, but there is a third possibility.
Instead of asking for a program or a trace we can instead ask students to make a decision.
This decision can be a simple binary choice as well as some multiple-choice questions.
The general approach for constructing such tasks is giving students two (or more) artifacts and asking them whether these artifacts originated from the same behavior.
To illustrate this type of task let us look at two examples.

First, a simple binary choice would be a task asking whether two given programs have the same behavior.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{7}{@{}>{\hspre}l<{\hspost}@{}}%
\column{10}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{desc1}\mathbin{::}\Conid{TaskTemplate}\;(\Conid{Specification},\Conid{Specification})\;\Conid{Bool}{}\<[E]%
\\
\>[B]{}\Varid{desc1}\mathrel{=}\Varid{forUnknown}\;\Varid{similarSpecifications}\mathbin{\$}\lambda (\Varid{spec1},\Varid{spec2})\to \mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{sameBehavior}\leftarrow \Varid{elements}\;[\mskip1.5mu \Conid{True},\Conid{False}\mskip1.5mu]{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}(\Varid{p1},\Varid{p2})\leftarrow {}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}\mathbf{if}\;\Varid{sameBehavior}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbf{then}\;\Varid{differentPrograms}\;\Varid{spec1}\;\Varid{spec1}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbf{else}\;\Varid{differentPrograms}\;\Varid{spec1}\;\Varid{spec2}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\mathbin{\$}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}({}\<[10]%
\>[10]{}\text{\ttfamily \char34 Do~the~following~two~programs~have~the~same~behavior?\char34}{}\<[E]%
\\
\>[5]{}\hsindent{2}{}\<[7]%
\>[7]{}\mathbin{\$\$}\Varid{p1}\mathbin{\$\$}\text{\ttfamily \char34 ---\char34}\mathbin{\$\$}\Varid{p2}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{})\mathbin{`\Varid{solveWith}`}\Varid{exactAnswer}\;\Varid{sameBehavior}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
For this template we need to generate two different programs from either the same or two different specifications.%
\footnote{
Technically we do not need to use \ensuremath{\Varid{differentPrograms}} to generate two programs with different behavior.
Any well behaving implementation of \ensuremath{\Varid{similarSpecification}} should not generate two specifications that can produce the same program.
}
Generating two different programs is straight forward given our primitives and the QuickCheck generator API.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{20}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{differentPrograms}{}\<[20]%
\>[20]{}\mathbin{::}\Conid{Specification}\to \Conid{Specification}{}\<[E]%
\\
\>[20]{}\to \Conid{Gen}\;(\Conid{Description},\Conid{Description}){}\<[E]%
\\
\>[B]{}\Varid{differentPrograms}\;\Varid{s1}\;\Varid{s2}\mathrel{=}\mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{p1}\leftarrow \Varid{haskellProgram}\;\Varid{s1}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{p2}\leftarrow \Varid{haskellProgram}\;\Varid{s2}\mathbin{`\Varid{suchThat}`}(\mathrel{/\hspace{-1ex}=}\Varid{p1}){}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\;(\Varid{p1},\Varid{p2}){}\<[E]%
\ColumnHook
\end{hscode}\resethooks
However, in practice one would probably want some additional control over how the two programs actually differ.

As mentioned, we can also build multiple-choice tasks.
Here it is again helpful to have randomness available when constructing a task instance.
We can define a generator that takes the number of choices we like to provide as well as two lists of correct and incorrect answers.
The generator then constructs a selection of choices and also provides the indices of the correct answers.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{8}{@{}>{\hspre}l<{\hspost}@{}}%
\column{10}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{multipleChoice}\mathbin{::}\Conid{Show}\;\Varid{a}\Rightarrow \Conid{Int}\to [\mskip1.5mu \Varid{a}\mskip1.5mu]\to [\mskip1.5mu \Varid{a}\mskip1.5mu]\to \Conid{Gen}\;(\Conid{Description},[\mskip1.5mu \Conid{Int}\mskip1.5mu]){}\<[E]%
\\
\>[B]{}\Varid{multipleChoice}\;\Varid{n}\;\Varid{rs}\;\Varid{ws}\mathrel{=}\mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\mathbf{let}\;{}\<[8]%
\>[8]{}\Varid{rs'}\mathrel{=}\Varid{map}\;(,\Conid{True})\;\Varid{rs}{}\<[E]%
\\
\>[8]{}\Varid{ws'}\mathrel{=}\Varid{map}\;(,\Conid{False})\;\Varid{ws}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{cs}\leftarrow \Varid{take}\;\Varid{n}\mathbin{<\hspace{-1.6pt}\mathclap{\raisebox{0.1pt}{\scalebox{.8}{\$}}}\hspace{-1.6pt}>}\Varid{shuffle}\;(\Varid{rs'}\mathbin{+\!\!\!+} \Varid{ws'}){}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\mathbf{let}\;{}\<[8]%
\>[8]{}\Varid{desc}\mathrel{=}\Varid{unlines}\mathbin{\$}\Varid{zipWith}\;(\lambda \Varid{i}\;\Varid{x}\to \Varid{show}\;\Varid{i}\mathbin{+\!\!\!+} \text{\ttfamily \char34 )~\char34}\mathbin{+\!\!\!+} \Varid{show}\;\Varid{x}){}\<[E]%
\\
\>[8]{}\hsindent{2}{}\<[10]%
\>[10]{}[\mskip1.5mu \mathrm{1}\mathinner{\ldotp\ldotp}\mskip1.5mu]{}\<[E]%
\\
\>[8]{}\hsindent{2}{}\<[10]%
\>[10]{}(\Varid{map}\;\Varid{fst}\;\Varid{cs}){}\<[E]%
\\
\>[8]{}\Varid{is}\mathrel{=}[\mskip1.5mu \Varid{i}\mid (\Varid{i},(\kern0.06em \vbox{\hrule\@width.5em} ,\Varid{correct}))\leftarrow \Varid{zip}\;[\mskip1.5mu \mathrm{1}\mathinner{\ldotp\ldotp}\mskip1.5mu]\;\Varid{cs},\Varid{correct}\mskip1.5mu]{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\;(\Varid{desc},\Varid{is}){}\<[E]%
\ColumnHook
\end{hscode}\resethooks
With such a generator we can then easily define a task that, for example, asks to choose which traces a given program can produce.
\begin{hscode}\SaveRestoreHook
\column{B}{@{}>{\hspre}l<{\hspost}@{}}%
\column{3}{@{}>{\hspre}l<{\hspost}@{}}%
\column{5}{@{}>{\hspre}l<{\hspost}@{}}%
\column{6}{@{}>{\hspre}l<{\hspost}@{}}%
\column{9}{@{}>{\hspre}l<{\hspost}@{}}%
\column{E}{@{}>{\hspre}l<{\hspost}@{}}%
\>[B]{}\Varid{desc2}\mathbin{::}\Conid{TaskTemplate}\;(\Conid{Specification},\Conid{Specification})\;[\mskip1.5mu \Conid{Int}\mskip1.5mu]{}\<[E]%
\\
\>[B]{}\Varid{desc2}\mathrel{=}\Varid{forUnknown}\;\Varid{similarSpecifications}\mathbin{\$}\lambda (\Varid{spec1},\Varid{spec2})\to \mathbf{do}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{p}\leftarrow \Varid{haskellProgram}\;\Varid{spec1}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{ts1}\leftarrow \Varid{exampleTraces}\;\Varid{spec1}\;\mathrm{5}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{ts2}\leftarrow \Varid{exampleTraces}\;\Varid{spec2}\;\mathrm{5}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}(\Varid{choices},\Varid{solution})\leftarrow \Varid{multipleChoice}\;\mathrm{7}\;\Varid{ts1}\;\Varid{ts2}{}\<[E]%
\\
\>[B]{}\hsindent{3}{}\<[3]%
\>[3]{}\Varid{return}\mathbin{\$}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{}({}\<[9]%
\>[9]{}\text{\ttfamily \char34 Which~of~the~given~trace~can~the~program~below~produce?\char34}{}\<[E]%
\\
\>[5]{}\hsindent{1}{}\<[6]%
\>[6]{}\mathbin{\$\$}\Varid{p}{}\<[E]%
\\
\>[5]{}\hsindent{1}{}\<[6]%
\>[6]{}\mathbin{\$\$}\Varid{choices}{}\<[E]%
\\
\>[3]{}\hsindent{2}{}\<[5]%
\>[5]{})\mathbin{`\Varid{solveWith}`}\Varid{exactAnswer}\;\Varid{solution}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks

\section{Related Work}
Tools for automatic task generation exist in a variety of different application areas, for example, general science questions~\cite{DBLP:conf/aclnut/WelblLG17}, math related exercises~\cite{DBLP:conf/latice/Kurt-KaraogluSS15} and programming tasks~\cite{DBLP:conf/norchip/MosbeckHJ18,DBLP:conf/csedu/StrieweBG09}.
Most approaches use some form of template together with a parameter generator to automatically create new task.
Especially for natural language tasks, some systems can generate tasks from existing databases of domain specific text~\cite{DBLP:conf/aclnut/WelblLG17}, but most systems for generating programming tasks rely on templates.
In contrast to our flexible EDSL approach most systems use external templates, with varying degrees of expressivness, that are provided as inputs to the task generation.
Consequently, they usually come coupled with a fully fleshed out e-learning environment.

\section{Conclusion \& Future work}
We have presented a framework that allows us to describe a diverse range of exercise templates on Haskell I/O.
All of the variety stems from the domain specific primitives providing the different artifacts the templates are built around.
These artifacts are used as stand-ins for verbal descriptions, allowing us to precisely state task requirements even in the case of randomization of the underlying behavior of a task.
Tasks in the presented style differ from traditional hand-written tasks, while still offering a lot of flexibility when designing tasks.
The difference to hand-written tasks is also not necessarily a restriction one has to work around, but enables new and interesting task designs.
To the best of our knowledge this usage of automatically derived artifacts is a novel approach to automatic task generation.

We have not yet had the opportunity to use the approach in practice.
We plan to do a systematic exploration of the design space of exercise task as preparation for the next iteration of our programming paradigms course.
Especially, we want to evaluate which tasks help students the most when learning Haskell I/O.
We plan to use the results of such a survey to craft automatic assessments for a range of different Haskell-I/O related skills that amplifies students understanding of the subject.
Also the question of how to write good generators for the specifications underlying each task is something that should be investigated further, as they have a big influence on the quality of task instances.

\bibliographystyle{splncs04}

redpajama_set_name: RedPajamaArXiv


text:
\section{Introduction}

\subsection{Definitions}
Let $T$ be either $\Z$ or $\R$, with the usual topology. A
\emph{Gaussian process} (GP) on $T$ is a random function $f:T\to \R$
whose finite marginals, that is $(f(t_1),\dots,f(t_n))$ for
      any $t_1,\dots,t_n\in T$, have multi-variate Gaussian distribution.
A GP on $\Z$ is called a \emph{Gaussian sequence}, while a GP on
$\R$ is called a \emph{Gaussian function}. In what follows, we
always assume continuity of Gaussian functions.

A GP on $T$ whose distribution is invariant with respect to shifts
by any element of $T$, is called \emph{stationary}. We abbreviate
GSP, GSS and GSF for Gaussian stationary processes, sequences and
functions respectfully.

For a GSP $f$ on $T$ define the \emph{covariance function} $r: \
T\to\R$ as
$$r(t)=\E(f(0) f(t)).$$
Observe that due to stationarity, for every $t,s\in T$ we have
$$\E \left[f(s) f(t)\right]=r(t-s). $$
It is not difficult to verify that $r(\cdot)$ is a positive-definite
continuous function (see Adler and Taylor ~\cite[Chapter
1]{AdlTay}). By Bochner's theorem, there is a finite non-negative
measure $\rho$ on $T^*$ such that
$$r(t) = \widehat {\rho}(t):=\int_{T^*} e^{-i\lm t}d\rho(\lm). $$
Here $T^*$ is the dual of $T$, i.e. $\Z^*\simeq [-\pi,\pi]$ and
$\R^*\simeq\R$. We use the notation $\M$ for the set of all finite
non-negative measures on $T^{*}$. The measure $\rho= \rho_f\in \M$
is called \emph{the spectral measure} of the process $f$. Notice
that $\rho$ must be symmetric, i.e., for any interval $I$:
$\rho(-I)=\rho(I)$. Any $\rho\in \M$ uniquely defines a GSP $f$.

Throughout the paper, we shall assume the following condition:
\begin{equation}\label{eq: cond moment}
  \exists \delta>0: \: \int_{T^*} |\lm|^\delta d\rho(\lm) <\infty.
\end{equation}
This condition is enough to ensure that the associated process $f$
will be continuous (see once again ~\cite[Chapter 1]{AdlTay}).
Notice that this holds trivially in case $T=\Z$.

\subsection{Results}
Let $f:T\to\R$ be a GSP. Define the "gap probability" of $f$ to be
$$H_f(N)=\Pro\left(\forall t\in[0,N)\cap T: \ f(t)>0\right), $$
where $N\in\R $ is a parameter. This describes the probability that
no sign-changes of $f$ occurred in a time interval of length $N$. We
study the asymptotics of this probability as $N\to \infty$. It makes
no essential difference to regard $N$ as an integer, and we usually
do so.

Our main results are the following. Let $f$ be a Gaussian stationary
process on $T=\Z$ or $T=\R$, with spectral measure
  $\rho\in \M$, satisfying ~\eqref{eq: cond moment}.

\begin{thm}[upper bound]\label{thm: upper bd gen}
Suppose that there exists $a>0$ and two positive numbers $M,m>0$
such that
$$ \text{for any interval } I\subset (-a,a), \:\: m|I|\le \rho(I)\le M|I|.$$
Then there exists $C=C(a,m,M)>0$ such that for all large enough $N$,
$$H_f(N)\le e^{-C N}.$$
\end{thm}

\begin{thm}[lower bound]\label{thm: lower bd gen}
Suppose that there exists $a>0$ and a number $m>0$ such that
$$ \text{for any interval } I\subset (-a,a), \:\: m|I|\le \rho(I).$$
Then there exists $c=c(a,m)>0$ such that for all large enough $N$,
$$H_f(N)\ge e^{-cN}.$$
\end{thm}

\begin{rmk}\label{rmk: up extend}
{\rm The condition in Theorem~\ref{thm: upper bd gen} may be
replaced by the following: There exist two intervals $J_1=(-a,a)$
and $J_2$, and two numbers $M,m>0$, such that
\begin{enumerate}[(i)]
  \item\label{item: up bd} for any interval $I\subset J_1$: $\rho(I)\le M|I|$, and
  \item\label{item: low bd} for any interval $I\subset J_2$: $m|I|\le \rho(I)$.
\end{enumerate}
The necessary changes in the proof are indicated in Section
\ref{sec: up-extend}. However, the authors believe condition
~\eqref{item: up bd} might be enough to ensure an upper exponential
bound on $H(N)$. }
\end{rmk}

\begin{rmk}\label{rmk: fast slow}
{\rm Examples for which $H(N)$ tends to zero slower than any
exponential in $N$ are known; Newell and Rosenblatt construct one in
~\cite{NR}.

Examples for which $H(N)$ tends to zero faster than any exponential
in $N$ are also known. A simple example was pointed out to us by M.
Krishnapur. Let $(Y_j)_{j\in\Z}$ be a GS with independent entries,
and define $X_j=Y_j-Y_{j-1}$ for all $j\in \Z$. Then $X$ is a GSS
with $H_X(N) = \frac 1{N!} \simeq e^{-C N\log N}$, for a suitable
constant $C>0$. Notice that the spectral measure has density
$2(1-\cos(\lm))$, $\lm \in [-\pi,\pi]$, which vanishes at $\lm=0$.}
\end{rmk}

\subsection{Overview}
The rest of the paper is organized as follows. Section~\ref{sec:
discuss} is devoted to discussion of the results. This includes an
historical background, and a simple yet useful observation that we
shall use (Observation~\ref{obs: rho1+rho2} below). The results are
then proved independently: Theorem~\ref{thm: upper bd gen} (an upper
exponential bound) is proved in Section~\ref{sec: up}, while
Theorem~\ref{thm: lower bd gen} (a lower exponential bound) is
proved in Section~\ref{sec: low}.

\subsection{Acknowledgements}
We thank Mikhail Sodin for introducing
us to the problem and for his advice throughout the research. We are
grateful to Ron Peled for a conversation which laid the foundations
to Theorem~\ref{thm: upper bd gen}. Discussions with Jeremiah
Buckley, Amir Dembo, Manjunath Krishnapur, Zakhar Kubluchko,
Jan-Fredrik Olsen and Ofer Zeitouni improved our understanding of
the problem, its applications and its relation to other works.

\section{Discussion}\label{sec: discuss}
\subsection{Background}
Gap probability, sometimes referred to by the name "persistence
probablity" or "hole probability", was studied extensively in the
1960's, by Slepian \cite{Slep}, Longuet-Higgins~\cite{LonHig},
Newell-Rosenblatt~\cite{NR} and others. In addition to proving some
bounds and inequlities (such as the well-known "Slepian
inequality"), they developed series expansions which approximate
this probability quite well for small intervals. In a few examples,
exact expressions for the gap probability were calculated (see
~\cite{Slep} and references therein).

In the last decade or two, physicists (such as Majumdar-Bray
~\cite{phys1} and Ehrhardt-Majumdar-Bray ~\cite{phys2}) proposed
some new methods of approximation, especially for the long-range
regime. Their predictions suggest that in many cases of interest the
gap probability $H(N)$ behaves asymptotically like $e^{-\theta N}$,
with some $\theta>0$. A rigorous derivation of such a result is
still lacking.

In case the covariance function $r(t)$ is non-negative, Dembo and
Mukherjee~\cite[Theorem 1.6]{DemMu} proved those predictions are
correct; namely, that the limit
$$\lim_{N\to\infty} \frac {-\log H_f(N)}{N}$$
exists (possibly infinite).
The case when $r(t)$ changes sign, as well as computation of the
limit, remain open. We note that the work last mentioned, along with
other works by physicists such as Schehr-Majumdar~\cite{ScMa}, draw
connections between gap probabilities of GSPs, those of
diffusion processes, and those of zeros of random polynomials.

In this work we are interested in the case where $r(t)$ changes sign.
A simple and interesting example is the cardinal sine covariance $r(t)=\frac{\sin (\pi t)}{t}$,
which corresponds to indicator spectral density $\ind_{[-\pi,\pi]}$. In
an elegant recent work, Antezana, Buckley, Marzo and Olsen ~\cite{Barc} give
exponential upper and lower bounds for $H_f(N)$ (see
Theorem~\ref{thm: barc} below). Our research may be viewed as a an
extension of their result to other \emph{stationary} Gaussian
processes. Recently Antezana, Marzo and Olsen were able to
generalize this same result in the direction of Gaussian analytic
functions over \emph{de-Branges spaces} ~\cite{AMO}.

Via private communication we learned of results by
Krishnapur-Maddaly regarding lower bounds for the gap
probability of a SGS. It seems that our conditions for a lower
exponential bound are currently stronger, but they have given very
mild conditions which ensure $H_f(N)\ge e^{-cN^2}$ (where $c>0$ is a
constant, and the inequality holds for large ehough $N$). Though the
results are similar in spirit, their methods seem to be very
different from ours.

Lastly we mention an analogous result for the planar Gaussian
analytic function
$$\sum_{n\in \Z}a_n \frac{z^n}{\sqrt{n!}}, \text{  where } a_n\sim \calN_\C (0,1) \text{ are i.i.d.}$$
Bounds concerning hole probabilities for this model were obtained by
Sodin and Tsirelson~\cite{ST3}, and later refined by
Nishry~\cite{Alon}. They showed that the probability of having no
zeroes in a ball of radius $R$ in the plane is asymptotically
$e^{-(e^2/4+o(1))R^4}$, as $R\to\infty$. For discussion of such
results and comparison to other point processes in the plane, see
\cite[Chapter 7]{GAF book}.

\subsection{A Key Observation}\label{sec: obs}
We include here the basic observation which will be used to prove
both Theorems ~\ref{thm: upper bd gen} and ~\ref{thm: lower bd gen}.
We use the symbol $\oplus$ to indicate the sum of two independent
processes or random variables.

\begin{obs}\label{obs: rho1+rho2}
Let $f$ be a GSP on $T$ with spectral measure $\rho\in \M$, and
Suppose $\rho=\rho_1+\rho_2$, where $\rho_1, \rho_2 \in \M$. Then
the following equality holds in distribution:
$$f\overset{d}{=}f_1 \oplus f_2, $$
where $f_j$ is a GSP with spectral measure $\rho_j$ ($j=1,2$), and
$f_1$ is independent (as a process) from $f_2$.
\end{obs}

\begin{proof}
We calculate the covariance function of $f_1\oplus f_2$ using the
independence of the processes:
\begin{align*}
\E \Big[\left(f_1(0)+f_2(0)\right)\left(f_1(t)+f_2(t)\right) \Big]
&= \E f_1(0)f_1(t) + \E f_2(0)f_2(t) \\
&=\widehat{\rho_1}(t)+\widehat{\rho_2}(t) = \widehat{\rho}(t).
\end{align*}
This covariance function is equal to that of $f$. As all processes
are Gaussian, the observation follows.
\end{proof}

\section{Upper bound: proof of Theorem~\ref{thm: upper bd gen}}\label{sec: up}
This section is devoted to the proof of Theorem~\ref{thm: upper bd
gen}.

Let $f$ be a GSF or GSS with spectral measure $\rho$, obeying the
conditions of Theorem~\ref{thm: upper bd gen}. Let $k\in\N$ be such
that $\frac \pi k \le a$, and denote $J:=[-\pi / k, \pi /k]\subset
[-a,a]$. We decompose the spectral measure as follows:
$$d\rho(\lm)=m\ind_{J}(\lm)d\lm + d\mu(\lm),$$
where $\mu\in \M$ is non-negative and there
exists $M'>0$ such that
\begin{equation}\label{eq: cond bound on mu}
\text{for any interval } I \subset (-a,a): \: \mu(I)\le M'|I|.
\end{equation}

By Observation~\ref{obs: rho1+rho2}, we may represent
$$f\overset{d}{=}S\oplus g$$
where $S$ and $g$ are independent processes, with spectral measures
$m\ind_{J}(\lm)$ and $\mu$ respectively.

Next, we observe that sampling $S$ in a certain lattice results in
independent random variables:

\begin{obs}[indicator spectrum]\label{obs: ind spec}
The GSP $(S(t))_{t\in T}$ having spectral density
\emph{$m\ind_{[-\pi / k, \pi /k]}$} has the property that
$(S(jk))_{j\in\Z}$ are i.i.d. Gaussian random variables.
\end{obs}

\begin{proof}
By taking the Fourier transform of the given measure, the covariance
function of $S$ is
$$\E\left[ S(s) S(t) \right]= \frac {\sin(\pi (t-s)/k)}{(t-s)}. $$
Thus $S(jk)$ and $S(mk)$ are uncorrelated for any $j,m\in\Z$, $j\neq
m$; as these are Gaussian random variables - independence follows.
\end{proof}

In order to apply Observation~\ref{obs: ind spec}, we look at a
certain translated lattice $\{jk+l: j\in \Z\}$ on which $S$ is
indeed independent. The translation (which we call "split") of the
sampled lattice will depend on $g$.

More precisely, fix a number $q>0$ (say, $q=1$), and define an event
$E$ depending only on the process $(g(t))_{t\in T}$ in the following
way:
\begin{align*}
\ds
E&=\left\{\frac 1 N \sum_{t=1}^N g(t) < q \right\}, \text{ if $g$ is GSS}\\
E&=\left\{\frac 1 N \int_{0}^N g(t) dt < q \right\}, \text{ if $g$ is GSF}
\end{align*}
Using the law of total probability we have:
\begin{align*}
  \ds \Pro &\left( f(t)=S(t)+g(t) >0, \ 0< t\le N\right) \notag \\
  &\le   \Pro\left(S(t) +g(t)>0, \ 0< t\le N \: \Big|\: E \right) +
  \Pro\left(E^c \right).
\end{align*}
It is enough to show that there exist $C_1,C_2>0$ such that for
large enough $N$,
\begin{enumerate}[(I)]
\item~\label{item: given E} $\Pro\left(S(t) +g(t)>0, \ 0< t\le N \: \Big|\: E \right)
\le e^{-C_1 N}$, and
\item~\label{item: E^c} $ \Pro\left(E^c \right)\le e^{-C_2 N}$.
\end{enumerate}
We proceed the proof for the function-case, noting the sequence-case
follows similar lines and is generally easier.

We begin by showing ~\eqref{item: given E}.
It is enough to show
that there is $C_1>0$ such that for any large enough $N$ and any
fixed $g\in E$,
\begin{equation*}
\Pro\left(S(t) +g(t)>0, \ 0< t\le N \right) \le e^{-C_1 N}.
\end{equation*}
Indeed, this would imply (using the independence of $g$ and $S$):
\begin{align*}
&\Pro\left(S(t) +g(t)>0, \ 0< t\le N |\  E \right)\\
&=\E\left(  \Pro\left(S(t) +g(t)>0, \ 0< t\le N \right) \ \big| \ E  \right) \le e^{-C_1 N},
\end{align*}
as required.

To that end, we use a
property which holds when the event $E$ occurs, stated below.
\begin{obs}\label{obs: split}
Let $g$ be a continuous function such that
$\frac 1 N \int_0^N g(t) dt <q$,
and assume $N\in \N$ is divisible by $k$,  then there exists a number $l\in[0,k)$ such that
$$\frac k N \sum_{j=0}^{N/k-1} g(jk+l)<q. $$
\end{obs}

\begin{proof}
  Else, for every $l\in[0,k)$ the reverse inequality holds.
  Integrating it over $l\in[0,k]$ yields a contradiction.
\end{proof}

Now, fix a
function $g\in E$.
We can find a special split $l_g$ whose existence
is guaranteed by Observation~\ref{obs: split}. Therefore:
\begin{align*}
&\Pro\left(S(t) +g(t)>0, \ 0< t\le N \right) \\
&\le\Pro\left(S(jk+l_g)+g(jk+l_g)>0, \ j=0,1,\dots, N/k -1\right),
\end{align*}
where $(S(jk+l))_{j\in\Z}$ are i.i.d Gaussians (whose variance is
independent of $l_g$), and $\frac k N
\sum_{j=0}^{N/k-1} g(jk+l_g)<q$.
The following inequality will give the desired
bound.

\begin{prop}\label{prop: Xj+aj>0}
  Let $X_1,\dots,X_N$ be i.i.d real centered Gaussian random variables, and let $q\in \R$.
  There is a constant $C_q>0$ such that
  for any numbers $b_1,\dots,b_N\in \R$ which obey $\frac 1 N\sum_{j=1}^N b_j<q$, the following holds:
  \[
  \Pro\left(X_j+b_j>0,\: 1\le j\le N\right)\le e^{-C_q N}.
  \]
\end{prop}

\begin{proof}
Without loss of generality assume $\text{var}(X_1)=1$. Denote by
$\Phi(b) =\Pro(X_1<b)$ the cumulative distribution function of
$X_1$. By symmetry, $\Phi(b)=\Pro(X_1>-b)$. Using the "i.i.d"
property of the variables $\{X_j\}_{j=1}^N$ we have:
\[
p=\Pro\left(X_j+b_j>0,\: 1\le j\le N\right)=\prod_{j=1}^N
\Pro\left(X_j>-b_j\right)=\prod_{j=1}^N \Phi(b_j).
\]
Taking logarithm and using the concavity and monotonicity of
$x\mapsto \log \Phi(x)$, we get:
\[
\log p = \sum_{j=1}^N \log \Phi(b_j)\le N \cdot\log
\Phi\left(\frac{\sum_j^N b_j}{N}\right)< N\cdot \log \Phi(q),
\]
and so $C_q=-\log \Phi(q)>0$ is the desired constant.
\begin{comment}
For completeness, we include the proof of concavity of $\log
\Phi(x)$. We compute two derivatives.
\begin{align*}
  \log \Phi(x) &= \log\left(\int_{-\infty}^x e^{-t^2/2}
  dt\right)-\log \sqrt{2\pi} \\
  (\log \Phi)'(x)&=\frac {e^{-x^2/2}}{\int_{-\infty}^x e^{-t^2/2}
  dt} = \left[e^{x^2/2}\cdot\int_{-\infty}^x e^{-t^2/2}dt
  \right]^{-1}\\
  (\log \Phi)^{\prime \prime}(x)&=-\left[e^{x^2/2}\cdot\int_{-\infty}^x e^{-t^2/2}dt
  \right]^{-2}\cdot \left( 1+xe^{x^2/2}\int_{-\infty}^x e^{-t^2/2}dt
  \right)
\end{align*}

In order to show concavity we need $\forall x\in\R: \:(\log
\Phi)^{\prime \prime}(x)<0$. For $x\ge 0$ this is clear from our
calculation. For $x<0$ write $y=-x>0$; we need to check that
\begin{equation}\label{eq: Gauss bound}
\forall y>0: \:\:\int_y^\infty e^{-t^2/2}dt <\frac 1 y e^{-y^2/2}.
\end{equation}
Indeed: $y\int_y^\infty e^{-t^2/2}dt\le\int_y^\infty t e^{-t^2/2}dt
  =e^{-y^2/2}.$
\end{comment}
\end{proof}

In order to prove ~\eqref{item: E^c}, we shall use the following:
\begin{prop}\label{prop: II}
$\frac 1 N \int_0^N g(t) dt\sim \calN_\R(0,\si_N^2)$, where
$\si_N^2\le \frac {C_0}{N}$ for all $N\in\N$ and some constant
$C_0>0$.
\end{prop}
\begin{proof}
The normality of the given integral follows from general arguments
of convergence of Gaussian random variables.
We focus on the bound on its variance. Recall that $\mu$ denoted the
spectral measure of $g$. We calculate the variance:
\begin{align*}
  \si_N^2&=\frac 1 {N^2} \E\left(\int_0^N g(t) dt\right)^2
  = \frac 1 {N^2}\iint_{[0,N]^2} \E (g(t) g(s)) dt\ ds\\
  &=\frac 1 {N^2} \int_0^N \int_0^N \widehat{\mu}(t-s) dt\ ds
  =\frac 1 {N} \int_{|t|<N}\left(1-\frac {|t|}{N}\right) \widehat{\mu}(t)dt.
\end{align*}
The change in order of integration and expectancy in the first
equality is easily justified by use of Fubini's theorem.

The inverse Fourier transform of $(1-\frac {|t|}{N})\ind_{[-N,N]}(t)$ is given by
$$K_N(\lm)=N\left(\frac {\sin(N\lm/2)}{N\lm/2}\right)^2 \le \min\left(N,\frac {\pi^2}{N\lm^2}\right)$$
Using first Plancherel's identity, and then condition~\eqref{eq: cond bound
on mu} on the boundness of $\mu$, we get:
\begin{align*}
\si_N^2 &= \frac 1 N \int_\R K_N(\lm) d\mu(\lm) \\
&\le \int_{|\lm|< \frac \pi N} d\mu(\lm) + \frac {\pi^2}{N^2}\left(\int_{\frac{\pi}{N}\le |\lm|<a} +\int_{|\lm|\ge a}\right)\frac 1 {\lm^2}d\mu(\lm)\\
& \le M'\cdot \frac {2\pi}{N} + \frac {\pi^2}{N^2}\left( M' \int_{\frac{\pi}{N}\le |\lm|<a} \frac {d\lm} {\lm^2} + \frac 1 {a^2} \mu(\left\{|\lm|>a\right\}) \right) \\
& \le \frac {C_0} N,
\end{align*}
where $C_0$ is a constant (depending on $\mu$).
\end{proof}

At last, we prove ~\eqref{item: E^c}. Denote by $\g$ a standard
Gaussian random variable (i.e., distributed $\calN(0,1)$). Using the
Proposition~\ref{prop: II} together with the well-known inequality
\begin{equation*}
\forall y>0: \:\:\Pro\left(\g >y\right) <\frac 1 {\sqrt{2\pi}y}
e^{-y^2/2},
\end{equation*}
we get:
\begin{align*}
  \Pro(E^c)=\Pro\left( \frac 1 N \int_{0}^N g(t) \ge q \right)& =
  \Pro(\si_N \cdot \g \ge q)=\Pro\left(\g \ge \frac q {\si_N}\right) \\
  &\le \frac 1 {\sqrt{2\pi} }\cdot \frac{\si_N}{q} e^{-\frac 1 2 \cdot
  \frac{q^2}{\si_N^2}}\\
  & \le \frac 1 q \sqrt{\frac { C_0}{2\pi N}}  e^{-\frac{q ^2}{2 C_0} N} \le e^{-C_2 N},
\end{align*}
for a suitable choice of $C_2>0$ (depending only on $q$ and $\mu$).
Theorem~\ref{thm: upper bd gen} is proved.
\qed

\subsection{Extension: Proof of Remark~\ref{rmk: up extend}}\label{sec: up-extend}
Remark~\ref{rmk: up extend} states a somewhat more general condition
under which the conclusion of Theorem~\ref{thm: upper bd gen} is
true. The proof is only a slight modification of the one presented.
First, choose $l,k\in \N$ so that
$$J:=\left[\frac {(2l-1)\pi}{k},\frac {2l\pi}{k}\right] \subset J_2\cup(-J_2) .$$
Now decompose the measure as follows:
$$d\rho(\lm)=m\ind_{J\cup -J}(\lm)d\lm + d\mu(\lm).$$
By the premise, $\mu\in \M$ obeys the boundedness condition~\eqref{eq:
cond bound on mu} (just as before). Applying Observation ~\ref{obs:
rho1+rho2} we get
$$f\overset{d}{=} S \oplus g,$$
where $S$ has spectral measure $m\ind_{J\cup -J}(\lm)d\lm$ and $g$
has spectral measure $\mu$. We define $E$ and strive to prove items
~\eqref{item: given E} and ~\eqref{item: E^c}. Item~\eqref{item:
E^c} follows from Proposition~\ref{prop: II} and the calculation
following it with no change. The only property used in order to
prove item~\eqref{item: given E} is the independence of
$(S(jk))_{j\in\Z}$ (i.e., Observation~\ref{obs: ind spec}). Let us
show this still holds.

One way to end the argument is by calculation of the Fourier
transform of $\ind_{J\cup -J}(\lm)d\lm$ and observing it vanishes at
$kj$, $j\in\Z$ (just as in the proof of Observation~\ref{obs: ind
spec}). We give here a more general argument, relying on two
observations:
\begin{obs}\label{obs: scale}
  Let $(f(t))_{t\in \R}$ be a GSF with spectral measure $\rho$,
  and $\al>0$. Then the GSF $x\mapsto f(\al x)$  has spectral measure $\rho_\al$, defined by
  $$\forall I\subset \R:\ \: \rho_\al(I)=\rho(\{x\in \R:\ \al x \in I\}) $$
\end{obs}

\begin{proof}
  $\E \left[f(\al t)f(\al s)\right] = \widehat{\rho}(\al(t-s)) = \widehat{\rho_\al}(t-s)$.
\end{proof}

\begin{obs}\label{obs: sample}
  If $(f(t))_{t\in \R}$ is a GSF with spectral measure $\rho$,
  then sampling the lattice $(f(j))_{j\in\Z}$
  has the folded spectral measure $\rho^*\in\mathcal{M^{+}}([-\pi,\pi])$ obtained by:
  $\rho^*(I) = \sum_{m\in \Z }\rho(I + 2\pi m)$.
\end{obs}

\begin{proof}
  $\rho^{*}$ is the unique measure in $\mathcal M^{+}([-\pi,\pi])$ such that $\widehat{\rho^{*}}(j)=\widehat{\rho}(j)$ for any $j\in\Z$.
\end{proof}

Combining the last two observations, we get that if $(S(t))_{t\in
T}$ has spectral density $m\ind_{J\cup -J}(\cdot)$, then the
spectral density of $(S(kj))_{j\in\Z}$ is
$m\ind_{[-\pi,\pi]}(\cdot)$. Now Observation~\ref{obs: ind spec}
leads to the desired conclusion.

\section{Lower bound: Proof of Theorem~\ref{thm: lower bd gen}}\label{sec: low}
\subsection{Reducing GSS to GSF}
Theorem~\ref{thm: lower bd gen} is easily reduced to the case of
functions, by noticing the following:
\begin{obs}\label{obs: Hf<Hx}
  Any finite measure $\rho\in \mathcal{M}^{+}([-\pi,\pi])$ generates a GSF
  $f$ and a GSS $X$. The distribution of $(X(j))_{j\in\Z}$ is the
  same as that of $(f(j))_{j\in\Z}$ (since their covariance functions coincide).
  Moreover, for any number $N$:
\begin{align*}
H_f(N)&=\Pro(f(x)>0, \: x\in [0,N)\cap \R) \\
&\le \Pro(f(j)>0, \: j\in
  [0,N)\cap \N)=H_X(N).
  \end{align*}
\end{obs}

Therefore, in order to bound $H_X(N)$ from below where $X$ is a GSS,
it is enough to bound $H_f(N)$ from below where $f$ is the GSF with
the same spectral measure as $X$.

\subsection{Proof for GSF}
Let $(f(t))_{t\in\R}$ be a GSF with spectral measure $\rho$, obeying
the condition of Theorem~\ref{thm: lower bd gen}. By scaling $f$
(and therefore scaling its spectral measure according to
Observation~\ref{obs: scale}, we may assume the condition is
satisfied with $a=\pi$.

Just as in the proof of Theorem~\ref{thm: upper bd gen}, we
decompose the spectral measure in the following manner:
$$d\rho=m\ind_{[-\pi,\pi]}(\lm)d\lm + d\mu.$$
Applying Observation~\ref{obs: rho1+rho2} we have
$$f\overset{d}{=}S\oplus g$$
where $S$ and $g$ are independent processes, and the spectral
measure of $S$ has density $m\ind_{[-\pi,\pi]}(\lm)$.

We have:
\begin{align}\label{eq: main low}
  H_f(N)&=\Pro\left(S(x)+g(x)>0,\:\: 0\le x< N\right) \notag \\
  & \ge \Pro\left(S(x)>d,\; 0\le x< N\right)\Pro\left(|g(x)|\le \frac d 2, \;0\le x< N\right),
\end{align}
where $d>0$ is a parameter of our choice. The first probability is
bounded from below by the following theorem:

\begin{thm} [Antezana, Buckley, Marzo, Olsen ~\cite{Barc}] \label{thm: barc}
Let $S(x)$ be the GSF with spectral measure {\rm
$d\rho(\lm)={\ind}_{[-\pi,\pi]}(\lm)d\lm$}. Then for any $d>0$ there
exists a constant $c_d>0$, such that for all $N\in\N$,
\[
\Pro\left(S(x)>d, \:\:0\le x< N\right) \ge e^{-c_d N}.
\]
\end{thm}

We turn to bound the second probability in ~\eqref{eq: main low},
i.e., the probability of the event $\{|g(x)|\le \ep, \:\: 0\le x< N
\}$. This is known in literature as a "small ball probability", and
is bounded from below by the following result:

\begin{lem}[Talagrand~\cite{Tal}, Shao and Wang ~\cite{ShaoWang}]\label{lem: ShW}
Let $(f(t))_{t\in I}$ be a centered Gaussian process on a finite
interval $I$. Suppose that for some $c>0$ and $0<\delta\le 2$,
\begin{equation*
d_f(s,t)^2:= \E |f(s)-f(t)|^2 \le c|t-s|^{\delta}, \:\:s,t\in I.
\end{equation*}
Then, for some $K>0$ and every $\ep>0$,
$$\Pro\left(\sup_{t\in I}{|f(t)|\le \ep} \right)
\ge \exp\left(-\frac{K |I|}{\ep^{2/\delta}}\right).$$
\end{lem}

The proof of Lemma~\ref{lem: ShW}, apart from being deduced from a
much more general result in Talagrand's paper, may be found in notes
by Ledoux~\cite[Ch. 7]{Led} (but in a slightly different version).
Shao and Wang decided to omit a proof from their paper as they
learned that Talagrand's result generalizes theirs; but they do
include the most close formulation to the one above.

We draw the following corollary:
\begin{cor}\label{cor: moment small ball}
Let $f$ be a Gaussian stationary function on $\R$ with spectral
measure $\rho$, obeying the moment condition~\eqref{eq: cond
moment}. Then for all $\ep>0$ there exists $C, K>0$ such that for
any interval $I$ and any $N\in\N$:
$$\Pro\left(\sup_{I} |f| <\ep \right)\ge C e^{-K|I|}.$$
\end{cor}

Applying the corollary to $f=g$, $I=[0,N)$ and $\ep=\frac d 2>0$,
will give the desired bound on the second factor in ~\eqref{eq: main
low}, thus ending the proof of Theorem~\ref{thm: lower bd gen}.

\begin{proof}[Proof of Corollary ~\ref{cor: moment small ball}]
First we notice that if the moment condition~\eqref{eq: cond moment}
is satisfied with a certain exponent ~$\delta>0$, then it is also
satisfied by any smaller positive exponent. Therefore we may assume
$0<\delta<2$.

We shall check that $f$ obeys the condition of Lemma ~\ref{lem: ShW}
with this same ~$\delta$, i.e. that there exists a constant $c>0$
such that
\begin{equation*
d_{f}(s,t)^2\le c|t-s|^{\delta}, \:\: s,t\in I.
\end{equation*}
Indeed:
\begin{align*}
  d_f(s,t)^2&= \E (f(s)-f(t))^2
  =2(r(0)-r(s-t))\\
  &=2\int_\R\Big(1-\cos(\lm( s-t)) \Big) d\rho(\lm)\le 2L|t-s|^\delta \int_\R |\lm|^\delta d\rho(\lm),
\end{align*}
where $L = \sup_{x\in\R} \frac {1-\cos(x)}{|x|^\delta}<\infty$. The
Corollary follows.
\end{proof}


text:
Metamesia is a genus of moths belonging to the subfamily Tortricinae of the family Tortricidae.

Species
Metamesia ametria Diakonoff, 1960
Metamesia catadryas (Meyrick, 1937)
Metamesia designata (Meyrick, 1921)
Metamesia dilucida Diakonoff, 1960
Metamesia elegans (Walsingham, 1881)
Metamesia endopyrrha (Meyrick, 1930)
Metamesia episema Diakonoff, 1960
Metamesia incepta (Meyrick, 1912)
Metamesia intensa (Meyrick, 1921)
Metamesia leptodelta Diakonoff, 1973
Metamesia leucomitra Diakonoff, 1960
Metamesia leucophyes Diakonoff, 1960
Metamesia metacroca Diakonoff, 1960
Metamesia nolens Diakonoff, 1960
Metamesia peracuta Diakonoff, 1960
Metamesia physetopa (Meyrick, 1932)
Metamesia ptychophora Diakonoff, 1960
Metamesia retrocitra Diakonoff, 1960
Metamesia synclysa Diakonoff, 1973

Former species
Metamesia octogona Bradley, 1965

See also
List of Tortricidae genera

References

 , 2005: World catalogue of insects volume 5 Tortricidae.
 , 2010: Tortricidae (Lepidoptera) from Ethiopia Journal of Entomological and Acarological Research Serie II 42 (2): 47–79. Abstract: .

External links
tortricidae.com

Archipini
Tortricidae genera


Although called red bush tea, rooibos tea does not have theine, which caffeine in teas. This suggests that this drink is not a tea, however an infusion. It is one of the most popular natural ingredients due to its slimming impacts.
It assists improve digestion. As a result, all the food you consume is digested appropriately, which means that there are no fat deposits that sit in your stubborn belly location making it look big.
Its antioxidant properties increase the metabolism of cells, making sure that nutrients are dispersed appropriately throughout your system.
It is a natural diuretic, which suggests that it effectively eliminates contaminants, decreases fluid retention and cleans the body.
It helps assist in digestive transit, avoiding irregularity, which is among the causes of weight gain.
It has actually been known to reduce your hunger, if you drink this tea on a regular basis. You'll feel full faster if you drink it 30 minutes or two prior to eating. It has been used for many centuries to make a person's brain signal the stomach that it's already complete.
It can enhance the quality of sleep, which accelerates your metabolic process and your body's fat burning capability. Being well rested would also increase your energy levels, providing you more endurance to exercise and keep yourself active at all times.
It helps relieve stress, which is one of the major causes of binge consuming and ultimately, weight gain. Plus, it has a calming result on the main anxious system, which assists you yearn for less for unhealthy food.