\documentclass{article}
%include polycode.fmt

\begin{document}

\subsection{Introduction}

In this post I'll try to convince you that optic laws (as in the \texttt{lens} package
documentation) are the right ones. For example recall the \href{http://hackage.haskell.org/package/lens-4.17/docs/Control-Lens-Lens.html#t:Lens}{|Lens|} laws
(law names come from alternative naming of the operations: |view = get| and |set = put|\footnote{but |get| and |put| clash with |MonadState| names}.

\begin{itemize}
\item \textbf{GetPut}: You get back what you put in:

\begin{code}
view l (set l v s) ≡ v
\end{code}

\item \textbf{PutGet}: Putting back what you got doesn't change anything:

\begin{code}
set l (view l s) s ≡ s
\end{code}

\item \textbf{PutPut}: Setting twice is the same as setting once:

\begin{code}
set l v' (set l v s) ≡ set l v' s
\end{code}
\end{itemize}

Why there are these three laws (why three?).
How we know these three laws are "enough",
or that they don't require too much?

The solution is to consider a representation which is obviously a |Lens| (or |Prism| or ...),
but for which also the "laws" are obvious. Then when we write conversion functions
between this and more concrete representation we'll find out what laws are needed.

The |Iso| - an isomorphism is the simplest example.
The concrete representation is

\begin{code}
data Iso s a = Iso { to :: s -> a, from :: a -> s }
\end{code}

But the an obvious definition in math world (citing \href{https://global.oup.com/academic/product/category-theory-9780199237180}{Awodey 2010}):
In any category $\mathcal{C}$, an arrow $f : S \to A$ is called an \emph{isomorphism},
if there is an arrow $g : A \to S$ in $\mathcal{C}$ such that
$g \circ f \equiv 1_S$ and $f \circ g \equiv 1_A$.
We write $S \cong A$ if there exists an isomorphism between $S$ and $A$.

Here, the mathematical conditions tells precisely which laws are required,
namely for a (\texttt{RecordWildCards} destructed) |Iso| fields:

\begin{code}
to . from ≡ id
from . to ≡ id
\end{code}

The isomorphism case is trivial. In the post I'll go through more interesting
optics:

\begin{itemize}
\item |Lens| and |Prism|. Their laws should not be surprising to people
    familiar with \texttt{lens} library.
    However we discover a new law for |Prism|.

\item |Affine| (traversal). I discussed affine traversal in
\href{2017-03-20-affine-traversal.html}{a post about them}
and give laws in
\href{2017-04-18-glassery.html#laws:affine-traversal}{Glassery}.
These laws were written based purely on intuition, which we'll now verify.

\item |Achroma|tic lens. There are
  \href{2018-03-28-achromatic-lens.html}{my quickly written notes from March 2018}.
  Turns out, I missed few bits.

\item |Grate|. Russel O'Connor gives laws for a van Laarhoven Grate (a two functor variant) in
\href{https://r6research.livejournal.com/28050.html}{Grate: A new kind of Optic}.
We formulate them for "normal" Grates: |grate :: ((s -> a) -> b) -> t|.
In the process we also learn more about |Grate|.
\end{itemize}

At the end we discuss a little about |Setter|s. And conclude the post
with general thoughts.

The \href{https://twanvl.nl/blog/haskell/isomorphism-lenses}{Isomorphism lenses} by Twan van Laarhoven
makes the same things for lenses; we do it for more classes of optics.

\subsection{Lens}

A \emph{Lens} is a purely functional reference. Concrete |Lens| representation
is

\begin{code}
data Lens s a = Lens { view :: s -> a, set :: s -> a -> s }
\end{code}

\textbf{Note:} Here and later we work with so called monomorphic optics,
i.e. optics which don't allow type change.
Hopefully I will lift this restriction in a future blog post.
Also we swapped the arguments for |set|
(in \texttt{lens} library |set :: ASetter s t a b -> b -> s -> t|).
This version is easier to do math with
(we will partially apply it).

We claim that a |Lens| \emph{should be} a subject to the following three laws:

\begin{itemize}
\item |view (set s a) ≡ a|
\item |set s (view s) ≡ s|
\item |set (set s a) a' ≡ set s a'|
\end{itemize}

Let's show that the laws are both \emph{sound} (not asking too much)
and \emph{complete} (asking enough). For that we'll need another representation
of lenses: an existential formulation:

\begin{equation}
\quad \mathsf{Lens}\;S\;A = \exists C, S \cong C \times A
\end{equation}

This formulation isn't novel (e.g. Russel O'Connor have a footnote about affine traversal in
\href{https://www.cambridge.org/core/journals/journal-of-functional-programming/article/representation-theorem-for-secondorder-functionals/4B782B0BB6EB53E53935D7A0F8432E8E}{\emph{A representation theorem for second-order functionals}}),
but I haven't seen it used to highlight the laws.
In \href{https://r6research.livejournal.com/23705.html}{Lenses Are Exactly the Coalgebras for the Store Comonad}
the laws of a coalgebra for a comonad are used.
As my category theory knowledge is not very deep, I find the "coalgebra for a comonad" intimidating.
I don't know if and how "Lenses are the coalgebras for the costate comonad" is generalisable to other optics.
Existential formulation on the other hand works \emph{for everything}.

\subsubsection{Soundness}

We can define |view| and |set| operations using existential formulation:

\begin{equation}
\begin{aligned}
\quad &\mathsf{view} : S \to A \\
      &\mathsf{view}\;s   = \mathsf{snd}\; (f\;s) \\
      &\mathsf{set} : S \to A \to S \\
      &\mathsf{set}\;s\;a = g\; (\mathsf{pair}\;(\mathsf{fst}\;(f\;s))\;a)
\end{aligned}
\end{equation}

where $f : S \to C \times A$ is the forward direction, and $g : C \times A \to S $ is the backward direction of the isomorphism.

We can prove that |Lens| laws are \emph{sound} by showing that this definitions satisfy the laws.
For example the first \textbf{GetPut} law:

\begin{equation}
\begin{aligned}
  \quad &\mathsf{view}\; (\mathsf{set}\; s\; a)                             &&\equivvia{ \text{definitions} } \\
      &\mathsf{snd}\;(f\; (g\; (\mathsf{pair}\;(\mathsf{fst}\;(f\;s))\;a))) &&\equivvia{ f \circ g \equiv 1 } \\
      &\mathsf{snd}\;(\mathsf{pair}\;(\mathsf{fst}\;(f\;s))\;a)             &&\equivvia{ \mathsf{snd}\;(\mathsf{pair}\;x\;y) = y } \\
      &a                                                                    &&\;\blacksquare
\end{aligned}
\end{equation}

The remaining \textbf{PutGet} and \textbf{PutPut} laws can be proved sound
using similar direct calculation. In fact, I mechanized the proofs in this post;
and soundness properties are proven mostly by a very simple tactic.

\subsubsection{Completeness}

The other direction, \emph{completeness}, is more interesting.
Given $\mathsf{view} : S \to A$ and $\mathsf{set} : S \to A \to S$, such that
the three laws above hold, construct a $\mathsf{Lens}\;S\;A$.

The key insight is to pick the right "left overs" type $C$.
I found the process of mechanizing proofs helpful, you will immediately rule out e.g. bare $S$.
The $A \to S$ (partially applied $\mathsf{set}$) is better, but still too "big".
We need to use a smaller, \emph{refined} type:

\begin{equation}
\quad C = \{ c : A \to S \mid \exists s : S, c \equiv \mathsf{set}\; s \}
\end{equation}

All three options make sense if you consider how |lens :: (s -> a) -> (s -> b -> t) -> Lens s t a b| can be defined using e.g. |Profunctor| encoding,
\href{2017-04-18-glassery.html#v:lens}{Glassery} uses:

\begin{code}
lens :: (s -> a) -> (s -> b -> t) -> Lens s t a b
lens getter setter pab = dimap
     -- c = s
     (\s -> (getter s, s))
     (\(b, s) -> setter s b)
     (first' pab)
\end{code}

but you can also write

\begin{code}
lens :: (s -> a) -> (s -> b -> t) -> Lens s t a b
lens getter setter pab = dimap
     -- c = b -> t; or refined
     -- c = { f : b -> t | exists s. f = setter s }
     (\s -> (getter s, setter s))
     (\(b, f) -> f b)
     (first' pab)
\end{code}

Anyway, after a $C$ is found, we can define an isomorphism.
The arrows are defined as in the latter |Profunctor| example:

\begin{equation}
\begin{aligned}
\quad &f : S \to C \times A \\
      &f\; s = \mathsf{pair}\; (\mathsf{set}\;s \mid \mathsf{exists}\;s\;\mathsf{refl})\; (\mathsf{view}\; s) \\
      &g : C \times A \to S \\
      &g\;(\mathsf{pair}\; (c \mid -)\; a) = c\; a
\end{aligned}
\end{equation}

The proof obligation for the $C$ refinement is not really an obligation at all, the equality is definitionally true ($\mathsf{refl}$).
However, next we need to prove the isomorphism properties.
The backwards direction is easy, it's essentially a \textbf{PutGet}-law:

\begin{equation}
\begin{aligned}
\quad &g\; (f\; s) &&\equivvia{\text{definition of $f$}} \\
      &g\; (\mathsf{pair}\; (\mathsf{set}\;s \mid \mathsf{exists}\;s\;\mathsf{refl})\; (\mathsf{view}\; s)) &&\equivvia{ \text{definition of $g$} } \\
      &\mathsf{set}\;s\;(\mathsf{view}\;s) &&\equivvia{ \text{\textbf{PutGet}-law} } \\
      &s &&\;\blacksquare
\end{aligned}
\end{equation}

Forward direction is more involved. Here we will use proof irrelevance for the refinement proofs.
Those proofs aren't computationally relevant, so this is justified.
I think it's possible to encode these proofs in \href{https://homotopytypetheory.org/2018/12/06/cubical-agda/}{Cubical Agda}.
I also use functional extensionality,
but it's a theorem in Cubical Agda
(it's used once in the following part of the proof, try to spot it!)
We will need to use both \textbf{PutPut} and \textbf{GetPut} laws,
for the both halves of the product (pair), respectively.

\begin{equation}
\begin{aligned}
\quad &f\; (g\; (\mathsf{pair}\;(c \mid \mathsf{exists}\; s\; \mathit{proof})\;a)) &&\equivvia{ \text{definition of $g$} } \\
      &f\; (c\; a) &&\equivvia{ \text{rewrite using $\mathit{proof}$} } \\
      &f\; (\mathsf{set}\;s\; a) &&\equivvia{ \text{definition of $f$} } \\
      &\mathsf{pair}\; (\mathsf{set}\; (\mathsf{set}\;s\; a) \mid -)\;(\mathsf{view}\;(\mathsf{set}\;s\;a)) &&\equivvia{ \text{definition of $g$} } \\
\end{aligned}
\end{equation}

We continue by proving the equality of pair halves separately:

\begin{equation}
\begin{aligned}
\quad &\mathsf{set}\; (\mathsf{set}\;s\; a)  &&\equivvia{ \text{\textbf{PutPut}-law} }  \\
      &\mathsf{set}\;s  &&\equivvia{ \text{rewrite using $\mathit{proof}$} }  \\
      &c &&\;\blacksquare_1
\end{aligned}
\end{equation}

and the other half:

\begin{equation}
\begin{aligned}
\quad &\mathsf{view}\;(\mathsf{set}\;s\;a) &&\equivvia{ \text{\textbf{GetPut}-law} }  \\
      &a &&\;\blacksquare_2
\end{aligned}
\end{equation}

This completes the completeness proof. We had to use all three laws.
I'm sure that intuitively thinking the \textbf{PutPut}-law is simple to forget,
but these calculations show that it is important.

\subsubsection{Calculation of the laws}

In previous subsections, we knew the right |Lens| laws (candidates) a priori.
We only needed to verify that they are in fact the right ones.
But what if we didn't knew the laws beforehand.
Could we somehow calculate them?

In the completeness proof, all branches ended up with the use of some law,
however we \emph{knew} which $C$ to pick, so we end up with a proof goal which
is exactly one of the laws we can use.
Therefore, trying to write a completeness proof is not a direct calculation
way, either we need to know the right existential type to use (in my opinion
not entirely obvious), or which laws there possibly are.

Unfortunately I don't have a definitive answer to this problem.
A sketch of the idea is however to start with

\begin{equation}
\quad S \cong C \otimes A
\end{equation}

isomorphism, and try to massage it into something without $C$.
Notice that we work in a category where all arrows are isomorphisms (\href{https://en.wikipedia.org/wiki/Groupoid}{groupoid}),
therefore we use $\otimes$-tensor, not $\times-$product.

If we assume that we have a
\href{https://ncatlab.org/nlab/show/closed+monoidal+category#symmetric_closed_monoidal_category}{Symmetric closed monoidal groupoid},
then we can use a property:

\begin{equation}
\quad    \mathrm{Hom}(C\otimes A, S) \cong \mathrm{Hom}(C,[ S, A] )
\end{equation}

to find out that $C$ and $[S,A] = A \leftrightarrow S$ are isomorphic.
(And this is the hand-wavy part: I don't understand groupoids enough to say if above reasoning make any sense at all).
We need to "remember" how to go back in $S \to A$ direction, not all $A \to S$ are invertible (e.g. |const someS|).
That's one reasoning to $c = \mathsf{set}\; s$ refinement of $C = A \to S$.

We could stop here, as we in fact calculated what existential $C$ we have to use
(or at least could try).
We can also continue, at this point we have

\begin{equation}
\quad S \cong (A \leftrightarrow S) \otimes A
\end{equation}

$C$-less representation of |Lens|. If we \emph{forget} the isomorphisms,
the remainders are $S \to A \to S$ and $S \to A$ arrows (when we split the product),
these are $\mathsf{set}$ and $\mathsf{view}$ arrows!
So we went from existential to concrete representation of |Lens| without
using Yoneda.
(\href{https://dl.acm.org/citation.cfm?id=3236779}{What you needa know about Yoneda} is an interesting reed anyway!)

But if we stick with this representation, then we can write down the laws too:
If left-to-right direction is $s \mapsto (\mathsf{set}\;s, \mathsf{view}\;s)$ and
the right-to-left is $(\mathsf{set}\;s, a) \mapsto \mathsf{set}\;s\;a$
(here I hand-wave about, what the first half of the pair should be!) then we get

\begin{equation}
\begin{aligned}
    s &= \mathsf{set}\;s\;(\mathsf{view}\;s)               && \textbf{PutGet} \\
    \mathsf{set}\;s &= \mathsf{set}\;(\mathsf{set}\;s\;a)  && \textbf{PutPut} \\
    a &= \mathsf{view}\;(\mathsf{set}\;a)                  && \textbf{GetPut}
\end{aligned}
\end{equation}

Note how the \textbf{PutPut}-law is in contracted form, the form
which we would preferred in the above |Lens| laws completeness proof
(spoiler: that's where I used functional extensionality, if you haven't spotted it).

\subsection{Prism}

In this section we do similar soundness and completeness proofs for prisms.
A \emph{Prism} is a \href{2018-06-19-prisms-are-constructors.html}{generalized constructor}.
Concrete |Prism| representation is
(|preview| and |review| are often called |match| and |build| respectively, I stick with \texttt{lens}-library vocabulary).

\begin{code}
data Prism s a = Prism { preview :: s -> Maybe a, review :: a -> s }
\end{code}

We claim that a |Prism| \emph{should be} a subject to the following two laws:

\begin{itemize}
\item \textbf{MatchBuild}: First, if I |review| a value and then |preview| I will get it back:

    |preview (review a) ≡ Just a|

\item \textbf{BuildMatch}: Second, if you can extract a value |a| using a |preview| from a value |s|, then the value |s| is completely described by a prism and |a|:

    |preview s ≡ Just a ⇒ review a ≡ s|
\end{itemize}

The existential formulation of a |Prism| is similar to |Lens|',
but instead of product we have a coproduct (or sum):

\begin{equation}
\quad \mathsf{Prism}\;S\;A = \exists D, S \cong D + A
\end{equation}

\subsubsection{Soundness}

First step is to define |preview| and |review| operations for existential representation:

\begin{equation}
\begin{aligned}
\quad &\mathsf{preview} : S \to \mathsf{Maybe}\;A \\
      &\mathsf{preview}\;s   = \mathsf{either}\;(d \mapsto \mathsf{nothing})\;(a \mapsto \mathsf{just}\;a) \; (f\;s) \\
      &\mathsf{review} : A \to S \\
      &\mathsf{review}\;a = g\; (\mathsf{right}\;a)
\end{aligned}
\end{equation}

where $f : S \to D + A$ is the forward direction, and $g : D + A \to S $ is the backward direction of the isomorphism.

As with |Lens| none of the laws is hard to show sound.
The first one is a direct computation:

\begin{equation}
\begin{aligned}
\quad &\mathsf{preview}\; (\mathsf{review}\; a)                               &&\equivvia{\text{definitions}} \\
      &\mathsf{either}\;(d \mapsto \mathsf{nothing})\;(a \mapsto \mathsf{just}\;a)\; (f\; (g\;(\mathsf{right}\; a)))  &&\equivvia{ f\circ g \equiv 1 } \\
      &\mathsf{either}\;(d \mapsto \mathsf{nothing})\;(a \mapsto \mathsf{just}\;a)\; (\mathsf{right}\; a)  &&\equivvia{\text{$\mathsf{either}$ computation}} \\
      &\mathsf{just}\;a  &&\;\blacksquare \\
\end{aligned}
\end{equation}

and for the second one we need to prepare the precondition.
The injectivity of constructors let's rule out the case when $f\;s = \mathsf{left}\;d$,
because $\mathsf{nothing} \not\equiv \mathsf{just}\;x$.

\begin{equation}
\begin{aligned}
\quad &\mathsf{preview}\; s\equiv\mathsf{just}\;a                 &&\equivvia{\text{definitions}} \\
      &\mathsf{either}\;(d \mapsto \mathsf{nothing})\;(a \mapsto \mathsf{just}\;a) (f \;s) \equiv \mathsf{just}\;a &&\equivvia{\text{injectivity of constructors}} \\
      &f\;s \equiv \mathsf{right}\;a  &&\;\blacksquare
\end{aligned}
\end{equation}

using which we can prove the law itself

\begin{equation}
\begin{aligned}
\quad & \mathsf{review}\; a          &&\equivvia{\text{definitions}} \\
      & g\; (\mathsf{right }\; a)    &&\equivvia{\text{rewrite using a precondition}} \\
      & g\; (f\; (\mathsf{just}\;s)) &&\equivvia{g \circ f \equiv 1}  \\
      & s &&\;\blacksquare
\end{aligned}
\end{equation}

\subsubsection{Completeness}

We need to construct a $\mathsf{Prism}\;S\;A$ from the given
$\mathsf{review} : A \to S$ and $\mathsf{preview} : S \to \mathsf{Maybe}\;A$.
Here also, as in the |Lens| case, we need to pick the right type for the existential.
And again we need a refined type:

\begin{equation}
    \quad D = \{ d : S \mid \mathsf{preview}\;d \equiv \mathsf{nothing} \}
\end{equation}

And yet again, that type can be found either from
\href{http://oleg.fi/gists/posts/2017-04-18-glassery.html#v:prism}{|Profunctor| definition of |prism|}
(the |either id setter|):

\begin{code}
prism :: (b -> t) -> (s -> Either t a) -> Prism s t a b
prism setter getter = dimap getter (either id setter) . right'}
\end{code}

or by informal calculation

\begin{equation}
\quad S \cong D + A \Rightarrow D \cong S - A
\end{equation}

Then we can define the arrows of the isomorphism (I'm quite sure that smart enough proof search would find exactly these definitions):

\begin{equation}
\begin{aligned}
\quad &f : S \to D + A \\
      &f\; s = \mathsf{maybe\textnormal{-}case}\;(\mathsf{preview}\;s)\; (\mathit{proof} \mapsto \mathsf{left}\;(s \mid \mathit{proof}))\; (a\; \mathit{proof} \mapsto \mathsf{right}\; a)\\
      &g : D + A \to S \\
      &g\;(\mathsf{left}\; (s \mid - )) = s \\
      &g\;(\mathsf{right}\; a) = \mathsf{review}\;a \\
\end{aligned}
\end{equation}

We use non-standard $\mathsf{maybe\textnormal{-}case}$\footnote{
Writing the definition using (dependent) |Maybe| elimination

\begin{equation}
\quad \mathsf{maybe\textnormal{-}elim} : \prod_{A : \star} \prod_{P : \mathsf{Maybe}\;A \to \star} \left( P\;\mathsf{nothing} \to \prod_{x : A} P\,(\mathsf{just}\;x) \to \prod_{m : \mathsf{Maybe}\;A} P\,m  \right)
\end{equation}

\begin{code}
maybe-elim : ∀ {a b} {A : Set a}
           → (P : Maybe A → Set b)
           → P nothing
           → ((x : A) → P (just x))
           → (m : Maybe A)
           → P m
maybe-elim P n j (just x) = j x
maybe-elim P n j nothing  = n
\end{code}

is a nice exercise. Something like that is what |remember| tactic would
do in \emph{Coq}. Knowing what happens helps to write definitions by hand.
}, which provides
the match proofs too.
yet it can be directly coded e.g. in Agda as:

\begin{code}
open import Data.Maybe
open import Relation.Binary.PropositionalEquality

maybe-case : ∀ {a b} {A : Set a} {B : Set b}
           → (m : Maybe A)
           → (m ≡ nothing → B)
           → ((a : A) → m ≡ just a → B)
           → B
maybe-case (just a) f g = g a refl
maybe-case nothing  f g = f refl
\end{code}

Next we can show that $f$ and $g$ are inverses. The $g \circ f \equiv 1$ case:

\begin{equation}
\begin{aligned}
\quad & g\; (f\; s)          &&\equivvia{\text{definition of $f$}} \\
      & g\; (\mathsf{maybe\textnormal{-}case}\;(\mathsf{preview}\;s)\; (\mathit{proof} \mapsto \mathsf{left}\;(s \mid \mathit{proof}))\; (a\; \mathit{proof} \mapsto \mathsf{right}\; a)) \\
\end{aligned}
\end{equation}

which we can split: $\mathsf{preview}\;s \equiv \mathsf{none}$

\begin{equation}
\begin{aligned}
\quad & g\; (\mathsf{left}\; (s \mid \mathit{proof}))  &&\equivvia{\text{definition of $g$}} \\
      & s &&\;\blacksquare_1
\end{aligned}
\end{equation}

and the second case $\mathsf{preview}\;s \equiv \mathsf{just}\;a$

\begin{equation}
\begin{aligned}
\quad & g\; (\mathsf{right}\;a))  &&\equivvia{\text{definition of $g$}} \\
      & \mathsf{review}\;a &&\equivvia{\text{\textbf{BuildMatch}-law}} \\
      & s &&\;\blacksquare
\end{aligned}
\end{equation}

The other direction of the isomorphism, $f \circ g \equiv 1$ has two cases:
first $\mathsf{right}\; a$:

\begin{equation}
\begin{aligned}
\quad & f\; (g\; (\mathsf{right}\;a))          &&\equivvia{\text{definition of $g$}} \\
      & f\; (\mathsf{review}\;a)               &&\equivvia{\text{definition of $f$}} \\
      & \mathsf{maybe\textnormal{-}case}\;(\mathsf{preview}\;(\mathsf{review}\;a))\; (\mathit{proof} \mapsto \mathsf{left}\;(s \mid \mathit{proof}))\; (a\; \mathit{proof} \mapsto \mathsf{right}\; a))
                                               &&\equivvia{\text{\textbf{MatchBuild}-law}} \\
      & \mathsf{maybe\textnormal{-}case}\;(\mathsf{just}\;a)\; (\mathit{proof} \mapsto \mathsf{left}\;(s \mid \mathit{proof}))\; (a\; \mathit{proof} \mapsto \mathsf{right}\; a))
                                               &&\equivvia{\text{calculation of $\mathsf{maybe\textnormal{-}case}$}} \\
      & \mathsf{right}\;a &&\;\blacksquare_1
\end{aligned}
\end{equation}

and $\mathsf{left}\;(s \mid \mathit{proof})$

\begin{equation}
\begin{aligned}
  \quad & f\; (g\; (\mathsf{left}\;(s \mid \mathit{proof}))       &&\equivvia{\text{definition of $g$}} \\
      & f\; s                                          &&\equivvia{\text{definition of $f$}}  \\
      & \mathsf{maybe\textnormal{-}case}\;(\mathsf{preview}\;s)\; (\mathit{proof} \mapsto \mathsf{left}\;(s \mid \mathit{proof}))\; (a\; \mathit{proof} \mapsto \mathsf{right}\; a)
                                                       &&\equivvia{\text{using the $\mathit{proof} : \mathsf{preview}\;s \equiv \mathsf{nothing} $}} \\
      & \mathsf{maybe\textnormal{-}case}\;\mathsf{nothing}\; (\mathit{proof} \mapsto \mathsf{left}\;(s \mid \mathit{proof}))\; (a\; \mathit{proof} \mapsto \mathsf{right}\; a)
                                               &&\equivvia{\text{calculation of $\mathsf{maybe\textnormal{-}case}$}} \\
      & \mathsf{left}\;(s \mid \mathit{proof}') && \equivvia{\text{proof-irrelevance}} \\
      & \mathsf{left}\;(s \mid \mathit{proof}) &&\;\blacksquare
\end{aligned}
\end{equation}

The |Prism| law proofs are only slightly more complex than |Lens| law;
mainly due the need to do case analysis.

One interesting point, is that we might want to have third, \textbf{MatchMatch} law.
There is polymorphic |matching :: APrism s t a b -> s -> Either t a| combinator.
Using its (non-existing in \texttt{lens}) monomorphic version |matching' :: APrism' :: s -> Either s a|
version we can write:
\begin{code}
matching' p s ≡ Left s' ⇒ s ≡ s'
\end{code}
The law is silly, as it completely defines the left part of the |Either|,
therefore we can have |Maybe| in |preview| without any loss.

For example take a bogus |_Right| |Prism| variant:

\begin{code}
_Right' :: Prism' (Either Int a) a
_Right' = prism Right $ \t ->  case t of
    Right x -> Right x
    Left i  -> Left (Left (i + 1))
\end{code}

which will pass the \texttt{QuickCheck} tests:

\begin{code}
matchBuild_prop :: Int -> Property
matchBuild_prop x = preview _Right' (review _Right' x) === Just x

buildMatch_prop :: T Int -> Property
buildMatch_prop s = case preview _Right' s of
    Just a  -> label "just" $ review _Right' a === s
    Nothing -> label "nothing" True
\end{code}

but behave weirdly

\begin{code}
λ> matching _Right' (Right 'a')
Right 'a'
λ> matching _Right' (Left 42)
Left (Left 43)
\end{code}

or

\begin{code}
λ> over _Right' id (Left 1)
Left 2
\end{code}

However, the \textbf{MatchMatch} property is clearly violated:

\begin{code}
matchMatch_prop :: T Int -> Property
matchMatch_prop s = case matching _Right' s of
    Right _ -> label "right" True
    Left  t -> label "left" $ t === s
\end{code}

as \texttt{QuickCheck} points out:

\begin{code}
λ> quickCheck matchMatch_prop
*** Failed! Falsifiable (after 1 test):
Left 0
Left 1 /= Left 0
\end{code}

Our verification process didn't needed this law, as we used |preview| and |review|
as core |Prism| primitives.
There are two ways to construct |Prism| in the \texttt{prism} library:

\begin{code}
prism  :: (b -> t) -> (s -> Either t a) -> Prism s t a b
prism' :: (b -> s) -> (s -> Maybe    a) -> Prism s s a b
\end{code}

we use the latter |prism'| way. As you see it can construct only semi-monomorphic |Prism|.
I think it's type should be restricted to return |Prism s s a a|.
And we need to check the \textbf{MatchMatch} law for |Prism|s constructed with |prism|.
If we used (even the monomorphic version of) |matching| in the completeness proof,
we would see the need for the law, if we actually use the left component of the result.

The \textbf{MatchMatch} law as stated doesn't generalize to the
polymorphic variant, as we require |s ~ t|, to be able to compare them.

My original idea of the law doesn't work, neither another way to state it:

\begin{code}
-- too weak
matching p s ≡ Left t ⇒ matching p t ≡ Left t'
matching p s `leftBind` matching p' ≡ matching p s
\end{code}

A counterexample is when all of negative matches are the same:

\begin{code}
_Right'' :: Prism' (Either Int a) a
_Right'' = prism Right $ \t ->  case t of
    Right x -> Right x
    Left i  -> Left (Left 0)
\end{code}

However one polymorphic variant of \textbf{MatchMatch} law:

\begin{code}
matching p s ≡ Left t ⇒ matching p t ≡ Left s
\end{code}

seems to catch all invalid examples I can come with.
It's not directly enough for \emph{Coq} (maybe it is).
The relaxation of strict equality to |matching p t ≡ Left s| is sensible,
as intuitively there should be a way to convert |t| back to |s|,
|t| (as well as positive match results) fully determine the original |s|.

\subsection{Affine}

 I discussed affine traversal previously in
\href{2017-03-20-affine-traversal.html}{a post about them}
and gave laws in
\href{2017-04-18-glassery.html#laws:affine-traversal}{Glassery}.

\begin{itemize}
\item You get back what you put in:

|preview l (set l v s)  ≡ v <$ preview l s|

\item If you can extract a value, and put it back, that doesn't change anything.

|preview l s ≡ Just a ⇒ set l s a ≡ s|

\item If you get nothing when extracting a value, then whatever you put in, the operation is no-op.

|preview l s ≡ Nothing ⇒  set l s a ≡ s|

\item Setting twice is the same as setting once:

|set l v' (set l v s) ≡ set l v' s|
\end{itemize}

Turns out that the third law is not required in monomorphic setting,
similarly as \textbf{MatchMatch}-laws isn't needed in the |Prism| case.

In this section I won't show the full proofs, you can check (step through) the
\emph{Coq} sources at \url{https://github.com/phadej/lens-laws}. Let's however
highlight the main points.

The concrete representation is the "worst" of |Lens| and |Prism|
(where |Iso| would be the best of both):

\begin{code}
data Affine s a = Affine { preview :: s -> Maybe a; set :: s -> a -> s }
\end{code}

The existential representation explains the naming of this optic:

\begin{equation}
\quad \mathsf{Affine}\;S\;A = \exists C, D, S \cong D + C \times A
\end{equation}

Defining operations using existential representation, and proving
soundness of the laws is a combination of above |Lens| and |Prism|
proofs.

The completeness proof is also similar to |Lens| and |Prism| proofs.
The interesting part is what's the value of existential types, given
$\mathsf{preview} : S \to \mathsf{Maybe}\;A$ and
$\mathsf{set} : S \to A \to S$ we use:

\begin{equation}
\begin{aligned}
\quad D &= \{ s : S \mid \mathsf{preview}\;s \equiv \mathsf{nothing} \} \\
      C &= \{ c : S \to A \mid \exists s : S, \mathsf{set}\;s \equiv c \land \exists (a : A), \mathsf{preview}\;s \equiv \mathsf{just}\;a \}
\end{aligned}
\end{equation}

These types are (again) very similar to the ones used in |Prism| and |Lens|
completeness proofs.

At this point it's good time to discuss some properties of optics we have covered so far:
\begin{itemize}
\item any |Iso| can be turned into |Lens|, |Prism| or |Affine|
\item any |Lens| can be turned into |Affine|
\item any |Prism| can be turned into |Affine|
\item each class of optics is closed under composition
\end{itemize}

First three properties should be easy to see.
For example take a $\mathsf{Lens}\;S\;A = \exists C, S\cong C \times A$,
we can turn it into $\mathsf{Affine}\;S\;A = \exists C, D, S\cong D + C \times A$
by taking $D = 0$ (|Void| in Haskell).

The last property is not hard to verify either. $\mathsf{Iso}$ is clearly
closed under composition (transitivity), $\mathsf{Lens}$ and $\mathsf{Prism}$
need associativity of $+$ and $\times$, i.e.

\begin{equation}
  \quad C \times (C' \times A) \cong (C \times C') \times A
  \qquad D + (D' + A) \cong (D + D') + A
\end{equation}

The |Affine| case is slightly trickier:

\begin{equation}
  \quad D + C \times (D' + C' \times A) \cong ((D + C \times D')+ (C \times C') \times A)
\end{equation}

Imagine (or try to write or look into \emph{Coq} proofs) the needed plumbing in the
existential (or concrete) lens representations! It's very neat, that
van Laarhoven and |Profunctor| encodings use uniform representations so
all optics can share the same composition combinator, and that encoding is
based on type-classes allowing automatic "upcasting" of optics.
The downside is that encoding internals leak into error messages (though
\url{https://github.com/mrkgnao/silica} demonstrates how that can be improved!)

\subsection{Grate}

Russell O'Connor and James Deikun discovered an optic called |Grate|
\footnote{Russell writes "Until Edward Kmett comes up with a better name, let's call this new optic a |Grate|",
three years later we are probably stuck with that name.}
in \href{https://r6research.livejournal.com/28050.html}{Grate: A new kind of
Optic}.  Using |Grate| we can |zipWithOf| structures. Essentially |Grate s a|
witnesses that |s| is some kind of |Distributive| or |Representable| containing
(always the same amount of) |a|s, and nothing else.

It sounds like |Traversal|, but this optic is very different.
The concrete representation is obscure:

\begin{code}
newtype Grate s a = Grate { grate :: ((s -> a) -> a) -> s }
\end{code}

However its existential representation is very simple
(I guess the |Closed| subclass of |Profunctor| helped the discovery to happen):

\begin{equation}
\quad \mathsf{Grate}\;S\;A = \exists I, S \cong I \to A
\end{equation}

Our tasks is to discover the laws. Russell doesn't formulate them for
concrete representation, checking
\href{http://hackage.haskell.org/package/distributive-0.6/docs/Data-Distributive.html}{|Distributive| documentation}
doesn't help either. We can write grate using existential encoding:

\begin{equation}
\begin{aligned}
  \quad & \mathsf{grate} : ((S \to A) \to A) \to S \\
        & \mathsf{grate}\; h = g\; (i \mapsto h\; (s \mapsto f\, s\, i))
\end{aligned}
\end{equation}
where $f : S \to I \to A$ is the forward direction,
$g : (I \to A) \to S $ is the backward direction of the isomorphism.
Having this around will help us to rule out too strong laws, the one
we cannot fulfill (i.e. violate soundness).

\subsubsection{Towards the laws: via completeness}

The journey starts with a short detour. It turns out that stating the laws
for |tabulate| and |index| construction is a lot easier.
Also in this case we stay in \emph{Haskell98} as we don't need type families
(|Representable| type-class has associated |type Rep f|)
the index type will be existentially hidden away, i.e:

\begin{code}
fromRepresentable :: (s -> i -> a) -> ((i -> a) -> s) -> Grate s a
fromRepresentable index tabulate = _exercise
\end{code}

is not hard to implement, and the required properties of |index| and |tabulate|
are almost obvious: they have to be inverses.

If you stare at |Grate| concrete representation and the types of |index| and
|tabulate| (and implementation of |fromRepresentable|), you might see
that |s -> a| acts as |i|, i.e. |grate| type is the similar to |tabulate|'s
"give me a function from indices to elements, and I'll return you a whole container".
This gives us something to try for a completeness proof:

\begin{equation}
\quad I_{\mathrm{wrong}} \stackrel{?}{=} S \to A
\end{equation}

We can define (possible) arrows of the isomorphism using $\mathsf{grate} : ((S \to A) \to A) \to S$,
the implementation is (suspiciously) simple:

\begin{equation}
\begin{aligned}
\quad &f : S \to I \to A \\
      &f\; s\; i= i\; s \\
      &g : (I \to A) \to S \\
      &g\; = \mathsf{grate}
\end{aligned}
\end{equation}

We have try to write a proof to see what properties $\mathsf{grate}$ must have
to make $f$ and $g$ actually inverses.

\begin{equation}
\begin{aligned}
\quad & g\; (f\; s) &&\equivvia{\text{definitions}} \\
  & \mathsf{grate}\;((s'\,i \mapsto i\; s')\;s)  &&\equivvia{\text{calculation}} \\
  & \mathsf{grate}\;(i \mapsto i\; s)
\end{aligned}
\end{equation}

This proof obligation seems reasonable to require as the law,
it's a special case of |tabulate (\i -> index a i)| |Representable| law:

\textbf{TabulateIndex}: Tabulating by indexing doesn't change anything.

\begin{code}
grate g (\i -> i s) ≡ s
\end{code}

So far everything looks good. Let's try the other direction:

\begin{equation}
\begin{aligned}
\quad & f\; (g\; h) &&\equivvia{\text{definitions}} \\
  & (s\,i \mapsto i \,s)\;(\mathsf{grate}\;h) &&\equivvia{\text{calculation}} \\
  & (i \mapsto i\; (\mathsf{grate}\;h)) &&\equivvia{\text{functional extensionality}} \\
  & i\;(\mathsf{grate}\;h) &&\stackrel{?}{\equiv} h\; i
\end{aligned}
\end{equation}

Can we require that as a law?

\textbf{IndexTabulate}: Indexing into tabulated value gives a value at the index?

\begin{code}
i (grate f) ≡ f i
\end{code}

The answer is yes and no. There is a but. If we require the law as such,
it's too strong. |i| and |f| are arbitrary!

\begin{code}
import Data.Functor.Identity
import Test.QuickCheck
import Test.QuickCheck.Instances

newtype Grate s a = Grate { grate :: ((s -> a) -> a) -> s }

identityGrate :: Grate (Identity a) a
identityGrate = Grate (\f -> Identity (f runIdentity))

law2example :: (Fun (Identity Int) (Int)) -> Property
law2example (Fn i) = i (grate identityGrate f) === f i where
    f :: (Identity Int -> Int) -> Int
    f = const 42
\end{code}

I claim that |identityGrate| is lawful |Grate|, however |lawExample| |Property|
doesn't hold, \texttt{QuickCheck} finds a counterexample immediately.
|f| indeed can be arbitrary, but the indices |i| aren't!
Only some values of $S \to A$ type are used, in |identityGrate| a single one: |runIdentity|.

We have make small amendments to the previous. Each grate will have a
associated predicate $\mathsf{P} : (S \to A) \to \mathsf{Prop}$ and the correct
existential type $I$ is

\begin{equation}
\quad  I = \{ i : S \to A \mid P\; i \}
\end{equation}

Definitions of $f$ needs to be slightly changed to forget the refinement:

\begin{equation}
\begin{aligned}
\quad &f : S \to I \to A \\
      &f\; s\; (i \mid -)= i\; s \\
\end{aligned}
\end{equation}

The $g$ need to be adjusted to, or actually the $\mathsf{grate}$ itself,
its more precise type is (remember that in "good" languages,
like \href{https://ucsd-progsys.github.io/liquidhaskell-blog/}{LiquidHaskell},
refinement types are erased, i.e. don't have any runtime cost!)

\begin{equation}
  \quad \mathsf{grate} : (\{ i : S \to A \mid P\; i \} \to A) \to S
\end{equation}

The user of the library "should" provide a predicate $P$
and verify that $\mathsf{grate}$ passes only functions satisfying
that predicate. The final versions of two laws are:

\begin{itemize}
  \item \textbf{TabulateIndex}: Tabulating by indexing doesn't change anything.

\begin{code}
grate g (\i -> i s) ≡ s
\end{code}

\item \textbf{IndexTabulate}: Indexing into tabulated value gives a value at the index?

\begin{code}
i (grate f) ≡ f i  -- forall i such that P i holds
\end{code}
\end{itemize}

For example, for above |Identity| example such predicate can be simply
$P\;i = i \equiv \mathtt{runIdentity}$
(can we encode this in LiquidHaskell, I  remember seeing some predicate support?)

\begin{code}
-- this holds (almost trivially)
law2proper :: Int -> Property
law2proper x = runIdentity (grate identityGrate f) === f runIdentity where
    f :: (Identity Int -> Int) -> Int
    f = const x
\end{code}

In general, for finite containers enumerating the all possible index values
is the simplest way to define predicate $P$.
There are \emph{some} indices, though the implementation hides them.

\subsubsection{Soundness}

Careful reader should notice that we have to do soundness proofs still.
 We have two parts of the isomorphism
$f : S \to I \to A$ is the forward direction,
$g : (I \to A) \to S $ is the backward direction for some existential index type $I$;
$\mathsf{grate}$ is defined using them.

\textbf{TabulateIndex} holds almost trivially:

\begin{equation}
\begin{aligned}
  \quad &\mathsf{grate}\;(i \mapsto i\; s) &&\equivvia{\text{definition of $\mathsf{grate}$}} \\
        & g\; (i \mapsto (i' \mapsto i' s) \; (s' \mapsto f\, s'\, i))
            &&\equivvia{\text{computation}} \\
        & g\; (i \mapsto f\;s\; i)
            &&\equivvia{g \circ f \equiv 1_S} \\
        &s  &&\;\blacksquare
\end{aligned}
\end{equation}

For \textbf{IndexTabulate} we have to define what is the $P$ for the
existential formulation. We check how $\mathsf{grate}$
is implemented (the unfolding is just in the above proof),
what's the argument of $\mathsf{grate}$ is called with:

\begin{equation}
\quad  P\;i = \exists i' : I, i \equiv (s \mapsto f\;s\;i')
\end{equation}

Using this predicate, we can verify the law:

\begin{equation}
\begin{aligned}
  \quad & i\;(\mathsf{grate}\;h) &&\equivvia{\text{definition of $\mathsf{grate}$}} \\
  &i\;(g\; (i' \mapsto h\; (s \mapsto f\, s\, i')))
      &&\equivvia{\text{rewrite using $P\,i = i \equiv (s \mapsto f\;s\;j)$ for some $j$}}\\
  &f\;(g\; (i' \mapsto h\; (s \mapsto f\, s\, i')))\;j
      &&\equivvia{f \circ g \equiv 1_{I\to A}} \\
  &h\; (s \mapsto f\, s\, j)
      &&\equivvia{\text{rewrite using $P$ in other direction}} \\
  &h\; i
      &&\;\blacksquare
\end{aligned}
\end{equation}

This completeness the soundness proof. We now have discovered the |Grate| laws.
As an exercise you could think about

\begin{code}
data V2 a = V2 a a

_V2 :: Grate (V2 a) a
_V2 = _grateExercise1

silly :: Grate () a
silly = _grateExercise2
\end{code}

\subsection{Achromatic}

Achromatic lenses are "A lost of the bidirectional programming folks like a variant of lenses that have get, put, and create." (quote by Russell O'Connor).
Originally, in \href{2018-03-28-achromatic-lens.html}{my notes} on achromatic lenses
the concrete representation has three operations, I rename them all to be nicely
six characters long and reflect the database-holy-grailness:
(update could have |s -> (a -> a) -> s| type as well).

\begin{code}
data Achroma s a = Achroma
    { select :: s -> a
    , update :: s -> a -> s
    , insert :: a -> s
    }
\end{code}

I listed four laws, three |Lens| laws and one from |Prism| or |Iso|
However I didn't notice to specify the last, fifth law.

\begin{itemize}
\item \textbf{SelectUpdate}

\begin{code}
select l (update l a s) ≡ a
\end{code}

\item \textbf{UpdateSelect}

\begin{code}
update l (select l s) s ≡ s
\end{code}

\item \textbf{UpdateUpdate}

\begin{code}
update l a' (update l a s) ≡ update l a' s
\end{code}

\item \textbf{SelectInsert}

\begin{code}
select l (insert l a) ≡ a
\end{code}

\item \textbf{UpdateInsert} \emph{new}

\begin{code}
update l (insert l a') a ≡ insert l a
\end{code}
\end{itemize}

The existential representation for achromatic lens is almost like lenses':

\begin{equation}
\quad \mathsf{Achroma}\;S\;A = \exists C, S \cong (1 + C) \times A
\end{equation}

It can be understood that $S$ might have $C$ or none of it at all.
It turned out that to create $\mathsf{Achroma}$ from three lawful components,
we also needed an extra bit:

\begin{equation}
\quad \mathsf{pristine} = \forall (s : S), \mathsf{Decidable}\;(\mathsf{update}\;s\equiv\mathsf{insert})
\end{equation}

in other words, we \emph{need to know}, for each $S$, whether it has those
extra bits $C$.

In the completeness proof I used the following value for $C$:

\begin{equation}
  \quad C = \{ c : A \to S \mid
  \exists s : S, c = \mathsf{update}\;s \land \mathsf{update}\; s \not\equiv \mathsf{insert}
  \}
\end{equation}

In Haskell implementation we don't care:

\begin{code}
type Optic p s t a b = p a b -> p s t

class Profunctor p => PointedCartesian p where
    pointedSecond :: p a b -> p (Maybe c, a) (Maybe c, b)

achroma
    :: forall p s t a b. PointedCartesian p
    => (s -> a)       -- ^ select
    -> (b -> t)       -- ^ insert
    -> (s -> b -> t)  -- ^ update
    -> Optic p s t a b
achroma sa bt sabt
    = dimap f g
    . pointedSecond
  where
    -- note: always Just.
    f :: s -> (Maybe (b -> t), a)
    f s = (Just (sabt s), sa s)

    g :: (Maybe (b -> t), b) -> t
    g (Nothing,  b) = bt b
    g (Just bt', b) = bt' b
\end{code}

We could try to implement a combinator checking whether the value is "pristine",
i.e. could been created with |insert| directly:

\begin{code}
newtype Pristine a b = Pristine { unPristine :: a -> Bool }

instance Profunctor Pristine where
    dimap f _ (Pristine h) = Pristine (h . f)

instance PointedCartesian Pristine where
    pointedSecond (Pristine h) = Pristine $ \(m, a) -> isNothing m && h a

pristine :: Optic Pristine s s a a -> s -> Bool
pristine o s = unPristine (o (Pristine (const True))) s
\end{code}

But with our current version of |achroma|, |pristine| always returns |False|.
If we add |pristine| to |achroma|:

\begin{code}
-- | better? concrete representation
data Achroma' s a = Achroma'
    { select'   :: s -> a
    , update'   :: s -> a -> s
    , insert'   :: a -> s
    , pristine' :: s -> Bool
    }

-- | better? Profunctor encoding constructor
achroma'
    :: forall p s t a b. PointedCartesian p
    => (s -> a)       -- ^ select
    -> (b -> t)       -- ^ insert
    -> (s -> b -> t)  -- ^ update
    -> (s -> Bool)    -- ^ pristine
    -> Optic p s t a b
achroma' sa bt sabt pris
    = dimap f g
    . pointedSecond
  where
    f :: s -> (Maybe (b -> t), a)
    f s = (if pris s then Nothing else Just (sabt s), sa s)

    g :: (Maybe (b -> t), b) -> t
    g (Nothing,  b) = bt b
    g (Just bt', b) = bt' b
\end{code}

then |pristine| combinator will work, whenever way the value is created or modified.
The |f| and |g| above are essentially the same as used in the completeness proof.

Addition of |pristine| will add two more laws (or one, depends on how you count),
which are encoded in the type of $\mathsf{pristine}$:

\begin{itemize}
\item \textbf{Pristine}
\begin{code}
pristine s ≡ True ⇒ update s a ≡ insert a
\end{code}

or alternatively as a variant of |Prism|'s \textbf{BuildMatch} law\footnote{
in my previous notes I though that law doesn't hold: I didn't notice to specify the precondition}

\begin{code}
 pristine s ≡ True ⇒ s ≡ insert (select s)
\end{code}

\item \textbf{NotPristine}

\begin{code}
pristine s ≡ False ⇒ s ≢ insert (select s)
\end{code}
\end{itemize}

The achromatic lens case illustrates again why it's important to think about laws;
when making \href{2018-03-28-achromatic-lens.html}{notes} I didn't think about |pristine| case, for example.

\subsection{Setter}

The last section is about |Setter|s. They have only single operation |over|:

\begin{code}
newtype Setter s a = Setter { over :: (a -> a) -> (s -> s) }
\end{code}

What is its existential representation? Look at all other different optics
we examined:

\begin{equation}
\begin{aligned}
  \quad \mathsf{Iso}\;S\;A     =&&  S &\cong A  \\
  \quad \mathsf{Lens}\;S\;A    =&& \exists C, S &\cong C \times A \\
  \quad \mathsf{Prism}\;S\;A   =&& \exists D, S &\cong D + A \\
  \quad \mathsf{Affine}\;S\;A  =&& \exists C, D, S &\cong D + C \times A \\
  \quad \mathsf{Grate}\;S\;A   =&& \exists I, S &\cong I \to A \\
  \quad \mathsf{Achroma}\;S\;A =&& \exists C, S &\cong 1 + C \times A
\end{aligned}
\end{equation}

Hopefully the pattern stands out, namely $S$ is isomorphic to $F\,A$ for some
\emph{endofunctor} $F$:

\begin{equation}
  \quad \mathsf{Setter}\;S\;A = \exists F : \mathsf{Endofunctor}, S \equiv F\,A
\end{equation}

Endofunctors can be composed, and there's an identity functor.
All optics seems to be given by some (closed under composition) class of
endofunctors. For example, |Traversal| is given by |Traversable| functors
(whatever these are in category theoretical terms).

That's the point which makes |Profunctor| encoding quite elegant,
Look at the (stripped of) classes behind the implementation
(there are no class for |Affine|, it's made simply by a |(Strong, Choice)|

\begin{code}
class Profunctor p where dimap    :: (a -> s) -> (t -> b) -> p a b -> p s t
class Strong p     where second'  :: p a b -> p (c, a) (c, b)
class Choice p     where right'   :: p a b -> p (Either d a) (Either d b)
class Closed p     where closed   :: p a b -> p (i -> a) (i -> b)
class PointedC p   where psecond' :: p a b -> p (Maybe c, a) (Maybe c, b)

-- Setter
class Mapping p    where map'      :: Functor f     => p a b -> p (f a) (f b)

-- Traversal
class Traversing p where traversed :: Traversable f => p a b -> p (f a) (f b)
\end{code}

It also explains why various |Functor| classes are enough to implement
\emph{simple} (i.e. not type-changing, as in this post) optics, described in
\href{2017-12-23-functor-optics.html}{Functor optics -post}.

Now is possible to consistently invent (or discover?) new optics,
for example |LensGrate|, combination of |Lens| and |Grate|
(I remember seeing it in some library):

\begin{equation}
  \quad \mathsf{LensGrate}\;S\;A = \exists C,I, S \cong C \times (I \to A)
\end{equation}

Does it have some handy operations still, or does it degenerate to |Setter|?
Or |PrismGrate|, which can still |review| things: try to verify that
this class is closed under composition:

\begin{equation}
  \quad \mathsf{PrismGrate}\;S\;A = \exists D,I, S \cong D + (I \to A)
\end{equation}

I think |AffineGrate| is practically as useful as |Setter|, but you can
still speak about it. |AchromaticGrate| seems to be useful
concept, we can implement a variant of |view| for it, as there's
a single index which stands out (and e.g |silly| isn't an |AchromaticGrate|):

\begin{equation}
  \quad \mathsf{AchromaticGrate}\;S\;A = \exists I, S \cong (1 + I) \to A
\end{equation}

Or we can have |FiniteGrate|, where index is some finite set.
Representing the index with $\mathsf{Fin}\;n$ gives us an \emph{order},
so we can |traverseOf| |FiniteGrate|. However, |Traversal|'s aren't
|FiniteGrate|s (exercise!)

\begin{equation}
  \quad \mathsf{FiniteGrate}\;S\;A = \exists n : \mathbb{N}, S \cong \mathsf{Fin}\,n \to A
\end{equation}

So many new or at least not well known optics, that we completely forgot
to talk about |Setter| laws. They aren't surprising, when we know
the existential encoding:

\begin{itemize}
\item \textbf{OverIdentity}
\begin{code}
over id ≡ id
\end{code}

\item \textbf{OverComposition}
\begin{code}
over f . over g ≡ over (f . g)
\end{code}
\end{itemize}

Proofs of their soundness and completeness can be found in
\href{https://github.com/phadej/lens-laws}{Coq formalisation},
or you can try to write them yourself as an exercise.

\subsection{Conclusion}

In this blog post we presented a results of formalization of some
optic laws. Thinking about laws paid off, among other things we

\begin{itemize}
\item discovered \textbf{MatchMatch}-law for |Prism|s,
\item discovered a \textbf{UpdateInsert} law, as well
  as possible new core operation |pristine| for |Achroma|s.
\item formulated laws for |grate| operation of |Grate|.
\end{itemize}

Also in the |Setter| section we discussed some other not well known optics.

A lot from a single insight of requiring the isomorphism
(and not only a equivalence, i.e. bare back and forth arrows without requiring them to be inverses)
in the existential formulation of optics.

Hopefully you enjoyed the read, and this blog post gave you some food for
thought!

\end{document}


\documentclass[12pt]{article}

% PACKAGES
\usepackage[utf8]{inputenc}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{float}
\usepackage{graphicx}
\usepackage{enumerate}
\usepackage{mathtools}
\usepackage{hyperref}

% CUSTOM STYLES
\providecommand{\e}[1]{\ensuremath{\times 10^{#1}}}
\DeclarePairedDelimiter\abs{\lvert}{\rvert}%

\usepackage{listings}
\usepackage{color}

\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

\lstset{frame=tb,
  language=Mathematica,
  aboveskip=3mm,
  belowskip=3mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\small\ttfamily},
  numbers=none,
  numberstyle=\tiny\color{gray},
  keywordstyle=\color{blue},
  commentstyle=\color{dkgreen},
  stringstyle=\color{mauve},
  breaklines=true,
  breakatwhitespace=true
  tabsize=3
}

\restylefloat{table}

% METADATA
\title{FPGA implementation of Cooley-Tukey}
\date{September 13, 2014}
\author{Garrett Massman and Cory Walker}

\begin{document}

  \maketitle

  \subsection*{Overview}
    In this project we will implement the Cooley-Tukey Fast Fourier Transform algorithm on a Xilinx FPGA using VHDL.
    The primary motivation for this is to digitally analyze musical signals, regardless of the instruments they come from.
    Using the Cooley-Tukey algorithm, we can calculate the FFT in $O(nlog_2(n))$ time.
    As far as implementation, the project will be split into several main components.
    First, we will need to sample our audio for a set amount of time and store the data into a buffer in memory.
    Then, the digital signal will need to be processed using a complex ALU.
    Finally, the data will need to be sent to an output buffer for application specific usage.
    This can be in the form of a 7-segment hex display (for instance if we want to print out the fundamental frequency), or a microcontroller (if we want to plot the entire sampled spectrum by passing it to a more capable platform such as Matlab).

  \subsection*{Theory}
    To understand the discrete Fourier Transform, one must first analyze its analogous continuous time form.
    This is written most simply as
    \begin{align*}
    &X(\omega) = \int_{-\infty}^{\infty} x(t) e^{-j \omega t}\,dt
    \end{align*}
    which defines the transform of a continuous time signal $f(t)$.
    Without getting into too many details, it defines the signal as a combination of sinusoids, making it a very useful tool for real world applications.\\

    The discrete Fourier Transform (DFT) is simply a reduction of the continuous Fourier Transform into a discrete sample space.
    In other words, if we let $x_n$ represent a sampled version of the continuous time function $x(t)$ with a total of $N$ samples, we can replace the integral with a summation over the series, as shown below.
    \begin{align*}
        &X_k = \sum\limits_{n=0}^{N-1} x_n e^{\frac{-j2\pi kn}{N}}
    \end{align*}
    As with the continuous time case, this series gives us a glimpse into the component elements of our original signal.
    However, it is rather costly to compute, requiring a time complexity of $O(N^2)$.
    For each value $X_k$, a series of values from $n = 0$ to $N - 1$ must be generated and summed, using up valuable computer resources.
    Thus, calculating the DFT in this way is very inefficient, and we must instead turn to the Fast Fourier Transform.\\

    In order to perform this operation more quickly, we can utilize the Cooley-Tukey FFT algorithm.
    However, the one requirement is that our input is strictly a power of 2.
    That is because the algorithm works by recursively finding the FFT of smaller and smaller sample sizes of $x_n$, which arises from the fact that the transform itself is periodic.
    The transform function can be broken into the sum of its even and odd components,
    \begin{align*}
        &X_k = \sum \limits_{m=0}^{N/2-1} x_{2m}e^{-\frac{2\pi i}{N} (2m)k} + \sum \limits_{m=0}^{N/2-1} x_{2m+1} e^{-\frac{2\pi i}{N} (2m+1)k}
    \end{align*}
    This equation can further be simplified by making the substitutions
    \begin{align*}
        &E_k = \sum \limits_{m=0}^{N/2-1} x_{2m}e^{-\frac{2\pi i}{N} (2m)k}
        &O_k = \sum \limits_{m=0}^{N/2-1} x_{2m+1} e^{-\frac{2\pi i}{N/2} mk}
    \end{align*}
    giving us
    \begin{align*}
        &X_k = E_k + e^{-\frac{2\pi i}{N} k} O_k
    \end{align*}
    The expression $e^{-\frac{2\pi i}{N}k}$ is commonly called the \textit{twiddle factor}.
    Furthermore, because of the periodicity of the transform, we can calculate respective even and odd components simultaneously
    \begin{align*}
        &E_k = E_{k+\frac{N}{2}}
        &O_k = O_{k+\frac{N}{2}}
    \end{align*}

  \subsection*{In Mathematica}
  We implemented a candidate version of the Cooley-Tukey algorithm in Mathematica. This code shows how it is possible to build a look up table of calculated results against the variables $N$ and $k$ for the calculation $e^{-\frac{2\pi i}{N} k}$. This table will not only save us time, but also complexity in the design by allowing us to simply look up this value.
  \begin{lstlisting}
myfasterft[x_, capn_, s_, precomputed_] := (
   result = {};
   If[capn == 1,
    result = Append[result, x[[1]]],
    result = Join[result, myfasterft[x, capn/2, 2 s, precomputed]];
    result =
     Join[result, myfasterft[x[[s + 1 ;;]], capn/2, 2 s, precomputed]];
    Do[
     t = result[[k + 1]];
     result[[k + 1]] =
      t + precomputed[[Log2[capn]]][[k + 1]]*
        result[[k + capn/2 + 1]];
     result[[k + capn/2 + 1]] =
      t - precomputed[[Log2[capn]]][[k + 1]]*
        result[[k + capn/2 + 1]];
     , {k, 0, capn/2 - 1}];
    ];
   result
   );

precomputedmatrix[datalen_] := (
  Table[
   Table[
    E^(-I*2*\[Pi]*k/capn),
    {k, 0., datalen/2 - 1}
    ],
   {capn, Table[2^x, {x, 1, Log2[datalen]}]}
   ]
  )

data = Table[N[Sin[30 2 Pi n/200] + (RandomReal[] - 1/2)], {n, 512}];
precomputed = precomputedmatrix[Length[data]];
ListLinePlot[Abs[myfasterft[data, Length[data], 1, precomputed]], PlotRange -> All]
  \end{lstlisting}
  \subsection*{FPGA considerations}
    %Describe how we plan to utilize SPI for communication, what type of logical blocks we will use and how they will work together.
    Because we are working with a very specialized type of hardware, we must take into consideration some limiting parameters.
    In particular, the Spartan3Es loaded on each BASYS board only have 72K RAM.
    This means that if we want a real time FFT, we will need to optimize the space we have available, and only write to it when it is absolutely necessary.
    We must also consider the quality of our digital input signal.
    There are various ADC PMODs available for the BASYS boards, which range in quality from 4.8 KHz to 1 MHz in speed.
    If we are only sampling audio, we probably won't need to gather any frequencies above 1200 Hz, so the lower end model should be fine.\\

    Once our data has been collected and stored inside the FPGA, it needs to be processed.
    This is perhaps the most challenging part of the project, as we need to implement what will be called an FTC (Fourier Transform Controller) as well as a CALU (Complex Arithmetic Logic Unit).
    The FTC's primary function will be to keep track of the data as it moves from RAM, into the CALU, and back out into the real world.
    This will require some heavily synchronized logic and very tight timing control to be carried out properly.
    It will work in sync with the SPI input master to tell the CALU when data is ready to be read, and then clear the input buffer once the CALU has finished.
    It will also need to monitor the movement of data from the output buffer into whatever output stream requires it.
    The data may be sent directly to a microcontroller or into another component for post processing.\\

    The CALU will have one primary job, which is to actually perform the FFT.
    From its perspective, it will only the addresses of an input buffer and an output buffer, as well as a flag that tells it when to start and stop.
    Because all of our calculations are being done inside of an FPGA, the CALU will be most composed of combinational logic at its core.
    In theory (i.e., provided enough gates), all that needs to be built is the base case of the FFT, which takes in two numbers and outputs their sum and difference.
    \begin{align*}
        &y_0 = x_0 + e^{-\frac{2\pi i}{N}k}x_1
        &y_1 = x_0 - e^{-\frac{2\pi i}{N}k}x_1
    \end{align*}
    This portion of the computation is commonly called a \textit{butterfly}, because the system's diagram looks vaguely like a butterfly.
    Each of the twiddle factors included above can be calculated beforehand and stored in a lookup table for quick access before each iteration.
    Because we will be performing the same type of computation with them each time, they are essentially constants in our system.\\

  \subsection*{Block diagram}
    Over the past few weeks we have taken what we know and outlined a basic block diagram of our device:
    \begin{figure}[H]
      \centering
      \includegraphics[trim=0 400 0 400,clip,width=140mm]{vhdl_fft.pdf}
      \caption{Block diagram of the FFT chip}
      \label{overflow}
    \end{figure}
    Now that we have divided the device into several logical blocks, we can now split up the work accordingly.
  \subsection*{Roadmap}
    In this section, we will describe the steps, in order, that need to be done. Our steps will essentially follow the data pipeline of the module:
    \begin{itemize}
      \item We will begin by implementing a deterministic "ADC" module that will emulate the interface of the real ADC but generate a fixed signal every time. We will use this signal in our own calculations so that we can then verify the outputs of the chip at each stage in the data pipeline. When we are ready to convert to real ADC signals, we can simply switch to an external ADC module using an SPI mux. This step has already been completed.
      \item We will then implement the capture controller. This controller will have an SPI master unit that can read from the ADC. It will be also connected to the block RAM to write the input buffer.
      \item Next, we will work on the heart of the chip, the FT controller. This controller will first simply implement the DFT algorithm. It will utilize the complex number ALU to make calculations.
      \item After the FT controller, we will implement an OUTBUF interpreter that is responsible for outputting the results of the FT controller to the rest of the world using SPI.
      \item Once all of this is done, we will finish converting our FT controller from using the simple DFT algorithm to the more complex Cooley-Tukey FFT algorithm.
      \item If we have further time still, we will build interesting demonstrations of this device.
    \end{itemize}
  \subsection*{Resources}
    While our aim is to implement the entire Fast Fourier Transform with our own code, there are some blocks of our design that are common practice to reuse. Those blocks are the SPI modules. While SPI is not a complicated communication protocol, it requires a fair amount of time to implement. There is a great resource online called OpenCores. This resource is essentially a collection of open source HDL modules that can be freely used in designs. We will be using the "SPI Master/Slave Interface" module a few times in our design to implement SPI.
  \subsection*{Possible extensions}
    The Fast Fourier Transform algorithm applied to an arbitrary voltage signal is very useful. Because our final chip will provide such capability, we have the option of using our newly-designed chip with multiple different applications. \\

    One of our ideas is to attach a microphone and a preamp and use the frequency-domain signal to determine the musical note being played. We could also build a tuner for a guitar. \\

    Additionally, we could connect an audio signal directly from any music player and display a graphic equalizer effect using either a computer or microcontroller that will interface with our chip. \\

    Finally, provided we are able to tune our chip and ADC to operate fast enough, we might be able to build a radio wave analyzer for very long wavelength signals. This device would almost certainly be too slow for any serious RF work, but it would still be an interesting extension. \\

    These extensions are not crucial to our design but might serve as a great demonstration when we present the device to the class.
  \subsection*{Conclusion}

  \begin{thebibliography}{9}
    \bibitem{doin}
      Doin, Jonny. \emph{SPI Master/Slave Interface.} OpenCores, 16 May 2011. Web. 13 Sept. 2014. \textless\url{http://opencores.org/project,spi_master_slave}\textgreater.
    \bibitem{reynwar}
      Reynwar, Ben. \emph{FFT on an FPGA.} FFT on an FPGA. N.p., n.d. Web. 13 Sept. 2014. \textless\url{http://www.reynwar.net/ben/docs/fft_dit/index.html}\textgreater.
    \bibitem{roberts}
      Roberts, Michael J. \emph{Signals and Systems: Analysis Using Transform Methods and MATLAB.} New York: McGraw Hill, 2012. Print.
    \bibitem{satoh}
      Satoh, Keiichi, Jubee Tada, Kenta Yamaguchi, and Yasutaka Tamura. \emph{Complex Multiplier Suited for FPGA Structure.} Computers and Communications (2008): 341-44. Web. 13 Sept. 2014.
    \bibitem{tukey}
      Wikipedia contributors. \emph{Cooley–Tukey FFT algorithm.} Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 27 Jun. 2014. Web. 13 Sep. 2014.
    \bibitem{dft}
      Wikipedia contributors. \emph{Discrete Fourier transform.} Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 2 Sep. 2014. Web. 13 Sep. 2014.
  \end{thebibliography}

\end{document}


\documentclass[12pt]{article}

\newcommand{\CN}{Principles of Sampling}
\newcommand{\Ti}{Homework 1}
\newcommand{\Pf}{Dr. Li}
\newcommand{\FN}{Zehao}
\newcommand{\LN}{Wang}

\input{settings.tex}
\usepackage{longtable}

\begin{document}
% \maketitle
\input{title.tex}

    \section{Homework 1}

    \begin{exercise}[Lohr-2.1]
        Let $N = 6$ and $n = 3$. For purposes of studying sampling distributions, assume that all population values are known.
        \[
            \begin{array}{cll}
            y_{1}=98 & y_{2}=102 & y_{3}=154 \\
            y_{4}=133 & y_{5}=190 & y_{6}=175
            \end{array}
        \]
        We are interested in \(\bar{y}_U\), the population mean. Two sampling plans are proposed.
        \begin{itemize}
            \item Plan 1. Eight possible samples may be chosen.
            \begin{center}
                \begin{tabular}{ccc}
                    Sample Number & Sample, \(\mathcal{S}\) & \(P(\mathcal{S})\) \\
                    \hline 1 & \(\{1,3,5\}\) & \(1 / 8\) \\
                    2 & \(\{1,3,6\}\) & \(1 / 8\) \\
                    3 & \(\{1,4,5\}\) & \(1 / 8\) \\
                    4 & \(\{1,4,6\}\) & \(1 / 8\) \\
                    5 & \(\{2,3,5\}\) & \(1 / 8\) \\
                    6 & \(\{2,3,6\}\) & \(1 / 8\) \\
                    7 & \(\{2,4,5\}\) & \(1 / 8\) \\
                    8 & \(\{2,4,6\}\) & \(1 / 8\)
                \end{tabular}
            \end{center}
            \item Plan 2. Three possible samples may be chosen.
            \begin{center}
                \begin{tabular}{ccc}
                    Sample Number & Sample, \(\mathcal{S}\) & \(P(\mathcal{S})\) \\
                    \hline 1 & \(\{1,4,6\}\) & \(1 / 4\) \\
                    2 & \(\{2,3,6\}\) & \(1 / 2\) \\
                    3 & \(\{1,3,5\}\) & \(1 / 4\)
                \end{tabular}
            \end{center}
        \end{itemize}
        \begin{enumerate}[(a)]
            \item What is the value of \(\bar{y}_U\)?
            \item Let \(\bar{y}\) be the mean of the sample values. For each sampling plan, find

            (i) \(E[\bar{y}]\);\
            (ii) \(V[\bar{y}]\);\
            (iii) \(\operatorname{Bias}(\bar{y})\)\
            (iv) \(\operatorname{MSE}(\bar{y})\).
            \item Which sampling plan do you think is better? Why?
        \end{enumerate}
    \end{exercise}

    \begin{solution}
        \begin{enumerate}[(a)]
            \item $\bar{y}_U=\frac{1}{N}\sum_{i=1}^N y_i=852/6=142. $
            \item
            \begin{itemize}
                \item Plan 1.
                \begin{enumerate}[(i)]
                    \item $\bar{y}_1=147.33$, $\bar{y}_2=142.33$, $\bar{y}_3=140.33$, $\bar{y}_4=135.33$, \\
                    $\bar{y}_5=148.67$, $\bar{y}_6=143.67$, $\bar{y}_7=141.67$, $\bar{y}_8=136.67$.\\
                    $E[\bar{y}]=\sum_{i=1}^8p_i\bar{y}_i=142$.

                    \item $Var[\bar{y}]=\sum_{i=1}^8p_i(\bar{y}_i-E(\bar{y}_i))^2=18.94$.

                    $V[\bar{y}]=SE[\bar{y}]/E[\bar{y}]=\sqrt{18.94}/142=3.06\%$.

                    \item $Bias(\bar{y})=E(\bar{y})-\bar{y}_U=0$.

                    \item $MSE(\bar{y})=\sum_{i=1}^8p_i(\bar{y}_i-\bar{y}_U)^2=18.94$.
                \end{enumerate}
                \item Plan 2.
                \begin{enumerate}[(i)]
                    \item $\bar{y}_1=135.33$, $\bar{y}_2=143.67$, $\bar{y}_3=147.33$.\\
                    $E[\bar{y}]=\sum_{i=1}^3p_i\bar{y}_i=142.5$.
                    \item $Var[\bar{y}]=\sum_{i=1}^3p_i(\bar{y}_i-E(\bar{y}_i))^2=19.36$.

                    $V[\bar{y}]=SE[\bar{y}]/E[\bar{y}]=\sqrt{19.36}/142.5=3.09\%$.
                    \item $Bias(\bar{y})=E(\bar{y})-\bar{y}_U=0.5$.
                    \item $MSE(\bar{y})=\sum_{i=1}^3p_i(\bar{y}_i-\bar{y}_U)^2=19.51$.
                \end{enumerate}
            \end{itemize}
            \item In general, Plan 1 is better because its bias, variance, and mean square error are smaller. However, in terms of convenience, Plan 2 has fewer kinds of samples.
        \end{enumerate}

    \end{solution}

    \begin{exercise}[Lohr-2.2]
        For the population in Example \(2.2\), consider the following sampling scheme:

        \begin{center}
            \begin{tabular}{cc}
                \(\mathcal{S}\) & \(P(\mathcal{S})\) \\
                \hline\(\{1,3,5,6\}\) & \(1 / 8\) \\
                \(\{2,3,7,8\}\) & \(1 / 4\) \\
                \(\{1,4,6,8\}\) & \(1 / 8\) \\
                \(\{2,4,6,8\}\) & \(3 / 8\) \\
                \(\{4,5,7,8\}\) & \(1 / 8\)
            \end{tabular}
            \begin{enumerate}
                \item Find the probability of selection \(\pi_{i}\) for each unit \(i\).
                \item What is the sampling distribution of \(\hat{t}=8 \bar{y}\) ?
            \end{enumerate}
        \end{center}
        Population in Example 2.2:
        \[
            y_i = \{1,2,4,4,7,7,7,8\}.
        \]
    \end{exercise}

    \begin{solution}
        \begin{enumerate}
            \item $\pi_1=P(S_1)+P(S_3)=1/4$, \\
            $\pi_2=P(S_2)+P(S_4)=5/8$, \\
            $\pi_3=P(S_1)+P(S_2)=3/8$, \\
            $\pi_4=P(S_3)+P(S_4)+P(S_5)=5/8$, \\
            $\pi_5=P(S_1)+P(S_5)=1/4$, \\
            $\pi_6=P(S_1)+P(S_3)+P(S_4)=5/8$, \\
            $\pi_7=P(S_2)+P(S_5)=3/8$, \\
            $\pi_8=P(S_2)+P(S_3)+P(S_4)+P(S_5)=1-1/8=7/8$.
            \item $\hat{t}_{S_1}=38$, $\hat{t}_{S_2}=42$, $\hat{t}_{S_3}=40$, $\hat{t}_{S_4}=42$, $\hat{t}_{S_5}=52$.
            \[
                \left\{\begin{array}{l}
                    P(\hat{t}=38)=1/8 \\
                    P(\hat{t}=40)=1/8 \\
                    P(\hat{t}=42)=1/4+3/8=5/8 \\
                    P(\hat{t}=52)=1/8
                \end{array}\right.
            \]
        \end{enumerate}
    \end{solution}

    \begin{exercise}[Levy-2.4]
        As part of an AIDS education program, 120 intravenous drug users seronegative for HIV (Human Immunodeficiency Virus) at a first screening were given instructions on sterilizing their needles with bleach and practicing ``safe sex." One year after the program's inception, a sample of 30 of these subjects was taken by numbering the participants from 1 to 120 and by taking all subjects whose numbers are divisible by 4 (e.g., \(4,8,12\), etc.).
        \begin{enumerate}[a.]
            \item What is the chance of each individual being chosen in the sample?
            \item If subjects \(1,3,4,8,29\), and 65 are seropositive for HIV, what is the proportion of seroconverted subjects for this population?
            \item What is the proportion of seroconverted subjects in the sample? Is this an unbiased estimate of the population proportion?
        \end{enumerate}
    \end{exercise}

    \begin{solution}
        \begin{enumerate}[a.]
            \item $p=30/120=0.25$.
            \item $p=6/120=0.05$.
            \item $\hat{p}=2/30=1/15$. It is not unbiased.
        \end{enumerate}
    \end{solution}

    \begin{exercise}[Levy-2.8]
        The following table shows the total number of laboratory determinations for the 10 sample patients described in Exercise 2.6 as well as the number of invalid determinations among the 10 determinations sampled: \\[0.5em]
        \begin{tabular}{ccc}
            \hline Subject & Total Laboratory Values & Total Invalid Laboratory Values in 10 Sampled \\
            \hline 1 & 103 & 1 \\
            2 & 123 & 0 \\
            3 & 93 & 2 \\
            4 & 200 & 0 \\
            5 & 128 & 1 \\
            6 & 165 & 0 \\
            7 & 132 & 0 \\
            8 & 189 & 0 \\
            9 & 176 & 0 \\
            10 & 180 & 1 \\
            \hline
        \end{tabular}
        \\[0.5em]
        Based on these data, estimate the proportion of laboratory values that are invalid using all of the data in the above table.
    \end{exercise}
    \begin{exercise}[Exercise 2.6]
        It is desired to perform a quality control audit of laboratory data from a large clinical trial for purposes of estimating the proportion of laboratory values in the database that are invalid. There are 394 patients in the clinical trial, and each person has had from 60 to 200 laboratory determinations during the course of the trial. The sampling plan chosen was to select a random sample of 10 patients and for each sample person take a random sample of 10 laboratory determinations, which would then be checked against the patient medical record for accuracy.
    \end{exercise}

    \begin{solution}
        Let $p_{i}$ be the proportion of invalid values for subject $i$. Then,

        $x_1=1/10\times103=10.3$, $x_2=0$, $x_3=2/10\times93=18.6$, $x_4=0$, $x_5=1/10\times128=12.8$.

        $x_6=0$, $x_7=0$, $x_8=0$, $x_9=0$, $x_{10}=1/10\times180=18$.

        So, the estimation is
        \[
            \hat{p}=\frac{\sum_{i=1}^{10} x_i}{103+123+93+200+128+165+132+189+176+180}=0.04009.
        \]
    \end{solution}

    \begin{exercise}[Levy-3.6]
        A survey of workers is to be taken in a large plant that makes products similar to those made in the plant from which the data in Table 3.8 are obtained. The purposes of the survey are to estimate
        \begin{enumerate}[(a)]
            \item the proportion of all workers having an fvc below $70\%$.
            \item the mean fvc among all workers.
            \item Estimates are needed within $5\%$ of the true value of the parameter being estimated. How large a sample of workers is required? The plant employs 5000 workers.
        \end{enumerate}
    \end{exercise}

    \begin{longtable}[c]{lllllllllll}
        \hline
        Worker & Exposure & Fvc & Popsize & Wtl &  & Worker & Exposure & Fvc & Popsize & Wtl \\ \hline
        \endfirsthead
        %
        \endhead
        %
        \hline
        \endfoot
        %
        \endlastfoot
        %
        1      & 3        & 81  & 1200    & 30  &  & 21     & 2        & 70  & 1200    & 30  \\
        2      & 3        & 64  & 1200    & 30  &  & 22     & 1        & 64  & 1200    & 30  \\
        3      & 2        & 85  & 1200    & 30  &  & 23     & 3        & 72  & 1200    & 30  \\
        4      & 2        & 91  & 1200    & 30  &  & 24     & 2        & 72  & 1200    & 30  \\
        5      & 3        & 60  & 1200    & 30  &  & 25     & 3        & 95  & 1200    & 30  \\
        6      & 1        & 97  & 1200    & 30  &  & 26     & 3        & 96  & 1200    & 30  \\
        7      & 1        & 82  & 1200    & 30  &  & 27     & 3        & 62  & 1200    & 30  \\
        8      & 1        & 99  & 1200    & 30  &  & 28     & 3        & 67  & 1200    & 30  \\
        9      & 3        & 96  & 1200    & 30  &  & 29     & 3        & 95  & 1200    & 30  \\
        10     & 3        & 91  & 1200    & 30  &  & 30     & 1        & 87  & 1200    & 30  \\
        11     & 1        & 71  & 1200    & 30  &  & 31     & 3        & 84  & 1200    & 30  \\
        12     & 3        & 88  & 1200    & 30  &  & 32     & 3        & 89  & 1200    & 30  \\
        13     & 2        & 84  & 1200    & 30  &  & 33     & 3        & 89  & 1200    & 30  \\
        14     & 3        & 85  & 1200    & 30  &  & 34     & 3        & 65  & 1200    & 30  \\
        15     & 3        & 77  & 1200    & 30  &  & 35     & 3        & 67  & 1200    & 30  \\
        16     & 3        & 76  & 1200    & 30  &  & 36     & 3        & 69  & 1200    & 30  \\
        17     & 3        & 62  & 1200    & 30  &  & 37     & 3        & 80  & 1200    & 30  \\
        18     & 3        & 67  & 1200    & 30  &  & 38     & 3        & 98  & 1200    & 30  \\
        19     & 3        & 91  & 1200    & 30  &  & 39     & 3        & 65  & 1200    & 30  \\
        20     & 2        & 99  & 1200    & 30  &  & 40     & 3        & 84  & 1200    & 30  \\ \hline
    \end{longtable}

    \begin{solution}
        \begin{enumerate}[(a)]
            \item There are $11$ workers  having an fvc below $70\%$. So, the estimation for the proportion is
            \[
                \hat{p}=11/40=27.5\%.
            \]
            \item Let $\bar{x}$ be the mean value of the population. $\hat{\bar{x}}=\frac{1}{n}\sum_{i=1}^{40}x_i=80.4$.

            \item \begin{itemize}
                \item For (a), let the confidence level be $95\%$, then $z=1.96$,
                \[
                    n\geqslant\frac{z^2Np(1-p)}{(N-1)0.05^2p^2+z^2p(1-p)}=2238.165.
                \]
                So, $n=2239$.
                \item For (a), let the confidence level be $95\%$, then $z=1.96$,
                \[
                    V^2=\frac{\sigma^2}{\bar{x}^2}=\frac{(n-1)s^2}{n\bar{x}^2}=\frac{39\times153.94}{40\times80.4^2}=0.0232,
                \]
                \[
                    n\geqslant\frac{z^2NV^2}{z^2V^2+(N-1)\cdot 0.05^2}=35.4,
                \]
                So, $n=36$.
            \end{itemize}
            Overall, I think the sample size should be $n=2239$.
        \end{enumerate}

    \end{solution}

    \begin{exercise}[Levy-4.2(a-c)]
        From the 120 homes of Exercise 1, suppose a one-in-five systematic sample was taken and suppose that the initial random number was 5 .
        \begin{enumerate}[a.]
            \item Estimate the proportion of homes with lead hazards from this sample.
            \item Obtain a \(95 \%\) confidence interval for the proportion of homes with lead hazards.
            \item What is the true variance of the distribution of the estimated proportion of lead hazardous homes? Compare this result with the estimated variance you used in part (b).
        \end{enumerate}
    \end{exercise}

    \begin{exercise}[Exercise 4.1]
        Suppose the local Childhood Lead Poisoning Prevention Council in a metropolitan area in western Tennessee undertakes the responsibility of determining the proportion of homes in a certain development of 120 homes with unsafe lead levels. Because of the great expense involved in performing spectrometric testing of interior walls, ceilings, floors, baseboards, cabinets, and other obvious lead hazards such as crib bars, as well as of exterior sidings, porches and porch rails, it was decided to select a sample of the homes under study. A good up-to-date frame exists for sampling purposes. This frame is a street listing containing the address and owner of each home for each of the streets in the target area. It was decided to select a one-in-three sample of homes. Let us assume that the only houses with serious lead hazard problems are the 26th, 27th, 28th, and 29th on the list.
    \end{exercise}

    \begin{solution}
        \begin{enumerate}[a.]
            \item The elements will be selected out are $5, 10, 15, 20, \cdots, 115, 120$. And the houses with hazard level are $26, 27, 28, 29$, so, the proportion is $0\%$.
            \item Because the sample points are all $0$. So, the variance should be $0$. And the confidence interval doesn't exist, or it is $[0,0]$.
            \item There are five possible values for $p$, $p_1=1/24$, $p_2=1/24$, $p_3=1/24$, $p_4=1/24$, $p_5=0$.
            \[
                E(p)=\frac{1}{5}\cdot\frac{4}{24}=1/30.
            \]
            \[
                Var(p)=\frac{1}{5}(4\times(1/24-1/30)^2+(-1/30)^2)=0.0002778.
            \]
            The estimated variance used in part (b) is $0$.
        \end{enumerate}
    \end{solution}
    % \printbibliography
\end{document}


\mychapter{19}{Lesson 19} %181130

\todo{E' venuto fuori un casino in questa lezione, ho cercato di riordinare le cose}

\subsubsection{Bilinear Map}

\begin{definition}

    Let's define a \emph{bilinear group} as $(G, G_t, q, g, \hat{e}) \pickUAR \mathcal{B}ilin(1^\lambda)$, where:
    \begin{itemize}
        \item $G, G_t$ are prime order groups (order $q$).
        \item $g$ is a generator of $G$ picked \uar.
        \item $(G, \cdot)$ is a multiplicative group and $G_t$ is the ``target'' group.
        \item $\hat{e}$ is an efficiently computable \emph{bilinear} map: $G \times G \rightarrow G_t$ defined as such:
        \[
            \forall h \in G \forall a, b \in \integer_q \implies \hat{e}(g^a, h^b) = \hat{e}(g, h)^{ab} = \hat{e}(g^{ab}, h)
        \]
        provided that $\hat{e}(g, g) \neq 1$, that is, $\hat{e}$ is \emph{non-degenerative}
    \end{itemize}

    To put it in simple words, the exponents can move.
\end{definition}

\todo{Venturi said something here related to Weil pairing over an elliptic curve. I found \href{https://www.math.auckland.ac.nz/~sgal018/crypto-book/ch26.pdf}{this}. Interesting but not useful.}

It must be noted that the \ddh{} assumption is easy for the group $G$: suffice to see that $\hat{e}(g^a, g^b) = \hat{e}(g, g^c)$ is true iff $c = ab$. On the other hand:

\begin{proposition}
    \cdh{} is hard for $G$.
\end{proposition}

\begin{proof}

    Now $\keygen(1^\lambda)$ will:
    \begin{itemize}
        \item Generate some params: $(G, G_t, g, q, \hat{e}) \pickUAR \mathcal{B}ilin(1^\lambda)$
        \item $a \pickUAR \integer_q$, then $g_1 = g^a$
        \item Pick $g_2 = g^b$ and $g_2, u_{0 \upto k} \pickUAR G$.
        \item Then output:
        \begin{itemize}
            \item $\pk = (\text{params}, g_1, g_2, u_0, ..., u_k)$
            \item $\sk = g_2^a = g^{ab}$
        \end{itemize}
    \end{itemize}

    $\textsf{\textup{Sign}}(\sk, m)$:
    \begin{itemize}
        \item Divide the message $m$ of length $k$ in single bits as follows: $m = (m_{1 \upto k})$
        \item Now define $\alpha(m) = u_0\prod_{i = 1}^k u_i^{m_i}$
        \item Pick $r \pickUAR \integer_q$ and output the signature $\sigma = (\sk \cdot \alpha(m)^r,g^r)=(\sigma_1,\sigma_2)$
    \end{itemize}

    $\textsf{\textup{Ver}}(\pk, m, (\sigma_1, \sigma_2))$:
    \begin{itemize}
        \item Check $\hat{e}(g, \sigma_1) = \hat{e}(\sigma_2, \alpha(m)) = \hat{e}(g_1, g_2)$
    \end{itemize}

    The scheme's correctness is proven my ``moving'' the exponents:
    \begin{align*}
        \hat{e}(g, \sigma_1) &= \hat{e}(g,g_2^a\cdot \alpha(m)^r)   & \\
        &= \hat{e}(g,g_2^a) \cdot \hat{e}(g,\alpha(m)^r)            & \text{(Bilinearity)}\\
        &= \hat{e}(g^a,g_2)\cdot \hat{e}(g^r,\alpha(m))             & \\
        &= \hat{e}(g_1,g_2)\cdot\hat{e}(\sigma_2,\alpha(m))         &
    \end{align*}

    We can say that we are moving the exponents from the "private domain" to the "public domain".

\end{proof}

\section{Waters signatures}
\begin{theorem}
    The Waters' signature scheme is \ufcma-secure
\end{theorem}

\begin{proof}
    \begin{quote}
        The trick is to ``program the 'u's'' (Venturi)
    \end{quote}

    \todo{Sequence is incomplete/incorrect, have to study it more...}

    % Original subjects: A, C_WDS, C_CDH
    \begin{cryptoredux}
        {reduxwaters}
        {Reducing Waters' scheme to \cdh}
        {cdh}
        {wds}

        \cseqdelay

        % Instantiate game
        \receive{\shortstack[l]{
            ``params''$ = $ \\
            $ \quad =(\G, \G_t, g, q, \hat{e})$ \\
            $ \quad \pickUAR BilGen(1^\lambda)$ \\
            $a, b \pickUAR \integer_q$
        }}{$(\text{``params''}, g^a, g^b)$}{}

        \cseqdelay

        % Send pk to adversary
        \invoke{Construct $u$ string}{$(\text{``params''}, g^a, g^b, u)$}{}

        % Adversary can query for signatures
        \cseqdelay
        \return{$m \in M$}{$m$}{}
        \invoke{$\sigma = \textit{Sign}()$}{$\sigma$}{}
        \cseqdelay

        % Adversary forges
        \return{$m^* \notin M$}{$(m^*, \sigma^*)$}{}

    \end{cryptoredux}

    \todo{The following explanation is roundabout, will rectify later}

    The following describes how the Waters' challenger constructs the $u$ string. The main idea is, given $k$ as the message length, to choose every single bit of $u$ from 1 up to $k$ such that:

    \begin{equation*}
        \alpha(m) = g_2^{\beta(m)}g^{\gamma(m)},\quad \beta(m) = x_0 + \sum_{i=1}^{k}m_ix_i,\quad \gamma(m) = y_0 + \sum_{i=1}^{k}m_iy_i
    \end{equation*}

    where $x_0 \pickUAR \{-kl, \dots, 0\}, x_{1\upto k} \pickUAR \{0, \dots, l\}, y_{0\upto k} \pickUAR \integer_q$

    In particular: $l = 2q_s$, where $q_s$ is the number of sign queries made by the adversary.

    Therefore, let $u_i = g_2^{x_i}g^{y_i} \quad \forall i \in [0, k]$. Then:

    \begin{equation*}
        \alpha(m) = g_2^{x_0}g^{y_0} \prod_{i=1}^k (g_2^{x_i}g^{y_i})^{m_i} = g_2^{x_0+\sum_{i=1}^k m_ix_i}g^{y_0+\sum_{i=1}^k m_iy_i} = g_2^{\beta(m)}g^{\gamma(m)}
    \end{equation*}

    \todo{Partizioni, doppi if.... qui non ci ho capito 'na mazza}

    \textbf{Step 1}: $\sigma = (\sigma_1, \sigma_2) = (g_2^a \alpha(m)^{\bar{r}}, g^{\bar{r}})$, for $\bar{r} \pickUAR \integer_q \quad \bar{r} = r - a\beta^{-1}$

    \begin{align*}
        \sigma_1 &= g_2^a\alpha(m)^{\bar{r}} \\
        &= g_2^a\alpha(m)^{r - a\beta^{-1}} \\
        &= g_2^a (g_2^{\beta(m)} g^{\gamma(m)})^{r - a\beta^{-1}} \\
        &= g_2^a g_2^{\beta(m)r-a} g^{\gamma(m)r - \gamma(m)a\beta^{-1}} \\
        &= g_2^{\beta(m)r} g^{\gamma(m)r} g^{-\gamma(m)\beta^{-1}}
    \end{align*}

\end{proof}

%https://eprint.iacr.org/2011/703.pdf