\section{Examples}\label{sec:example}

In this section,  we will present several examples of
\texttt{copilot-kind} applied to verify properties on Copilot
monitors.

 First, let us reexamine the Copilot program from
 Section~\ref{sec:co-intro} that generates the  Fibonacci
 sequence. A fundamental property of this program is that it produces
 a stream of values that are always positive. We express this as follows:
\begin{lstlisting}[frame=single]
spec = prop "pos" (fib > 0)
  where
    fib :: Stream Word64
    fib = [1, 1] ++ (fib + drop 1 fib)
\end{lstlisting}
This  invariant property is clearly inductive and is easily
discharged. Note that, as discussed in Section~\ref{sec:structure},
64-bit words are modelled by integers and eventual overflow problems are ignored here.

The next example uses \texttt{copilot-kind} to prove properties
relating two different specifications. Consider the following
specification:
\begin{lstlisting}[frame=single, deletekeywords={reset}]
intCounter :: Stream Bool -> Stream Word64
intCounter reset = time
  where
    time = if reset then 0
           else [0] ++ if time == 3 then 0 else time + 1
\end{lstlisting}
that acts as a counter performing modulo arithmetic, but is
reset when the \texttt{reset} stream value is true.  Now consider the
specification
\begin{lstlisting}[frame=single, deletekeywords={reset}]
greyTick :: Stream Bool -> Stream Bool
greyTick reset = a && b
  where
    a = (not reset) && ([False] ++ not b)
    b = (not reset) && ([False] ++ a)
\end{lstlisting}
After a reset, \texttt{greyTick}'s output stream forms a cycle of
Boolean values with the third item in the cycle having value
\texttt{true} and the rest being \texttt{false}.  Thus, the two
specifications both have a cyclic structure and with a cycle that
begins when the \texttt{reset} stream is set to \texttt{true}.

\begin{figure}[ht!]
\begin{lstlisting}[frame=single, deletekeywords={gt}]
spec = do
  prop "iResetOk"   (r ==> (ic == 0))
  prop "eqCounters" (it == gt)
  where
    ic = intCounter r
    it = ic == 2
    gt = greyTick r
    r  = extern "reset" Nothing
\end{lstlisting}
  \caption{\texttt{Spec} listing.\label{fig:spec}}
\end{figure}

From the above observations we
conjecture that given the same input stream, when \texttt{reset} is
true,  the \texttt{intCounter} is $0$ and \texttt{greyTick} is
\texttt{true} when \texttt{intCounter} is $2.$ (Extern streams are
uninterpreted; see Section~\ref{sec:structure}.) We formalize these two
properties in our framework as shown in Figure~\ref{fig:spec}.
These predicates are discharged using the proof scheme
\begin{lstlisting}[frame=single]
scheme :: ProofScheme
scheme = do
  check "iResetOk"
  check "eqCounters"
\end{lstlisting}

\subsection{Boyer-Moore Majority Vote}~\label{sec:mvote}
Earlier research on Copilot  has investigated fault-tolerant
runtime verification~\cite{pike-isse-13}. Fault-tolerant algorithms often
include a variant of a majority vote over values (e.g., sensor values, clock
values, etc.). The Boyer-Moore
Majority Vote Algorithm is a linear-time voting
algorithm~\cite{MooreBoyer82,Hesselink2005}. In this case-study, we verify the
algorithm.

The algorithm operates in two passes, first it chooses a candidate and the
second pass verifies that the candidate is indeed a majority.  The algorithm is
subtle and the desire to apply formal verification to our Copilot implementation
helped motivate the effort described here.

%\begin{figure*}[ht]
%\begin{lstlisting}[frame=single]
%majority :: (Eq a, Typed a) => [Stream a] -> Stream a
%majority []     = error "majority: empty list!"
%majority (x:xs) = majority' xs x 1

%majority' []     can _   = can
%majority' (x:xs) can cnt =
% local
%    (if cnt == 0 then x else can) $
%      \ can' ->
%        local (if cnt == 0 || x == can then cnt+1 else cnt-1) $
%          \ cnt' ->
%           majority' xs can' cnt'
%\end{lstlisting}
%\caption{The first pass of the majority vote algorithm in Copilot.}
%\label{fig:majority}
%\end{figure*}

%\begin{figure*}[ht]
%\begin{lstlisting}[frame=single]
%aMajority :: (Eq a, Typed a) => [Stream a] -> Stream a -> Stream Bool
%aMajority xs can = aMajority' 0 xs can > (fromIntegral (length xs) `div` 2)
%
%aMajority' cnt []     _   = cnt
%aMajority' cnt (x:xs) can =
%  local
%    (if x == can then cnt+1 else cnt) $
%      \ cnt' ->
%        aMajority' cnt' xs can
%\end{lstlisting}
%\caption{The second pass of the majority vote algorithm in Copilot.}
%\label{fig:amajority}
%\end{figure*}

%The Copilot implementations of the first a
%pass is  given in Figure \ref{fig:majority} and Figure \ref{fig:amajority}
%respectively.

Two versions of this algorithm were checked with
\texttt{copilot-kind}. The first algorithm was the one implemented as
part of the aforementioned research on fault tolerance and flew on a
small unmanned aircraft. This algorithm
is a parallel implementation, where at each tick, the
algorithm takes $n$ inputs from $n$ distinct streams and is fully
executed. The second version of the algorithm  is a sequential version, where the inputs are
delivered one by one in time and where the result is updated at each
clock tick. Both can be checked with the basic k-induction algorithm,
but the proofs involved are different.

\subsubsection{The parallel version.} The core of the algorithm is the
following:

\begin{lstlisting}[frame=single, deletekeywords={ls}]
majorityVote :: (Typed a, Eq a) => [Stream a] -> Stream a
majorityVote [] = error "empty list"
majorityVote (x : xs) = aux x 1 xs
  where
  aux p _s [] = p
  aux p s (l : ls) =
    local (if s == 0 then l else p) $ \ p' ->
    local (if s == 0 || l == p then s + 1 else s - 1) $ \ s' ->
    aux p' s' ls
\end{lstlisting}
Let us denote $A$ as the set of the elements that can be used as
inputs for the algorithm.  Assume $l$ is a list and $a \in A$, we
denote $|l|_a$ as the number of occurrences of $a$ in $l$. The total
length of a list $l$ is simply written $|l|$. The
\texttt{majorityVote} function takes a list of streams $l$ as its
input and returns an output $maj$ such that: \[ \forall a \in A, \;
\left( a \neq maj \right) \Longrightarrow \left(|l|_a \leq |l| /
  {2}\right) \] Given that quantifiers are handled poorly by \textsc{smt}
solvers and their use is restricted in most model-checking tools,
including \texttt{copilot-kind}, we use a simple trick to write
and check this property.  If $P(n)$ is a predicate of an integer $n$,
we have $\forall n \, . \, P(n)$ if and only if $\neg P(n)$ is
unsatisfiable, where $n$ an unconstrained integer, which can be solved
by a \textsc{smt} solver. The corresponding Copilot specification can be
written as:
\begin{lstlisting}[frame=single]
okWith :: (Typed a, Eq a) =>
          Stream a -> [Stream a] -> Stream a -> Stream Bool
okWith a l maj = (a /= maj) ==> ((2 * count a l) <= length l)
  where
    count _e [] = 0
    count e (x : xs) = (if x == e then 1 else 0) + count e xs

spec = prop "OK" (okWith (arbitraryCst "n") ss maj)
  where
    ss = [ arbitrary ("s" ++ show i) | i <- [1..10] ]
    maj = majorityVote
\end{lstlisting}
The function \texttt{arbitrary} is provided by the \texttt{copilot-kind} standard library and introduces an arbitrary stream. In the same way, \texttt{arbitraryCst} introduces a stream taking an unconstrained but constant value.

Note that we prove the algorithm for a fixed number of $N$ inputs
(here $N=10$). Therefore, no induction is needed for the proof and the
invariant of the Boyer-Moore algorithm does not need to be made explicit. However, the size of the problem discharged to the \textsc{smt} solver grows in proportion to $N$.

\subsubsection{The serial version.} Now, we discuss an implementation
of the algorithm where the inputs are read one by one in a single
stream and the result is updated at each clock tick. As the number of
inputs of the algorithm is not bounded anymore, a proof by induction
is necessary and the invariant of the Boyer-Moore algorithm, being
non-trivial, has to be stated explicitly. As stated in
Hesselink~\cite{Hesselink2005}, this invariant is:
\[ \begin{array}{c}
\forall m \in A, \;\;\; \left(m \neq p\right) \Longrightarrow \left( s + 2|l|_m \,\leq\, |l| \right) \;\; \wedge \;\; \left(m = p\right) \Longrightarrow \left( 2|l|_m \,\leq\, s + |l| \right)
\\

\end{array} \]
where $l$ is the list of processed inputs, $p$ is the intermediary result and $s$ is an internal state of the algorithm. The problem here is that the induction invariant needs universal quantification to be expressed. Unfortunately, this quantifier cannot be removed by a similar trick like the one seen previously. Indeed, when an invariant is of the form $\forall x. P(x, s)$, $s$ denoting the current state of the world, the induction formula we have to prove is:
\[ \forall x. P(x, s) \,\wedge\, T\left(s, s' \right) \,\models\, \forall x. P(x, s') \]
Sometimes, the stronger entailment
\[ P(x, s) \,\wedge\, T\left(s, s' \right) \,\models\, P(x, s') \]
holds and the problem becomes tractable for the \textsc{smt} solver by replacing a universally quantified variable by an unconstrained one. In our current example, it is not the case.

Our solution to the problem of dealing with quantifiers is restricted to the
case where $A$ is finite and we replace each formula of the form $\forall x \in A
\; P(x)$ by $\bigwedge_{x \in A} P(x)$. This can be done with the help of the
\texttt{forAllCst} function provided by the \texttt{copilot-kind} standard library. It is
defined as:
\begin{lstlisting}[frame=single, deletekeywords={conj}]
forAllCst ::(Typed a) =>
  [a] -> (Stream a -> Stream Bool) -> Stream Bool
forAllCst l f = conj $ map (f . constant) l
  where conj = foldl (&&) true
\end{lstlisting}
The code for the serial Boyer-Moore algorithm and its specification is then:

\begin{lstlisting}[frame=single]
allowed :: [Word8]
allowed = [1, 2]

majority :: Stream Word8 -> (Stream Word8, Stream Word8, Stream Bool)
majority l = (p, s, j)
  where
    p  = [0] ++ if s <= 0 then l else p
    s  = [0] ++ if p == l || s <= 0 then s + 1 else s - 1
    k  = [0] ++ (1 + k)

    count m = cnt
      where cnt = [0] ++ if l == m then cnt + 1 else cnt

    j = forAllCst allowed $ \ m ->
          local (count m) $ \ cnt ->
          let j0 = (m /= p) ==> ((s + 2 * cnt) <= k)
              j1 = (m == p) ==> ((2 * cnt) <= (s + k))
          in j0 && j1

spec = do
  prop "J" j
  prop "inRange" (existsCst allowed $ \ a -> input == a)
  where
    input = externW8 "in" Nothing
    (p, s, j) = majority input

scheme = assuming ["inRange"] $ check "J"
\end{lstlisting}

We make the hypothesis that all the elements
manipulated by the algorithm are in the set \texttt{allowed}, which is
finite. The \textsc{smt} proofs are generally exponential with respect to the number of
variables, so this approach does not scale well.


\documentclass{beamer}

\usepackage{verbatim}
\usepackage{fancyvrb}
\usepackage{amsmath}
\usepackage{mathtools}
\usepackage{booktabs}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{calc}
\usepackage{color}
\usepackage{multicol}
\usepackage{wrapfig}
\usepackage{natbib}
\usepackage[ruled,vlined]{algorithm2e}
\usepackage{animate}
\usepackage{mathtools}
\usepackage{listings}
\usepackage{xfrac}

% two col: two columns
\newenvironment{twocol}[4]{
\begin{columns}[c]
\column{#1\textwidth}
#3
\column{#2\textwidth}
#4
\end{columns}
}

\makeatletter
\setbeamertemplate{theorem begin}
{%
\begin{\inserttheoremblockenv}
  {}{\usebeamerfont*{block title}\usebeamercolor[fg]{block title}%
  \inserttheoremname
  %\inserttheoremnumber
  \ifx \inserttheoremaddition \empty \else\ (\inserttheoremaddition)\fi
  \inserttheorempunctuation}
  \normalfont
  }
  \setbeamertemplate{theorem end}{\end{\inserttheoremblockenv}}
\makeatother

\newcommand{\E}{\mathrm{E}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\Cov}{\mathrm{Cov}}
\newcommand{\s}{\mathrm{s}}
\newcommand{\Corr}{\mathrm{Corr}}
\newcommand{\rank}{\mathrm{rank}}
\newcommand{\nullspace}{\mathrm{null}}
\newcommand{\myspan}{\mathrm{span}}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\softmax}{softmax}

\definecolor{darkgreen}{rgb}{0,0.5,0}

\newtheorem{prop}[theorem]{Proposition}
\newtheorem{exe}{Exercise}
\newtheorem{remark}{Remark}
\newtheorem{myproof}{Proof}

\definecolor{darkgreen}{rgb}{0,0.5,0}

\title{Inferences in Regression Analysis}
\author{Zhenisbek Assylbekov}
\institute{Department of Mathematics}
\date{Regression Analysis}

\AtBeginSection[]
{
  \begin{frame}<beamer>
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}
  \titlepage
\end{frame}

\begin{frame}{Normal Errors Regression}
    Throughout this chapter we assume
    \begin{align*}
    &Y_i=\beta_0+\beta_1 x_i+\epsilon_i,\\
    &\epsilon_1,\ldots,\epsilon_n\stackrel{\rm iid}{\sim}\mathcal{N}(0,\sigma^2).
    \end{align*}
    \pause{This is equivalent to
    $$
    Y_i\,{\stackrel{\text{ind}}{\sim}}\,\mathcal{N}(\beta_0+\beta_1 x_i,\sigma^2)
    $$}
\end{frame}

\section{Inferences on $\beta_1$ and $\beta_0$}

\begin{frame}{Unbiasedness of $b_1$}
\begin{prop}
$b_1$ is an unbiased estimator of $\beta_1$.
\end{prop}
\pause\begin{proof}
Recall that $b_1=\frac{\sum(x_i-\bar{x})(Y_i-\bar{Y})}{\sum(x_i-\bar{x})^2}\pause{\stackrel{\text{?}}{=}\frac{\sum(x_i-\bar{x})Y_i}{\sum(x_i-\bar{x})^2}}$.

\pause Expected value of the numerator is
\begin{align}
&\E\left[\sum(x_i-\bar{x})Y_i\right]=\sum(x_i-\bar{x})\E[Y_i]=\sum(x_i-\bar{x})(\beta_0+\beta_1 x_i)\notag\\
&=\beta_0\sum x_i-n\bar{x}\beta_0+\beta_1\sum x_i^2-n\bar{x}^2\beta_1=\beta_1\left(\sum x_i^2-n\bar{x}^2\right)\label{eq:numerator}\notag
\end{align}
\pause The denominator is
$$
\sum(x_i-\bar{x})^2=\sum x_i^2-2\sum x_i\cdot\bar{x}+\sum\bar{x}^2=\sum x_i^2-n\bar{x}^2
$$
\pause Hence, $\E[b_1]=\beta_1$.
\end{proof}
\end{frame}

\begin{frame}{Variance of $b_1$}
\begin{prop}
$\Var[b_1]=\frac{\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}.$
\end{prop}
\pause\begin{proof}
\begin{align*}
\Var[b_1]&=\Var\left[\frac{\sum_i (x_i-\bar{x})Y_i}{\sum_i (x_i-\bar{x})^2}\right]\onslide<3->{=\frac{\sum_i(x_i-\bar{x})^2\Var[Y_i]}{\left[\sum_i(x_i-\bar{x})^2\right]^2}\\}
\onslide<4->{&=\frac{\sigma^2\sum_i(x_i-\bar{x})}{\left[\sum_i(x_i-\bar{x})^2\right]^2}}\onslide<5->{=\frac{\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}.}
\end{align*}
\end{proof}
\onslide<6->{\begin{remark}
\begin{itemize}
    \item<6-> $\sum_i{(x_i-\bar{x})^2}\uparrow$\quad $\Rightarrow$\quad $\Var[b_1]\downarrow$
    \item<7-> $n\uparrow\quad\Rightarrow\quad\Var[b_1]\downarrow$
\end{itemize}
\end{remark}}
\end{frame}

\begin{frame}{Distribution of $b_1$}
Rewrite $b_1$ as
$$
b_1=\frac{\sum_i(x_i-\bar{x})Y_i}{\sum_i(x_i-\bar{x})^2}=\sum_{i=1}^n\left[\frac{(x_i-\bar{x})}{\sum_{j=1}^n(x_j-\bar{x})^2}\right]Y_i
$$%
\onslide<2->{Thus, $b_1$ is a linear combination of $n$ independent normal random variables $Y_1,\ldots,Y_n$.} \onslide<3->{Therefore}
$$
\onslide<3->{b_1\sim}\onslide<4->{\mathcal{N}\left(\beta_1,\frac{\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right),}
$$%
\onslide<5->{and
$$
\frac{b_1-\beta_1}{\sqrt{\Var[b_1]}}\sim\mathcal{N}(0,1).
$$}%
\onslide<6->{We never know $\sigma^2$, we estimate it by MSE $=\frac{1}{n-2}\sum_i (Y_i-\hat{Y}_i)^2$.}
\end{frame}

\begin{frame}{Distribution of $\frac{b_1-\beta_1}{\s[b_1]}$}
Denote $
\s[b_1]=\sqrt{\frac{\text{MSE}}{\sum_{i=1}^n(x_i-\bar{x})^2}}$.
\pause\begin{theorem}
$\frac{b_1-\beta_1}{\s[b_1]}\sim t_{n-2}.$
\end{theorem}
\begin{proof}
\begin{align*}
\frac{b_1-\beta_1}{\s[b_1]}&=\overbrace{\frac{b_1-\beta_1}{\sqrt{\Var[b_1]}}}^{Z}\cdot\frac{\sqrt{\Var[b_1]}}{\s[b_1]} =Z\cdot\sqrt{\frac{\sigma^2}{\frac{1}{n-2}\sum_i{e_i^2}}}\\
&=\frac{Z}{\sqrt{\frac{\sfrac{\sum_i e_i^2}{\sigma^2}}{n-2}}}=\frac{Z}{\sqrt{\frac{\chi^2_{n-2}}{n-2}}}\sim t_{n-2},
\end{align*}
where we used the fact that $\frac{\sum_i e_i^2}{\sigma^2}\sim\chi^2_{n-2}$ (Ch 5).
\end{proof}
\end{frame}

\begin{frame}{Confidence interval for $\beta_1$ and testing $\rm{H}_0:\beta_1=\beta_{10}$}
A $(1-\alpha)\cdot100\%$ CI for $\beta_1$ has endpoints
$$
b_1\pm t_{n-2,1-\alpha/2}\cdot\s[b_1].
$$

\pause Under $\rm{H}_0:\beta_1={\beta_{1,0}}$,
$$
T=\frac{b_1-\beta_{1,0}}{\s[b_1]}\sim t_{n-2}.
$$
\pause P-values are computed as usually.\\~\\

\pause In simple linear regression
$$
Y_i=\beta_0+\beta_1 x_i+\epsilon_i
$$
of particular interest is $\rm{H}_0:\beta_1=0$, that $Y_i$ and \textit{does not depend on} $x_i$.
\end{frame}

\begin{frame}{Expectation, Variance, and Distribution of $b_0$}
\begin{exe}
Show that
\begin{itemize}
    \item $\E[b_0]=\beta_0$
    \item<2->$\Var[b_0]=\left[\frac1n+\frac{\bar{x}^2}{\sum_i(x_i-\bar{x})^2}\right]\sigma^2$
    \item<3->$b_0\sim\mathcal{N}\left(\beta_0,\left[\frac1n+\frac{\bar{x}^2}{\sum_i(x_i-\bar{x})^2}\right]\sigma^2\right)$
    \item<4->$\frac{b_0-\beta_0}{\s[b_0]}\sim t_{n-2}$, where $\s[b_0]$ is obtained from $\sqrt{\Var[b_0]}$ when replacing $\sigma^2$ by MSE
\end{itemize}
\end{exe}%
\onslide<5->{CI and Hypothesis Test for $\beta_0$ are as usual.}
\end{frame}

\begin{frame}{Table of regression coefficients}
Regression output typically produces a table like:

\vspace{10pt}
\begin{small}
\begin{tabular}{c c c c c}
Parameter & Estimate & Standard error & $t^\ast$ & p-value\\
\hline
Intercept $\beta_0$ & $b_0$ & $\s[b_0]$ & $t^\ast_0=\frac{b_0}{\s[b_0]}$ & $\Pr(|T|>|t_0^\ast|)$\\
Slope $\beta_1$ & $b_1$ & $\s[b_1]$ & $t_1^\ast=\frac{b_1}{\s[b_1]}$ & $\Pr(|T|>|t_1^\ast|)$\\
\hline
\end{tabular}
\end{small}

\vspace{10pt}
\pause where $T\sim t_{n-p}$ and $p$ is the number of parameters used to estimate the mean, here $p=2$: $\beta_0$ and $\beta_1$. Later $p$ will be the number of predictors in the model plus one.

\vspace{10pt}
\pause The two p-values in the table test $\rm{H}_0:\beta_0=0$ and $\rm{H}_1:\beta_1=0$ respectively. The test for zero intercept is usually not of interest.
\end{frame}

\begin{frame}[fragile]{Regression Output in \texttt{R}: Poverty vs HS grad rate}

{\color{blue} \url{https://raw.githubusercontent.com/zh3nis/MATH440/main/chp01/poverty.R}}
\begin{small}
\begin{verbatim}
Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept) 64.78097    6.80260   9.523 9.94e-13 ***
Graduates   -0.62122    0.07902  -7.862 3.11e-10 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.082 on 49 degrees of freedom
\end{verbatim}
\end{small}
\pause We reject $\rm{H}_0:\beta_1=0$ at any reasonable significance level. There is a significant linear association between HS graduation and poverty rates.
\end{frame}

\section{Inferences on $\E[Y]$ and $\hat{Y}$}

\begin{frame}{Inference on $\E[Y]=\beta_0+\beta_1 x$}
Let $x$ be any value of the \textit{predictor}; we want to estimate the mean of all responses in the \textit{population} that correspond to $x$. This is given by
$$
\E[Y]=\beta_0+\beta_1 x.
$$
Our estimator of $\E[Y]$ is
\begin{align*}
\hat{Y}&=b_0+b_1 x\\
&=\sum_{i=1}^n\left[\frac1n-\frac{\bar{x}(x_i-\bar{x})}{\sum_{j=1}^n(x_j-\bar{x})^2}+\frac{(x_i-\bar{x})x}{\sum_{j=1}^n(x_j-\bar{x})^2}\right]Y_i\\
&=\sum_{i=1}^n\left[\frac1n+\frac{(x-\bar{x})(x_i-\bar{x})}{\sum_{j=1}^n(x_j-\bar{x})^2}\right]Y_i
\end{align*}
\end{frame}

\begin{frame}{Distribution of $\hat{Y}$}
\textit{Again} we have a linear combination of independent normals as our estimator. This leads, after some math, to
\begin{equation}
b_0+b_1 x\sim \mathcal{N}\left(\beta_0+\beta_1 x,\sigma^2\left[\frac1n+\frac{(x-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right]\right).\label{eq:reg_dist}
\end{equation}

\pause As before, this leads to a $(1-\alpha)\cdot100\%$ CI for $\beta_0+\beta_1 x$
$$
b_0+b_1 x\pm t_{n-2,1-\alpha/2}\cdot\s[b_0+b_1 x],
$$
\pause where $\s[b_0+b_1 x]=\sqrt{\text{MSE}\cdot\left[\frac1n+\frac{(x-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right]}.$\\~\\

\pause For what value of $x$ is the CI narrowest? \pause For $x=\bar{x}$.

\begin{exe}
Prove \eqref{eq:reg_dist}. \pause Solution is on pp. 53--54 of the textbook.
\end{exe}
\end{frame}

\begin{frame}{Prediction intervals}
\begin{itemize}
\item We discussed constructing a CI for the unknown $\E[Y]$ at $x$.
\item<2-> What if we want to find an interval that the actual \textit{value} $Y$ is in (versus only its mean) with fixed probability?
\item<3-> If we knew $\beta_0$, $\beta_1$, and $\sigma^2$ this would be easy, because
$$
Y=\beta_0+\beta_1 x+\epsilon\sim\mathcal{N}(\beta_0+\beta_1 x,\sigma^2),
$$
and a $(1-\alpha)\cdot100\%$ CI for $\E[Y]$ would be
$$
\beta_0+\beta_1 x\pm z_{1-\alpha/2}\cdot\sigma.
$$
\item<4-> Unfortunately, we don't know $\beta_0$, $\beta_1$, and $\sigma$, but we can estimate all three of these.
\end{itemize}
\end{frame}

\begin{frame}{Variability of $b_0+b_1 x + \epsilon$}
An interval that contains $Y$ with $(1-\alpha)$ probability needs to account for
\begin{itemize}
\pause\item the variability of the estimators $b_0$ and $b_1$
\pause\item the natural variability of response $Y$ built into the model: $\epsilon\sim\mathcal{N}(0,\sigma^2)$.
\pause We have
\begin{align*}
\Var[b_0+b_1 x + \epsilon]&=\Var[b_0+b_1 x]+\Var[\epsilon]\\
 &=\sigma^2\left[\frac1n+\frac{(x-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}\right]+\sigma^2\\
 &=\sigma^2\left[\frac1n+\frac{(x-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}+1\right]
\end{align*}
\end{itemize}
\end{frame}

\begin{frame}{Prediction interval}
Estimating $\sigma^2$ by MSE we obtain that a $(1-\alpha)\cdot100\%$ \textbf{prediction interval} (PI) for $Y$ is
$$
b_0+b_1 x\pm t_{n-2,1-\alpha/2}\sqrt{{\text{ MSE}}\left[\frac1n+\frac{(x-\bar{x})^2}{\sum_{i=1}^n(x_i-\bar{x})^2}+1\right]}.
$$
\begin{remark} As $n\to\infty$, we have $b_0\stackrel{\rm P}{\to}\beta_0$, $b_1\stackrel{\rm P}{\to}\beta_1$, $t_{n-2,1-\alpha/2}\to z_{1-\alpha/2}$, and ${\rm MSE}\stackrel{\mathrm{P}}{\to}\sigma^2$.\\~\\

I.e., as the sample size grows, the PI converges to
$$
\beta_0+\beta_1 x\pm z_{1-\alpha/2}\cdot\sigma.
$$
\end{remark}
\end{frame}

\begin{frame}{Example: Poverty vs HS Graduation data}
\begin{itemize}
\item Find a 95\% CI for the mean poverty rate $\E[Y]$ in a state with HS graduation rate $x=80$.
\item Find a 95\% PI for the poverty rate $Y$ in a state with HS graduation rate $x=80$.
\item \texttt{R} code follows\ldots
\end{itemize}
\end{frame}

\begin{frame}[fragile]
\frametitle{\texttt{R} code}
{\color{blue}\url{https://raw.githubusercontent.com/zh3nis/MATH440/main/chp02/pov_predict.R}}
\begin{footnotesize}
\begin{verbatim}
> poverty = read.table("path/to/poverty.txt", h = T, sep = "\t")

> my_model = lm(Poverty ~ Graduates, data=poverty)

> new_x = data.frame(Graduates=80)

> predict.lm(my_model, new_x, interval="confidence", level=0.95)
       fit     lwr      upr
1 15.08363 13.9636 16.20365

> predict.lm(my_model, new_x, interval="prediction", level=0.95)
       fit     lwr      upr
1 15.08363 10.7527 19.41455
\end{verbatim}
\end{footnotesize}

\pause A 95\% CI for $\E[Y]$ given $x=80$ is $[13.96, 16.20]$.\\
\pause A 95\% PI for $Y$ given $x=80$ is $[10.75, 19.41]$.
\end{frame}

\begin{frame}{Confidence Band for Regression Line}
\begin{columns}
\begin{column}{0.45\textwidth}
\textbf{Working-Hotelling} confidence band:
$$
\hat{Y}\pm W\cdot\s[\hat{Y}],
$$
where $W^2=2\cdot F_{1-\alpha; 2, n-2}$. \pause Gives region that \textit{entire} regression line lies in with certain confidence.
\end{column}%
\begin{column}{.55\textwidth}
\pause \begin{figure}
    \centering
    \includegraphics[width=\textwidth]{plots/conf_band.pdf}
\end{figure}
\end{column}
\end{columns}
\pause {\color{blue}\url{https://raw.githubusercontent.com/zh3nis/MATH440/main/chp02/pov_cb.R}}
\end{frame}

\section{Analysis of Variance Approach}

\begin{frame}{Decomposing $Y_i-\bar{Y}$}
Notice that
$$
\overbrace{Y_i-\bar{Y}}^{\text{Total}} = \overbrace{Y_i-\hat{Y}_i}^{\text{Error}}+\overbrace{\hat{Y}_i-\bar{Y}}^{\text{Regression}}
$$
\begin{figure}
    \centering
    \includegraphics[width=.6\textwidth]{plots/ssto.png}
\end{figure}
\end{frame}

\begin{frame}{Partitioning SST}
\begin{equation}
\overbrace{Y_i-\bar{Y}}^{\text{Total}} = \overbrace{Y_i-\hat{Y}_i}^{\text{Error}}+\overbrace{\hat{Y}_i-\bar{Y}}^{\text{Regression}}\label{eq:tot_decomp}
\end{equation}
\pause Squaring \eqref{eq:tot_decomp} and summing over $i$, we have
\pause$$
\underbrace{\sum(Y_i-\bar{Y})^2}_{\text{SST}}=\underbrace{\sum(Y_i-\hat{Y}_i)^2}_{\text{SSE}}+\underbrace{\sum(\hat{Y}_i-\bar{Y})^2}_{\text{SSR}}
$$
\pause because (see Chapter 1)
$$
\sum(Y_i-\hat{Y}_i)(\hat{Y}_i-\bar{Y})=\sum e_i(\hat{Y}_i-\bar{Y})=\sum e_i\hat{Y}_i-\bar{Y}\sum e_i=0
$$
\end{frame}

\begin{frame}{Degrees of Freedom of SS}
In Chapter 5 we will show that
\begin{itemize}
    \item $\frac{\text{SSE}}{\sigma^2}\sim\chi^2_{n-2}$\onslide<2->{\quad$\Rightarrow$\quad SSE has $n-2$ df}
    \item<3->$\frac{\text{SSR}}{\sigma^2}\sim\chi^2_1$ (noncentral)\onslide<4->{\quad$\Rightarrow$\quad SSR has $1$ df}
    \item<4->SSE and SSR are indep.\onslide<5->{\quad$\Rightarrow$\quad$\frac{\text{SST}}{\sigma^2}=\frac{\text{SSE}}{\sigma^2}+\frac{\text{SSR}}{\sigma^2}\sim\chi^2_{n-1}$}\onslide<6->{\pause\quad$\Rightarrow$\quad$\text{SST}$ has $n-1$ df}
\end{itemize}
\end{frame}

\begin{frame}{Expectations of SS}

\begin{prop}
$\E[\text{MSE}]=\sigma^2$.
\end{prop}
\pause{\begin{proof}
$\E[\text{MSE}]=\E\left[\frac{\text{SSE}}{n-2}\right]=\frac{\sigma^2}{n-2}\E\left[\frac{\text{SSE}}{\sigma^2}\right]=\frac{\sigma^2}{n-2}(n-2)=\sigma^2$.
\end{proof}}
\end{frame}

\begin{frame}{Expectations of SS}
\begin{prop}
$\E[\text{MSR}]=\sigma^2+\beta_1^2\sum_i{(x_i-\bar{x})^2}$.
\end{prop}
\begin{myproof}
\begin{align*}
\onslide<2->{\text{SSR}&=\sum_i(\hat{Y}_i-\bar{Y})^2=\sum_i(b_0+b_1 x_i-\bar{Y})^2\\}
\onslide<3->{&=\sum_i(\bar{Y}-b_1\bar{x}+b_1 x_i-\bar{Y})^2\pause=b_1^2\sum_i(x_i-\bar{x})^2.\\}
\onslide<4->{\E[\text{MSR}]&=\E\left[\frac{\text{SSR}}{1}\right]=\E\left[b_1^2\sum_i(x_i-\bar{x})^2\right]\\}
\onslide<5->{&=\sum_i(x_i-\bar{x})^2(\Var[b_1]+(\E[b_1])^2)\\}
\onslide<6->{&=\sigma^2+\beta_1^2\sum_i(x_i-\bar{x})^2.}
\end{align*}
\end{myproof}
\end{frame}

\begin{frame}{Analysis of Variance (ANOVA) table}
\begin{footnotesize}
\begin{table}
\begin{tabular}{l c c c c}
\toprule
Source & SS & df & MS & $\E[\text{MS}]$\\
\midrule
Regression & $\text{SSR}=\sum(\hat{Y}_i-\bar{Y})^2$ & 1 & $\frac{\text{SSR}}{1}$ & $\sigma^2+\beta_1^2\sum(x_i-\bar{x})^2$\\
Error & $\text{SSE}=\sum(Y_i-\hat{Y})^2$ & $n-2$ & $\frac{\text{SSE}}{n-2}$ & $\sigma^2$\\
\midrule
Total & $\text{SST}=\sum(Y_i-\bar{Y})^2$ & $n-1$\\
\bottomrule
\end{tabular}
\end{table}
\end{footnotesize}
\pause\begin{remark}
$\frac{\E[\text{MSR}]}{\E[\text{MSE}]}=
\begin{cases}
1\quad&\text{if }\beta_1=0\\
>1&\text{if }\beta_1\ne0.
\end{cases}$\\~\\\pause Loosely, we expect MSR to be larger than MSE when $\beta_1\ne0$.
\end{remark}
\end{frame}

\begin{frame}{F-test for $\beta_1$}
\begin{prop}
Under $\text{H}_0:\,\beta_1=0$, we have $\frac{\text{MSR}}{\text{MSE}}\sim F_{1,n-2}$.
\end{prop}
\pause\begin{proof}
$$\frac{\text{MSR}}{\text{MSE}}=\frac{\frac{\text{SSR}}{\sigma^2}/1}{\frac{\text{SSE}}{\sigma^2}/(n-2)}=\frac{\chi^2_1/1}{\chi^2_{n-2}/(n-2)}\sim F_{1,n-2},$$
\pause because SSR and SSE are statistically independent (Ch 5).
\end{proof}
\pause\begin{remark}
The F-test and t-test for $\text{H}_0:\beta_1=0$ vs $\text{H}_1:\beta_1\ne0$ are equivalent, since\pause
$$
\frac{\text{MSR}}{\text{MSE}}=\frac{b_1^2\sum(x_i-\bar{x})^2}{\text{MSE}}=\frac{b_1^2}{\text{MSE}/\sum(x_i-\bar{x})^2}=\frac{b_1^2}{\s^2[b_1]}=\left(\frac{b_1-0}{\s[b_1]}\right)^2.
$$
\pause and both are generalized likelihood ratio tests (GLRT).
\end{remark}
\end{frame}

\begin{frame}[fragile]{ANOVA F-test in \texttt{R}}
\begin{small}
\begin{verbatim}
> poverty = read.table("path/to/poverty.txt", h = T, sep = "\t")
> my_model = lm(Poverty ~ Graduates, data=poverty)
> anova(my_model)
Analysis of Variance Table

Response: Poverty
          Df Sum Sq Mean Sq F value    Pr(>F)
Graduates  1 267.88 267.881  61.809 3.109e-10 ***
Residuals 49 212.37   4.334

> summary(my_model)
Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept) 64.78097    6.80260   9.523 9.94e-13 ***
Graduates   -0.62122    0.07902  -7.862 3.11e-10 ***
\end{verbatim}
\end{small}
\pause p-values of F-test and t-test for $\text{H}_0:\beta_0=0$ are same.
\end{frame}

\section{Coefficient of Determination}

\begin{frame}{Coefficient of Determination}
\begin{definition}
The \textbf{coefficient of determination} is
$$
R^2=\frac{\rm SSR}{\rm SST}=1-\frac{\rm SSE}{\rm SST},
$$
the proportion of total sample variation in $Y$ that is explained by its linear relationship with $x$.
\end{definition}
\pause Note:
\begin{itemize}
\item $0\le R^2\le 1$.
\item $R^2=1\Rightarrow$ data perfectly linear.
\item $R^2=0\Rightarrow$ regression line horizontal ($b_1=0$).
\end{itemize}
\pause The closer $R^2$ is to one, the greater the linear relationship between $x$ and $Y$.
\end{frame}

\begin{frame}{$R^2$ for different data sets}
\begin{center}
\includegraphics[scale=0.4]{plots/R-squared}
\end{center}
\end{frame}

\begin{frame}{Sample correlation $r$}
Let
$$
r=\frac{\sum_{i=1}^n(x_i-\bar{x})(Y_i-\bar{Y})}{\sqrt{\sum_{i=1}^n(x_i-\bar{x})^2\sum_{i=1}^n(Y_i-\bar{Y})^2}}
$$
be the sample correlation between $x$ and $Y$.

\pause\begin{exe} Show that
\begin{itemize}
    \item $R^2=r^2$,
    \item $\mathrm{sgn}(r)=\mathrm{sgn}(b_1)$.
\end{itemize}
\end{exe}

\pause\begin{remark}
\begin{itemize}
\item $r\approx 0\Rightarrow$ little linear association b/w $x$ and $Y$
\item $r\approx 1\Rightarrow$ strong positive, linear association b/w $x$ and $Y$
\item $r\approx -1\Rightarrow$ strong negative, linear association b/w $x$ and $Y$.
\end{itemize}
\end{remark}
\end{frame}

\begin{frame}[fragile]
\frametitle{Cautions about $R^2$ and $r$}
It is possible that
\begin{itemize}
\item $R^2\approx 1$, but the $\E[Y_i]$ may not lay on a line (Why?)
\item\pause $R^2\not\approx1$, but a line is best for $\E[Y_i]$ (Why?)
\item\pause $R^2\approx 0$, but $x$ and $Y$ are highly related (Why?)
\end{itemize}

\vspace{20pt}
\pause Poverty vs HS Graduation data:
\begin{small}
\begin{verbatim}
> summary(my_model)
...
Residual standard error: 2.082 on 49 degrees of freedom
Multiple R-squared:  0.5578, Adjusted R-squared:  0.5488
F-statistic: 61.81 on 1 and 49 DF,  p-value: 3.109e-10
\end{verbatim}
\end{small}
\end{frame}

\begin{frame}{Cautions about regression}
\begin{itemize}
\item Concluding that $x$ and $Y$ are linearly related (that $\beta_1\ne0$) does not imply a causal relationship between $x$ and $Y$. \onslide<2->{(Correlation does not imply causation!)}
\item<3-> Beware of extrapolation: predicting $Y$ for $x$ far outside the range of $x$ in the data. The relationship may not hold outside of the observed $x$-values.
\begin{itemize}
\item<4-> Sometimes the intercept might be an extrapolation.
\end{itemize}
\end{itemize}
\onslide<5->{\begin{center}
\includegraphics[width=.8\textwidth]{plots/poverty_hsgrad_line_wide}
\end{center}}
\end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{frame}
\frametitle{Examples of extrapolation}

\begin{center}
\includegraphics[width=\textwidth]{plots/extrapolation}
\end{center}

\end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{frame}
\frametitle{Examples of extrapolation}

\begin{center}
\includegraphics[width=\textwidth]{plots/womenOutsprint}
\end{center}

\end{frame}

\end{document}

Slides/02 Inferences in Regression Analysis/main.tex


\documentclass[t]{beamer}
\usetheme{Copenhagen}
\setbeamertemplate{headline}{} % remove toc from headers
\beamertemplatenavigationsymbolsempty

\usepackage{amsmath, tikz, pgfplots, tcolorbox, xcolor, array, bm}
\usetikzlibrary{calc}
\pgfplotsset{compat = 1.16}

\tikzstyle{input} = [circle, text centered, radius = 1cm, draw = black]
\tikzstyle{function} = [rectangle, text centered, minimum width = 2cm, minimum height = 1cm, draw = black]

\title{Transforming Functions}
\author{}
\date{}

\AtBeginSection[]
{
  \begin{frame}
    \frametitle{Objectives}
    \tableofcontents[currentsection]
  \end{frame}
}

\begin{document}

\begin{frame}{}
  \maketitle
\end{frame}

\begin{frame}{Parent Functions}
\begin{tabular}{cc}
    $y=x$   &   $y = x^2$   \\[11pt]
    \begin{tikzpicture}[scale=0.675]
    \begin{axis}
    [
    xmin = -5,
    xmax = 5,
    ymin = -5,
    ymax = 5,
    grid,
    axis lines = middle
    ]
    \addplot[<->, >=stealth, color=blue, line width = 1.5, domain=-4:4] {x};
    \end{axis}
    \end{tikzpicture}
    &
    \begin{tikzpicture}[scale=0.675]
    \begin{axis}
    [
    xmin = -5,
    xmax = 5,
    ymin = -5,
    ymax = 5,
    grid,
    axis lines = middle
    ]
    \addplot[<->, >=stealth, color=blue, line width = 1.5, domain=-2.25:2.25] {x^2};
    \end{axis}
    \end{tikzpicture}
\end{tabular}
\end{frame}

\begin{frame}{Parent Functions}
\begin{tabular}{cc}
$y = x^3$   &   $y = |x|$   \\[11pt]
    \begin{tikzpicture}[scale=0.675]
    \begin{axis}
    [
    xmin = -5,
    xmax = 5,
    ymin = -5,
    ymax = 5,
    grid,
    axis lines = middle
    ]
    \addplot[<->, >=stealth, color=blue, line width = 1.5, smooth, domain=-1.65:1.65] {x^3};
    \end{axis}
    \end{tikzpicture}
    &
    \begin{tikzpicture}[scale=0.675]
    \begin{axis}
    [
    xmin = -5,
    xmax = 5,
    ymin = -5,
    ymax = 5,
    grid,
    axis lines = middle
    ]
    \addplot[<->, >=stealth, color=blue, line width = 1.5, domain=-4:4] {abs(x)};
    \end{axis}
    \end{tikzpicture}
\end{tabular}
\end{frame}

\begin{frame}{Parent Functions}
\begin{tabular}{cc}
$y = \sqrt{x}$  &   $y = \sqrt[3]{x}$   \\[11pt]
    \begin{tikzpicture}[scale=0.67]
    \begin{axis}
    [
    xmin = -5,
    xmax = 5,
    ymin = -5,
    ymax = 5,
    grid,
    axis lines = middle
    ]
    \addplot[->, >=stealth, color=blue, line width = 1.5, smooth, domain=0:4.5] {sqrt(x)};
    \end{axis}
    \end{tikzpicture}
    &
    \begin{tikzpicture}[scale=0.67]
    \begin{axis}
    [
    xmin = -5,
    xmax = 5,
    ymin = -5,
    ymax = 5,
    grid,
    axis lines = middle
    ]
    \addplot[<->, >=stealth, color=blue, line width = 1.5, smooth, samples = 100, domain=-4:4] {x/abs(x)*abs(x)^(1/3)};
    \end{axis}
    \end{tikzpicture}
\end{tabular}
\end{frame}

\section{Perform vertical and horizontal shifts of functions}

\begin{frame}{$f(x) \pm d$}
\textsc{Investigation:}	\newline\\

For the function $f(x) = |x|$, examine the effects of
\[g(x) = |x| \pm d \]
where $d$ is a real number.
\end{frame}

\begin{frame}{Vertical shifts}
A \alert{vertical shift} (a.k.a. \alert{vertical translation}) moves each point on the graph either up or down by a given number of spaces.	\newline\\	\pause
\begin{center}

\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c|c}
\textbf{Equation}	&	\textbf{Shift Graph} & \textbf{Visually} \\ \hline
$f(x) + d$	& Up	& Add $d$ to $y$-coordinates	\\[6pt]	\hline
$f(x) - d$ & Down & Subtract $d$ from $y$-coordinates \\
\end{tabular}
\end{center}
\end{frame}

\begin{frame}{Example 1}
For each, list the parent function and indicate the vertical translation. \emph{Be specific}.	\newline\\
(a) \quad $g(x) = x^2 + 3$	\newline\\	\pause
Parent function: $f(x) = x^2$	\newline\\	\pause
Transformation: Shift up 3 units
\end{frame}

\begin{frame}{Example 1}
(b) \quad $g(x) = x^3 - 2$	\newline\\	\pause
Parent function: $f(x) = x^3$	\newline\\	\pause
Transformation: Shift down 2 units
\end{frame}

\begin{frame}{Example 1}
(c) \quad $g(x) = |x| + 4.7$	\newline\\	\pause
Parent function: $f(x) = |x|$	\newline\\	\pause
Transformation: Shift up 4.7 units
\end{frame}

\begin{frame}{$f(x \pm c)$}
\textsc{Investigation:}	\newline\\

For the function $f(x) = |x|$, examine the effects of
\[g(x) = |x \pm c| \]
where $c$ is a real number.
\end{frame}

\begin{frame}{Horizontal shifts}
A \alert{horizontal shift} (a.k.a. \alert{horizontal translation}) moves each point on the graph either left or right by a given number of spaces.	\newline\\	\pause
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c}
\textbf{Equation}	&	\textbf{Shift Graph} \\ \hline
$f(x+c)$	& Left	\\[6pt]	\hline
$f(x-c)$ & Right \\
\end{tabular}
\end{center}
\vspace{8pt} \pause
{\color{blue}\textbf{Note:}} With horizontal shifts, the addition or subtraction is done \underline{inside} the function. \pause \newline\\

You will need to use ( ) for functions like $x^2$ and $x^3$.
\end{frame}

\begin{frame}{Example 2}
For each, list the parent function and indicate the horizontal translation. \emph{Be specific.}	\newline\\
(a) \quad $g(x) = (x-2)^2$	\newline\\	\pause
Parent function: $f(x) = x^2$ \newline\\	\pause
Transformation: Shift right 2 units
\end{frame}

\begin{frame}{Example 2}
(b) \quad $g(x) = |x+3|$	\newline\\	\pause
Parent function: $f(x) = |x|$ \newline\\	\pause
Transformation: Shift left 3 units
\end{frame}

\begin{frame}{Example 2}
(c) \quad $g(x) = \sqrt{x-6}$	\newline\\	\pause
Parent function: $f(x) = \sqrt{x}$ \newline\\	\pause
Transformation: Shift right 6 units
\end{frame}

\section{Perform vertical stretches and compressions of functions}

\begin{frame}{$a \cdot f(x)$}
\textsc{Investigation:}	\newline\\

For the function $f(x) = \sin x$, examine the effects of
\[g(x) = a \cdot \sin x \]
where $a > 1$ and also where $0 < a < 1$.
\end{frame}

\begin{frame}{Vertical Stretches and Compressions}
A \alert{vertical stretch} or \alert{vertical compression} is obtained by vertically pulling on (or vertically pressing on) the graph.	\newline\\	\pause

A vertical stretch pulls the points away from the $x$-axis, while a vertical compression pushes the points towards the $x$-axis. \newline\\ \pause

Algebraically, we obtain vertical stretches and compressions by multiplying the entire function by a positive value.
\end{frame}

\begin{frame}{Vertical Stretches and Compressions}
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c|c}
\textbf{Value of $\bm{a}$} & \textbf{Stretch or Compress?} & \textbf{Factor} \\ \hline
$a > 1$ & Stretch & $a$ \\[6pt] \hline
$0 < a < 1$ & Compression & $\frac{1}{a}$ \\
\end{tabular}
\end{center}
\pause \vspace{11pt}
Vertical stretches and compressions will multiply the $y$-coordinates  of the function by whatever the value of $a$ is.	\newline\\	\pause

\alert{\emph{Note:}} The factor is expressed as a value greater than 1.
\end{frame}

\begin{frame}{Example 3}
List the parent function and indicate if there is a vertical stretch or vertical compression. Then list the factor.	\newline\\
(a) \quad $g(x) = 2\sqrt{x}$	\newline\\	\pause
Parent function: $f(x) = \sqrt{x}$ \newline\\	\pause
Transformation: Vertical stretch by factor of 2
\end{frame}

\begin{frame}{Example 3}
(b) \quad $g(x) = 3.5\sqrt[3]{x}$	\newline\\	\pause
Parent function: $f(x) = \sqrt[3]{x}$ \newline\\	\pause
Transformation: Vertical stretch by factor of 3.5
\end{frame}

\begin{frame}{Example 3}
(c) \quad $g(x) = \dfrac{1}{3}x^2$	\newline\\	\pause
Parent function: $f(x) = x^2$ \newline\\	\pause
Transformation: Vertical compression by factor of 3
\end{frame}

\begin{frame}{Example 3}
(d) \quad $g(x) = \dfrac{2}{5}|x|$	\newline\\	\pause
Parent function: $f(x) = |x|$ \newline\\	\pause
Transformation: Vertical compression by factor of $\dfrac{5}{2}$
\end{frame}

\section{Perform reflections of functions across the x- and y-axes}

\begin{frame}{Reflections of Functions Across the x- and y-axes}
\textsc{Investigation:}	\newline\\
For the function $f(x) = \sqrt{x}$, examine the effects of
\[g(x) = -\sqrt{x} \]
\end{frame}

\begin{frame}{Reflections of Functions Across the x- and y-axes}
\textsc{Investigation:}	\newline\\
For the function $f(x) = \sqrt{x}$, examine the effects of
\[g(x) =\sqrt{-x} \]
\end{frame}

\begin{frame}{Reflections Across the x- and y-Axes}
When we multiply by $-1$, we reflect our graph across either the $x$-axis or the $y$-axis.	\newline\\	\pause
\begin{center}
\setlength{\extrarowheight}{6pt}
\begin{tabular}{c|c}
\textbf{Function} & \textbf{Reflect Across} \\ \hline
$-f(x)$ & $x$-axis \\[6pt] \hline
$f(-x)$ & $y$-axis
\end{tabular}
\end{center}
\end{frame}

\begin{frame}{Example 4}
List the parent function and indicate the axis it is reflected across. \newline\\
(a) \quad $g(x) = \sqrt[3]{-x}$	\newline\\	\pause
Parent function: $f(x) = \sqrt[3]{x}$ \newline\\ \pause
Transformation: Reflected across the $y$-axis
\end{frame}

\begin{frame}{Example 4}
(b) \quad $g(x) = -|x|$	\newline\\	\pause
Parent function: $f(x) = |x|$ \newline\\ \pause
Transformation: Reflected across the $x$-axis
\end{frame}

\section{Perform multiple transformations of functions}

\begin{frame}{Order of Multiple Transformations}
For multiple transformations, perform them in the following order:	\newline\\	\pause
\begin{enumerate}
\item<+-> Horizontal translations (shift left or right) \newline\\
\item<+-> Reflections \newline\\
\item<+-> Vertical stretches or compressions \newline\\
\item<+-> Vertical translations (shift up or down)	\newline\\
\end{enumerate}
\onslide<6->{{\color{blue}\textbf{Note}:} Steps 2 and 3 may be switched without penalty.}
\end{frame}

\begin{frame}{Example 5}
Determine the parent function and list all transformations done to it. Be specific.	\newline\\
(a) \quad $g(x) = -2\sqrt{x-3}$ \newline\\
\onslide<2->{Parent function: $f(x) = \sqrt{x}$} \newline\\
\onslide<3->{Shift right 3 units \quad $\sqrt{x-3}$} \newline\\
\onslide<4->{Reflect across $x$-axis \quad $-\sqrt{x-3}$} \newline\\
\onslide<5->{Vertical stretch by factor of 2 \quad $-2\sqrt{x-3}$}
\end{frame}

\begin{frame}{Example 5}
(b) \quad $g(x) = \frac{1}{2}(x+3)^2$ \newline\\
\onslide<2->{Parent function: $f(x) = x^2$} \newline\\
\onslide<3->{Shift left 3 units \quad $(x+3)^2$} \newline\\
\onslide<4->{Vertical compression by a factor of 2 \quad $\frac{1}{2}(x+3)^2$}
\end{frame}

\begin{frame}{Example 5}
(c) \quad $g(x) = |-x+2|+1$ \newline\\
\onslide<2->{Parent function: $f(x) = |x|$} \newline\\
\onslide<3->{Shift left 2 units \quad $|x+2|$} \newline\\
\onslide<4->{Reflect across $y$-axis \quad $|-x+2|$} \newline\\
\onslide<5->{Vertical shift up 1 unit \quad $|-x+2|+1$}
\end{frame}

\begin{frame}{Example 5}
(d) \quad $g(x) = -\dfrac{1}{5}(-x-7)^3-4$ \newline\\
\onslide<2->{Parent function: $f(x) = x^3$} \newline\\
\onslide<3->{Shift right 7 units \quad $(x-7)^3$} \newline\\
\onslide<4->{Reflect across $y$-axis \quad $(-x-7)^3$} \newline\\
\onslide<5->{Vertical compression by factor of 5 \quad $\frac{1}{5}(-x-7)^3$} \newline\\
\onslide<6->{Reflect across $x$-axis \quad $-\frac{1}{5}(-x-7)^3$} \newline\\
\onslide<7->{Vertical shift down 4 units \quad $-\frac{1}{5}(-x-7)^3-4$}
\end{frame}

\end{document}


\chapter{Preliminaries}
\label{chap:background}

% Describe what we need to understand to move forward
In this chapter, we discuss the preliminaries of encryption and quantum computation, the two ideas key to understanding the BB84 quantum KEP.

\section{Encryption}
Consider two parties, Alice and Bob, trying to communicate through a channel.
If the channel is insucure, a malicious third party, Eve, can listen and eavesdrop on the communication.
Encryption helps to make the communication more secure, by obscuring data such that an eavesdropper cannot gain any information by intercepting communication \cite{encrypt}.

\begin{figure}[htp]
\centering
\includegraphics[scale=0.4]{images/classical_communication.png}
\caption{Alice and Bob communicate over an insecure channel with an eavesdropper, Eve.}
\label{foo bar}
\end{figure}

In practice, information is encrypted by manipulating the data according to some encryption algorithm.
The goal of a good encryption algorithm is simple: allow the sender and receiver to access the information, but make the data useless to anyone else.
In order to allow for information to be decrypted by the receiver, the algorithm must be reversible with the use of a secret key, also known as the encryption key.
The basic encryption process is as follows:

\begin{itemize}
\item A sender, Alice, composes a message, referred to as the plaintext.
\item Alice encrypts the plaintext into cyphertext using some encryption algorithm and an encryption key.
\item Alice then sends the cyphertext to the receiver, Bob.
\item Bob uses the encryption key to decrypt the cyphertext into plaintext.
\item Note that the plaintext was never transmitted over the network.
\end{itemize}

Encryption appears in two general forms: symmetric and asymmetric, each with their own benefits and drawbacks.

\subsection{Symmetric Encryption}
% Symmetric Encryption
Symmetric encryption is an encryption method in which both the sender and receiver, Alice and Bob respectively, use the same key.
\begin{figure}[htp]
\centering
\includegraphics[scale=0.350]{images/symmetric_ecryption.png}
\caption{Alice and Bob symmetrically encrypt and decrypt a message.}
\label{}
\end{figure}
While symmetric encryption is the oldest form of encryption, being used for over 4000 years, it does have a major drawback -- the key distribution problem: if Alice and Bob are separate parties (as in the case of secure message transmission), they must exchange their key beforehand using a secure channel \cite{cryptography}.
If an eavesdropper, Eve, were to intercept the key in transit, then the encryption is compromised and there is not necessarily any way for Alice or Bob to know.

% Talk about AES encryption and/or stream cyphers

% One Time Pad in depth
A One-Time-Pad (OTP) is a symmetric key that is used only once.
The OTP, as the name suggests, is discarded after a singe message has been sent, and the OTP is usually the same length as the message it encrypts.
This allows bit-wise encryption techniques, such as parity manipulation, to add to the security of the key.
This also ensures that common frequency hacking techniques are not possible \cite{cryptography}.
In order to make the OTP truly secure, the bits used must be generated using a true-random number generator to avoid a malicious party guessing the pad.
Data is encrypted using a OTP with the following bitwise operation:
\begin{center}
\begin{tabular}{rc}
Plaintext: & $X = \{x_0, x_1, ..., x_n\}$ \\
OTP:  & $P = \{p_0, p_1, ..., p_n\}$  \\
Cyphertext:  & $Y = \{y_0, y_1, ..., y_n \}$\\
where  & $y_i = (x_i + p_i) \mod 2 $\\
\end{tabular}
\end{center}

For example, in binary:

\begin{center}
\begin{tabular}{rc}
Plaintext: &  \plaintext{ 0 1 0 0 1 0 1 0} \\
OTP: &        \plaintext{ 0 1 1 0 1 0 0 1} \\
Cyphertext: & \plaintext{ 0 0 1 0 0 0 1 1} \\
\end{tabular}
\end{center}

To decrypt the cyphertext using the OTP, you simply apply the same opperation:

\begin{center}
\begin{tabular}{rc}
Cyphertext: & \plaintext{ 0 0 1 0 0 0 1 1} \\
OTP: &        \plaintext{ 0 1 1 0 1 0 0 1} \\
Plaintext: &  \plaintext{ 0 1 0 0 1 0 1 0} \\
\end{tabular}
\end{center}
Under the proper conditions, a OTP is considered to be ``perfect encryption", and is only susceptible to brute force attacks, where one has to try every possible combination of the secret key, which is computationally infeasible \cite{cryptography}.
% One Time Pad caveats
However OTPs suffer from the same issue as all symmetric encryption protocols, the key distribution problem.
In the case of the OTP, not only does one encryption key have to be distributed, a unique key must be distributed for each message, making it highly impractical in most cases.

\subsection{Asymmetric Encryption}
% Asymmetric encryption
Asymmetric encryption, also known as public-key cryptography, is the process of encrypting data using one key in such a way that it can be decrypted using a different key.
In effect, Alice can encrypt a message using a key that is publicly accessible, and the message can only be decrypted by Bob, who has the secret key corresponding to the public key used.
This method requires a key to be compromised of two parts, a public and private key-pair: $k = (k_{pub}, k_{priv})$ where $k_{pub} = f(k_{priv})$ \cite{cryptography}.

% Diffie Hellman in depth
The Diffie-Hellman KEP was the first asymmetric KEP to be proposed, and is still widely used in various forms today.
It is performed as follows:

\begin{enumerate}
\item The sender and receiver, Alice and Bob respectively, establish a line of communication.
\item Some large prime $p$ and an integer $\alpha \in \{2,3,4,...,p-2\}$ are shared between Alice and Bob.
\item Alice and Bob each compute their own private keys, $a$ and $b$, respectively. $$k_{priv,A} = a \in \{2,3,4,...,p-2\}$$
\item Alice sends Bob her public key: $A = \alpha^a \mod p$.
\item Bob sends Alice his public key: $B = \alpha^b \mod p$.
\item Alice computes a session key: $k_{AB} = B^a \mod p$.
\item Bob computes a session key: $k_{AB} = A^b \mod p$.
Because $(\alpha^a)^b = (\alpha^b)^a = \alpha^{ab}$, both Alice and Bob's computations result in the same session key despite never knowing each other's private keys.
\end{enumerate}

This method is cryptographically secure due to the Discreet Logarithm Problem (DLP), finding $x$ such that $B = \alpha^x \mod p$.
This problem is computationally not feasible to solve using classical computers; although it has never been shown to be NP-Hard, nor has it been shown to be polynomial time solvable \cite{Shor_1997}.
Although these techniques are currently secure, they become easy to solve in the quantum space.

\section{Quantum Computation}

% Basic properties
Just as classical computation involves bits, quantum computation is computation using quantum bits, qubits, which are usually denoted using ``bra-ket" notation as $\ket{\psi}$ \cite{qc:agi}.
A ket is simply a representation of a vector, and in this case the vector is the ``state-vector" of the qubit, as further explained.
Similar to classical bits having the value 0 or 1, qubits' state can be the analogous $\ket{0}$ or $\ket{1}$.
These states will be used as binary, just like 0 and 1, for computation.
%	State
It is commonly said that while a classical bit exists in either the state 0 or 1, qubits can exist in both $\ket{0}$ and $\ket{1}$ at once.
There is an element of truth in this, but a more accurate description would be that as a qubits state-vector exists in a 3-dimensional space, and it can point somewhere in between the states $\ket{0}$ and $\ket{1}$.
This state-vector is described as a linear combination $\ket{\psi} = \alpha\ket{0} + \beta\ket{1}$, where $|\alpha|^2 + |\beta|^2 = 1$, and $\alpha$ and $\beta$ are referred to as the amplitude of probability for their respective kets.

%	Measurement
Just as a computer can read the value of a classical bit, we can read, or``measure", a qubit.
The measurement is performed against two states, $\ket{0}$ and $\ket{1}$, and upon measurement the qubits state collapses to one of the two values.
The probability with which a vector will collapse into one of two states is the square of the amplitude for that state.
It is unknown exactly what causes the superposition to collapse, but it has been derived from empirical observations and is the single most important property of quantum mechanics \cite{qc:agi}.
For example, consider the qubit $\ket{\psi} = \frac{1}{\sqrt{3}}\ket{0} + \sqrt{\frac{2}{3}}\ket{1}$.
The probability that $\ket{\psi}$ will be $\ket{0}$ when measured is $(\frac{1}{\sqrt{3}})^2$, or $\frac{1}{3}$.
Any qubit or state-vector is defined using this vector formula.
If $\ket{\psi}$ is a linear combination of $\ket{0}$ and $\ket{1}$, and neither amplitudes are zero, the qubit is said to be in a superposition of $\ket{0}$ and $\ket{1}$.
Superposition is one of the fundamental properties of quantum computation \cite{qc:agi}.

We use quantum operators called gates to manipulate $\alpha$ and $\beta$ probabilities.
For example, the Z gate inverts the qubit: $$\ket{\psi} = \ket{0} \rightarrow \textSq{Z} \rightarrow \ket{\psi} = \ket{1}$$ $$\ket{\psi} = \ket{1} \rightarrow \textSq{Z} \rightarrow \ket{\psi} = \ket{0}$$
Similarly, the Hadamard Gate, or H gate, performs what is called a ``quarter turn"; it maps $$\ket{\psi} = \ket{0} \rightarrow \textSq{H} \rightarrow \ket{\psi} = \frac{1}{\sqrt{2}}\ket{0} + \frac{1}{\sqrt{2}}\ket{1}$$  $$\ket{\psi} = \ket{1} \rightarrow \textSq{H} \rightarrow \ket{\psi} = \frac{1}{\sqrt{2}}\ket{0} - \frac{1}{\sqrt{2}}\ket{1}$$ \cite{qc:agi}.
Since after the H gate is applied $\alpha$ and $\beta$ both equal $\frac{1}{\sqrt{2}}$, and ${(\frac{1}{\sqrt{2}})}^2 = \frac{1}{2}$, if we measure $\ket{\psi}$ in this state it would collapse to $\ket{0}$ and $\ket{1}$ with equal probability.
It is worth noting that this is one way to create a true-random number generator.

%	Bases
Because the state vector exists in a 3D state space, we do not necessarily have to measure against $\ket{0}$ and $\ket{1}$, in fact we can measure against any two vector values that are opposite each other on the unit circle.
A set of vectors to measure against is known as a basis, and there are many possible bases.
The set $\{\ket{0},\ket{1}\}$ is known as the standard basis or computational basis, as it is analogous to classical bits \cite{qcftgu}.
Another common basis is the Hadamard Basis, which is denoted $\ket{+} = \frac{1}{\sqrt{2}}\ket{0} + \frac{1}{\sqrt{2}}\ket{1}$ and $\ket{-} = \frac{1}{\sqrt{2}}\ket{0} - \frac{1}{\sqrt{2}}\ket{1}$.
The H gate will put $\ket{0} \rightarrow \ket{+}$ and $\ket{1} \rightarrow \ket{-}$, but it will also revert the Hadamard basis into the standard basis: $\ket{+} \rightarrow \ket{0}$ and $\ket{-} \rightarrow \ket{1}$.

Because the standard basis and the Hadamard basis are perpendicular to each other they are referred to as orthonormal bases.
That is to say if a $\ket{\psi} = \ket{+}$ or $\ket{\psi} = \ket{-}$ then it will have a 50\% change of being $\ket{0}$ or $\ket{1}$ if measured in the standard basis, and vice-versa \cite{qc:agi}.

\begin{figure}[H]
\centering
\begin{tabular}{cc|cccc}
	&&\multicolumn{4}{|  c }{Measured value}  \\
	   &       &$ \ket{0}$ & $\ket{1}$ & $\ket{+}$ & $\ket{-}$\\ \hline
	\multirow{4}{*}{Qubit state} &$\ket{0}$ & $100\%$   & $0\%$     & $50\%$    &  $50\%$  \\
	&$\ket{1}$ & $0\%$     & $100\%$   & $50\%$    &  $50\%$  \\
	&$\ket{+}$ & $50\%$    & $50\%$    & $100\%$   &  $0\%$   \\
	&$\ket{-}$ & $50\%$    & $50\%$    & $0\%$     &  $100\%$ \\
\end{tabular}
\caption{The probability of measuring a qubit from one basis state into another}
\end{figure}

In effect, if you have a vector in one orthonormal basis it is useless if measured in another orthonormal basis.
Further, since the vector state changes when measured, if a value is encoded in one orthonormal basis, that information is destroyed by measuring in another orthonormal basis \cite{qcftgu}.

% Guarantees
% 	Copy-ability
Another crucial property of qubits is that they cannot be cloned.
It is impossible to have an operator that clones the state of an input qubit into an output qubit without knowing the basis of the input \cite{qc:agi}.
This property proves crucial in the BB84 protocol, as explained in Chapter~\ref{chap:bb84}.

Using these properties and others, it has been shown that quantum computers can solve the DLP on an $n$-bit number in only $O(n^2\log n \log \log n)$ time \cite{MikeAndIke}.
Therefore, the growing popularity of quantum computers poses a serious threat to the securities of the Diffie Hellman KEP and asymmetric encryption.
The BB84 protocol is a quantum key distribution (QKD) protocol that allows two parties to co-create a shared key, using a verifiably secure channel, that can then be used to symmetrically encrypt messages.


%!TEX root = ../../dissertation.tex

\section{Computation}
\label{app:computation_BCFL}

{\textit Scaling.\/}
One challenge with computing solutions to this model is that the
productivity process (\ref{eq:lom-z}) has a unit root.
We deal with that by scaling: We divide everything by
$\bar{z}_t$ and label it with a tilde:  $\ti{c}_{1t} = c_{1t} / \bar{z}_t$,
$\ti{c}_{2t} = c_{2t} / \bar{z}_t$, $\wt{U}_{t} = U_{t} / \bar{z}_t$, and so on.
We define the relevant state by $ \ti{s}_t = (\wh{z}_t, v_t)$
and the growth rate in the scaling variable $\bar{z}_t$ by
$g_{t+1} = \bar{z}_{t+1} / \bar{z}_t $.

Since the functions are all hd1, we can divide the whole Bellman equation by $\bar{z}_t$:
\begin{eqnarray*}
    \wt{J}(\wt{U}_t, \ti{s}_t) &=& \max_{\{\ti{c}_{1t}, \wt{U}_{t+1} \}}
            V \big\{ \ti{c}_{1t}, \mu_t [g_{t+1} \wt{J}(\wt{U}_{t+1}, \ti{s}_{t+1})] \big\} \\
    \mbox{s.t.} && V \big\{ \ti{c}_{2t}, \mu_t (g_{t+1} \wt{U}_{t+1}) \big\} \;\geq\; \wt{U}_{t} \\
            && \mbox{plus resource constraints and shocks} .
\end{eqnarray*}
The scaled resource constraints become
\begin{eqnarray*}
    \ti{a}_{1t} + \ti{a}_{2t} &=&  \ti{y}_{1t} \;\;=\;\; \wh{z}_{t} \\
    \ti{b}_{1t} + \ti{b}_{2t} &=&  \ti{y}_{2t} \;\;=\;\; 1/\wh{z}_{t} \\
    \ti{c}_{1t} &=&  h (\ti{a}_{1t}, \ti{b}_{1t} ) \\
    \ti{c}_{1t} &=&  h (\ti{a}_{1t}, \ti{b}_{1t} ) .
\end{eqnarray*}
The laws of motion for the shocks are
(\ref{eq:lom-zhat}) for $\wh{z}_t$ and (\ref{eq:lom-v}) for $v_t$ (no scaling required).
$\bar{z}_t$ drops out, leaving us with a lower-dimensional state.

Given a solution to the scaled problem, we can multiply by $\bar{z}_t$
to produce a solution to the original problem.
The notation is horrendous.
The point is simply that we can
convert our problem to one with stable shocks.

{\textit Algorithm.\/}
Another challenge in computing a solution is that at each date $t$,
we need to choose utility promises $\wt{U}_{t+1}$
for every state the following period.
We adapt the algorithm of Collin-Dufresne, Johannes, and Lochstoer (2015) to a multi-good setting.
Their algorithm has three essential features.
First, they make a clever choice of state variable.
Second, they use backward recursion, starting at a terminal date and computing the value
function recursively at earlier dates.
Third, they use the first-order conditions to solve for the optimal policies,
rather than simply choosing the best from a finite set of possibilities.

Consider the state variable.  We characterize the problem using promised utility $\wt{U}_t$ as the state.
Collin-Dufresne, Johannes, and Lochstoer replace promised utility with the consumption share.
Given the consumption share, the conditions of the solution give us promised utility as a byproduct.
We use the (additive) Pareto weight the same way,
with consumption shares and promised utility as byproducts.
The choice of variable does not change the solution,
but in our experience it can have a large effect on the performance of the algorithm.

Now consider backward recursion.
We approximate an infinite-horizon dynamic programming problem
with a long finite-horizon problem.
At the terminal date $T$, we set utility equal to current consumption.
This corresponds to a steady state in which consumptions are constant at these values
at all future dates.
If we know the Pareto weight $\lambda_T^*$ in all states at $T$,
we can compute allocations of the two intermediate goods and consumptions of the two agents
in the same states.
The consumptions then give us value functions for each state at $T$ that we can use the
previous period.

In any previous period $t$, we need to choose both current consumptions
and continuation values for the Pareto weight in every succeeding state at $t+1$.
We choose the Pareto weights $\lambda^*_{t+1}$ and current consumptions $c_{jt}$
by solving the planner's first-order conditions.
We speed this up by pre-solving the allocation problem on a fine grid for the state variables.
Given a solution for the Pareto weight, the time aggregators then give us current utility
and the value function.

We repeat this process until the value function and decision rules converge.
We use a sup norm criterion over both value functions and decision rules.

We use a discrete grid for the Pareto weight and other state variables.
Our numerical implementation has 451 points for the Pareto weight,
13 points each for the exogenous state variables $\wh{z}_t$ and $v_t$,
and 5 quadrature nodes for each of the three future innovations ---
a total of 9.5 million points.
We then use Hermite quadrature to compute the expectations in the certainty equivalent functions.
If the number of points seems high, we have found that it's essential to have a fine grid
over the Pareto weight to describe its dynamics adequately.
We could probably work with a less fine grid,
but this gives us some confidence that the solutions are accurate.

We do all of these calculations in Julia,
a programming language that combines the convenience of
dynamic vector-based languages like Matlab with the speed of compiled languages like C or Fortran.


\documentclass{article}

\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{amsfonts}
\usepackage{tikz}
\usepackage{physics}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{hyperref}

\usetikzlibrary{automata,positioning}

%
% Basic Document Settings
%

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\chead{\hmwkClass\ : \hmwkTitle}
\rhead{\firstxmark}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}

%
% Create Problem Sections
%
\newcommand{\be}{\begin{equation}}
\newcommand{\ee}{\end{equation}}
\newcommand{\bes}{\begin{equation*}}
\newcommand{\ees}{\end{equation*}}
\newcommand{\bea}{\begin{flalign*}}
\newcommand{\eea}{\end{flalign*}}

\newcommand{\enterProblemHeader}[1]{
    \nobreak\extramarks{}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
}

\newcommand{\exitProblemHeader}[1]{
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \stepcounter{#1}
    \nobreak\extramarks{Problem \arabic{#1}}{}\nobreak{}
}

\setcounter{secnumdepth}{0}
\newcounter{partCounter}
\newcounter{homeworkProblemCounter}
\setcounter{homeworkProblemCounter}{1}
\nobreak\extramarks{Problem \arabic{homeworkProblemCounter}}{}\nobreak{}

%
% Homework Problem Environment
%
% This environment takes an optional argument. When given, it will adjust the
% problem counter. This is useful for when the problems given for your
% assignment aren't sequential. See the last 3 problems of this template for an
% example.
%
\newenvironment{homeworkProblem}[1][-1]{
    \ifnum#1>0
        \setcounter{homeworkProblemCounter}{#1}
    \fi
    \section{Problem \arabic{homeworkProblemCounter}}
    \setcounter{partCounter}{1}
    \enterProblemHeader{homeworkProblemCounter}
}{
    \exitProblemHeader{homeworkProblemCounter}
}

%
% Homework Details
%   - Title
%   - Due date
%   - Class
%   - Section/Time
%   - Instructor
%   - Author
%

\newcommand{\hmwkTitle}{Assignment\ \#6}
\newcommand{\hmwkDueDate}{Due on 28th November, 2018}
\newcommand{\hmwkClass}{Advanced Quantum Mechanics}
\newcommand{\hmwkClassTime}{}
\newcommand{\hmwkClassInstructor}{}
\newcommand{\hmwkAuthorName}{\textbf{Aditya Vijaykumar}}

%
% Title Page
%

\title{
    %\vspace{2in}
    \textmd{\textbf{\hmwkClass:\ \hmwkTitle}}\\
    \normalsize\vspace{0.1in}\small{\hmwkDueDate\ }\\
%    \vspace{3in}
}

\author{\hmwkAuthorName}
\date{}

\renewcommand{\part}[1]{\textbf{\large Part \Alph{partCounter}}\stepcounter{partCounter}\\}

%
% Various Helper Commands
%

% Useful for algorithms
\newcommand{\alg}[1]{\textsc{\bfseries \footnotesize #1}}

% For derivatives
\newcommand{\deriv}[1]{\frac{\mathrm{d}}{\mathrm{d}x} (#1)}

% For partial derivatives
\newcommand{\pderiv}[2]{\frac{\partial}{\partial #1} (#2)}

% Integral dx
\newcommand{\dx}{\mathrm{d}x}

% Alias for the Solution section header
\newcommand{\solution}{\textbf{\large Solution}}

% Probability commands: Expectation, Variance, Covariance, Bias
\newcommand{\E}{\mathrm{E}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\Cov}{\mathrm{Cov}}
\newcommand{\Bias}{\mathrm{Bias}}

\begin{document}

\maketitle
(\textbf{Acknowledgements} - I would like to thank Chandramouli Chowdhury, Sarthak Duary and Junaid Majeed for discussions.)
\\

\begin{homeworkProblem}[1]
	The wave equation can be written as,
	\begin{align*}
	\pdv[2]{f}{t} &= c^2 \laplacian{f} \\
	\implies \pdv[2]{f}{t} &= c^2 \qty[\dfrac{1}{r}\pdv{r} \qty(r \pdv{f}{r}) + \dfrac{1}{r^2} \pdv[2]{f}{\theta} ]
	\end{align*}
	We write $ f(r, \theta, t)  = R(r) \Theta(\theta) T(t)$. We then have,
	\begin{equation*}
	\dfrac{1}{T}\dv[2]{T}{t} = c^2 \qty[\dfrac{1}{Rr} \dv{r} \qty(r \dv{R}{r}) + \dfrac{1}{r^2 \Theta} \dv[2]{\Theta}{\theta} ] = constant = - \lambda^2
	\end{equation*}
	where $ - \lambda^2 $ comes from the fact that LHS is only dependent on $ t $, and the RHS does not depend on $ t $. Let's look at the RHS.
	\begin{align*}
	c^2 \qty[\dfrac{1}{Rr} \dv{r} \qty(r \dv{R}{r}) + \dfrac{1}{r^2 \Theta} \dv[2]{\Theta}{\theta} ] &= -\lambda^2 \\
	\implies \dfrac{r}{R} \dv{r} \qty(r \dv{R}{r}) + \dfrac{1}{\Theta} \dv[2]{\Theta}{\theta}  &= -\dfrac{\lambda^2 r^2}{c^2}\\
	\implies \dv[2]{\Theta}{\theta} = - \mu^2 \Theta \qq{and} \dfrac{r}{R} \dv{r} \qty(r \dv{R}{r}) &= - \dfrac{\lambda^2 r^2}{c^2} + \mu^2	\\
	\implies \Theta =A  e^{i \mu \theta} + B e^{-i \mu \theta} \qq{and} {r R''(r)+R'(r)} + &\qty(\frac{\lambda ^2 r^2}{c^2} - \mu^2) R(r) = 0 \\
	\implies \Theta =A  e^{i \mu \theta} + B e^{-i \mu \theta} \qq{and} R &= C J_\mu \qty(\frac{\lambda R}{c})
	\end{align*}
	So $ \mu, \lambda $ are the required quantum numbers.
	\\

	\textbf{Part (b)}\\
	We have to find $ G(\vb{x},\vb{x'}) $. We know that the Green's function in momentum space is $ \tilde{G}(\vb{p}) = \dfrac{1}{E - p^2/2m} $

	\begin{align*}
	\therefore  G(\vb{x},\vb{x'})  &= \mel{\vb{x}}{\tilde{G}(\vb{p})}{\vb{x'}} \\
	&= \int \dfrac{d^2 \vb{p'}}{(2\pi)^2} \dfrac{d^2 \vb{p}}{(2\pi)^2}  \ip{\vb{x}}{\vb{p}}  \mel{\vb{p}}{\tilde{G}(\vb{p})}{\vb{p'}} \ip{\vb{p'}}{\vb{x'}}\\
	&= \int \dfrac{d^2 \vb{p'}}{(2\pi)^2} \dfrac{d^2 \vb{p}}{(2\pi)^2}  \ip{\vb{x}}{\vb{p}}  \dfrac{1}{E - p^2/2m} \delta_{\vb{p},\vb{p'}} \ip{\vb{p'}}{\vb{x'}}
	\end{align*}

	\begin{align*}
	G(\vb{x},\vb{x'})   &= \int  \dfrac{d^2 \vb{p}}{(2\pi)^2}  \ip{\vb{x}}{\vb{p}}  \dfrac{1}{E - p^2/2m}  \ip{\vb{p}}{\vb{x'}}\\
	&= \int  \dfrac{d^2 \vb{p}}{(2\pi)^2}  e^{i \vb{p} \cdot (\vb{x} - \vb{x'})}  \dfrac{1}{E - p^2/2m}\\
	&= \dfrac{1}{4 \pi^2} \int_0^\infty  p dp  \int_{-1}^1 d(\cos \theta) e^{i p \abs{\vb{x} - \vb{x'}}}  \dfrac{1}{E - p^2/2m}\\
	&= \dfrac{2m}{4 \pi^2} \int_0^\infty  p dp  \int_{-1}^1 d(\cos \theta) e^{i p \abs{\vb{x} - \vb{x'}} \cos \theta}  \dfrac{1}{2mE - p^2}\\
	&=\dfrac{2m}{4 \pi^2} \int_0^\infty  p dp \dfrac{ e^{i p \abs{\vb{x} - \vb{x'}} } - e^{-i p \abs{\vb{x} - \vb{x'} }}}{i \abs{\vb{x} - \vb{x'} }}  \dfrac{1}{2mE - p^2}\\
	&= \dfrac{2m i}{4 \pi^2  \abs{\vb{x} - \vb{x'} }} \int_{-\infty}^\infty  p dp  e^{i p \abs{\vb{x} - \vb{x'}} } \dfrac{1}{ p^2  - 2 m E - i \epsilon} \\
	&= \dfrac{2m i}{4 \pi^2  \abs{\vb{x} - \vb{x'} }} ( \pi i) e^{i k \abs{\vb{x} - \vb{x'}}} \\
	G(\vb{x},\vb{x'})  &= - \dfrac{m  e^{i k \abs{\vb{x} - \vb{x'}}} }{2 \pi  \abs{\vb{x} - \vb{x'} }}
	\end{align*}
	The integration is exactly the same as the contour integral which was described in class, and so is the Green's function.
\end{homeworkProblem}

\begin{homeworkProblem}[2]
	We first lay out our notation. From the Lippmann-Schwinger equation, we know,
	\begin{equation*}
	\psi(\vb{r}) = e^{i \vb{k} \cdot \vb{r}} +  \dfrac{e^{i k r}}{r} \underbrace{\qty[\qty(\dfrac{-m}{2 \pi}) \int d^3 \vb{r'} e^{-i k \vu{r} \cdot \vb{r'}} V(\vb{r'}) \psi (\vb{r'})]}_{f(\vb{k},\vb{k'})}
	\end{equation*}
	To solve this order by order, we use the ansatz $ \psi(\vb{r}) = e^{i \vb{k} \cdot \vb{r}}  + \sum_{n = 1}^{\infty} \phi_n(\vb{r})$. Substituting in the above equation, we get the recurrence relation,
	\begin{align*}
	\phi_{n+1}(\vb{r}) &=   \int d^3 \vb{r'} G_0^+ (k,\vb{r} - \vb{r'}) V(\vb{r'})\phi_n(\vb{r})\\
	\qq{in particular} \phi_{1}(\vb{r}) &= \qty(\dfrac{-m}{2 \pi}) \dfrac{e^{i k r}}{r} \int d^3 \vb{r'} e^{-i k \vu{r} \cdot \vb{r'}} V(\vb{r'}) e^{i \vb{k} \cdot \vb{r'}} = \qty(\dfrac{-m}{2 \pi}) \dfrac{e^{i k r}}{r}  \int d^3 \vb{r'} e^{i (\vb{k} - \vb{k'}) \cdot \vb{r'}} V(\vb{r'})
	\end{align*}
	In the Born approximation, one sets $ \psi(\vb{r}) = e^{i \vb{k} \cdot \vb{r}}  + \phi_1 (\vb{r}) \implies f^{(1)}(\vb{k},\vb{k'}) = \qty(\dfrac{-m}{2 \pi}) \int d^3 \vb{r'} e^{i (\vb{k} - \vb{k'}) \cdot \vb{r'}} V(\vb{r'}) $

	\begin{align*}
	\dv{\sigma}{\Omega} &= \abs{f^{(1)} (\vb{k},\vb{k'})}^2 \\
	&= \dfrac{m^2}{4 \pi^2} \int d^3 \vb{x}  d^3 \vb{x'} V(\vb{x}) V(\vb{x'}) e^{-i \vb{k'} \cdot \vb{x}} e^{i \vb{k} \cdot \vb{x}} e^{-i \vb{k} \cdot \vb{x'}} e^{i \vb{k'} \cdot \vb{x}}\\
	\dv{\sigma}{\Omega} &= \dfrac{m^2}{4 \pi^2} \int d^3 \vb{x}  d^3 \vb{x'} V(\vb{x}) V(\vb{x'}) e^{i (\vb{k} - \vb{k'}) \cdot (\vb{x} - \vb{x'})}
	\end{align*}
	The total cross-section $ \sigma_{T} $ can be obtained by integrating over outgoing momenta and averaging over ingoing momenta.

	\begin{align*}
	\implies \sigma_T  &= \dfrac{m^2}{4 \pi^2} \int d^3 \vb{x}  d^3 \vb{x'} \dfrac{d \Omega_{\vb{k}}}{4 \pi} d \Omega_{\vb{k'}} V(\vb{x}) V(\vb{x'}) e^{i (\vb{k}) \cdot (\vb{x} - \vb{x'})}  e^{i (-\vb{k'}) \cdot (\vb{x} - \vb{x'})}   \\
	&= \dfrac{m^2}{4 \pi^2} \int d^3 \vb{x}  d^3 \vb{x'} V(\vb{x}) V(\vb{x'})\int_{0}^{2 \pi} \dfrac{d\phi_{\vb{k}}}{4 \pi} \int_{-1}^{1}  d (\cos \theta_{\vb{k}}) e^{i \abs{\vb{k}} \abs{\vb{x} - \vb{x'}} \cos \theta_{\vb{k}}}  \int_{0}^{2 \pi} d\phi_{\vb{k'}} \int_{-1}^{1}  d (\cos \theta_{\vb{k'}}) e^{-i \abs{\vb{k'}} \abs{\vb{x} - \vb{x'}} \cos \theta_{\vb{k'}}}
	\end{align*}
	\begin{align*}
	\sigma_T &= \dfrac{m^2}{4 \pi} \int d^3 \vb{x}  d^3 \vb{x'} V(\vb{x}) V(\vb{x'}) \dfrac{e^{i k \abs{\vb{x} - \vb{x'}}} - e^{-i k \abs{\vb{x} - \vb{x'}} }}{i k \abs{\vb{x} - \vb{x'}}} \cross  \dfrac{e^{-i k \abs{\vb{x} - \vb{x'}}} - e^{i k \abs{\vb{x} - \vb{x'}} }}{-i k \abs{\vb{x} - \vb{x'}}} \impliedby (\abs{\vb{k}} = \abs{\vb{k'}} = k) \\
	&= \dfrac{m^2}{4 \pi} \int d^3 \vb{x}  d^3 \vb{x'} V(\vb{x}) V(\vb{x'})   \dfrac{2 \sin  k \abs{\vb{x} - \vb{x'}} }{k \abs{\vb{x} - \vb{x'}}} \cross \dfrac{2 \sin  k \abs{\vb{x} - \vb{x'}} }{k \abs{\vb{x} - \vb{x'}}} \\
	\sigma_{T} &= \dfrac{m^2}{ \pi} \int d^3 \vb{x}  d^3 \vb{x'} V(\vb{x}) V(\vb{x'})   \dfrac{ \sin^2  k \abs{\vb{x} - \vb{x'}} }{k^2 \abs{\vb{x} - \vb{x'}}^2}
	\end{align*}
	 The optical theorem tells us,
	 \begin{equation*}
	 \sigma_T = \dfrac{4 \pi}{k} 	\Im f(\vb{k}, \vb{k}) = - \dfrac{2 m L^3}{k}  \Im \mel{\vb{k}}{T}{\vb{k}}
	 \end{equation*}
	 Upto first order the imaginary part is zero. Hence,
	 \begin{align*}
	 \sigma_T &=  - \dfrac{2 m L^3}{k}  \Im \mel{\vb{k}}{V \dfrac{1}{E -  H_0 + i \epsilon} V}{\vb{k}} \\
	 &=  - \dfrac{2 m L^3}{k}  \Im \int d^3 \vb{x} d^3 \vb{x'}  \mel{\vb{k}}{V}{\vb{x}} \mel{\vb{x}}{ \dfrac{1}{E -  H_0 + i \epsilon} }{\vb{x'}} \mel{\vb{x'}}{V}{\vb{k}} \\
	 &=  - \dfrac{2 m }{k}  \Im \int d^3 \vb{x} d^3 \vb{x'}  V(\vb{x}) V(\vb{x'})  e^{-i \vb{k} \cdot (\vb{x} - \vb{x'})} \qty(\dfrac{-m}{2 \pi})  \dfrac{e^{i k \abs{\vb{x} - \vb{x'} }}}{\abs{\vb{x} - \vb{x'} }}\\
	 &=  - \dfrac{2 m }{k}  \Im \int d^3 \vb{x} d^3 \vb{x'}  V(\vb{x}) V(\vb{x'})  e^{-i \vb{k} \cdot (\vb{x} - \vb{x'})} \qty(\dfrac{-m}{2 \pi})  \dfrac{e^{i k \abs{\vb{x} - \vb{x'} }}}{\abs{\vb{x} - \vb{x'} }}\\
	 &= \dfrac{m^2}{ \pi} \int d^3 \vb{x}  d^3 \vb{x'} V(\vb{x}) V(\vb{x'})   \dfrac{ \sin^2  k \abs{\vb{x} - \vb{x'}} }{k^2 \abs{\vb{x} - \vb{x'}}^2}
	 \end{align*}
\end{homeworkProblem}

\begin{homeworkProblem}
	From equation $ (6.4.54) $ of Sakurai, we have,
	\begin{equation*}
	\tan \delta_l = \dfrac{k R j'_l(kR)  - \beta_l j_l(kR)}{kR n'_l(kR) - \beta_ln_l (kR)} \qq{where} \beta_l = \eval{\qty(\dfrac{r}{A_l} \dv{A_l}{r})}_{r = R}
	\end{equation*}
	Here, $ j_l $ and $ n_l $ are the spherical Bessel function, and $ A_l $ is the solution to the wave equation in the region $ r >R $.

	From Sakurai $ (6.4.55) $, we have, for $ r<R $ and $ u_l(r) = r A_l(r) $
	\begin{equation*}
	\dv[2]{u_l}{r} + \qty(k^2 - 2mV_0	 - \dfrac{l(l+1)}{r^2}) u_l = 0
	\end{equation*}
	Solving the above differential equation in Mathematica, we get,
	\begin{equation*}
	u_l(r) = c_1 \sqrt{r} j_{\frac{1}{2} (2 l+1)}\left(-i r \sqrt{2 \text{mV}_0-k^2}\right)+c_2 \sqrt{r} n_{\frac{1}{2} (2 l+1)}\left(-i r \sqrt{2 \text{mV}_0-k^2}\right)
	\end{equation*}
	The above solution is subject to boundary condition $ u_l(0) = 0 $. From the properties of Bessel functions, we know that this can only be possible when $ l=0 $. Then, using $ l=0 $ and applying $ u_0(0) = 0 $,
	\begin{equation*}
	A_0(r) = \frac{i \sqrt{\frac{2}{\pi }} c_1 \sinh \left(x \sqrt{2 \text{mV}_0-k^2}\right)}{x \sqrt{-i \sqrt{2 \text{mV}_0-k^2}}} \implies \beta_0(r) = \dfrac{r}{A_0} \dv{A_0}{r} = r \sqrt{2 \text{mV}_0-k^2} \coth \left(r \sqrt{2 \text{mV}_0-k^2}\right) - 1
	\end{equation*}
\end{homeworkProblem}

\begin{homeworkProblem}
	\textbf{Part (a)}\\
	If $ \rho $ is the density matrix of a pure state, we know that $ \rho^2 = \rho \implies \log \rho^2 = \log \rho \implies 2 \log \rho - \log \rho = 0 \implies \log \rho = 0 \implies \rho \log \rho  = 0 \implies S = - \Tr(\rho \log \rho) = 0 $  \\

	\textbf{Part (b)}\\
	We know that $ \Tr \rho = 1 $ and that $ \rho $ is positive semi-definite $ \implies \lambda_i \le 1 $, where $ \lambda_i $ are the eigenvalues. In the basis where the density matrix in diagonal, $ \lambda_i $'s are the diagonal elements.
	\begin{equation*}
	\Tr \rho = \sum_{i=1}^{d} \lambda_i = 1 \qq{and} -\Tr \rho \log \rho = \sum_{i=1}^{d} \lambda_i \log \dfrac{1}{\lambda_i} = \sum_{i=1}^{d}  \log \dfrac{1}{\lambda_i^{\lambda_i}} = \log \prod_{i=1}^d\dfrac{1}{  \lambda_i^{\lambda_i}}
	\end{equation*}
	The  Arithmetic Mean (AM) - Geometric Mean (GM) inequality with weights \footnote{https://www.jstor.org/stable/24340414} says that for non-negative numbers $ n_i $ and non-negative weights $ w_i $, the following holds
	\begin{equation*}
	\dfrac{\sum_{i} w_i n_i}{\sum_{i} w_i} \ge
	\qty(\prod_{i=1} n_i^{w_i})^{\frac{1}{\sum_{i} w_i}}
	\end{equation*}
	Using $	n_i = \dfrac{1}{\lambda_i}$ and $ w_i = \lambda_i $,
	\begin{align*}
	\dfrac{\sum_{i=1}^d \lambda_i \frac{1}{\lambda_i}}{\sum_{i}^d \lambda_i} &\ge
	\qty(\prod_{i=1}  \frac{1}{\lambda_i^{\lambda_i}})^{\frac{1}{\sum_{i}^d \lambda_i}} \\
	\implies \prod_{i=1}  \frac{1}{\lambda_i^{\lambda_i}} &\le  d \\
	\implies \log \prod_{i=1}  \frac{1}{\lambda_i^{\lambda_i}} &\le  \log d \\
	\implies S \le \log d
	\end{align*}
	Hence the maximum value of $ S $ is $ \log d $.
\end{homeworkProblem}

\begin{homeworkProblem}[5]
	Given,
	\begin{align*}
	\ket{\psi} &= \kappa \sum_E e^{- \frac{\beta E}{2}} \ket{E} \ket{\tilde{E}}\\
	\implies \ip{\psi}{\psi} &= \kappa^2 \sum_{E,E'} e^{- \frac{\beta (E + E')}{2}} \ip{E}{E'}  \ip{\tilde{E}}{\tilde{E'}}\\
	\implies 1 &= \kappa^2 \sum_{E,E'} e^{- \frac{\beta (E + E')}{2}} \delta_{E,E'}\\
	\implies \kappa &= \qty(\dfrac{1}{\sum_E e^{-{\beta E}}})^{1/2} = \dfrac{1}{\sqrt{Z(\beta)}}
	\end{align*}

	The density matrix corresponding to $ \ket{\psi} $ is,
	\begin{align*}
	\rho &= \op{\psi}{\psi}\\
	&= \kappa^2 \sum_{E,E'} e^{- \frac{\beta (E + E')}{2}} \ket{E} \ket{\tilde{E}} \bra{E'} \bra{\tilde{E'}}
	\end{align*}

	We need to find $ \rho_1 = \Tr_2 \rho = \ev{\rho}{\tilde{E}} $,
	\begin{align*}
	\rho_1  &= \kappa^2 \sum_{E,E'} e^{- \frac{\beta (E + E')}{2}} \ket{E} \ip{\tilde{E}}{\tilde{E}} \bra{E'} \ip{\tilde{E'}}{\tilde{E}} \\
	&= \kappa^2 \sum_{E,E'} e^{- \frac{\beta (E + E')}{2}} \ket{E} \bra{E'} \delta_{E,E'}\\
	\rho_1 &= {\sum_{E} \kappa^2 e^{- {\beta E}} \ket{E} \bra{E}}
	\end{align*}
	We see that $ \rho_1 $ is diagonal in the energy basis. Hence, taking the trace of a function of $ \rho_1 $ is straightforward,
	\begin{align*}
	S = -\Tr \rho_1 \log \rho_1 &= {\sum_{E} \dfrac{e^{-\beta E}}{Z} (\log Z + \beta E)}\\
	&=  {\sum_{E} \dfrac{e^{-\beta E}}{Z} \beta( E - F)}\\
	&\sim S_{th}
	\end{align*}
	where $ S_{th} $ is the thermal entropy of the system.
\end{homeworkProblem}

\end{document}