\documentclass[10pt,a4paper,twocolumn]{article}

\usepackage{microtype,nowidow}
\usepackage{fontenc,newtxtext}
\usepackage[left=1.5cm,right=1.5cm,top=2cm,bottom=2cm]{geometry}

\usepackage{amsmath, amssymb, bm}
\usepackage{graphicx}
\usepackage{booktabs}
\usepackage{hyperref}
\usepackage{cleveref}
\usepackage[backend=biber]{biblatex}
\addbibresource{references.bib}

\usepackage{macros}

\title{\bfseries Approximate inference for Bayesian neural networks}
\author{Silas Brack}
\date{Spring 2022}

\begin{document}
\maketitle

\begin{abstract}
\end{abstract}

\section{Introduction}

Neural networks (NNs) suffer from calibration issues and overconfidence, catastrophic interference in continual learning, and sensitivity to hyper-parameter selection.

Bayesian neural networks (BNNs) arise from applying Bayesian inference on the layer weights of NNs and are one solution for these issues, allowing an estimate of the uncertainty of the output of NNs.

However, BNNs networks are expensive and cumbersome to train.
Their posteriors are intractable in an exact sense, meaning these have to be approximately inferred.
Furthermore, they inherit having a large parameter space (and consequently a large optimisation space) from their non-Bayesian counterparts, leading to slow inference for traditional methods such as Markov-chain Monte Carlo (MCMC), which, while yielding very competitive results~\cite{izmailov2021bayesian}, scales poorly with increasing dimensionality.
Other Bayesian and quasi-Bayesian methods have been proposed as alternatives, such as variational inference, SWAG/MultiSWAG, deep ensembles, and Laplace approximations.

This paper analyses these alternative approximate inference methods.

\section{Previous Work}\label{sec:priorwork}

\subsection{Bayes’ Theorem}

Bayes' theorem describes the probability of an event occurring with respect to prior knowledge of that event.
Specifically for modelling, it can be used to relate latent variables with observed variables, allowing distributions on these latent variables to be inferred.
We therefore want to find the posterior \(p(\bm \theta \given \bm y)\) by
According to Bayes' theorem, the posterior is given by
\begin{align}\label{eq:bayes}
    p(\bm \theta \given \bm y) = \frac{p(\bm y \given \bm \theta) p(\bm \theta)}{p(\bm y)}
\end{align}
where \(p(\bm y \given \bm \theta)\) is the likelihood, parameterised by the model parameters \(\bm \theta\), \(p(\bm \theta) = p(\bm \theta; \bm \alpha)\) is the prior, parameterised by its hyper-parameters \(\bm \alpha\), and \(p(\bm y) = p_{\bm \theta}(\bm y)\) is the marginal likelihood (or evidence).

Furthermore, the posterior predictive probability, i.e., the probability of observing some new data, is given by
\begin{align}
    p(\bm y_{\mathrm{new}} \given \bm y) = \int p(\bm y_{\mathrm{new}} \given \bm \theta) p(\bm \theta \given \bm y)\,d\bm\theta.
\end{align}

\subsection{Variational Inference}

In variational inference (VI), we minimise the KL divergence between the posterior distribution \(p(\bm \theta \given \bm y)\) and a variational distribution \(q_{\bm \phi}(\bm \theta)\) parameterised by its variational parameters \(\bm \phi\) by optimising with respect to these parameters.
The KL divergence is defined as
\begin{align}
    \kld{q(\bm \theta)}{p(\bm y \given \bm \theta)} \equiv{} & \E{q}{\log \frac{q(\bm \theta)}{p(\bm y \given \bm \theta)}} \label{eq:kld}  \\
    ={}                                                      & \E{q}{\log q(\bm \theta)} - \E{q}{\log p(\bm y \given \bm \theta)} \nonumber
\end{align}
However, the KL divergence contains the evidence term \(\log p(\bm y)\), which is intractable.
Instead, we define a lower bound on this marginal likelihood term that is surrogate to the KL divergence (known as the ELBO).
Since they are surrogate, ELBO fulfils the property \(\elbo = \log p(\bm y) - \kld{q(\bm \theta)}{p(\bm y \given \bm \theta)}\), minimising the KL divergence is equivalent to maximising the ELBO, which is defined as
\begin{align}
    \elbo \equiv{} & \E{q}{\log p(\bm y, \bm \theta)} - \E{q}{\log q(\bm \theta)} \nonumber                                                                                \\
    ={}            & \E{q}{\log p(\bm y \given \bm \theta)} + \E{q}{\log p(\bm \theta)} - \E{q}{\log q(\bm \theta)} \nonumber                                                                             \\
    ={}            & \E{q}{\log p(\bm y \given \bm \theta)} - \underbrace{- \E{q}{\log p(\bm \theta)}}_{\text{Cross-entropy}} + \underbrace{- \E{q}{\log q(\bm \theta)}}_{\text{Entropy}} \label{eq:elbo}
\end{align}
where the first term \(\E{q}{\log p(\bm y \given \bm \theta)}\) is the data (or likelihood) term, \(-\E{q}{\log p(\bm \theta)}\) is the cross-entropy of the prior with respect to the variational approximation, and \(-\E{q}{\log q(\bm \theta)}\) is the entropy term of the variational approximation.
The last two terms can be interpreted as the regularising KL divergence (or relative entropy) from the prior to the variational approximation, \(\kld{q(\bm \theta)}{p(\bm \theta)} = \E{q}{\log q(\bm \theta)} - \E{q}{\log p(\bm \theta)}\).
Furthermore, the likelihood term in the ELBO can be decomposed further by assuming independence in the observations as \(\E{q}{\log p(\bm y \given \bm \theta)} = \frac{1}{N} \sum^N_i \E{q}{\log p(y_i \given \bm \theta)}\).

Overall BNNs have been found to be relatively ineffective unless the number of observations is greater than the number of model parameters.

\subsection{Methods}\label{ssec:methods}

Multiple approximate inference methods were implemented.
Specifically, VI with mean-field, full-rank, low-rank and radial variational families, \emph{maximum a posteriori} estimation, Laplace approximation, deep ensembles, and MultiSWAG.
These methods will be described in the following sections.

\subsubsection{Mean-Field Variational Approximation}\label{ssec:mfvi}
% Mean-field VI
One common variational family used to approximate the real posterior is a product of independent Gaussian distributions (the mean-field approximation) such that each model parameter is sampled from a normal distribution and is independent of all other model parameters, yielding a Gaussian with a diagonal covariance matrix.
In mean-field VI, model weights are sampled from an approximate posterior \(q(\bm \theta) \approx p(\bm{\theta} \given \bm y)\) as
\begin{align}
    q(\bm \theta) ={} & \normal(\bm \mu, \bm \sigma) \Leftrightarrow \nonumber         \\
    \bm{\theta} ={}   & \bm{\mu} + \bm{\sigma} \circ \bm{\epsilon}\label{eq:meanfield}
\end{align}
where \(\bm{\epsilon} \sim \normal(\bm 0, \identity)\).
For this type of variational approximation, the entropy term is given by \(\entropy{q(\theta)} = - \sum_i \log \sigma_i\) and the cross-entropy of the prior relative to the variational approximation is calculated via Monte Carlo simulation by taking the prior log probability with respect to mean-field posterior samples as
\begin{align}
    \crossentropy{q(\bm\theta)}{p(\bm\theta)} ={} & - \int q(\bm\theta) \log(p(\bm\theta))\,d\bm\theta \nonumber \\
    \approx{}                                     & \frac{1}{S} \sum^S_{s=1} \log p(\bm{\theta}^{(s)})
\end{align}
where \(\bm{\theta}^{(s)} \sim \normal(\bm{\mu}, \bm{\sigma})\).

The use of the mean-field approximation for BNNs has been found to be unreliable~\cite{wu2018deterministic} and, for increasingly wide neural networks, converges towards the prior~\cite{coker2021wide}.
However, for \emph{deep} BNNs, the mean-field assumption may be reasonable~\cite{farquhar2020liberty}.

\subsubsection{Full-Rank Variational Approximation}
% Full-rank VI
As an alternative to the mean-field approximation, a full-rank approximation doesn't assume independence of the model parameters, instead being sampled as
\begin{align}\label{eq:fullrank}
    q(\bm \theta) = \normal(\bm \mu, \bm \Sigma_{\mathrm{FR}}).
\end{align}
For \(D\) model parameters, while a mean-field approximation has scale parameters \(\bm \sigma^D\), the full-rank approximation has scale parameters \(\bm \Sigma_{\mathrm{FR}}^{D \times D}\).
The variational parameters scale quadratically with the number of model parameters, which constitutes a significant limitation of this type of approximation.
Specifically, since NNs are typically overparameterised, possessing from thousands to millions to billions of parameters, the quadratic number of variational parameters makes this type of VI infeasible for BNNs.
As such, full-rank VI is not applied in any of the experiments in this paper.

\subsubsection{Low-Rank Variational Approximation}
% Low-rank VI
As a compromise between the last two methods, a low-rank variational approximation uses a low-rank approximation of the covariance matrix \(\bm \Sigma_{\mathrm{LR}}\).
\begin{align}\label{eq:lowrank}
    q(\bm \theta) = \normal(\bm \mu, \bm \Sigma_{\mathrm{LR}}).
\end{align}
\emph{Need more}

\subsubsection{Radial Variational Approximation}
% Radial BNN
\textcite{farquhar2020radial} introduces the Radial approximate posterior,
For each NN layer, this posterior is sampled similarly to the mean-field (\cref{eq:meanfield}), but by normalising the \(\bm{\epsilon}\) term (projecting it onto a hypersphere) yielding a direction term \(\bm{\epsilon}/\norm{\bm{\epsilon}}\), and scaling it by the distance \(r\).
Therefore, for each layer, the radial posterior is sampled as
\begin{align}\label{eq:radial}
    q(\bm \theta) ={} & \mathrm{Radial}(\bm{\mu}, \bm{\sigma}) \nonumber                          \\
    \bm{\theta} ={}   & \bm{\mu} + \bm{\sigma} \circ \frac{\bm{\epsilon}}{\norm{\bm{\epsilon}}} r
\end{align}
where \(\bm{\epsilon} \sim \normal(\bm{0}, \identity)\), \(r \sim \normal(0, 1)\).
The entropy term of the Radial approximate posterior is given
\(\entropy{q(\theta)} = - \sum_i \log \sigma_i + \mathrm{const}\) (and is therefore approximately equal to the entropy of the mean-field approximation up to a constant) and the cross-entropy term is calculated via Monte Carlo simulation as for mean-field VI (\cref{ssec:mfvi}), but by sampling from a radial posterior \(\bm{\theta}^{(s)} \sim \mathrm{Radial}(\bm{\mu}, \bm{\sigma})\) as in \cref{eq:radial}.

\subsubsection{Maximum a Posteriori Estimation}
% MAP
The \emph{maximum a posteriori} solution finds a point estimate of the posterior given by its maximum.
\begin{align}
    \bm \theta_\mathrm{MAP}
    ={} & \max_{\bm \theta} p(\bm \theta \given \bm y) \nonumber                                           \\
    ={} & \max_{\bm \theta} \log p(\bm \theta \given \bm y) \nonumber                                      \\
    ={} & \max_{\bm \theta} \left[ \log p(\bm y \given \bm \theta) + \log p(\bm \theta) \right] \nonumber  \\
    ={} & \max_{\bm \theta} \left[ \sum_{i=1}^N \log p(y_i \given \bm \theta) + \log p(\bm \theta) \right]
\end{align}
\emph{Need more}

\subsubsection{Laplace Approximation}
% Laplace
In the Laplace approximation (LA)~\cite{daxberger2021laplace}, the posterior is approximated by a Gaussian, similarly to mean-field VI.
However, instead of finding the optimal Gaussian distribution locations and scales by maximising the ELBO, the LA finds the location by computing the MAP solution \(\bm \theta_{\text{MAP}}\), and the scale by performing a Taylor expansion around this solution and determining the curvature via its Hessian matrix.
\begin{align}
    p(\bm{\theta} \given \bm y) \approx \normal(\bm \theta_{\text{MAP}}, \bm\Sigma)
\end{align}
where \(\bm\Sigma = -(\left.\nabla^2_{\bm \theta} \mathcal{L}(\bm y; \bm \theta) \right|_{\bm \theta_\mathrm{MAP}})^{-1}\).

\emph{Need more}

\subsubsection{Deep Ensembles}
% Deep ensembles
In deep ensembles~\cite{lakshminarayanan2017simple}, \(M\) neural networks are trained with different initialisations.
To make predictions with a deep ensemble,
\begin{align}
    p(\bm y_{\mathrm{new}} \given \bm y)
    ={} & \int p(\bm y_{\mathrm{new}} \given \bm \theta) p(\bm \theta \given \bm y)\,d\bm\theta \nonumber \\
    ={} & \frac{1}{M} \sum_{m=0}^M p(\bm y_{\mathrm{new}} \given \bm\theta^{(m)})
\end{align}
where \(\bm\theta^{(m)}\) are the model parameters for ensemble \(m\), and \(p(\bm y_{\mathrm{new}} \given \bm\theta^{(m)})\) are therefore the normalised logits, or predicted probabilities, from the model.

\emph{Need more}

\subsubsection{MultiSWAG}

% SWA
Stochastic weight averaging estimates the final point estimate of the weights of a neural network \(\bm \theta_\mathrm{SWA}\) as the average of the model parameters \(\bm \theta\) at the end of each epoch obtained via gradient descent after convergence has been reached.

% SWAG
For SWAG~\cite{maddox2019simple}, the standard deviation of the model parameters \(\bm \sigma_\mathrm{SWA}\) is also calculated.
From the SWA estimates of the model parameter mean and standard deviations, the posterior is then estimated as
\begin{align}\label{eq:swag}
    p(\bm \theta \given \bm y) \approx{} & \normal(\bm \theta_\mathrm{SWA}, \bm \sigma_\mathrm{SWA}).
\end{align}

% MultiSWAG
For MultiSWAG~\cite{wilson2020bayesian}, \(M\) SWAG estimates are obtained from different initialisations, as in deep ensembles.
Then, a Gaussian mixture model (GMM) is formed as a combination of each Gaussian SWAG posterior estimate \(\normal(\bm \theta_\mathrm{SWA}^{(m)}, \bm \sigma_\mathrm{SWA}^{(m)})\) where each component is equally weighed, as
\begin{align}\label{eq:multiswag}
    p(\bm \theta \given \bm y) \approx{} & \sum_{m=1}^M \normal(\bm \theta_\mathrm{SWA}^{(m)}, \bm \sigma_\mathrm{SWA}^{(m)}).
\end{align}
\emph{Need more}

% \subsection{Evaluation}\label{ssec:eval}

% % Marginal likelihood maximization
% % a.k.a.
% % Empirical Bayes
% % a.k.a.
% % The evidence framework
% \cite{mackay1992evidence, bernardo2009bayesian, bishop2006pattern_evidence}

\subsection{Continual Learning and Active Learning}

% Introduction
In continual learning, data is collected and used to train the model continually, with the inferred posterior \(p(\bm \theta \given \bm y)\) (or its approximation) being used as the prior to the new posterior as in \cref{eq:bayes}.

% BNNs in continual learning
\cite{nguyen2017variational}

% VI for continual learning
In continual learning using VI, the computation of the ELBO (\cref{eq:elbo}) requires the entropy of the variational approximation and the cross-entropy from the prior to the variational approximation.
The entropy term is calculated in the same way for continual learning, however the cross-entropy term requires determining the cross-entropy between the variational approximation and itself (with different sets of parameters).

% Active learning introduction
Active learning is...

For active learning

The \emph{Max Entropy} acquisition function~\cite{gal2017deep,shannon1948mathematical} for selecting images maximises the entropy of the posterior predictive
\begin{align}
    \entropy{p(\bm y \given \bm y_{\text{train}})} = -\sum_c p(y = c \given \bm y_{\text{train}}) \log p(y = c \given \bm y_{\text{train}})
\end{align}

The \emph{BALD} acquisition function maximises information gain
\begin{align}
    \mathbb{I} = \entropy{p(\bm y \given \bm y_{\text{train}})} - \E{p(\bm \theta \given \bm y_{\text{train}})}{\entropy{p(\bm y \given \bm \theta)}}
\end{align}

\section{Experiments}

\cite{paszke2019pytorch}
Pyro is a probabilistic programming language (PPL) built upon Pytorch~\cite{bingham2018pyro}.
Tyxe is a BNN library built upon Pyro~\cite{ritter2021tyxe}.

\subsection{MNIST and SVHN}\label{ssec:exp-mnist}

% Method
MNIST~\cite{lecun1998gradient} and CIFAR~\cite{krizhevsky2009learning}

The methods described in \cref{ssec:methods} were trained on

To verify the confidence of the model on out-of-distribution (OOD) data, a network was trained on MNIST and tested on CIFAR-10.
The NN had two fully-connected hidden layers with 64 neurons each, a ReLU non-linearity and a dropout layer after each.

The 95\% uncertainty interval of calibration plots was calculated as per the binomial proportion confidence interval, i.e., \(\pm 2 \sqrt{p (1-p) / n}\), where \(p\) is the true probability of a bin and \(n\) is the number of predictions in a bin.

% Results
It can be seen that the radial approximation is more confident on MNIST than the Laplace and mean-field approximations (\cref{fig:cross-calibration}).
However, since MNIST is a relatively easy problem, all of the algorithms have an accuracy above 90\%.
Since most algorithms are overconfident, this leads to most predictions being made with very high confidence, and thus the overconfident algorithms have low ECE.
Furthermore, all algorithms perform similarly on out-of-distribution (OOD) data, since all are extremely overconfident in their predictions on the SVHN dataset.
This may suggest that the problem is with the model being used and not the inference algorithm.
%
\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/MnistVI.pdf}
    \caption{Caption}
    \label{fig:cross-calibration}
\end{figure}
\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/MnistEnsembles.pdf}
    \caption{Caption}
    \label{fig:cross-calibration-ensemble}
\end{figure}
\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/MnistMultiSwag.pdf}
    \caption{Caption}
    \label{fig:cross-calibration-multiswag}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/ensemble_comparison.pdf}
    \caption{Ensembles with between one and twenty maximum likelihood NNs, trained on MNIST. We find that two ensembles have an increase of 0.8\% accuracy relative to a single neural network.}
    \label{fig:ensembles}
\end{figure}

% Discussion

\subsection{MURA}\label{ssec:mura}

% Method
\textcite{rajpurkar2017mura} introduced MURA, a musculoskeletal radiograph X-ray dataset for computer vision.
They trained a DenseNet with 169 layers~\cite{huang2017densely}
Due to computational constraints, we trained with the same dense neural network as in \cref{ssec:exp-mnist}.

% Results
%
\begin{figure}
    \centering
    \includegraphics[width=0.55\linewidth]{figures/MuraVI.pdf}
    \caption{Caption}
    \label{fig:mura-cc-vi}
\end{figure}
\begin{figure}
    \centering
    \includegraphics[width=0.55\linewidth]{figures/MuraEnsembles.pdf}
    \caption{Caption}
    \label{fig:mura-cc-ensembles}
\end{figure}
\begin{figure}
    \centering
    \includegraphics[width=0.55\linewidth]{figures/MuraMultiSwag.pdf}
    \caption{Caption}
    \label{fig:mura-cc-multiswag}
\end{figure}

% Discussion

\subsection{Active learning}

\textcite{foong2020expressiveness} found that, for shallow mean-field VI BNNs, active learning doesn't show improvement over random acquisition.
This may be because VI for BNNs typically converge to a reasonable approximation of the mean faster than they approximate the variance.
This means that, since we start the active learning process on an untrained BNN, the initial iterations quantify uncertainty poorly and therefore their predictive entropy has little value.

\begin{figure*}
    \centering
    \includegraphics[width=\textwidth]{figures/active_comparison_mnist.pdf}
    \caption{Trained on MNIST.}
    \label{fig:active=mnist}
\end{figure*}

\begin{figure*}
    \centering
    \includegraphics[width=\textwidth]{figures/active_comparison_mura.pdf}
    \caption{Trained on MURA.}
    \label{fig:active-mura}
\end{figure*}

\section{Conclusion}

% Summary and key results

% Outlook and future developments
\textbf{Ideas for experiments}
\begin{enumerate}
    \item Is it better to train an Nx bigger neural network, or to ensemble N neural networks? Mind you, deep ensembles have the advantage of easy parallelization.
    \item See what happens with a mean field when you increase width or depth
    \item See what happens with VI if you increase the size of the model, over-parameterise
\end{enumerate}

\printbibliography

\clearpage
\onecolumn
\appendix
\section{Appendix}
\begin{table}[h]
    \centering
    \input{tables/mnist_full}
    \caption{Trained on MNIST.}
    \label{tab:mnist}
\end{table}
\begin{table}[h]
    \centering
    \input{tables/mura_full}
    \caption{Trained on MURA.}
    \label{tab:mura}
\end{table}

\end{document}


\documentclass[a4pper,11pt]{article}
\usepackage{lipsum} %This package just generates Lorem Ipsum filler text.
\usepackage{graphicx}
\usepackage{natbib}
\usepackage{geometry}
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage[linesnumbered,ruled,vlined]{algorithm2e}
\usepackage[noend]{algpseudocode}
\usepackage[usenames]{color}
\usepackage{amsmath,amsthm,amssymb,amsfonts}
\usepackage{dsfont}
\usepackage{mathrsfs}
\usepackage{tabu}
\usepackage{multirow}
\usepackage{enumerate}
\usepackage{listings}
\usepackage{subfigure}
\usepackage{float}
\usepackage{authblk}
\newtheorem{thm}{Theorem}[section]
\newtheorem{defn}{Definition}[section]
\newtheorem{prop}[thm]{Proposition}
\title{Notes on Stochastic Programming}

\begin{document}

\author[1]{\small Chuwen Zhang}
\author[1]{\small Jingyuan Yang}
\affil[1]{\footnotesize Stochastic Programming Reading Group}
\maketitle

\section{Mathematical Background}
\subsection{Functions}
\begin{defn}
    An extended real valued function $f: \mathbb R^n \to \bar {\mathbb R}$ is \textit{proper} if $f(x)>-\infty$ for all $x\in \mathbb R^n$ and its domain, dom $f$, is nonempty.
\end{defn}
\begin{defn}
    An extended real valued function $f: \mathbb R^n \to \bar {\mathbb R}$ is lower semicontinuous if at every point $x_0\in \mathbb R^n$, $f(x_0)\leq \lim \inf_{x\to x_0} f(x)$.
\end{defn}
$f$ is lower semicontinuous iff its epigraph is a closed subset of $\mathbb R^{n+1}$.
\begin{defn}
    An extended real valued function $f: \mathbb R^n \to \bar {\mathbb R}$ is polyhedral if it is proper convex and lower semicontinuous, its domain is a convex closed polyhedron, and $f(\cdot)$ is piecewise linear on its domain.
\end{defn}
\begin{defn}
    We refer $\mathscr G$ which is a mapping from $\Omega$ into the set of subsets of $\mathbb R^n$ as a multifunction.
\end{defn}
\begin{defn}
    The function $F(x,\omega)$ is random lower semicontinuous if the associated epigraphical multifunction $\omega \mapsto \text{epi} \ F(\cdot,\omega)$ is closed valued and measurable.
\end{defn}
\begin{thm}
    \label{thm-1}
    Let $F: \mathbb R^n \times \Omega \to \bar {\mathbb R}$ be a random lower semicontinuous function. Then the optimal value function $v(\omega)$ and the optimal solution multifunction $X^*(\omega)$ are both measurable.
\end{thm}
\section{Linear Two-Stage Problems}
\subsection{Basic Properties}
We discuss two-stage stochastic linear programming problems of the form
\begin{equation}
    \label{first}
    \begin{array}{ll}
        \min_{x}    & c^Tx+\mathbb E[Q(x,\xi)] \\
        \text{s.t.} & Ax=b, x\geq 0,
    \end{array}
\end{equation}
where $Q(x,\xi)$ is the optimal value of the second-stage problem
\begin{equation}
    \begin{array}{ll}
        \min_y      & q^Ty               \\
        \text{s.t.} & Tx+Wy=hy, y\geq 0.
        \label{second-stage}
    \end{array}
\end{equation}
The dual problem of problem \eqref{second-stage} can be written in the form
\begin{equation}
    \begin{array}{ll}
        \max_\pi    & \pi^T(h-Tx)    \\
        \text{s.t.} & W^T\pi \leq q.
        \label{dual}
    \end{array}
\end{equation}
We define
$$
    s_q(\chi)\triangleq \inf\{q^Ty: Wy=\chi,y\geq 0\}.
$$
Clearly, $Q(x,\xi)=s_q(h-Tx)$. In addition, by duality theory,
$$
    s_q(\chi)=\sup_{\pi\in \Pi(q)}\pi^T\chi,
$$
where $\Pi(q)\triangleq\{\pi:W^T\pi\leq q\}$.
\begin{prop} For any given $\xi$, $Q(\cdot,\xi)$ is convex. Moreover, if $\{\pi:W^T\pi\leq q\}$ is nonempty and problem \eqref{second-stage} is feasible for at least one $x$, then the function $Q(\cdot,\xi)$ is polyhedral.
    \label{prop-1}
\end{prop}
\begin{prop}
    Suppose that for given $x=x_0$ and $\xi\in \Xi$, the value $Q(x_0,\xi)$ is finite. Then $Q(\cdot,\xi)$ is subdifferentiable at $x_0$ and
    $$
        \partial Q(x_0,\xi)=-T^T\mathfrak D(x_0,\xi),
    $$
    where $\mathfrak D(x_0,\xi)\triangleq \text{arg}\max_{\pi\in \Pi(q)}\pi^T(h-Tx)$ is the set of optimal solutions of the dual problem \eqref{dual}.
\end{prop}
\subsubsection{The domain of $s_q(\chi)$}
The \textit {positive hull} of a matrix $W$ is defined as
$$\text{pos} \ W\triangleq \{\chi:\chi=Wy, y\geq0\}.$$
The recession cone of $\Pi(q)$ is equal to
$$\Pi_0\triangleq \{\pi:W^T\pi\leq 0\}.$$
Denote the polar cone to $\Pi_0$ as $\Pi_0^*$.
We have
$$\text{dom} \ s_q=\text{pos}\ W=\Pi_0^*.$$

\subsection{The Expected Recourse Cost}
$$\phi(x)\triangleq \mathbb E[Q(x,\xi)].$$
With discrete distribution of $\xi$, if for at least one scenario, the corresponding second-stage problem is infeasible, $\phi(x)$ is $+\infty$.

To ensure that $\phi(x)$ is well defined, we have to verify two conditions:
\begin{enumerate}
    \item $Q(x,\cdot)$ is measurable [followed by Theorem \ref{thm-1}];
    \item either $\mathbb E[Q(x,\xi)_+]$ or $\mathbb E[(-Q(x,\xi))_+]$  is finite.
\end{enumerate}
\begin{prop}
    Suppose that the recourse is fixed and
    $$\mathbb E[||q||||h||]<+\infty \ \text{and} \ \mathbb E[||q||||T||]<+\infty.$$
    Consider a point $x\in \mathbb R^n$. Then $\mathbb E[Q(x,\xi)_+]$ is finite iff $h-Tx\in \text{pos}\ W$ holds w.p. 1.
\end{prop}
\paragraph{Proof.} By Hoffman's lemma.\\

Moreover, if the recourse is complete, $\phi(\cdot)$ is well deﬁned and is less than $+\infty$. Since the function $\phi(\cdot)$ is convex, we have that if $\phi(\cdot)$ is ﬁnite valued in at least one point, then $\phi(\cdot)$ is ﬁnite valued on the entire space $\mathbb R^n$.
\begin{prop}
    Suppose that the probability distribution of $\xi$ has finite support $\Xi=\{\xi_1,\dots, \xi_K\}$ and $\phi(\cdot)$ has a finite value in at least one point $\bar \xi\in \mathbb R^n$. Then the function $\phi(\cdot)$ is polyhedral, and for any $\xi_0\in \text{dom} \ \phi$,
    $$\partial \phi(x_0)=\sum_{k}p_k\partial Q(x_0,\xi_K).$$
\end{prop}
\paragraph{Proof.} By Proposition \ref{prop-1}.
\paragraph{\textit{Remark.}} $\phi$ is differentiable at $x_0$ iff for every $\xi_k$, the corresponding second-stage dual problem has a unique optimal solution.

\begin{prop}
    Suppose that the expectation function $\phi(\cdot)$ is proper and its domain has a nonempty interior. Then for any $x_0\in \text{dom}\ phi,$,
    $$\partial \phi(x_0)=-\mathbb E[T^T\mathfrak D(x_0,\xi)]+\mathcal N_{\text{dom}\ \phi} (x_0),$$
    where
    $$\mathfrak D(x_0,\xi) \triangleq \text{arg}\max_{\pi\in \Pi(q)}\pi^T(h-Tx).$$
    Moreover, $\phi$ is differentiable at $x_0$ iff $x_0$ belongs to the interior of $\text{dom} \ \phi$ and the set $\mathfrak D(x_0,\xi)$ is a singleton w.p. 1.
\end{prop}
\begin{prop}
    Suppose that (i) the recourse is ﬁxed, (ii)for a.e. $q$ the set $\Pi(q)$ is nonempty, (iii) condition $\mathbb E[||q||||h||]<+\infty \ \text{and} \ \mathbb E[||q||||T||]<+\infty$ holds, (iv) the conditional distribution of $h$, given $(T,q)$ is absolutely continuous for almost all $(T,q)$. Then $\phi$ is continuously differentiable on the interior of its domain.
\end{prop}
In the case of a continuous distribution of $\xi$, the expectation operator smoothes the piecewise linear function $Q(\cdot, \xi)$.

\begin{thm}
    Let $\bar x$ be a feasible solution of problem \eqref{first}. Then $\bar x$ is an optimal solution iff there exist $\pi_k \in\mathfrak D(\bar x,\xi_k), k=1,\dots, K$, and $\mu\in \mathbb R^m$ such that
    \begin{equation}
        \begin{aligned}
            \sum_{k} p_kT_k^T\pi_k+A^T\mu\leq c
            \\
            \bar x^T(c-\sum_{k} p_kT_k^T\pi_k-A^T\mu)=0.
        \end{aligned}
    \end{equation}
\end{thm}

If we deal with general distributions of the problem’s data, additional conditions are needed to ensure the subdifferentiability of the expected recourse cost and the existence of Lagrange multipliers.
\begin{thm}
    Let $\bar x$ be a feasible solution of problem \eqref{first}. Suppose that the expected recourse cost function $\phi(\cdot)$ is proper, $\text{int(dom}\ \phi) \cap X$ is nonempty, and $\mathcal N_{\text{dom} \ \phi}  (\bar x) \subset \mathcal N_X (\bar x)$. Then $\bar x$ is an optimal solution iff there exist a measurable function $\pi(\omega)\in \mathfrak (x,\xi(\omega))$ , and a vector $\mu\in \mathbb R^m$ such that
    \begin{equation}
        \begin{aligned}
            \mathbb E[T^T\pi]+A^T\mu\leq c
            \\
            \bar x^T(c-\mathbb E[T^T\pi]-A^T\mu)=0.
        \end{aligned}
    \end{equation}
\end{thm}

The assumptions can be substituted by i) the recourse is ﬁxed, (ii)for a.e. $q$ the set $\Pi(q)$ is nonempty, (iii) condition $\mathbb E[||q||||h||]<+\infty \ \text{and} \ \mathbb E[||q||||T||]<+\infty$ holds and $T$ is deterministic.
\end{document}


\section{Anomalies}
Last time we mentioned the types of TQFTs.
There's one more TQFT that you might encounter.
There's something called an extended TQFT.\@
When I define TQFT,
I assign a path integral to a $(d+1)$-dimensional manifold
and we have a path integral
$Z\left( M^{d+1} \right)\in \mathbb{C}$.

For a closed $d$-manifold $V(\Sigma^d)$.

For a closed $(d-1)$-manifold,
we can associate a $1$-category.
This is called once-extended.
The objects get more and more abstract as you go down.

And then for $(d - 2)$-manifold,
you can attach a 2-category.
For a $(d-3)$-manifold,
you attach a 3-category.

I'm not going to define these.
And then for a 0-manifold,
you get a $d$-category.

Useful to note that if you have a fully extended TQFT,
all the way down to $d$-category,
this is going to come up for us,
we won't necessarily use it,
but we will encounter them often.
Fully extended TQFTs are situations where you can write down exactly solvable
Hamiltonians for your topological phases as sums of commuting projectors.
And then the path integral can be written as an exact combinatorial state sum.

We'll encounter examples eventually.
To define the path integral,
triangular the manifold
define simplexes 1 2 3,
assign labellings,
and them sum over all labellings.
That's an exact combinatorial space.
They're useful to work with and have a lot to say about them.
Physicists don't have a good understanding of what these higher structures mean
physically,
so it's an interesting question.

\section{Invertible vs non-invertible TQFT}
The next distinction is between invertible and non-invertible TQFTs.

\begin{definition}
    A TQFT is \emph{invertible} if the magnitude of the path integral has unit
    magnitude
    \begin{align}
        |Z(M^{d+1})| = 1
    \end{align}
    on any closed manifold.
\end{definition}
In particular, if you look at $Z(\Sigma^d\times S^1) = 1
= |\dim V(\Sigma^d)|$.
This is interpreted as the dimension of the Hilbert space on this space
Physically,
this means that there is always a unique state on any closed $\Sigma^d$.
A topological state of matter described by an invertible TQFT always has a
unique ground state on any closed manifold.
Not always the case,
depending on the topology of the space,
and those will be described by non-invertible TQFTs.

The reason it's called invertible,
is that if the path integral is a phase,
there's always a well-defined inverse of the phase,
which is the complex-conjugate of the phase.

Let's say if you have a TQFT,
then the ``inverse'' TQFT has a path integral like
\begin{align}
    Z_{T^{-1}}\left( M^{d+1} \right)
    =
    Z_T^*\left( M^{d+1} \right)
\end{align}
Invertible TQFTs come up a lot.
They describe integer quantum hall states.
Topological insulators and superconductors.
All $(1+1)$-dimensional topological phases of matter are describable by
invertible TQFTs.
Any TQFT that is not invertible is a \emph{non-invertible} TQFT.\@

Actually, one more thing before I get to that.
Invertible TQFTs form an Abelian group.
Suppose that I have the path integral for two theories $T_1$ and $T_2$,
and I want the path integral for $T_1\times T_2$,
then I just multiply their path integrals.
\begin{align}
    Z_{T_1T_2}\left( M^{d+1} \right)
    =
    Z_{T_1}\left( M^{d+1} \right)
    Z_{T_2}\left( M^{d+1} \right)
\end{align}

\begin{question}
    Does the partition function define the TQFT uniquely?
\end{question}
Well $V(\Sigma^d)$ is always going to be a one-dimensional vector space.
Even if it has some boundaries,
it's just going to be a number,
a multiple of that state,
so just an amplitude.
We can extend that definition to the full TQFT.
I'm specifically talking about the invertible ones.

Even in $M^{d+1}$ is open,
the state on the boundary is one-dimensional,
I can just take the inverse theory to give me the complex conjugate.

If a theory is not invertible,
it is a non-invertible TQFT,
and it will have a non-trivial degeneracy on different manifolds.
One thing that is true,
but I'm not sure if there is a proof,
I doubt there's a counterexample,
but I think that this always goes the other way.
There's also an arrow going the other way.
If the path integral magnitude is 1,
then every path integral on the manifold is 1.
I've seen the proof for once-extended,
but I'm not sure if it's general.

So a non-invertible TQFT should have
$Z|\left( \Sigma^d\times S^1 \right)\ne 1|$
for some $\Sigma^d$.
Using ``should'' loosely here.

Let me give you a quick example of an invertible TQFT.\@
This is actually going to be an interesting example,
because it's going to show you a TQFT that is non-trivial,
but it's going to describe a trivial phase of matter.
Imagine we take our path integral on a two-manifold
and have some $U(1)$ gauge field $A$ with field strength $F$.
Then
\begin{align}
    Z\left( M^d, A \right)
    =
    e^{-\theta \int_{M^2} F}
    e^{i\phi \chi\left( M^d \right)}
\end{align}
where
\begin{align}
    \chi\left( M^d \right) = \frac{1}{2\pi}\int_{M^2}R
\end{align}
is the Euler characteristic.

Phases of matter are 1 to 1 correspondence with deformation classes of TQFTs.
In the absence of pinned phases of $\phi$,
this is an example of an invertible TQFT.\@

The first most important question studying a new phase of matter is to ask if
it's describable by an invertible TQFT or a non-invertible TQFT.\@

\section{'t Hooft anomaly}
One big concept about TQFTs I want to mention are called
\emph{'t Hooft anomaly}.
This is something that applies a general QFT,
not just TQFT.
We can consider a QFT which has a symmetry group $G$.
This is the global symmetry group of the QFT.
This may be completely well-defined in the absence of any background gauge field
associated with $G$i.
Then you try to turn on a background gauge field and see something is wrong
with the theory.
Consider your path integral $Z(M^{d+1})$ and turn on $A$,
and you find that the path integral is not gauge-invariant.
\begin{align}
    Z\left( M^{d+1}, A + d\lambda \right)
    \ne
    Z\left( M^{d+1}, A \right)
\end{align}
when $A\ne 0$.
It's not the end of the world.
You might be able to redefine the path integral and make it gauge invariant.
You may be able to define a new gauge-invariant path integral
\begin{align}
    \tilde{Z}\left( M^{d+1}, A \right)
    = Z\left( M^{d+1}, A \right)
    e^{i\int_{M^{d+1}} f(A)}
\end{align}
where the $e^{i\int_{M^{d+1}} f(A)}$
is called a local counter-term that cures the gauge invariance.

\begin{question}
    You mean the action of $G$?
\end{question}
You look at all the correlation functions and you're applying $G$ to all the
operators.
The ground state of the QFT is invariant under $G$.

\begin{question}
    When you couple the background gauge field,
    should we be doing minimal coupling?
\end{question}
It depends how you define this quantity.
Suppose you have some action that is a single Dirac fermion with a mass.
You integrate out the mass
and find a Chern-Simons term with some action.
You wouldn't have written this term down anyway if you were experienced.
But you could actually just add another local functional to this,
and now the coefficients add up to 1 and they are gauge invariant.
If you're not following this discussion it's fine.
I haven't introduced Chern-Simons theory yet.
But you're right.
Usually, you would have defined your path integral in the first place.

\begin{question}
    Is it an anomaly in global symmetry?
\end{question}
Yes. This is the only anomaly worth having a name.
Gauge anomaly means you have a 't Hooft anomaly you try to gauge.
You may not always be able to add counter terms.
I haven't defined 't Hooft anomaly yet.

My point is that a theory with a 't Hooft anomaly is a real theory.
A theory with a gauge anomaly is a theory that doesn't make sense in the first
place.

There's something called a framing anomaly.
There's gravitational anomalies.
Anomaly is an overloaded term in physics,
used everywhere for different things.

There's no such thing as a gauge theory in QM.
It's a classical redundancy in your description.
A theory can have a global symmetry that is anomalies.
That is a 't Hooft anomaly.

You write down a Lagrangian with classical gauge symmetry.
You quantize it and find the symmetry does not hold when quantized,
and that's an anomaly.
There's a whole story.

There could be situations where you can't cancel this gauge apology by adding
counter terms.
You can cancel it not with a local counter term,
but thinking your theory as the surface of a higher dimensional invertible
TQFT.

So,
we can have a situation where the gauge non-invariance is cancelled by a
$(d+2)$-dimensional invertible TQFT.
An invertible TQFT that exists in a 1-higher dimension.
You can think of your system as living on the $(d+1)$-dimension surface of a
$(d+2)$-dimensional bulk and this bulk is an invertible TQFT.
And then you can have some bulk theory
$Z_{\text{bulk}}\left( W^{d+2}, A \right)|_{\text{boundary cond.}}
= Z\left( \partial W^{d+2}, A|_{\partial W} \right)$
perfectly well-defined,
but if you evaluate with some boundary condition.
And the gauge transformation would be governed by the higher-dimensional theory.
So
\begin{align}
    Z\left( M^{d+1}, A + d\lambda \right)
    Z\left( M^{d+1}, A \right)
    = Z_{\text{bulk}}\left( M^{d+1}\times I, \tilde{A} \right)
\end{align}
So the amplitude of going from $A$ to $A + d\lambda$,
there's an extra phase that comes from the invertible TQFT in the bulk,
which cancels out the gauge non-invariance here,
but cancelling in a way that cannot be done by just a local counter-term.

So then 't Hooft anomalies can be classified by TQFTs.

\begin{question}
    Why only just one higher dimension?
\end{question}
If you have to go to 2 higher dimensions,
you can always compactify one of the dimensions
and then you're back to $+1$ extra dimensions.

Let me just explicitly write it out

A QFT has a 't Hooft anomaly if
\begin{enumerate}
    \item It has a global symmetry $G$
    \item The path integral $Z$ is not gauge invariant under gauge
        transformaiton $A\to A + d\lambda$
    \item Gauge non-invariance cannot be cancelled by a local counter-term,
        but it can be cancelled by a $(d+1)$-dimensional TQFT.
\end{enumerate}
In topologcial pahses of matter,
some are described at low energy witha 't Hooft anolmaly
and what that tells us is that it cannot be realised ni $(d+1)$ spacetime
dimensions,
but has to be the surface of a higher dimensional theory.
In condensed matter,
a theory witha 't Hooft anomaly can only be raelised microscopically
and have the symmetry act on-site,
it arises as the $(d+1)$-dimensional surface of a $(d+2)$-dimensional
topological phase.

On-site means the symmetry means a tensor product of on-site symmetry opreators
separately.
That is,
$g=\bigotimes_i g^{(i)}$.

The symmetry group dictates that the gauge field is a connetion on a principle
$G$-bundle,
so it's a gague field of global symmetry group $G$,
so it controls what gauge transfomatinos you have.
What I'm saying is
your therory may be defined for correlation functions without $A$,
but when you turn on the $A$ and things go wrong.
Let me just mention a few examples of 't Hooft anomalies in case you may have
seen examples before.

If you haven't seem examples before,
these examples will come up in the course.

Famous example.
You can have a $(1+1)$-dimensional chiral fermion
with $U(1)$ symmetry.
It can only exist in the boundary of an integer quantum Hall state,
which is a TQFT.

Another example.
You can have a single $(2+1)$-dimensional massless Dirac fermion
with $U(1)$ and time-reversal symmetry.
But this suffers from the famous parity anomaly.
So htis can only exist at the surface of a $(3+1)$-dimensional topological
insulator.

In particular physics,
there's a famous axial anomaly,
which is an example of a 't Hooft anomaly.

These are the most famous ones.

\begin{question}
    Are massless Dirac fermions topological?
\end{question}
No,
I'm just saying the ntion of a 't Hooft anomaly applies to QFTs in general.
These examples are not topological.

\begin{question}
    Gauge-gravity duality?
\end{question}
It's related in an either loose way or such a profound way that I don't have an
aswer.
Definitely related,
but it's hard to yeah.
It's related,
but it's one of things things,
as time goes on
we're seeing more and more hints.
At the surface it seems pretty good.

\begin{question}
    Is there some version for CFTs of diffe.rent dimensions?
\end{question}
A CFT can certainly have a 't Hooft anomaly.
They must live at the surface of a higher-dimensional TQFT.
But a higher-dimensiona CFT doesn't really make sense,
because the boundary theory only makes snese
if there are no additional degrees of freedom in the bulk.
If the bulk has degrees of freedom,
there can be leaking of these.
Here the boundary has degrees of freemdom that cannot leak into the bulk.

\begin{question}
    Why should the bulk by invertible?
\end{question}
There's two answers.
People do talk abbout non-invertible anomalies.
There are examples,
but it suffers from the thing I was describing.
If your bulk is non-inverible,
it means there are non-trivial degrees of freedom that are also non-trivial i
hte bulk,
they can leak out and there is no clean separation between the bluk and the
boundary.

An integer quantum Hall state is well-defined,
but the boundary mode cannot exist on its own.

We can open up the bulk and have it open up the space and there is a boundary
theory there.
.
\begin{question}
    So you have abulk theorey
    You restrict it to the boundary,
    you evaluate it on hte boundary?
\end{question}
It's better to htink I have a bublk theory,
and if the manifold has a boundary,
then I will specify some boundary conditions.

\begin{question}
    When you specify the boundary condition,
    what do you mean exactly?
\end{question}
It's only when I specify boundary conditions that I can get a number $Z$.

What is the bulk?
$Z_{\text{bulk}}\left( W^{d+2} \right)\in V(\partial W^{d+2})$
is just a state,
which I take an inner product with something to specify the boundary condition.

It might be more useful that we go to examples later on.
This discussion is a bit abstract.
Different theories may correspond to different boundary conditions.
In all cases, the boundary theory will have a 't Hooft anomaly.

\begin{question}
    TFQT exist on the boundary of a therory in the sense you want to put it on a
    lattice?
    If we don't put it on a lattice and don't couple it to a gauge field?
\end{question}
Call that a macroscopic realisation.

A single fermion is not emergable from a one-dimensional system.

\begin{question}
    The Standard model is not emergable?
\end{question}
If a theory has a 't Hooft anomaly,
if you're willing to give up the symmetry,
you can realise it microscopically.
If you dedmanded you wanted to preserve axial symmetry,
you have to go to the surface of a one-higher dimensino.

In the standard mdoel,
there is actually 2 anomoalies.
There's a gauge $U(1)$ anomaly for chiral fermions,
and there's a gravitaitonal anomaly,
that you cannot give up.
It's tied to energy enregy conservation.
I don't even know what that means,
because then you have a time-dependent Hamiltonian.

I think we're getting ahead of ourselves with this example.

\begin{question}
    In a 1D condensed matter system,
    in a Luttinger liquid for example?
\end{question}
That anomaly you can think of as an axial anomaly in 1+1 dimensinos.
The theory emerges out ofa sitautiaon where it doesn't exist microscopically.

If there are no more questions about TQFTs,
we're going to talk about actual topological quantum phases of matter.

\section{1D topological superconductor}
This is describable by an invertible spin TQFT.

We're going to start off with a $(1+1)$-dimensional
Majorana-Kitaev chain.
Start with a simple toy model that is the 1d spineless $p$-wave superconductor.
That's going to be described by the following.
Imagine you have 1D array of $N$ sites.
Write the free fermion Hamiltonian,
which you can think of as a free field for the semiconductor.

\begin{align}
    H &=
    \sum_{i=1}^{N}
    \left[
    -t\left( c_i^\dagger c_{i+1} + c_{i+1}^\dagger c_i\right)
    - \mu\left( c_i^\dagger c_i - \frac{1}{2} \right)
    + \Delta c_i c_{i+1}
    + \Delta^* \chi_{i+1}^\dagger c_i
    \right]
\end{align}
and consider periodic boundary conditions $c_{i+N}=c_i$.

Consider momentum space.
\begin{align}
    \tilde{c}_k &=
    \frac{1}{\sqrt{N}}
    \sum_{j=1}^{N}
    c_j e^{ik_j}
\end{align}
where $k = 2\pi n/N$,
$n=0,\ldots,N$.
The Brillouin zone is $[0,2\pi] \simeq [-\pi, \pi]$.
$\Delta$ is just any complex number here.

Define this spinner with some redundancy.
\begin{align}
    \Phi_k =
    \begin{pmatrix}
        c_k\\
        c_{-k}^\dagger
    \end{pmatrix}
\end{align}
then the Hamiltonian is
\begin{align}
    H &=
    \frac{1}{2}\sum_{k\in BZ}
    \Psi_k^\dagger \mathcal{H}_k \Psi_k
\end{align}
So then the Bloch Hamiltonian is
\begin{align}
    \mathcal{H}_k =
    \begin{pmatrix}
        \mathcal{E}_k & \Delta_k^*\\
        \Delta_k & -\mathcal{E}_k
    \end{pmatrix}
\end{align}
where
\begin{align}
    \mathcal{E}_k &= -2t\cos k - \nu\\
    \Delta_k &= -2i\Delta\sin k
\end{align}
The dispersion relation is
\begin{align}
    E_k &= \pm
    \sqrt\left\{ \mathcal{E}_k^2 + |\Delta_k|^2 \right\}\\
    &= \pm
    \sqrt{
        (\mu + 2t\cos k)^2
        + 4|\Delta|^2\sin^2 k
    }
\end{align}2
$E_k$ is always gapped away from $k=0,\pi$.

If $\mu=-2t$,
then it is gapless at $k=0$.
If $\mu=2t$,
then it is gapless $k=\pi$

Let's draw some pictures.

[pictures]
It's called p-wave because $\Delta_k$ is linear in $k$ for small $k$.

If the Hampton is
\begin{align}
    \mathcal{H}_k &=
    h_k^0 \mathbf{1}
    + \vec{h}_k \cdot \vec{\sigma}
\end{align}
Then the combo.
\begin{align}
    h_k^x = -h_{-k}^x
    \qquad
    h_y^y = -h_{-k}^y
    \qquad
    h_k^z = h^z_{-k}
\end{align}

If we know $\vec{h}_k$ from $[0,\pi]$,
then we know is everywhere.
\begin{align}
    \hat{h}_{k=0} &= S_0 \hat^{z}\\
    \hat{h}_{k=\pi} &= \underbrace{S_{\pi}}_{\pm 1}\hat{Z}
\end{align}
depending on the sign of $\mathcal{E}_\pi$.
Define $\nu = S_0 S_\pi$.

The two spaces $S_0 = S_{\pi}=2$
and $S_0 = -S_{T}$.
They are distinct so long as the energy gap stays open so $h_k$ is well-defined.

[picture]

Everything is region is topological.
Everything outside is topological.
As we tune the chemical potential through kinetic energy,
if we have an odd number of Fermi points,
they we're on a topological phase.


\chapter{One Option per Thread}
\label{chapter:oneoptionperthread}
This chapter describes the first parallel approach that exploits only outer parallelism, i.e. it computes a batch of options in parallel where one thread prices a single option.
This algorithm is therefore similar to the sequential implementation with some caveats concerning GPGPU architectures that were tackled in an iterative process producing multiple versions of the code.

\section{Sequential Implementation to CUDA}
\label{section:oneoption:seqtocuda}
\subsection*{Global memory setup}
It was necessary to identify arrays being used and consider how to store them in device memory. The input is a structure of arrays of size number of options, each array representing one parameter of the options. This structure is ideal for coalesced memory access because consecutive threads will load contiguous memory, optimizing the number of memory transactions and thus speed. The algorithm itself requires two arrays of tree width size for $\mathit{Qs}$ and $\mathit{QsCopy}$, and one array of tree height size for $\mathit{alphas}$ per option. We place the three arrays in GPU's global memory, where each thread uses a single part of each array which size depends on the option being computed.

\subsection*{Global memory accesses}
Array accesses had to be analyzed as well to avoid race conditions and optimize performance. The sequential implementation pre-computes rates and probabilities for each node along the width and saves them. They are reused later both during the forward and backward propagations. In contrast, for this CUDA implementation we would have to store the values in global memory, resulting in slower access times. Instead, since they were accessed too few times, we opted to re-compute the values every time they were used in order to save query time and reduce the global memory consumption.

 More importantly, in the forward propagation computing $\mathit{Qs}$ at the next time step was done by a single computation on the current node and adding the value to three nodes in the next step as shown in figure~\ref{fig:seqforward}. This performs 1 read for the computation and 3 reads followed by 3 writes for the addition. For all parallel implementations, this \textbf{scatter} pattern was replaced by a \textbf{gather} pattern illustrated in fig.~\ref{fig:scattervsgather}, which performs 3 reads/computations and 1 write in this scenario.

 This makes it necessary to pre-compute and save the values for $\mathit{Qs}$ first to avoid computing the same value multiple times. The final result makes 1 read/computation/write in the pre-computation step followed by 3 reads and 1 write in computing the next time step. This approach eliminates 1 write and the need to have atomic additions, which would be necessary when exploiting inner parallelism described in chapter~\ref{chapter:multoptionsperthreadblock}. However, since the tree is trinomial (fig.~\ref{fig:treeconststage1}), a value can be computed from up to 5 values from the previous time step. Thus all possible types of branching have to be enumerated which makes the code more verbose and more difficult to maintain.

\begin{figure}[H]
    \centering
    \def\svgwidth{0.8\textwidth}
	\caption{Comparison of scatter and gather operations}
    \input{img/scattervsgather.pdf_tex}
	\source{Compiled by the authors}
	\label{fig:scattervsgather}
\end{figure}

\section{Implementations}
We iteratively implemented 4 versions of this first parallel approach. They all share the same kernel but they store and access the three global arrays $\mathit{Qs}$, $\mathit{QsCopy}$ and $\mathit{alphas}$ in different ways.

The general steps for all these versions are as follows:
\begin{enumerate}
    \item Load all options to GPU device memory.
    \item Compute widths and heights for all options.
    \item Allocate in global GPU memory two (expanded) arrays $\mathit{Qs}$ and $\mathit{QsCopy}$ which are large enough to hold option-width elements for all the options in the batch. What \enquote{large enough} means will be specified for each version.
    \item Similarly, allocate in global GPU memory one array $\mathit{alphas}$ which is large enough to hold option-height elements for all options in the batch.
    \item Price all options using our CUDA kernel.
    \item Copy results to host memory.
\end{enumerate}

All pre-processing is performed on the GPU and is implemented using CUDA's Thrust library~\footnote{\url{https://developer.nvidia.com/thrust}}.

\subsection*{Version 1 - Naive}
The first version stores arrays in a simple way where one thread gets contiguous parts of memory with sizes that match the computed option's width/height. Table~\ref{table:cuda-option-memory-naive} shows an example of storing $\mathit{alphas}$ for 3 options of heights 2-4-3 computed by 3 threads in a single flat array. Each thread needs to know only the start index of its array chunk along with the option's width/height and then it can access array elements consecutively. The indices can be easily computed by running inclusive scans on widths and heights, obtaining also total sizes for the arrays in the process.

\begin{table}[H]
\centering
\caption{Memory alignment in version 1}
\source{Compiled by the authors}
\label{table:cuda-option-memory-naive}
\begin{tabular}{|l|l|
>{\columncolor[HTML]{EFEFEF}}l |
>{\columncolor[HTML]{EFEFEF}}l |
>{\columncolor[HTML]{EFEFEF}}l |
>{\columncolor[HTML]{EFEFEF}}l |l|l|l|}
\hline
T1         & T1         & T2         & T2         & T2         & T2         & T3         & T3         & T3         \\ \hline
$\alpha_0$ & $\alpha_1$ & $\alpha_0$ & $\alpha_1$ & $\alpha_2$ & $\alpha_3$ & $\alpha_0$ & $\alpha_1$ & $\alpha_2$ \\ \hline
\end{tabular}
\end{table}

This approach is very efficient in terms of storage space, however, it is very inefficient when it comes to performance. When we analyze how array elements are accessed in forward (alg.~\ref{alg:sequential-forward}) and backward propagation (alg.~\ref{alg:sequential-backward}), we find out that all threads access their $\alpha_0$ at the same time, then move to $\alpha_1$ and so on. This results in strided, un-coalesced access to global memory which ineffectively uses GPU hardware. The next 3 versions tackle this problem by padding and transposing the arrays on different levels, so as to ensure coalesced accesses to global memory whenever possible.

\subsection*{Version 2 - Global-level Padding}
In order to make array access coalesced, the second version stores arrays padded to the maximum width/height across all options. Continuing with the example from above, the new alignment is illustrated in table~\ref{table:cuda-option-memory-global}. This obviously leads to some array elements not being used, unless the widths/heights are equal across options. However, when threads in a warp access the same array index at the same time, the access is now coalesced and can be performed in fewer memory transactions, greatly improving performance.

\begin{table}[H]
\centering
\caption{Memory alignment in version 2}
\source{Compiled by the authors}
\label{table:cuda-option-memory-global}
\begin{tabular}{|l|l|l|
>{\columncolor[HTML]{EFEFEF}}l |
>{\columncolor[HTML]{EFEFEF}}l |
>{\columncolor[HTML]{EFEFEF}}l |l|l|l|
>{\columncolor[HTML]{EFEFEF}}l |
>{\columncolor[HTML]{EFEFEF}}l |
>{\columncolor[HTML]{EFEFEF}}l |}
\hline
T1         & T2         & T3         & T1         & T2         & T3         & T1 & T2         & T3         & T1 & T2         & T3 \\ \hline
$\alpha_0$ & $\alpha_0$ & $\alpha_0$ & $\alpha_1$ & $\alpha_1$ & $\alpha_1$ &    & $\alpha_2$ & $\alpha_2$ &    & $\alpha_3$ &    \\ \hline
\end{tabular}
\end{table}

To compute sizes of the arrays, it is necessary just to find out the maximum width/height
and multiply them by the number of all options. Indexing to array elements can be simply computed as $\mathit{index} * \mathit{optionsCount} + \mathit{optionIndex}$.

The only downside is that the global padding might require very large memory chunks to be allocated but unused, especially if the dataset is skewed, i.e. with a small number of options that have very large widths/heights. According to our tests, discussed in detail in chapter~\ref{chapter:experimentalresults}, this version is up to \textasciitilde$10\times$ faster than version 1 but the memory footprint is up to \textasciitilde$7\times$ larger. The next two versions try to minimize the memory footprint by padding arrays on a smaller scale.

\subsection*{Version 3 - Block-level Padding}
The third version is designed to save memory compared to the second version, while keeping memory access coalesced. Here we look at what options get computed in a single CUDA block (of up to 1024 threads). The maximum width/height of the options is computed per block and the total size of an array is then the sum of all block maxima multiplied by the block size.

The pre-processing of options is thus more complex. It is implemented using $\mathit{reduce\_by\_key}$ and $\mathit{transform\_inclusive\_scan}$ Thrust routines with custom operations. As a result, arrays $\mathit{QsInds}$ and $\mathit{alphasInds}$ storing indices to the respective arrays are computed, created using two addition helper arrays, all of size \\ $\lceil \mathit{optionsCount} / \mathit{blockSize} \rceil$. The computed indices represent the start of an array part specific to a block, elements can then be accessed as $\mathit{blockSize} * \mathit{index} + \mathit{threadId}$.

The downsides of this version are that if options in the dataset are not sorted by widths and heights, the amount of saved memory compared to version 2 may be very small while this version requires more pre-processing time and intermediate arrays. With correct sorting applied and the block size kept small, on our datasets this version uses only up to 4\% more memory than version 1, while keeping up to \textasciitilde$10\times$ performance lead like version 2. The last version further reduces the array padding even on bigger block sizes.

\subsection*{Version 4 - Warp-level Padding}
The fourth version is similar to version 3 with a difference that we look at what options get computed by a single warp of 32 threads instead of the whole block.  This is motivated by the fact that coalesced accesses to global memory are supported by hardware at the (half) warp level, i.e., the threads in a warp execute in lock step, and need to access consecutive memory locations in a SIMD instruction. In effect, array padding is performed at a lower granularity---that of a warp---while still preserving coalesced accesses. Depending on a dataset, the added warp-level padding might be smaller than block-level padding, in exchange for $\mathit{blockSize} / 32$ more indices to be stored. When comparing version 4 with version 3, we achieved up to 70\% decrease in memory for a CUDA block of size 1024.

\subsection*{Futhark implementation}
This one option per thread parallel approach was also implemented in Futhark as a proof of concept and is equivalent to version 2 since Futhark uses global padding on arrays to ensure that memory accesses are coalesced. This version will be discussed further in chapter~\ref{chapter:fullflattening}.

\subsection*{Optimizations}
\paragraph{Choosing thread block size}
Since this implementation prices one option per thread, the thread block size represents the number of options being priced in parallel. Those options can have different widths and heights, leading to thread divergence on outer parallelism (heights) and inner parallelism (widths). It follows that bigger blocks might contain options that vary in widths/heights more than in small blocks, thus causing more threads to wait for completion of the block execution. On the other hand, smaller block sizes result in more thread blocks being scheduled, what adds extra overhead. Depending on the dataset, block sizes of either 128 or 256 are preferred.

\paragraph{Sorting input}
In order to (easily) reduce thread divergence, we can sort options before computation by either heights or widths. This would make one warp/block of threads compute options with similar number of execution steps and reduce the amount of time the threads have to wait for each other's execution. Our experiments show that sorting does have a large positive impact on performance, up to \textasciitilde$10\times$ faster than no sorting, but the choice of sorting by height or width depends on a dataset with a difference of up to 20\%.

\section{Validation}\label{section:option:validation}
To validate computed results from the CUDA implementations, we created a test case that uses the example mentioned in section~\ref{section:sequential:validation}. 100 instances of this option with gradually more time steps are first computed on the CPU using the sequential implementation from chapter~\ref{chapter:sequential} and then compared with results computed from all 4 versions. The test case was written using Catch2~\footnote{\url{https://github.com/catchorg/Catch2}} test framework for its simplicity.

The differences between floating point GPU results and CPU results must be interpreted carefully, since there are many reasons why the same sequence of operations may not be performed on the GPU and CPU, e.g. because of fused multiply-add on the GPU, rearranging operations for parallelization, higher than expected precision for computations on the CPU and rounding not required by common math operations by the IEEE 754 standard~\cite[pg. 16]{whitehead2018}.

Despite this, we successfully validated our results using a small epsilon value of std::numeric\_limits<\textbf{real}>::epsilon() $* 1000$,
where \textbf{real} is either single or double precision floating point number. This equals to $0.000119209$ for single and $0.00000000000022204$ for double precision, which makes double precision much more reliable.

\section*{Summary}
This chapter provided an overview of our parallel one option per thread implementation using CUDA with focus on explaining the challenges of parallelizing the sequential implementation from chapter~\ref{chapter:sequential}. It introduced 4 versions of this parallel implementation, each with its own advantages and disadvantages. Finally, it described how the GPU results were validated against the CPU results and the challenges of doing that. The following chapter will describe how this implementation was adapted to compute multiple options in a single CUDA thread block.


\documentclass[12pt]{article}
\usepackage[margin=0.8in]{geometry}
\usepackage[utf8]{inputenc}
\usepackage{tikz}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage[version=4]{mhchem}
\usepackage{siunitx}
\usepackage{longtable,tabularx}
\usepackage{cleveref}
\setlength\LTleft{0pt}
\usepackage{subcaption}
\usepackage{relsize}
\usepackage{float}
\usepackage{booktabs}
\usepackage{bm}
\usepackage{comment}
\graphicspath{ {./Figures/} }
\usepackage{amssymb}
\usepackage{listings}
\usepackage{color} %red, green, blue, yellow, cyan, magenta, black, white
\definecolor{mygreen}{RGB}{28,172,0} % color values Red, Green, Blue
\definecolor{mylilas}{RGB}{170,55,241}

\date{}
\title{Project 1 \large \\
        Statistical Patter Recognition}

\author{Girguis Sedky}
\begin{document}
\maketitle

\section{ML/Bayes rule estimation}
In this method, the image pixel values were assumed to have a gaussian distribution. In addition, the prior probability of all the classes were assumed to be equal during training and testing. Under this assumption, the Bayesian classification rule reduces to assigning a state to the class that maximizes

\begin{equation}
P(\bm{x}|\omega_i) = \frac{1}{((2\pi)^d |\Sigma|)^{1/2}}\exp\left(-\frac{1}{2}(\bm{x}-\bm{\mu})^{\text{T}}\Sigma^{-1}(\bm{x}-\bm{\mu})\right),
\end{equation}
where $d=784$ is the length of the state vectors. This is equivalent to maximizing
\begin{equation}
\log\left(P(\bm{x}|\omega_i)\right) = -\frac{1}{2}\log(|\Sigma|)- \frac{1}{2}(\bm{x}-\bm{\mu})^{\text{T}}\Sigma^{-1}(\bm{x}-\bm{\mu}).
\end{equation}
The sample mean and covariances for each class are
\begin{equation}
\bm{\hat{\mu}}_i = \frac{1}{n}\sum_{j=1}^{n}\bm{x}_j,
\end{equation}
\begin{equation}
\hat{\Sigma}_i = \frac{1}{n}\sum_{j=1}^{n}(\bm{x}_j-\bm{\hat{\mu}}_i)(\bm{x}_j-\bm{\hat{\mu}}_i)^{\text{T}}.
\end{equation}
A training script was used on the training set comprised of 60,000 images to determine $\bm{\hat{\mu}}_i$ and $\hat{\Sigma}_i$ of every class and a testing script in turn used these values and the Bayesian classification algorithm to classify the test images.

Since the covariance matrices are large and some are singular, special computational techniques were used to ensure good results. A pseudo inverse operation was used to inverse $|\Sigma_i|$ and small multiple of the identity was added to $|\Sigma_i|$ before obtaining its determinant. \textbf{The classification error on the testing set came to about $35 \%$}

\section{Nearest Neighborhood classifier}
In this method, the image pixel values are arranged in a single column vector that lives in $\mathbb{R}^{784}$. In this space, each training image constitutes a prototype. The euclidiean distance between each testing image and all the training images is found, and the testing image is assigned to the class of the training image that is closest to it. \textbf{The classification error on the testing set came to about $15 \%$}

\section{PCA/LDA}
\subsection{PCA}
Principal component analysis was used to reduce the dimensionality of the problem. The low-order state vectors were then fed into the Bayesian and Nearest Neighborhood classifier. Figure~\ref{fig:PCA} demonstrates the percentage error in classification based on the number of dimensions kept. \textbf{The minimum classification error came to about $ 20.5\%$ for 50 kept components in the case of the Bayesian classifier and $14.7 \%$ for 150 components kept in the case of the Nearest Neighborhood classier.}
 \begin{figure}[h]
 \centering
 \begin{subfigure}[H]{0.49\textwidth}
\centering
  \includegraphics[width=\linewidth]{ML_PCA.pdf}
  \caption{Bayesian classifier}

\end{subfigure}
 \begin{subfigure}[H]{0.49\textwidth}
\centering
  \includegraphics[width=\linewidth]{NN_PCA.pdf}
  \caption{Nearest neighborhood classifier}

  \end{subfigure}
  \caption{Principal component analysis}
  \label{fig:PCA}
\end{figure}
\subsection{LDA}
Linear discriminant analysis was used to reduce the image state vectors from $\mathbb{R}^{784}$ to $\mathbb{R}^{M-1}$ where $M$ is the number of classes. The built-in MATLAB function was used for this purpose. Again, the low-order state vectors were then fed into the Bayesian and Nearest Neighborhood classifier. \textbf{The minimum classification error came to about $ 25.4\%$ in the case of the Bayesian classifier and $28 \%$ in the case of the Nearest Neighborhood classier.}
\newpage
\section{Summary}
Table~\ref{table:1} shows the classification error of the different methods utilized in this study. The most accurate method of classification was to use principal component analysis to reduce the dimensionality of the state vectors to 50 and then to apply NN to the transformed data.
\begin{table}[H]
\centering
\caption{Summary of classification results}
\label{table:1}
\begin{tabular}{cc}
\hline
Method     & Percentage Error \\ \hline
Bayes      & 35\%             \\
NN         & 15\%             \\
PCA, Bayes (50) & 20.5\%           \\
PCA, NN  (150)  & 14.7\%           \\
LDA, Bayes & 25.4\%           \\
LDA, NN    & 28\%             \\ \hline
\end{tabular}
\end{table}

\lstset{language=Matlab,%
    %basicstyle=\color{red},
    breaklines=true,%
    morekeywords={matlab2tikz},
    keywordstyle=\color{blue},%
    morekeywords=[2]{1}, keywordstyle=[2]{\color{black}},
    identifierstyle=\color{black},%
    stringstyle=\color{mylilas},
    commentstyle=\color{mygreen},%
    showstringspaces=false,%without this there will be a symbol in the places where there is a space
    numbers=left,%
    numberstyle={\tiny \color{black}},% size of the numbers
    numbersep=9pt, % this defines how far the numbers are from the text
    emph=[1]{for,end,break},emphstyle=[1]\color{red}, %some words to emphasise
    %emph=[2]{word1,word2}, emphstyle=[2]{style},
}

\newpage
\section*{Appendix}
\subsection*{ML/Bayes rule estimation}
\subsubsection*{Training script}
\lstinputlisting{TrainingScript_ML.m}

\subsubsection*{Testing script}
\lstinputlisting{TestingScript_ML.m}

\subsection*{NN}
\lstinputlisting{NN.m}

\subsection*{PCA}
\subsection{ML/Bayes rule estimation with PCA}
\lstinputlisting{ML_PCA.m}

\subsection{NN with PCA}
\lstinputlisting{NN_PCA.m}

\subsection*{LDA}
\lstinputlisting{LDA_Script.m}

\end{document}