\section{Background Modelling}
\label{sec:background_model}

\subsection{Background modeling strategy}
\label{ssec:bkg_strategy}

The strategy for the continuous background modelling follows what traditionally done in $H\to\gamma\gamma$ measurements, with the addition of some dedicated procedures to cope with specific issues related to insufficient MC statistics or very low-statistics categories. This section only contains a simplified description, more details could be found in HGam coupling analysis ~\ref{ANA-HIGG-2020-16-INT1}.

The steps comprising the background modelling strategy can be summarised as follows:
\begin{itemize}

  \item The composition of the continuous background in term of \yy, \yjet and \jetjet components is estimated with data driven technique for each category entering the measurement, as detailed in \Sect{\ref{ssec:bkg_composition}}.
  \item The background fractions are used to build \myy background templates for each category entering the measurement, where the \yy component is obtained from MC, while the \yjet and \jetjet components are measured from data driven control regions and used to reweight the \yy MC, as detailed in \Sect{\ref{ssec:bkg_templates}}.

  \item In order to improve the background modeling and reduce the impact of limited MC statistics, a smoothing procedure is being developed to be eventually applied to the background template, provided that the improvements on the results outweight the possible associated systematic uncertainties.

  \item The Spurious Signal approach is used in the smoothed \yy templates to select a functional form to describe each template, as well as the associated bias that will enter the measurement as a systematic uncertainties, as detailed in \Sect{\ref{ssec:spurious_signal}}. The Spurious Signal criteria are defined in order to cope with the limited MC statistics that in some case leads to non-physical large bias measurements.

  \item In very low-statistic categories, defined as those categories with \textcolor{red}{less than 10 events per bin in the \myy sidebands}, the smoothing procedure tends to introduce large non-physical structures in the templates, and it is therefore not suited to be used for them. For this reason, for this categories we initially select the statistically-justified function performing a Wald-test on the data sidebands using a family of function based on a exponential of increasing complexity. We accept the simplest function justifies by the Wald-test, and perform the Spurious Signal measurement with this function tolerating a bias that could be larger than the criteria used for the high-statistic categories, but would have a small impact on the overall sensitivity. This procedure is described in \Sect{\ref{sssec:bkg_functions}}.

\end{itemize}

\subsection{Background compositions}
\label{ssec:bkg_composition}

The dominant background entering the invariant mass spectrum comes from SM continuum diphoton production. A large number of photons are produced inside jets due to de decays of neutral mesons to photon pairs. Thus, photon-jet and di-jet events in which the jets are mis-identified as photons represent a non-negligible source of background. Photon-pairs can be also faked from Drell-Yan events in which both electrons are misidentified as photons. However, this background only contributes with a small fraction ($<1\%$).

The number of \yy, \yjet and \jetjet events entering each category after the final selection is estimated by means of a double two-dimensional sideband ABCD method~\cite{ATL-COM-PHYS-2012-592}. This data-driven method extrapolates the fraction of fake photons within the signal region from the composition of the side-band control regions, built by inverting photon identification and isolation requirements. The fractions are calculated individually in each categories and OO bins for the background template building. Table ~\ref{tab:yyfraction} lists the \yy fraction in all categories. More details are shown in Appendix ~\ref{appendix:2x2DSB}.

\begin{table}[htbp]
  \centering
  \begin{tabular}{c|cccc}
  \toprule
           &  TT  &  TL  &  LT  &  LL  \\
  \toprule
  OO bin1  & $0.76 \pm 0.16$ & $0.90 \pm 0.10$ & $0.93 \pm 0.03$ & $0.86 \pm 0.05$ \\ \hline
  OO bin2  & $0.89 \pm 0.10$ & $0.81 \pm 0.10$ & $0.81 \pm 0.06$ & $0.87 \pm 0.07$ \\ \hline
  OO bin3  & $0.79 \pm 0.15$ & $0.75 \pm 0.12$ & $0.88 \pm 0.03$ & $0.83 \pm 0.06$ \\ \hline
  OO bin4  & $0.92 \pm 0.08$ & $0.81 \pm 0.09$ & $0.84 \pm 0.05$ & $0.90 \pm 0.05$ \\ \hline
  OO bin5  & $0.78 \pm 0.14$ & $0.81 \pm 0.10$ & $0.86 \pm 0.05$ & $0.88 \pm 0.07$ \\ \hline
  OO bin6  & $0.88 \pm 0.10$ & $0.87 \pm 0.09$ & $0.88 \pm 0.05$ & $0.83 \pm 0.04$ \\
  \bottomrule
  \end{tabular}
  \caption{\yy fraction calculated with double two-dimensional sideband ABCD method. \yjet and \jetjet components are merged together in following analysis, so the fraction is $1-f_{\yy}$. }
  \label{tab:yyfraction}
\end{table}

\begin{figure}[tbp]
  \centering
  \includegraphics[width=0.45\linewidth]{figure/bkgtmpls/TT/yyfrac_TT.png}
  \includegraphics[width=0.45\linewidth]{figure/bkgtmpls/TL/yyfrac_TL.png} \\
  \includegraphics[width=0.45\linewidth]{figure/bkgtmpls/LT/yyfrac_LT.png}
  \includegraphics[width=0.45\linewidth]{figure/bkgtmpls/LL/yyfrac_LL.png}
  \caption{\yy, \yjet, \jetjet fraction in TT(upper left), TL(upper right), LT(bottom left), LL(bottom right) categories, as a function of OO. }
  \label{fig:yyfraction}
\end{figure}

\clearpage
\subsection{Background templates}
\label{ssec:bkg_templates}

Considering the low \jetjet fraction and similar performance of \yjet and \jetjet, those two components are merged together as \yjet+\jetjet for the background estimation. The shape is determined with a data control region, in TT, TL and LT categories it is defined as at least one of the two selected photons fail the tight identification and loose isolation requirement due to insufficient MC statistics, and in LL category it is defined as inverting the identification and isolation of exactly one of two photon candidates.
Contamination from \yy has been tested to be ignorable as shown in Figure ~\ref{fig:yjCR}, and a \textcolor{red}{linear} reweighting is derived to match the \yy and control region shapes. The bin width for ratio function smoothing varies from 5GeV, 2GeV, 1GeV and 0.5GeV respectively in TT, TL, LT, LL categories to match the MC statistics for an un-biased result. Uncertainty of this reweighted template contains two part: from the choice of smoothing function and from \yy fraction, while the latter is dominant.
These derived shapes are combined with the measured relative event fractions to obtain the total background shape. The \yy fraction is measured as mentioned in ~\ref{ssec:bkg_composition} \\

The obtained templates show a good agreement when comparing them with events in data passing the nominal selection (with tight photon identification and isolation cuts, TI) in the sidebands, as shown in \Fig{\ref{fig:BkgData-MC}}.\\

\clearpage
\begin{figure}[htbp]
  \centering
  \subfloat[TT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/CR/b1.png} }
  \subfloat[TT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/CR/b2.png} }
  \subfloat[TT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/CR/b3.png} } \\
  \subfloat[TT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/CR/b4.png} }
  \subfloat[TT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/CR/b5.png} }
  \subfloat[TT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/CR/b6.png} } \\
  \subfloat[TL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/CR/b1.png} }
  \subfloat[TL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/CR/b2.png} }
  \subfloat[TL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/CR/b3.png} } \\
  \subfloat[TL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/CR/b4.png} }
  \subfloat[TL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/CR/b5.png} }
  \subfloat[TL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/CR/b6.png} } \\
\end{figure}
\clearpage
\begin{figure}[htbp]
\centering
  \subfloat[LT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/CR/b1.png} }
  \subfloat[LT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/CR/b2.png} }
  \subfloat[LT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/CR/b3.png} } \\
  \subfloat[LT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/CR/b4.png} }
  \subfloat[LT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/CR/b5.png} }
  \subfloat[LT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/CR/b6.png} } \\
  \subfloat[LL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/CR/b1.png} }
  \subfloat[LL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/CR/b2.png} }
  \subfloat[LL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/CR/b3.png} } \\
  \subfloat[LL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/CR/b4.png} }
  \subfloat[LL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/CR/b5.png} }
  \subfloat[LL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/CR/b6.png} } \\
  \caption{Control region data in all categories and OO bins.}
  \label{fig:yjCR}
\end{figure}
\clearpage

\begin{figure}[htbp]
  \centering
  \subfloat[TT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/reweight/b1.png} }
  \subfloat[TT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/reweight/b2.png} }
  \subfloat[TT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/reweight/b3.png} } \\
  \subfloat[TT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/reweight/b4.png} }
  \subfloat[TT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/reweight/b5.png} }
  \subfloat[TT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/reweight/b6.png} } \\
  \subfloat[TL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/reweight/b1.png} }
  \subfloat[TL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/reweight/b2.png} }
  \subfloat[TL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/reweight/b3.png} } \\
  \subfloat[TL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/reweight/b4.png} }
  \subfloat[TL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/reweight/b5.png} }
  \subfloat[TL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/reweight/b6.png} } \\
\end{figure}
\clearpage
\begin{figure}[htbp]
\centering
  \subfloat[LT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/reweight/b1.png} }
  \subfloat[LT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/reweight/b2.png} }
  \subfloat[LT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/reweight/b3.png} } \\
  \subfloat[LT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/reweight/b4.png} }
  \subfloat[LT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/reweight/b5.png} }
  \subfloat[LT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/reweight/b6.png} } \\
  \subfloat[LL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/reweight/b1.png} }
  \subfloat[LL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/reweight/b2.png} }
  \subfloat[LL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/reweight/b3.png} } \\
  \subfloat[LL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/reweight/b4.png} }
  \subfloat[LL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/reweight/b5.png} }
  \subfloat[LL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/reweight/b6.png} } \\
  \caption{Reweight \yy MC with a linear function to match the \yjet+\jetjet shape in each bins and categories.}
  \label{fig:yyReweight}
\end{figure}
\clearpage

\begin{figure}[htbp]
  \centering
  \subfloat[TT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/raw_tmpl/b1.png} }
  \subfloat[TT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/raw_tmpl/b2.png} }
  \subfloat[TT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/raw_tmpl/b3.png} } \\
  \subfloat[TT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/raw_tmpl/b4.png} }
  \subfloat[TT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/raw_tmpl/b5.png} }
  \subfloat[TT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/raw_tmpl/b6.png} } \\
  \subfloat[TL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/raw_tmpl/b1.png} }
  \subfloat[TL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/raw_tmpl/b2.png} }
  \subfloat[TL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/raw_tmpl/b3.png} } \\
  \subfloat[TL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/raw_tmpl/b4.png} }
  \subfloat[TL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/raw_tmpl/b5.png} }
  \subfloat[TL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/raw_tmpl/b6.png} } \\
\end{figure}
\clearpage
\begin{figure}[htbp]
\centering
  \subfloat[LT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/raw_tmpl/b1.png} }
  \subfloat[LT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/raw_tmpl/b2.png} }
  \subfloat[LT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/raw_tmpl/b3.png} } \\
  \subfloat[LT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/raw_tmpl/b4.png} }
  \subfloat[LT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/raw_tmpl/b5.png} }
  \subfloat[LT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/raw_tmpl/b6.png} } \\
  \subfloat[LL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/raw_tmpl/b1.png} }
  \subfloat[LL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/raw_tmpl/b2.png} }
  \subfloat[LL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/raw_tmpl/b3.png} } \\
  \subfloat[LL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/raw_tmpl/b4.png} }
  \subfloat[LL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/raw_tmpl/b5.png} }
  \subfloat[LL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/raw_tmpl/b6.png} } \\
  \caption{Final background templates.}
  \label{fig:BkgData-MC}
\end{figure}
\clearpage

\subsubsection{Background smoothing}
\label{ssec:bkg_smoothing}
\textcolor{red}{Need Simplification} \\

Because of the limited statistics of the simulation samples used in the calculation, the value
of the spurious signal systematic uncertainty is subject to significant statistical fluctuations within
many of the analysis categories~\cite{Hyneman:2712576}. Statistical fluctuations in the background-only sample may cause
signal-like bumps, which are then fit as signal by the spurious signal procedure. These statistical fluctuations do
not capture the shape mismodeling from the analytic function, and they often drastically inflate the
value of the systematic.

Although simply producing additional simulation samples would alleviate the issue of statistical
fluctuations fitted as spurious signal, producing more simulated events is computationally
expensive. Additionally, producing events which fall into specific phase spaces is often highly inefficient.
Therefore, an alternative solution using the available simulation samples is preferred.

A Gaussian Process (GP) is defined as a set of random processes, where all finite subsets of these
processes have a multivariate normal distribution~\cite{ebden2015gaussian}.
Given a finite dataset – such as the bin contents of a smooth histogram – with corresponding
mean and covariance matrices, a Gaussian Process may be defined. The “correct” mean and the
covariance, however, are not necessarily well defined, as they encode specific assumptions about
the underlying dataset. In practice, the two quantities are fit to a finite dataset using a minimization
algorithm. In the case of a one-dimensional histogram with a finite number of bins, the mean can
be interpreted as a “rough” description of the underlying shape. The diagonal elements of the
covariance matrix represent the error of each bin while the off-diagonal elements specify
how “similar” the bin content of two different bins should be.

The covariance matrix can be simplified through the introduction of a kernel, which analytically
determines the level of correlation between two distinct points (i.e., the length scales in X at which points are expected to influence one another in Y). Two useful kernels are the Radial Basis Function (RBF) kernel and the Gibbs kernel~\cite{3569,Gibbs}.

The RBF kernel has one hyperparameter, the constant length scale $l$, and it is defined as

\begin{equation}
K_\text{RBF}(x,x') = exp\left(\frac{-(x-x')^2}{2l^2}\right)
\end{equation}

The RBF kernel is useful for mostly-flat functions. However, for smoothly-falling functions, it is likely that nearby points will be more correlated in some regions than in others, so a constant length scale is a suboptimal model. The Gibbs kernel allows the length scale $l(x)$ to vary linearly as a function of $x$, and thus has two hyperparameters: the initial length scale and the length scale slope. The Gibbs kernel function is:

\begin{equation}
K_\text{Gibbs}(x, x') = \frac{\sqrt{2l(x)l(x')}}{l(x)^2 + l(x')^2 } \cdot exp\left( \frac{-(x-x')^2}{l(x)^2 + l(x')^2} \right)
\end{equation}

The background templates used in the spurious signal test for the analysis categories are all smooth,
roughly exponentially falling distributions with statistical fluctuations. Fitting a background template
using Gaussian Process Regression (using the Gibbs kernel with the errors as determined by the initial templates)
offers a consistent method of estimating the underlying smooth shape of the template, without the problematic fluctuations.
Notably, the GP smoothing technique makes no assumption on the underlying distribution other than that it is smooth and falling, hence the choice of functional form from the spurious signal test will not be biased.

The hyperparameters (initial length scale and length scale slope) are allowed to vary over a range specified by the user; the optimal hyperparameters within this range are determined by the Gaussian Process fitting procedure.

A GP is fit to the original (noisy) background template in each category. The GP mean in the fits is defined as an exponential function,
the parameters of which are obtained by a fit to the original background template. The exponential
shape has been observed to be a sufficiently close guess for the categories used by the analysis.
However, in cases where the input template has very few statistics (less than about ten events per
bin on average), the resulting GP fit may be nearly identical to the mean exponential shape. This
issue occurs when the statistical uncertainties of the original template are so large that the template
is fully compatible with the preliminary exponential shape. Although the exponential shape
is technically an adequate descriptor of the template shape, the choice of the exponential mean
does bias the functional choice of the spurious signal test in this case. Therefore, a check has been
added to re-perform the GP fit using a flat mean in cases where the resulting GP shape and the
mean exponential shape disagree with a $\chi^2/DoF < 0.1$.

The resulting smoothed shape obtained from the GP fit is then saved as a new histogram. This
smoothed histogram is passed as the background template to the spurious signal test, which then
determines the background functional form and spurious signal systematic uncertainty as described
in \Sect{\ref{ssec:spurious_signal}}.

Note that based on previous study, the GPR method can only remain effectively unbiased for smoothly-falling templates containing more than an average of 20 Monte Carlo events/ bin ~\cite{ATLAS-CONF-2020-026}.
The TT and TL categories in this analysis even do not have sufficient statistics for GPR. Only LT and LL categories used this method, and the treatment of TT and TL is discussed in Section ~\ref{sssec:bkg_functions}.

Examples of the smoothed templates are presented in \Fig{\ref{fig:exampleGPR}} for a sample category with a high
level of statistics and for a category with a very low level of statistics.
The original templates are shown as well, for comparison. The data sidebands are also shown for
validation, although the GP smoothing technique does not take into account the data sidebands.

\begin{figure}[htbp]
  \centering
  \subfloat[LT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/smooth_tmpl/b1.png} }
  \subfloat[LT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/smooth_tmpl/b2.png} }
  \subfloat[LT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/smooth_tmpl/b3.png} } \\
  \subfloat[LT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/smooth_tmpl/b4.png} }
  \subfloat[LT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/smooth_tmpl/b5.png} }
  \subfloat[LT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LT/smooth_tmpl/b6.png} } \\
  \subfloat[LL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/smooth_tmpl/b1.png} }
  \subfloat[LL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/smooth_tmpl/b2.png} }
  \subfloat[LL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/smooth_tmpl/b3.png} } \\
  \subfloat[LL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/smooth_tmpl/b4.png} }
  \subfloat[LL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/smooth_tmpl/b5.png} }
  \subfloat[LL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/LL/smooth_tmpl/b6.png} } \\
  \caption{LT and LL category background templates after GPR smooth.}
  \label{fig:exampleGPR}
\end{figure}

%The smoothed background templates used in this analysis and the spurious signal values extracted from them can be found in the \App{\ref{sec:GPR_templates}}.

%Extensive validation tests were performed with the GP smoothing technique in order to ensure that the smoothing itself does not introduce a substantial bias. These tests primarily use “toy” templates – randomly-generated background templates constructed from either simulated diphoton events or from the probability distribution function of a known analytic function. We find that GPR remains effectively unbiased for smoothly-falling templates containing more than an average of 20 Monte Carlo events/ bin. These procedures to validate the GPR smoothing are reported in \App{\ref{sec:GPR_validation}}.

\subsection{Spurious signal test}
\label{ssec:spurious_signal}

The background \myy shape, for each analysis category, is described using an analytic function whose parameters and normalization are fitted to data. The choices for the analytic function that has been considered are:
\begin{itemize}
\item Exponential Function: $f(\myy) = e^{c\cdot \myy}$
\item Exponential Function of $2^{nd}$ Order Polynomial: $f(\myy) = e^{c_1\cdot m^2_{\gamma\gamma}+c_2\cdot \myy}$
\item Exponential Function of $3^{nd}$ Order Polynomial: $f(\myy) = e^{c_1\cdot m^3_{\gamma\gamma}+c_2\cdot m^2_{\gamma\gamma}+c_3\cdot \myy}$
\item Bernstein polynomial of order $N$: $B_{N}(\myy) = \sum_{i=0}^N c_i\cdot b_{i,N}$ with $b_{i,N} = \begin{pmatrix}N\\i\end{pmatrix}\myy^i (1-\myy)^{N-i}$, N=3, 4, 5
\item First-Order Power Law Function: $f(\myy) = \myy^c$
\end{itemize}
The method to select the functional form in each category is based on the spurious signal. The spurious signal test is described in more detail in \cite{Hasib:2238687}.

To perform the spurious signal test, the full analytic signal plus background model is fitted to a background-only template for each category separately. The fit is performed in the nominal diphoton mass range of $105 \leq \myy \leq \SI{160}{\GeV}$. The number of fitted signal events as a function of the Higgs mass is computed in intervals of \SI{1}{\GeV} within the diphoton mass range of $120 \leq \myy \leq \SI{130}{GeV}$. Negative fluctuations are taken into account. The number of spurious signal events $N_{sp}$ is then defined as the maximum of the absolute value of the fitted number of signal events within the signal diphoton mass sub-range. The statistical error is always taken into account.

The choice of analytic function is done such that the function satisfies at least one of the following criteria:
\begin{itemize}
\item $N_{sp}$ < 10\% $N_{s,exp}$, where $N_{s,exp}$ is the expected number of signal events within the SM in the category ($\mu_{sp} = N_{sp} / N_{s,exp}$)
\item $N_{sp}$ < 20\% $\sigma_{bkg}$, where $\sigma_{bkg}$ is the statistical uncertainty on the fitted number of signal events, when fitting the signal+background model to a background only Asimov dataset ($Z_{sp} = N_{sp} / \sigma_{bkg}$).
\end{itemize}

If multiple functions pass the test, then the one with the least number of parameters is chosen.

%The study including all the categories (also non-ttH) will be performed in the future in accordance with the method described in the previous iteration of the analysis. Due to the higher statistics, new functions (with more parameters) might be needed for the high statistics event categories, as for example the ggH categories. For the ttH only analysis, this does not need to be done.
%These should pass at least the
%Due to the limitation of MC statistic amount,
    A relaxed spurious signal criteria, that are made to accommodate $2\sigma$ local statistical fluctuations in the MC template, in case the MC statistics is small compared to the expected signal. This is done by defining a new variable:
\begin{align}
    \zeta_{sp} =
   \begin{cases}
N_{sp} + 2\Delta_{MC},~~~~ N_{sp} + 2\Delta_{MC}<0\\
N_{sp} - 2\Delta_{MC},~~~~ N_{sp} - 2\Delta_{MC}>0\\
0, ~~~~~~~~~~~~~~~~~~~~~~~\text{otherwise}\\
\end{cases}
 \label{eq:relSpuriousSignalMetric}
\end{align}
where $\Delta_{MC}$ is a local statistical fluctuation of the MC background template. This is illustrated in \Fig{\ref{fig:relaxed_ss_cartoon}}. This newly defined variable should then pass the criteria as $N_{sp}$ before.

For categories with at least 100 data events in the sideband regions defined as $105 < m_{\gamma\gamma} < 120$ GeV or $130 < m_{\gamma\gamma} < 160$ GeV, the background functions are selected to result in a $|S_{\text{spur}}|$ that is smaller than either 10\% of the total expected Higgs boson signal event ($S_{\text{exp}}$) or 20\% of the statistical uncertainty of the fitted signal yield ($\sigma_{\text{exp}}$). The fit of the analytic function to the background template is required to yield a $\chi^2$ probability of at least $1\%$~\footnote{The $\chi^2$ is computed with a background template uniformly binned over $105<m_{\gamma\gamma}<160$ GeV. The number of bins is 22, and the degrees of freedom used in the computation is 21 - $N_{\text{pars}}$, the number of free function parameters.}.

If multiple functions pass the requirements, the one with the smallest number of degrees of freedom is chosen. If no function passes, the spurious signal requirement is relaxed to
 $\zeta_{sp}$ < 10\% $N_{s,exp}$ or $\zeta_{sp}$ < 20\% $\sigma_{bkg}$.
For 28 out of the 88 categories, these relaxed conditions were used.
It's to be emphasized that even the relaxed condition is used, the background uncertainty is still $N_{sp}$ from the nominal approach of spurious signal study.

\subsubsection{Treatment of low-statistic cetegories}
\label{sssec:bkg_functions}

The determination of analytical functions is part of the result of spurious signal study, however, for low stastics categories with a background template suffering of very large statistical flucttuation we introduce a preliminary step. The straight use of the spurious signal procedure on these templates, in fact, would potentially lead to select a function with large number of degrees of freedom to accomodate these statistical fluctuation, and these function can prove to be dangerous when used on data. For this reason, for categories with less then 100 events in the \myy\ sidebads, we restrict the selection of the modeling function to exponential of first, second and third order, collectively described by:
\begin{equation}
  f_i{x} = e^{\sum_{i=0}^2 a_i x^i}
\end{equation}
where $x = \myy$.

We accept as modeling function the $f_i$ function with the lowest number of degrees of freedom for which the addition of more degrees of freedom does not lead to a significant improvement in the quality of the fit of the \myy\ sidebands.
To this purpose, we checked the likelihood ratio between the fits to the side-band data with two different background models:
\begin{equation}
\lambda_{(1,2)} = -2ln(L_{1}/L_{2})
\end{equation}
Where $L_{1}$ and $L_{2}$ are the likelihood values of the two fits.

The distributions of these test statistics are computed with \myy\
 side-band data, fitting it with two models and calculating the value of the test statistics.
 Once the distributions of the test-statistics
are computed, we can reject the simpler (less parameters) background
model by checking the P-value $P(\lambda >= \lambda_{\mathrm{data}})$ with the $\lambda$ measured in data ($\lambda_{\mathrm{data}}$).
This procedure to reject simpler background function is called Wald test.

Figure ~\ref{fig:Wtest} shows plots of the Wald-test on the low-statistic categories. All of the categories selected exponential function as the final decison.
Where the lowest P-value among all categories are around 3$\%$, corresponding to a significance of 2.2 $\sigma$ standard deviations.
The background uncertainty for these categories will be the spurious signal from exponential function.

As a cross-check, a F-test is also performed in Appendix ~\ref{append:ss}, and we draw the same conclusion that all of the categories can select exponential function.
The reason we use Wald test to decide the background function is the fits on side-band data are un-bin fit (so that it's consistent with the final result),
    there might be bias if we calculate the $\chi^2$ values in an artifical bin-size for the F-test. In this case the Wald test based on likelihood ratio is more reliable.

\begin{figure}[htbp]
  \centering
  \subfloat[TT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/wald/b1.png} }
  \subfloat[TT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/wald/b2.png} }
  \subfloat[TT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/wald/b3.png} } \\
  \subfloat[TT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/wald/b4.png} }
  \subfloat[TT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/wald/b5.png} }
  \subfloat[TT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TT/wald/b6.png} } \\
  \subfloat[TL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/wald/b1.png} }
  \subfloat[TL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/wald/b2.png} }
  \subfloat[TL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/wald/b3.png} } \\
  \subfloat[TL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/wald/b4.png} }
  \subfloat[TL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/wald/b5.png} }
  \subfloat[TL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgtmpls/TL/wald/b6.png} } \\
  \caption{Wald test in TT and TL categories. In all OO bins the first order exponential function shows the lower p value and is selected.}
  \label{fig:Wtest}
\end{figure}

\subsection{Background modelling results}
\label{ssec:bck_results}

The determined analytical functions modeling the baground in each category entering the measureument are found in Table~\ref{tab:spurious_sig} , as well as the corresponsing Spurious Signal bias.
The background only fit on the background templates based on MC samples is shown in Figure ~\ref{fig:bkgonly}

\begin{table}[htbp]
\centering
\small
\begin{tabular}{l|l|l|l|l|l|l|l|l}
\hline
           & \multicolumn{2}{c|}{TT} & \multicolumn{2}{c|}{TL} & \multicolumn{2}{c|}{LT} & \multicolumn{2}{c|}{LL} \\ \hline
           & Selected func. & maxS   & Selected func. & maxS   & Selected func. & maxS   & Selected func. & maxS   \\ \hline \hline
bin0       & Exp            & -0.29  & Exp            & -0.468 & Exp            & 0.87   & Exp            & 0.885  \\ \hline
bin1       & Exp            & 0.582  & Exp            & -0.882 & Exp            & -0.534 & Exp            & 1.54   \\ \hline
bin2       & Exp            & 0.235  & Exp            & -0.355 & Pow            & 0.437  & ExpPoly2       & -1.64  \\ \hline
bin3       & Exp            & 0.409  & Exp            & -0.304 & Pow            & 0.642  & Exp            & -0.718 \\ \hline
bin4       & Exp            & -0.404 & Exp            & 0.779  & Exp            & 0.775  & Exp            & -1.19  \\ \hline
bin5       & Exp            & 0.293  & Exp            & 0.408  & Exp            & 0.627  & Exp            & 0.52   \\ \hline
\end{tabular}
\caption{The final background modelling decision and the size of spurious signal uncertainties.}
\label{tab:spurious_sig}
\end{table}

\begin{figure}[htbp]
  \centering
  \subfloat[TT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TT/b1.pdf} }
  \subfloat[TT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TT/b2.pdf} }
  \subfloat[TT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TT/b3.pdf} } \\
  \subfloat[TT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TT/b4.pdf} }
  \subfloat[TT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TT/b5.pdf} }
  \subfloat[TT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TT/b6.pdf} } \\
  \subfloat[TL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TL/b1.pdf} }
  \subfloat[TL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TL/b2.pdf} }
  \subfloat[TL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TL/b3.pdf} } \\
  \subfloat[TL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TL/b4.pdf} }
  \subfloat[TL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TL/b5.pdf} }
  \subfloat[TL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/TL/b6.pdf} } \\
\end{figure}
\clearpage
\begin{figure}[htbp]
\centering
  \subfloat[LT category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LT/b1.pdf} }
  \subfloat[LT category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LT/b2.pdf} }
  \subfloat[LT category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LT/b3.pdf} } \\
  \subfloat[LT category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LT/b4.pdf} }
  \subfloat[LT category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LT/b5.pdf} }
  \subfloat[LT category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LT/b6.pdf} } \\
  \subfloat[LL category OO bin1]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LL/b1.pdf} }
  \subfloat[LL category OO bin2]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LL/b2.pdf} }
  \subfloat[LL category OO bin3]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LL/b3.pdf} } \\
  \subfloat[LL category OO bin4]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LL/b4.pdf} }
  \subfloat[LL category OO bin5]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LL/b5.pdf} }
  \subfloat[LL category OO bin6]{ \includegraphics[width= 0.30\textwidth]{figure/bkgfit/LL/b6.pdf} } \\
  \caption{Background only fit on the background templates based on MC samples. }
  \label{fig:bkgonly}
\end{figure}


\section{The Model}

Protocol participants are represented as parties---formally Interactive Turing Machine instances (ITIs)---in a multi-party computation. We assume a central adversary who  corrupts miners and  uses them to attack the protocol. The adversary is {\em adaptive}, i.e., can corrupt (additional) parties  at any point  and depending on his current view of the protocol execution. Our protocols are synchronous (G)UC protocols~\cite{C:BMTZ17,TCC:KMTZ13}:  parties have access to  a (global) clock setup, denoted by~$\gclock,$ and can communicate over a network of authenticated multicast channels. We assume instant and {\em fetch-based}  delivery channels~\cite{TCC:KMTZ13,AC:CGHZ16}. Such channels, whenever they receive a message from their sender, they record it and deliver it to the receiver upon his request with a ``fetch'' command. In fact, all functionalities we design in this work will have such  fetch-based delivery of their outputs. We remark that the instant-delivery assumption is without loss of generality as the channels are only used for communicating the timestamped object to the verifier which can anyway happen at any point after its creation. However, our treatment trivially applies also to the setting where parties communicate over  bounded-delay channels as in~\cite{C:BMTZ17}.

We adopt the {\em dynamic availability} model implicit in~\cite{C:BMTZ17} which was fleshed out in~\cite{CCS:BGKRZ18}. We next sketch its main components: All functionalities, protocols, and global setups have a dynamic party set. i.e., they all include special instructions allowing parties to register,  deregister, and allowing the adversary to learn the current set of registered parties. Additionally,  global setups allow any other setup (or functionality) to register and deregister with them, and they also allow other setups to learn their set of registered parties. For more details on the registration process we refer the reader to Appendix~\ref{se:registration}.
\paragraph{The Clock Functionality} $\gclock$ (cf. Fig.~\ref{fig:gclock}).
%As in~\cite{BMTZ17} we  assume a global UC-functionality $\gclock$  that allows the parties to ensure that the protocol they are running proceeds in synchronized rounds.
The \emph{clock functionality}  was initially proposed in~\cite{TCC:KMTZ13} to enable synchronous execution of UC protocols. Here we adopt its global-setup version, denoted by $\gclock$, which was proposed by~\cite{C:BMTZ17} and was used in the (G)UC proofs of the ledger's security.\footnote{As a global setup, $\gclock$ also exists in the ideal world and the ledger connects to it to keep track of rounds.}  $\gclock$  allows  parties (and functionalities) to ensure that the protocol they are running proceeds in synchronized rounds; it keeps track of  round variable whose value can be retrieved by parties (or by functionalities) via sending to it the pair: $\gread$. This value is increased when every honest party has sent to the clock a command $\gupdate$.
The parties use the clock as follows.  Each party starts every operation by reading the current round from $\gclock$ via the command $\gread$. Once any party  has executed all its instructions
for that round it instructs the clock to advance by sending a $\gupdate$ command, and gets in an idle mode where it simply reads the clock time in every activation until the round advances. %The clock advances to the next round when all the parties have requested to advance to the next round.
%In~\cite{KZZ16,BMTZ17}, this idea was transfered to the (G)UC setting, by assuming that the clock is a global setup. This allows  different protocols to use the same clock and we will also use the latter model in this paper. In particular, we model the \emph{time aspect} of all the functionalities that we propose, in terms of number of rounds. Therefore, when  $\gclock$ proceeds from one round to another, we say that $\gclock$ has \emph{ticked}.
%For completeness, we have included the global clock functionality in Appendix~\ref{app:funcs} We refer  readers to Fig.~\ref{fig:gclock} for a formal description of the functionality.
To keep more compact the description of our functionalities that rely on $\gclock$, we implicitly assume that
whenever an input is received the command $\gread$ is sent to $\gclock$ to retrieve the current round. Moreover, before giving the output, the functionalities request to advance the clock by sending $\gupdate$ to $\gclock$.

\subsection{Ledger Consensus: Model}\label{se:ledgermodel}

In this section, we define our notion of protocol execution following~\cite{EC:GarKiaLeo15,FOCS:Canetti01}.
The execution of a protocol $\Pi$ is driven by an
environment program $\env$ that may spawn multiple instances running the protocol $\Pi$. The programs
in question can be thought of as interactive Turing machines (ITM) that have communication,
input and output tapes. An instance of an ITM running a certain program will be referred to as
an interactive Turing machine instance or ITI. The spawning of new ITI's by an existing ITI as
well as the interaction between them is at the discretion of a control program which is also an ITM
and is denoted by $C$. The pair $(\env, C)$ is called a system of ITM's, cf.~\cite{FOCS:Canetti01}.
Specifically, the execution driven by $\env$ is defined with respect to a protocol $\Pi$, an adversary $\adv$
(also an ITM) and a set of parties $P_1,\dots,P_n$; these are hardcoded in the control program $C$.
Initially, the environment $\env$ is restricted by $C$ to spawn the adversary $\adv$. Each time the
adversary is activated, it may send one or more messages of the form $(\corrupt, P_i)$ to $C$. The control
program $C$ will register party $P_i$ as corrupted, only provided that the environment has previously
given an input of the form $(\corrupt, P_i)$ to $\adv$ and that the number of corrupted parties is less or
equal $\tcorrupt$, a bound that is also hardcoded in $C$.

We divide time into discrete units called \emph{time slots} or {\emph round}. Players are equipped with (roughly)
synchronized clocks $\gclock$ that indicate the current slot: we assume that any clock drift is subsumed in the slot
length.
%Each slot $\slr$ is indexed by an integer $r\in\{1,2,\dots\}$.

%The first ITI party to be spawned running protocol
%$\Pi$ is restricted by $C$ to be party $P_1$. After a party $P_i$ is activated, the environment is restricted to
%activate party $P_{i+1}$, except when $P_n$ is activated in which case the next ITI to be activated is always
%the adversary $\adv$. Note that when a corrupted party $Pi$ is activated the adversary $\adv$ is activated
%instead.

\subsubsection{Ledger Consensus.}
Ledger consensus (a.k.a. ``Nakamoto consensus'') is the problem where a set of nodes
(or parties) operate continuously accepting inputs
that are called transactions and incorporate them in a public data
structure called the {\em ledger}.
A ledger (denoted in calligraphic-face, e.g. $\ledger$) is a mechanism for maintaining a sequence of transactions, often
stored in the form of a blockchain. In this work, we denote with $\ledger$
the algorithms used to maintain the sequence, and with $\state$ all the views of the
participants of the state of these algorithms when being executed. For example, the (existing) ledger Bitcoin
consists of the set of all transactions that ever took place in the Bitcoin network, the current UTXO set, as
well as the local views of all the participants.
In contrast, we call a \emph{ledger state} a concrete sequence of transactions $\tx_1,\tx_2,\dots$ stored in the stable
part of a ledger state $\state$, typically as viewed by a particular party. Hence, in every blockchain-based ledger $\ledger$,
every fixed chain $\mathcal{C}$ defines a concrete ledger state by applying the interpretation rules given as a part of the
description of $\ledger$. In this work, we assume that the ledger state is obtained from the blockchain by dropping the last $k$ blocks
and serializing the transactions in the remaining blocks. We refer to $k$ as the \emph{common-prefix parameter}.
We denote by $\state^P[t]$ the ledger state of a ledger $\ledger$ as viewed by a party $P$ at the beginning of a time slot $t$
and by ${\check \state}^{P}[t]$ the complete state of the ledger (at time $t$) including all pending transactions that are not stable yet. $\state^P[t]$ can be obtained from  ${\check \state}^{P}[t]$ by dropping the last $k$ block.
%ledger states (or, more generally, any sequences), we denote by the prefix relation.
% and by $\hat P^\ledger[t]$ the complete state of the ledger (at time $t$) including all pending transactions that are not stable yet.

For two ledger states (or, more generally, any sequences), we denote by $\preceq$ the prefix relation.
Recall the definition of secure ledger protocol given in~\cite{RSA:GarKia20}.

\begin{definition}\label{de:ledger} A ledger protocol $\ledger$ is secure if it enjoys the following properties.

\ignore{
\begin{itemize}
\item {\em Consistency:} This property mandates that if a client queries
an honest node's ledger at round $t_1$
and receives the response $\mc{L}_1$,
then a client querying an honest node's ledger at round $t_2\geq t_1 $
 receives a response $\mc{L}_2$ that satisfies $\mc{L}_1 \preceq \mc{L}_2$,
where $\preceq$ denotes the standard prefix operation.

\item {\em Liveness:} If a transaction $tx$ is given as input to all honest
nodes continuously for a certain number of rounds denoted by $u$,
and a client queries any honest node's ledger
after these $u$ rounds, then the node responds with a value
$\mc{L}$ that includes $tx$.
%\end{itemize}
}

\begin{myitemize}
  \item[]{\bf Consistency.}
    For any two honest parties $\party_1,\party_2$ and two time slots $t_1\leq t_2$,
    it holds  $\LView{\state}{\party_1}{t_1} \preceq
    \LView{\check{\state}}{\party_2}{t_2}$.

  \item[]{\bf Liveness.}
    If all honest parties in the system attempt to include a  transaction $\tx$
    then, at any slot $t$ after $s$ slots (called the
    liveness parameter), any honest party $\party$, if queried,
    will report $\tx \in \LView{\state}{\party}{t}$.
\end{myitemize}

\end{definition}

In this work we also explicitly rely on the properties of \emph{Common Prefix (CP)}, \emph{Chain Growth (CG)}
and  \emph{Chain Quality (CQ)}.

\begin{myitemize}
  \item[] {\bf Common Prefix (CP); with parameters $k\in\mathbb{N}$}
states that for any pair of honest players  $P_1,P_2$ at rounds $r_1\leq r_2$ respectively, it holds that  ${ \state}^{P_1}[r_1] \preceq  {\check \state}^{P_2}[r_2] $.
  \item[] {\bf Chain Growth (CG); with parameters $\cg\in (0,1]$ and $s\in\mathbb{N}$}. Consider the chain $\mathcal{C}$
  adopted by an honest party at the onset of a slot and any portion of $\mathcal{C}$ spanning $s$ prior slots; then the number of
  blocks appearing in this portion of the chain is at least $\tau s$.

  \item[] {\bf Chain Quality (CQ) with parameters $\mu\in\mathbb{R}$
and $\ell\in\mathbb{N}$}. For any honest party $P$ with chain $\mathcal{C}$  it holds that for any $\ell$ consecutive blocks of $\mathcal{C}$ the ratio of honest blocks is at least $\mu$.
 \end{myitemize}

We consider a setting where a set of parties run a protocol maintaining a ledger $\ledger_1$. Following~\cite{SP:GazKiaZin19}, we denote by $\asmp_1$
the assumptions for $\ledger_1$.
 That is, if the assumption $\asmp_1$ holds, then ledger $\ledger_1$ is secure under the Definition~\ref{de:ledger}.
Formally, $\asmp_i$ for a ledger $\ledger_i$ is a sequence of events $\asmp_i[t]$ for each time slot $t$ that can assume value $1$,
if the assumption is satisfied, and $0$ otherwise.
For example, $\asmp_i$ may denote that
there has never been a majority of hashing power (or stake in a particular
asset, on this ledger or elsewhere) under the control of the adversary; that a
particular entity (in case of a centralized ledger) was not corrupted; and so
on.
Without loss of generality, we say that the assumption $\asmp_1$ for the ledger $\ledger_1$ holds if and only if the fraction of corrupted parties (the parties
that received the input $(\corrupt, \cdot)$) is below the threshold $\tcorrupt_1$ (where $\tcorrupt_1$ is part of the control function as described in the beginning
of this section).

\paragraph{Chain selection rule and block validation.} We sometimes assume that a ledger protocol describes a \emph{chain selection rule} that we denote with $\chainsel$.
That is, we assume that  each party in each round of the execution of the protocol collects all chains that come from the network and runs the algorithm $\chainsel$ to decide whether to keep his current local chain $\cloc$, or adopt one of the newly received chains.
Following~\cite{C:BMTZ17} we also assume
that before applying the chain-selection rule, any given chain is tested
 using the procedure $\isvalid$. $\isvalid$ checks filters the
 valid chains among all the chain received from the network and only
 the valid chain are used as input for $\chainsel$.
 $\chainsel$ in turns rely on the algorithm $\isvalidb$.
 $\isvalidb$ take as input a block $B$ of $\cloc$ and outputs $1$ if $B$ is a valid block (i.e., the structure of
 the block is correct) and  $0$ otherwise.

 We note that by assuming that a ledger protocol is always equipped with the algorithms $\chainsel$,
 $\isvalid$ and $\isvalidb$ make some of our results less general. However, we will show that it is possible to
obtain a better updatable ledger in the case when the two ledgers (the current ledger) and the new ledger have the same chain selection rule (among other similarities).

\subsection{Genesis Block Functionality}
The ledger protocols that we consider in this work are equipped with the description of an algorithm $\genesis$ that, on input a random value of appropriate length, outputs a valid genesis block (i.e., the first block of the chain). The security of most of the known ledger protocols holds
under the additional assumption that the genesis block is correct. That is, the genesis block has been generated
accordingly to $\genesis$ using appropriate randomness.
Multiple ways have been presented to generate a correct genesis block in the literature (i.e., by relying on a trusted authority, use
unpredictable information (like in bitcoin), run a multi-party computation (MPC) protocol~\cite{zcash},
rely on PoW~\cite{PKC:GKLP18} assumptions and so on and so forth).
In this work we abstract the generation of the genesis block by means of an ideal functionality.
The ideal functionality that one might expect, upon being activated from the adversary or from an honest party, should sample a random string and use it to run the algorithm $\genesis$. Unfortunately this simple functionality does not cover real world scenarios where an adversarial party might see the genesis block before the honest parties do. This, for example, can happen in the case when $\genesis$ is realized via an MPC protocol and a rushing adversary\footnote{A rushing adversary waits to receive the messages from all the honest parties and then computes its reply. Note that this means that, in general, the adversary is always able to see the output of the computation before the honest parties do.} could hold the genesis block (the output of the computation) for some bounded amount of time $\delay$ before the honest parties can see it.
We note that an adversary can use this strategy to take an advantage on the generation of the blocks that extend the genesis block.
 Therefore, the first modification that we consider for our ideal functionality is to allow the adversary to see the genesis block up to $\delay$ rounds earlier than the honest parties.
 The second relaxation allows the adversary to see up to
 $\maxblocks$ honestly generated genesis blocks and consequently decide which of these blocks will become the
 genesis block.
 We propose the formal description of our genesis functionality $\fgenesis$ in Fig.~\ref{fig:genesis}.
 We note that the case where $\delay=0$ and $\maxblocks=1$ corresponds to the case where there is only one candidate genesis block and all the parties can see it at the same round.

\begin{figure}[]
\small
\setlength{\fboxsep}{8pt}
\begin{center}
\begin{boxedminipage}{12cm}
{\bf Genesis Functionality for $\ledger$}

{\bf Parameters.} The functionality is parametrized by $\delay$, the maximum number of candidate genesis block $\maxblocks$, the genesis block $\bgen$ initialized with a default value $\bot$ and the procedure $\genesis()$.
We assume the functionality to be registered to $\gclock$ and that it maintains a set of registered parties $\parties$.
On any input $I$ the functionality queries $\gclock$, and we denote with  $\now$ be the response obtained by $\gclock$.

\begin{description}
\item[-] If $I=\gengenesis$ is received  from the adversary $\adv$ then  set $\tau:=\now$, generate $\maxblocks$
 genesis blocks (each block is generated by running the procedure $\genesis()$) $\GB:=\{\bgen_1,\dots, \bgen_\maxblocks\}$ for $\ledger$, and send $\GB$ to the adversary.

 \item[-] If $I=\getgenesis$ is received from an honest party $p_i\in\parties$ do the following
 \begin{itemize}
 	\item If $\bgen\neq\bot$ then return $\bgen$ to $p_i$.
	\item If $\bgen =\bot$ and $R-\tau>\delay$ then set generate a genesis block $\tilde \bgen$ by running
	 $\genesis$, set
	$\bgen\leftarrow \tilde \bgen$ and send $\bgen$ to $p_i$.
\end{itemize}
 \item[-] If $I=(\setgenesis,{\bgen}')$ is received from the adversary do the following
 \begin{itemize}
 	\item If $(R-\tau)\leq\delay$ and ${\bgen}'\in\GB$ then set $\bgen:={\bgen}'$.
	\item Else, return $\bot$ to the adversary.
 \end{itemize}

\end{description}

\end{boxedminipage}
\end{center}
\caption{The genesis functionality $\fgenesis$.}
\label{fig:genesis}
\end{figure}


\section{Introduction}\label{sec:intro}

Fish population (aka "stock")  assessment models determine the impact of past fishing on the historical and current abundance of the population, evaluate sustainable rates of removals (catch), and project future levels of catch reflecting one or more risk-averse catch rules.  These catch rules are codified in regional Fishery Management Plans according to requirements of the Sustainable Fisheries Act.  In the U.S., approximately 500 federally managed fish and shellfish populations are managed under approximately 50 Fishery Management Plans.  About 200 of these populations are assessed each year, based on a prioritized schedule for their current status. Despite this, many minor species have never been quantitatively assessed.  Although the pace is slower than that for weather forecasting, fish stock assessments are operational models for fisheries management.

Assessment models typically assimilate annual catches, data on fish abundance from diverse surveys and fishery sources, and biological information regarding fish body size and proportions at age.  A suite of models is available depending on the degree of data availability and unique characteristics of the fish population or its fishery.  Where feasible, environmental time series are used as indicators of changes in population or observation processes, especially to improve the accuracy of the projections of abundance and sustainable catch into the future.  Such linkages are based principally on correlations given the challenge of conducting field observations on an appropriate scale.  The frontier of model development is in the rapid estimation of parameters to include random temporal effects, in the simultaneous modeling of a suite of interacting species, and in the explicit treatment of the spatial distribution of the population.

Assessment models are loosely coupled to other models. For example, an ocean-temperature or circulation model or benthic-habitat map may be directly included in the pre-processing of the fish abundance survey.  A time series of a derived ocean factor, like the North Atlantic Oscillation, can be included as an indicator of a change in a population process.  Output of a multi-decadal time series of derived fish abundance can be an input to ecosystem and economic models to better understand cumulative impacts and benefits.

Stock Synthesis is an age- and size-structured assessment model in the class of models termed integrated analysis models. Stock Synthesis has evolved since its initial inception in order to model a wide range of fish populations and dynamics.  The most recent major revision to Stock Synthesis occurred in 2016, when version 3.30 was introduced. This new version of Stock Synthesis required major revisions to the input files relative to earlier versions (see the \hypertarget{ConvIssues}{Converting Files} section for more information). The acronym for Stock Synthesis has evolved over time with earlier versions being referred to as SS2 (Stock Synthesis v.2.xx) and older versions as SS3 (Stock Synthesis v.3.xx).

SS3 has a population sub-model that simulates a stock's growth, maturity, fecundity, recruitment, movement, and mortality processes, an observation sub-model estimates expected values for various types of data, a statistical sub-model characterizes the data’s goodness of fit and obtains best-fitting parameters with associated variance, and a forecast sub-model projects needed management quantities.  SS3 outputs the quantities, with confidence intervals, needed to implement risk-averse fishery control rules. The model is coded in C++ with parameter estimation enabled by automatic differentiation (\href{http://www.admb-project.org}{admb}).  Windows, Linux, and iOS versions are available.  Output processing and associated tools are in R, and a graphical interface is in QT.  SS3 is available from NOAA’s VLAB. The rich feature set in SS3 allows it to be configured for a wide range of situations.  SS3 has become the basis for a large fraction of U.S. assessments and many other assessments around the world.

This manual provides a guide for using SS3. The guide contains a description of the input and output files and usage instructions. An overview and technical description of the model itself is in \citet{methotstock2013}. However, SS3 has continued to evolve and grow since the publication in 2013, with this manual reflecting the most up to date information regarding SS3.  The model and a graphical user interface are available from the NOAA \href{https://vlab.noaa.gov/web/stock-synthesis}{VLAB}. The VLAB site also provides a user forum for posting questions and for accessing various additional materials.  An output processor package, r4ss, in R is available for download from CRAN or \href{https://github.com/r4ss/r4ss}{GitHub}.

Additional guidance for new users can be found in the SS3 \href{https://vlab.noaa.gov/web/stock-synthesis/document-library}{document library} within the NOAA VLAB website.  The "Begin Here - Introduction to Stock Synthesis" folder located in the Document Library contains step-by-step guidance for running Stock Synthesis.

  \subsection{How To Cite}

Please cite Stock Synthesis as:

Methot, R.D. and Wetzel, C.R. (2013). Stock Synthesis: A biological and statistical
framework for fish stock assessment and fishery management. Fisheries Research,
142: 86-99. \href{https://doi.org/10.1016/j.fishres.2012.10.012}{https://doi.org/10.1016/j.fishres.2012.10.012}

Please cite the Stock Synthesis User Manual as:

Methot, R. D., Jr., C. R. Wetzel, I. G. Taylor, and K. Doering. (2020). Stock Synthesis User Manual Version 3.30.15. U.S. Department of Commerce, NOAA Processed Report NMFS-NWFSC-PR-2020-05.\href{https://doi.org/10.25923/5wpn-qt71}{https://doi.org/10.25923/5wpn-qt71}

\pagebreak

\section{File Organization}\label{FileOrganization}
	\subsection{Input Files}
	\begin{enumerate}
		\item starter.ss: required file containing filenames of the data file and the control file plus other run controls (required).
		\item datafile: file containing model dimensions and the data (required)
		\item control file: file containing set-up for the parameters (required)
		\item forecast.ss: file containing specifications for reference points and forecasts (required)
		\item ss.par: previously created parameter file that can be read to overwrite the initial parameter values in the control file (optional)
		\item wtatage.ss: file containing empirical input of body weight by fleet and population and empirical fecundity-at-age (optional)
		\item runnumber.ss: file containing a single number used as run number in output to CumReport.sso and in the processing of profilevalues.ss (optional)
		\item profilevalues.ss: file contain special conditions for batch file processing (optional)
	\end{enumerate}

	\subsection{Output Files}
	\begin{enumerate}
		\item data\_echo.ss\_new: Contains the input data as read by the model. In model versions prior to 3.30.19 a single data.ss\_new file was created that included the echoed data, the expected data values (data\_expval.ss), and any bootstap data files seleected (data\_boot\_x.ss).
		\item data\_expval.ss: Contains the expected data values given the model fit. This file is only created if the value for "Number of datafiles to produce" in the starter file is set to 2 or greater.
		\item data\_boot\_x.ss: A new data file filled with bootstrap data based on the original input data and variances. This file is only created if the value in the "Number of datafiles to produce" in the starter file is set to 3 or greater. A separate bootstrap data file will be written for the number of bootstrap data file requests where x in the file name indicates the bootstrap simulation number (e.g., data\_boot\_001.ss, data\_boot\_002.ss,...).
		\item control.ss\textunderscore new: Updated version of the control file with final parameter values replacing the initial parameter values.
		\item starter.ss\textunderscore new: New version of the starter file with annotations.
		\item Forecast.ss\textunderscore new: New version of the forecast file with annotations.
		\item warning.sso: This file contains a list of warnings generated during program execution.
		\item echoinput.sso: This file is produced while reading the input files and includes an annotated echo of the input. The sole purpose of this output file is debugging input errors.
		\item Report.sso: This file is the primary report file.
		\item ss\_summary.sso: Output file that contains all the likelihood components, parameters, derived quantities, total biomass, summary biomass, and catch. This file offers an abridged version of the report file that is useful for quick model evaluation. This file is only available in SS3 v.3.30.08.03 and greater.
		\item CompReport.sso: Observed and expected composition data in a list-based format.
		\item Forecast-report.sso: Output of management quantities and for forecasts.
		\item CumReport.sso: This file contains a brief version of the run output, output is appended to current content of file so results of several runs can be collected together.  This is useful when a batch of runs is being processed.
		\item Covar.sso: This file replaces the standard ADMB ss.cor with an output of the parameter and derived quantity correlations in database format.
		\item ss.par: This file contains all estimated and fixed parameters from the model run.
		\item ss.std, ss.rep, ss.cor etc.:  Standard ADMB output files.
		\item checkup.sso: Contains details of selectivity parameters and resulting vectors.  This is written during the first call of the objective function.
		\item Gradient.dat: New for SS3 v.3.30, this file shows parameter gradients at the end of the run.
		\item rebuild.dat: Output formatted for direct input to Andre Punt's rebuilding analysis package.  Cumulative output is output to REBUILD.SS (useful when doing MCMC or profiles).
		\item SIS\_table.sso: Output formatted for reading into the NMFS Species Information System.
		\item Parmtrace.sso: Parameter values at each iteration.
		\item posteriors.sso, derived\_posteriors.sso, posterior\_vectors.sso: Files associated with MCMC.
	\end{enumerate}

\pagebreak

\section{Starting Stock Synthesis}
SS3 is typically run through the command line interface, although it can also be called from another program, R, the Stock Synthesis Interface, or a script file (such as a DOS batch file). SS3 is compiled for Windows, Mac, and Linux operating systems. The memory requirements depend on the complexity of the model you run, but in general, SS3 will run much slower on computers with inadequate memory. See the Running Stock Synthesis section on page \pageref{sec:RunningSS} for additional notes on methods of running SS3.

Communication with the program is through text files. When the program first starts, it reads the file starter.ss, which typically must be located in the same directory from which SS3 is being run. The file starter.ss contains required input information plus references to other required input files, as described in the File Organization section on page \pageref{FileOrganization}. The names of the control and data files must match the names specified in the starter.ss file. File names, including starter.ss, are case-sensitive on Linux and Mac systems but not on Windows.The echoinput.sso file outputs how the executable read each input file and can be used for troubleshooting when trying to setup a model correctly. Output from SS3 consists of text files containing specific keywords. Output processing programs, such as the SSI, Excel, or R can search for these keywords and parse the specific information located below that keyword in the text file.

\pagebreak


\chapter{Differentiation}
\section{Gradients}
\section{Differentiation from first principles}
\section{Differentiation of standard functions}
\section{Tangents, normals, stationary points, increasing and decreasing functions}
\section{Techniques of differentiation}
\section{Parametric and implicit differentiation}
\section{Constructing differential equations}


\documentclass[a4paper,10pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{graphicx}
%
%opening
\title{}
\author{}

\begin{document}

\maketitle

%\begin{abstract}

%\end{abstract}

\section{Automatic generation of neural mass model kernels}
\subsection{High level description of a simulation of a network of neural masses}
Neural mass models (NMMs) are mathematical tools to describe ensabled behaviour of groups of neurons through time.
These models contain a set of internal states which describe the system and a set of coupled differential equations which define how the states of the system evolve.
An instance of these models is called a node.
The output of the model consists of a set of observables which identify states of interest for other nodes.
Nodes are linked to each other using a coupling function.
This coupling function is the equivalent of a synapse in a point neuron network.
Coupling defines how the input coming from other nodes should be treated.
Usually the coupling involves weighting the incoming signals by a factor and then passing this through a simple function.
The weights used for the coupling are of such level of detail that are matchable to connectivity maps which can be obtained from experimental measurements, e.g. tractography.

Certain observables of a neural mass can be post-processed to produce simulated BOLD, EEG and EMG signals among others.
These signals can afterwards be input to an analysis step where a measure of the ability of the model to reproduce given aspects of the signal is performed.
The amount of open degrees of freedom in the Neural Mass Models generates a vast parameter space.
The nature of the workflow described above allows the iterative modification and exploration of parameters in this admissible space.
Brute force of more intelligent approaches can be used to optimize the behaviour of the models with respect to fitness functions defined by experts and related to the essential characteristics of the higher level signals.

A general description of the simulation can be seen in Figure \ref{fig}
\begin{figure}
  \begin{center}
    \includegraphics[scale=0.4]{DiagramNMM.eps}
  \end{center}
  \caption{Diagram of the interaction between the different computation stages in a neural mass model simulation.}
  \label{fig}
\end{figure}

\subsection{High level description of neural mass models}
In order to ease the high level description of the computation models used in this workflow, we have implemented the BaseModel class.
This class defines a set of attributes:
\begin{enumerate}

\item State: Stores the internal states of the model.
\item Auxex: Auxiliary mathematical expressions which are used for internal calculations in the model.
\item Input: Stores the input comming from other neural masses into this neural mass.
\item Drift: Set of equations which transfer the model from a state at t-1 to another at time t.
\item Diffs: Differentiable variables in the system.
\item Observ: Observers are the data structures which contain inner states which must be available to the outside of the model. They represent the output of the model.
\item Const: Constant values specifically defined for a each model.
\item Param: Parameters provided to an specific model.
\item Limit: Defines the min and max limits within which the state values must be wrapped to ensure mathematical consistency.

\end{enumerate}

\subsection{Loopy as automatic code generation library}
Loopy \cite{loopy2014} is a python library intended to aid the automatic generation of parallel kernels for different target hardware platforms.
It includes targets for CUDA, OpenCL, KNL and CPUs.
Parallel code in Loopy is generated by enclosing a set of functions into an independent execution domains.
The dimensions of these domains are specified using inames.
These inames represent the number of parallel instances that one can process at the same time of the given set of functions.

Other feature of loopy is the integration of multiple kernels into a single one.
The set of independent kernels is combined using a data flow which defines how variables are sent from one kernel into the next as input.
This allows the creation of complex kernels where data dependencies are assured.

Loopy automatically analizes the data structures used within the domain and the access patterns to them.
The user can specify type and ranges for values and access limits to the data structures to control the way the data is handled.
Loopy assambles the computation within a loop-like environment, where each iteration is independent.
The code can then be specifically ported to a target platform using its specific language.

\subsection{Implementation of neural mass models with Loopy}
Our objective is to have a high level description of neural mass models which can then be used to automatically generate high performance parallel code for multiple platforms.
To achieve this goal with Loopy, we need to match the high level description language of the models to the specific API of Loopy.

The BaseModel class has functions which translate the information provided in the attributes of a model instance
Generate the data structures
Define the kernel domain
Define the types for the attributes of the model
Generate expressions for constant deffinition
We unpack the values for the input, state and parameters.
Generate expressions for the auxiliary expressions
Generate expressions to store observables
We use pybolic to build a set of symbolic expressions representing the drift, diffs and observables.
We wrap the output within certain limits to avoid numerical inaccuracies.

\bibliographystyle{abbrvnat}
\bibliography{references}

\end{document}


\chapter{\MSTlong{} Methodology}\label{chap:methodology}
An effective, automated \mstlong{} (\mst{}) methodology is key to \cplop{}'s success as a tool to aid fecal contamination investigators.
Until recently, \cp{} researchers performed most \mst{} by hand \cite{moritz2015application, shapiro2015source}.
The goal for any \mst{} method, when used to source fecal contamination, is to take fecal matter or a substance contaminated with fecal matter and determine, or classify, the \spec{} that provided the fecal matter; \libdep{} \mst{} methods leverage the known-\spec{} information stored in their library, usually digital representations of \fiblong{} (\fib{}) stored in a database.
\cplop{} is such a \libdep{} technique that aims to support \mst{} using \pyros{} of both \itsshort{} regions of the \fib{} \ecoli{}.
Given a \fib{} \isol{} from an unknown-\spec{}\footnote{often referred to as the ``the unknown'' or ``the unknown \isol{}''}, a \libdep{} \mst{} technique determines the \spec{} of the unknown using the information in the library.
Towards this end, we built and investigated two \mst{} techniques, one that approaches \spec{} classification from the perspective of strains in the database and another that directly uses \isols{} present in the database.
This Chapter outlines the abstract approaches we chose to take, while \autoref{chap:clustering} and \autoref{chap:krap} detail the specific techniques we used for the strain-based and \isol{}-based approaches respectively.

\section{Strain-Based}
Strain typing is central to \libdep{} \mst{} methods and building a \spec{} classification technique that uses strains directly is an intuitive approach to take.
If an unknown-\spec{} matches a strain in the library, then we can make a reasonable assertion as to its \spec{} if the strain has a dominant \spec{}.
Thus, given an unknown-\spec{} \isol{}:
\begin{enumerate}
    \item Incorporate the unknown-\spec{} \fib{} \isol{} into \cplop{}
    \item Build strains of \fib{} using the \isols{} in \cplop{}
    \item Classify the source \spec{} of the \isol{} as the dominant \spec{} of the strain it ended up in
\end{enumerate}
Strain construction can happen in many ways, but from a computational perspective, it is very amenable to clustering.
If one can imagine a coordinate space that encapsulates mathematical representations (vectors) of \fib{} \isols{}, then strains are the close groupings (clusters) of these \isol{} representations.
For \cplop{}, this means constructing strains from the \pyros{} of collected \ecoli{} \isols{}.

As explained in \autoref{chap:clustering}, we use \dbscan{}, a \dbased{} clustering algorithm that we introduce in \autoref{sec:background:dbscan}, which builds clusters using a similarity metric, allowing for the concept of noise --- datapoints that remain unclustered.
Dense groupings of similar \isols{} fits our intuition of bacterial \isol{} strains because closely related ``families'' of \isols{} will appear in the same cluster.
A primary limitation of \dbscan{} is that it may not cluster some \isols{}, which still aligns with our notion of strains; sometimes, \isols{} are not part of any strain present in \cplop{}.
As a result, we need a fallback \mst{} method that can work for every \isol{}.

\section{\Isol{}-Based}
Using \isols{} directly can give us a level of flexibility that strict strain typing may not allow.
An unknown-\spec{} may not fit within the sometimes strict definition of a strain that a particular library may have, but we may still be able to make a reasonable assertion as to its \spec{} based on the \isols{} that are most similar to the unknown.
Thus, given an unknown-\spec{} \isol{}:
\begin{enumerate}
    \item Find $k$ known-\spec{} \isols{} from \cplop{} most similar to it, called the \knnlong{}
    \item Classify the source \spec{} of the \isol{} as the dominant \spec{} of the \knnlong{}
\end{enumerate}

The described approach is the \kNNlong{} classification algorithm (\kNN{}) introduced in \autoref{sec:background:knn}, however, due to the multiple \compfuncs{} needed between \isols{}, \kNN{}, in its natural form, will not work with \cplop{}.
\autoref{sec:background:isolates} details why the comparison of two \isols{} to each other requires multiple \compfuncs{}.
\autoref{chap:krap} explains the \compfunc{} resolution strategy employed by the \kraplong{}, the \isol{}-based \mst{} method we built and investigated.


\documentclass[12pt]{article}
\setlength{\evensidemargin}{5pt}
\setlength{\marginparwidth}{0pt}
\setlength{\marginparsep}{0pt}
\setlength{\textheight}{9in}
\setlength{\tabbingsep}{0pt}
\setlength{\headsep}{30pt}
\setlength{\fboxsep}{40pt}
\setlength{\fboxrule}{4pt}
\setlength{\footnotesep}{20pt}
\setlength{\parindent}{0pt}
\setlength{\parskip}{8pt plus 5pt}
\setlength{\topmargin}{-0.5in}
\setlength{\oddsidemargin}{0in}
\setlength{\textwidth}{6.5in}

\usepackage{times}
\usepackage{url}
\usepackage{graphics}
\usepackage{graphicx}      % extended graphics package
\usepackage{epsfig}        % wrapper for graphicx package
\usepackage[noend]{algpseudocode}
\algrenewcommand\algorithmicprocedure{\textbf{ALGORITHM}}

\begin{document}

\title{An Empirical Study of the Brute-Force Closest Pairs Algorithm}

\author{Jim Teresco\\
jteresco@siena.edu\\
\and
My Partner\\
partner@fake.email}

\maketitle

\begin{abstract}
  In this study, which is intended as an example of how to put
  together the writeup for an empirical study of algorithm
  performance, we present and analyze timings and operation counts for
  a straightforward brute-force closest pairs implementation.  Points
  are generated randomly in a specified range, and the closest pair is
  computed and reported by a Java program.
\end{abstract}

\section{Introduction}

This empirical analysis study focuses on the brute-force closest pairs
algorithm in two dimensions.  We will consider a straightforward
implementation of the algorithm in Java, instrumented to count basic
operations and gather timings.

The remainder of this report is organized as follows.
Section~\ref{sec:environment} describes the computing environment
used.  Section~\ref{sec:bfcp} describes the algorithm, our
expectations based on the theoretical analysis, presents our results,
and relates them back to the theory.  We conclude with some additional
discussion in Section~\ref{sec:conclusions}.

\section{Computing Environment}
\label{sec:environment}

For this study, a Java program was developed that implements the
algorithm of interest.  Random inputs of varying sizes and other
characteristics are generated, and the solution is computed.

All runs are using the following Java version:

\begin{verbatim}
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
\end{verbatim}

The computing environment is a MacBook Pro (15-inch, 2016) with a 2.6
GHz Intel Core i7, 16 GB of 2133 MHz LPDDR3 memory, and running macOS
Sierra Version 10.12.6.  There are 4 cores, each with 256 KB L2 cache,
and these cores share a 6 MB L3 cache.

\section{Brute-Force Closest Pairs}
\label{sec:bfcp}

The brute-force closest pairs algorithm as shown in
Figure~\ref{fig:alg} was implemented in Java, and instrumented to
count the number of times the distance between pairs of points is
computed and to report the elapsed time for the main loops to execute.

\begin{figure}[htb]
  \centering
\begin{algorithmic}
  \Procedure{BruteForceClosestPoints}{$P$}
  \State //Input: a set of points $P[0..n-1]$
  \State $d_{min} \gets \infty$
  \For{$i \gets 0 .. n-2$}
  \For{$j \gets i+1 .. n-1$}
  %\State $d \gets distance(P[i], P[j])$
  \State $d \gets \sqrt{(P[i].x - P[j].x)^2 +(P[i].y - P[j].y)^2}$
  \If {$d < d_{min}$}
  \State $d_{min} \gets d$
  \State $index_1 \gets i$
  \State $index_2 \gets j$
  \EndIf
  \EndFor
  \EndFor
  \State \textbf{return} $(index_1, index_2)$
\EndProcedure
\end{algorithmic}
\caption{The brute-force closest pairs algorithm, as implemented for
  this study.}
\label{fig:alg}
\end{figure}

The number of points for the runs is all powers of 2 from
$2^{10}=1024$ to $2^{19}=524,288$.  Points for each run are generated
randomly, within a range specified as a parameter to the program.
Points are generated within the square with x- and y-coordinates no
further than that range from the origin.

Each combination of the number of points and range of point positions
is run a total of 5 times.  The number of distance calculations is
expected to be identical for each of the runs for a given number of
points.  Run times will vary, however, and the minimum time for a
given combination of the number of points and range of point positions
is chosen to minimize the effects from other computations that could
be in process on the computer during the study.

\subsection{Expectations}

The theoretical expectation is for both the number of distance
calculations and elapsed time to be $\Theta(n^2)$ in the best,
average, and worst cases.  There are no shortcuts out of the loops,
which execute about $\frac{n^2}{2}$ times.

\subsection{Results}

The raw results of the study are available in the file
\texttt{timings.dat} in the GitHub repository.  Each line, such as

{\small
\begin{verbatim}
BFCP 262144 64.0 92375.0 34359607296 (99936,203911) 4.069483000753879E-4
\end{verbatim}
}

has 7 space-separated fields.  The first field indicates the algorithm name,
and is followed by the number of points, the range of coordinate
values (each of x and y are within this distance of the origin), the
elapsed time in milliseconds, the number of distance calculations, the
indices of the two points that were found to be the closest pair, and
the distance between those points.  Of these, we are interested in the
number of points, the elapsed time, and distance calculation count.

\begin{figure}[htb]
  \centering
  \includegraphics{bfcp-times.pdf}
  \caption{Times in milliseconds for the brute-force closest pairs to
    complete its computation for problem sizes from $2^{10}$ to
    $2^{19}$ points.}
  \label{fig:timings}
\end{figure}

\begin{figure}[htb]
  \centering
  \includegraphics{bfcp-opcounts.pdf}
  \caption{Number of distances between pairs of points computed the
    brute-force closest pairs to complete its computation for problem
    sizes from $2^{10}$ to $2^{19}$ points.}
  \label{fig:opcounts}
\end{figure}

\begin{table}[htb]
  \centering
  \begin{tabular}{|c|c|c|}
    \hline
    Problem Size & Time (ms) & Distance Calculations \\ \hline \hline
1024 & 11.0 & 523776 \\ \hline
2048 & 15.0 & 2096128 \\ \hline
4096 & 35.0 & 8386560 \\ \hline
8192 & 81.0 & 33550336 \\ \hline
16384 & 298.0 & 134209536 \\ \hline
32768 & 1171.0 & 536854528 \\ \hline
65536 & 4607.0 & 2147450880 \\ \hline
131072 & 18496.0 & 8589869056 \\ \hline
262144 & 84123.0 & 34359607296 \\ \hline
524288 & 368682.0 & 137438691328 \\ \hline
  \end{tabular}
  \caption{Actual times and operation counts used to create the plots
    in Figure~\ref{fig:timings} and Figure~\ref{fig:opcounts}.}
  \label{tab:rawdata}
\end{table}

Figure~\ref{fig:timings} shows the times in milliseconds (minimum
across 5 runs of each) and Figure~\ref{fig:opcounts} shows the number
of computations of the distance between two points for this study.
Unsurprisingly, the timings are very similar and the operation counts
identical for the different ranges of point placements, so
Table~\ref{tab:rawdata} shows the data as plotted in the figures but
only for the range value of 1.0.

\subsection{Discussion}

Both the timings and distance calulation counts match well with the
expected $\Theta(n^2)$ behavior of this algorithm.  The graphs in
Figures~\ref{fig:timings} and~\ref{fig:opcounts} show the expected
parabolic shape.  The actual numbers in Table~\ref{tab:rawdata} align
well.  The number of distance calculations is exactly as predicted.
Any variation there would have indicated an error in the algorithm or
in the instrumentation that counted the operations.  For the timings,
small problem sizes are subject to errors due to the accuracy of the
millsecond timer over short timespans.  However, when we look at the
larger problem sizes, we see exactly what we would expect: when we
double the problem size, the time taken to compute the solution
increased by a factor of four.

\section{Conclusions}
\label{sec:conclusions}

In this simple study, we found that all of our results matched the
expected behavior from the theory.  In this particular algorithm,
since there are no asymptotic differences among the best-, average-,
and worst-case behavior, this should be unsurprising.  The small
variations in run time can perhaps be explained by how soon the
algorithm finds the closest pair (or at least a very close pair).  The
more times that a new closest pair is found, the more often the body
of the \texttt{if} statement following the distance calculation will
need to execute.  There is also the issue that the computer used for
the study is a laptop system that had many processes such as web
browsers with many tabs and terminal windows in regular interactive
use during the timing studies.  This is partially accounted for by
taking the minimum time across 5 runs, but as each run has its own set
of random points generated, the previously-mentioned effect from the
different numbers of times the body of the \texttt{if} statement is
executed will potentially skew the results more toward a best case
behavior.  Any such variations are likely minor, and do not decrease
the confidence in our conclusions that the timing results match
closely with theoretical expectations.

\end{document}