\section{Mapping to Different Formalisms}
\label{sec:Intended_Use}
We want to simulate, verify and execute SLCO models.
Therefore, we have implemented a number of model transformations that transform an SLCO model into models on different platforms to enable simulation, verification, and execution.
Each of these target platforms has different characteristics.
Table~\ref{tab:platform_characteristics} shows the characteristics of the different platforms.
\begin{table}[hbt]
\centering
\begin{tabular}{l||c|c|c}
              & Synchronous/asynchronous channels & Lossless/lossy channels & Concurrent objects \\
\hline\hline
 SLCO         &   synchronous and asynchronous    &   lossless and lossy    &     unlimited      \\
\hline
 Simulation   &            synchronous            &        lossless         &     unlimited      \\
\hline
 Verification &   synchronous and asynchronous    &        lossless         &     unlimited      \\
\hline
 Execution    &           asynchronous            &          lossy          &      limited       \\
\end{tabular}
\caption{Platform characteristics}
\label{tab:platform_characteristics}
\end{table}
The first column lists the different platforms.
The second column indicates whether communication over channels is synchronous or asynchronous on the corresponding platform.
In case communication is synchronous, both sender and receiver need to be available for message exchange to occur.
In this way, sender and receiver synchronize on message exchange.
In case communication is asynchronous, a sender can send a message even though the receiver is not yet ready to receive it.
The third column indicates whether channels are lossless or lossy.
In case a channel is lossless, a message that is sent will always arrive at the receiver.
In case a channel is lossy, a message that is sent may get lost.
We assume that not all messages get lost.
We also assume that messages, if they arrive, arrive as they are sent, i.e., undamaged.
The fourth column lists the amount of controllers that are available on the platform.
A controller is a device on which an (implementation of) an SLCO object can be deployed.
For SLCO (specification), simulation, and verification this number is unlimited since these controllers are virtual.
On the implementation platform, actual controllers need to be available.
Therefore, this number is limited.
We have also implemented a number of model transformations that transform SLCO models to bridge the semantic platform gaps.
An overview of the model transformations is given in Figure~\ref{fig:trafo_overview}.
\begin{figure}[hbt]
 \centering
 \includegraphics[width=.8\textwidth]{figs/trafo_overview}
 \caption{Overview of the transformations}
 \label{fig:trafo_overview}
\end{figure}
In the remainder of this section, we explain how an SLCO model can be transformed into models for simulation, verification, and execution.
We also explain in detail how we dealt with the differences in platform characteristics.

\subsection{Simulation}
The Parallel Object-Oriented Specification Language (POOSL) is a formal modeling language for simulation and performance analysis~\cite{Theelen2007}.
The syntax of POOSL is based on the syntax of the formal language CCS~\cite{Milner1989}.
The main constructs of POOSL are cluster classes, process classes and data classes.
Each of these classes can be instantiated and the resulting objects can be combined to specify a system.
A cluster class consists of a number of process classes that are connected by channels.
A process class consists of variables, ports, process methods and instantiation parameters.
The ports of a process class can be connected to the ports of other process classes by means of channels.
A pair of process classes connected by a channel can communicate by sending synchronous signals over the channel.
The variables are instances of data classes and are used to store data.
Instantiation parameters are special kinds of variables, whose initial value can be specified.
The behavior of process classes is specified using process methods.
A process method takes a number of parameters as input and can pass output through output parameters.
The behavior of a process method is defined using a number of statements, each of which can be guarded by an expression.
Among others, there are statements for sending and receiving signals, assigning values to variables or parameters, and calling methods of data objects.
These statements can be composed both sequentially and in parallel.
POOSL also has a statement for conditional iteration, a statement for conditional execution, and a statement for non-deterministic execution of a number of statements.

The language is supported by two tools: SHESim and Rotalumis.
SHESim offers a graphical user interface for defining the structure of models as well as editors for editing of the textual definitions of process classes and data classes.
Models can be interactively simulated using the built-in POOSL interpreter of SHESim.
Rotalumis is a command-line tool that can simulate POOSL models at high speed, by compiling them to byte code that is executed on a virtual machine implemented in C++.

Transformation~$T_s$ depicted in Figure~\ref{fig:trafo_overview} transforms SLCO models containing synchronous, lossless channels only to POOSL models.
This transformation transforms each class to a process class.
The state machines of each class are transformed to a number of process methods.
Each process method represents a state.
It contains a number of guarded statements that are executed non-deterministically.
The guards of the statements represent the guards and triggers of outgoing transitions, and the statements represent the effects of those transitions.
Each guarded group of statements ends with a process method call that calls the method representing the target state of the transition represented by this guarded group of statements.

\begin{figure}[hbt]
\centering
\subfloat[Partial state machine]
         {
          \label{subfig:slco_poosl_slco}
          \includegraphics[scale=.2]{figs/slco_poosl_slco}
         }
\hspace{.2\textwidth}
\subfloat[Corresponding \textsc{Promela} code]
         {
          \label{subfig:slco_poosl_poosl}
          \includegraphics[scale=.3]{figs/slco_poosl_poosl}
         }
\caption{Transforming state machines into \textsc{POOSL}}
\label{fig:slco_poosl}
\end{figure}

To transform SLCO models containing asynchronous channels into POOSL, the SLCO model should be transformed such that it contains synchronous channels only.
This is done by transformation~$T_1^\prime$ depicted in Figure~\ref{fig:trafo_overview}.
In this transformation buffers are added, such that asynchronous communication is simulated using synchronous channels.
To transform SLCO models containing lossy channels into POOSL, the SLCO model should be transformed such that it contains lossless channels.
This is done by transformation~$T_2^\prime$ depicted in Figure~\ref{fig:trafo_overview}.
In this transformation, an object is added that represents a lossy channel.
Messages are no longer sent over a channel directly, but via this object.
Upon reception of a message, the object either sends the message to its destination, or it disposes of it.
In this way, a lossy channel is simulated.
An SLCO model with a limited number of controllers can be transformed into POOSL without any problems, since it allows an unlimited number of (virtual) controllers.
Therefore, in transformation~$T_3^\prime$ nothing is changed.

\subsection{Verification}
\label{subsec:verification}
Model checking is an automated verification technique that checks whether a formally specified property holds for a model of a system~\cite{Clarke1999}.
An exhaustive state space search is performed by an automated model checker to determine whether a property holds in a finite state model of a system.
Model checking is commonly used for verifying concurrent systems.
Before automated model checking can be applied, two tasks need to be performed.
First, the system to be verified needs to be specified in a formalism that is accepted by an automated model checker.
Second, the properties that the system should satisfy have to be formulated in a logical formalism that is accepted by an automated model checker.
Usually a form of temporal logic is used for this.
Hereafter, an automated model checker can be used to verify whether the property holds in the model.
If this is not the case, a counterexample in the form of a trace violating the property is often provided.

In order to enable verification of SLCO models, we implemented a model transformation that transforms SLCO models into \textsc{Promela}.
This is transformation~$T_v$ in Figure~\ref{fig:trafo_overview}.
\textsc{Promela} is the input formalism for the model checker \textsc{Spin}~\cite{Holzmann2003}.
\textsc{Spin} can check a model for deadlocks, unreachable code, or determine whether it satisfies an LTL property.

The \textsc{Promela} language has constructs for modeling selections and loops.
These are based on Dijkstra's guarded commands.
\textsc{Promela} has primitives for message passing between processes over channels either using queues, or handshaking.
This enables the modeling of both asynchronous and synchronous communication respectively.
The syntax of expressions and assignments in the \textsc{Promela} language is inherited from \textsf{C}.

Channels in \textsc{Promela} are lossless and can be either synchronous (handshake) or asynchronous (queue).
Therefore, our transformation can transform SLCO models that do not contain lossy channels into \textsc{Promela} code.
Every state machine contained in a class instantiated as an object in an SLCO model is transformed into a \textsc{Promela} process.
State machines can be implemented using an imperative programming style in multiple ways.
We chose to implement them as jump tables using goto-statements.
An example of part of the transformation is shown in Figure~\ref{fig:sm2pomela}.
State~$S_0$ depicted in Figure~\ref{subfig:sm} is transformed into the code depicted in Figure~\ref{subfig:code}.
\begin{figure}[hbt]
\centering
\subfloat[Partial state machine]
         {
          \label{subfig:sm}
          \includegraphics[scale=.2]{figs/sm2promela_1}
         }
\hspace{.2\textwidth}
\subfloat[Corresponding \textsc{Promela} code]
         {
          \label{subfig:code}
          \includegraphics[scale=.3]{figs/sm2promela_2}
         }
\caption{Transforming state machines into \textsc{Promela}}
\label{fig:sm2pomela}
\end{figure}
A state is transformed into an if-statement with a label and a goto-statement that jumps to that label.
Every outgoing transition of state~$S_0$ is transformed into an alternative of the if-statement.
The guard and statements of a transition are transformed into \textsc{Promela} code in a straightforward way.
When the guard condition holds and the statements are executed, the transition to state~$S_1$ has been completed and the state machine is \emph{in} state~$S_1$.
Therefore, the goto-statement that jumps to the label representing state~$S_1$ is added after the transformed statements in the code.

To verify an SLCO model that contains lossy channels, transformation~$T_2^\prime$ should be performed such that messages may be disposed of.
Note that verification of models with lossy channels requires extra care since traces in which all messages are disposed of should be avoided.

\subsection{Execution}
To execute the SLCO models, an implementation platform is required.
We chose to use the Lego Mindstorms platform for this~\cite{Lego}.
The key part of this platform is a programmable Lego brick, called RCX.
This RCX has an infrared port for communication and is connected to sensors and motors for interacting with its environment.
The language used to program an RCX is Not Quite C (NQC)~\cite{Baum2003}.
To enable execution, we implemented a model transformation that transforms SLCO models into NQC code.

NQC, as the name suggests, resembles \textsf{C}.
NQC is severely restricted to fit the Lego Mindstorms platform.
However, there is also an API consisting of a set of constants, functions, values, and macros that provide access to various capabilities of the Lego Mindstorms platform such as sensors, outputs, timers, and communication.

Every state machine contained in a class instantiated as an object in an SLCO model is transformed into an NQC task.
The transformation from state machines to NQC code strongly resembles the transformation from state machines to \textsc{Promela} described in Section~\ref{subsec:verification}.
The only difference is the syntax of the if-statement.

An RCX is a controller as listed in Table~\ref{tab:platform_characteristics}.
RCX's can communicate with each other by sending and receiving messages via an infrared port.
Communication via the infrared port is asynchronous.
Since an infrared signal can be interrupted, this communication is lossy.
This restricts the type of SLCO models that can be transformed.
In order to enable execution of all SCLO models, a number of model transformations, i.e., $T_1$, $T_2$, and~$T_3$ depicted in Figure~\ref{fig:trafo_overview}, were implemented to overcome the platform differences.
In model transformation~$T_1$, the synchronous signals are replaced by asynchronous signals.
In order to maintain semantic equivalence, acknowledgement messages are added for synchronization.
In model transformation~$T_2$, an implementation of the alternating bit protocol~\cite{Bartlett1969} is added to every class.
The state machines are adapted in such a way that they do not send a message directly, but instead offer it to the alternating bit protocol for transmission.
The protocol will ensure that the message will eventually arrive at it destination, under the premise that not all messages get lost.
In this way, lossless message exchange over a lossy channel is achieved.
In model transformation~$T_3$, objects are merged to match the number of available controllers.
Communication between merged objects is implemented using shared variables, since SLCO and NQC do not provide channels internal to objects.


\chapter{Liquid Nitrogen Cooling in IR Thermography applied to steel specimen}
The results of this study were firstly presented at an oral session of SPIE Commercial+ Scientific Sensing and Imaging. International Society for Optics and Photonics, 2017.
Then it was accepted for publication in Proceedings Volume 10214, Thermosense: Thermal Infrared Applications XXXIX; 102140T (2017).

%
\section{Résumé}
La thermographie pulsée (PT) est l'une des méthodes les plus courantes dans les procédures de thermographie active de la thermographie pour NDT \& E (essais non destructifs  \& évaluation), en raison de la rapidité et la commodité de cette technique d'inspection. Des éclairs ou des lampes sont souvent utilisés pour chauffer les échantillons dans la PT traditionnelle. Cet article explore principalement exactement la stimulation externe opposée en IR Thermographie: refroidissement au lieu de chauffage. Un échantillon d'acier avec des trous à fond plat de différentes profondeurs et tailles a été testé. De l'azote liquide (LN$_2$) est répandu sur la surface de l'échantillon et l'ensemble du processus est capturé par une caméra thermique. Pour obtenir une bonne comparaison, deux autres techniques classiques de CND, la thermographie pulsée et la thermographie verrouillée, sont également utilisées. En particulier, la méthode Lock-in est implémentée avec trois fréquences différentes. Dans la procédure de traitement d'image, la méthode de thermographie en composantes principales (PCT) a été effectuée sur toutes les images thermiques. Pour les résultats Lock-in, les images de phase et d'amplitude sont générées par la transformée de Fourier rapide (FFT). Les résultats montrent que toutes les techniques présentaient en partie les défauts tandis que la technique LN $_2$ affichait les défauts seulement au début du test. De plus, un poste-traitement de seuil binaire est appliqué aux images thermiques, et en comparant ces images à une carte binaire de l'emplacement des défauts, les courbes caractéristiques de fonctionnement du récepteur (ROC) correspondantes sont établies et discutées. Une comparaison des résultats indique que la meilleure courbe ROC est obtenue en utilisant la technique flash avec la méthode de traitement PCT.

%Cette recherche étudie une stimulation externe - refroidissement au lieu de chauffer en thermographie infrarouge pour NDT \& E. Un spécimen en acier est utilisé afin de  tester trois stimulations différentes sur les images thermiques et également une comparaison d'analyse ROC. Les résultats montrent que toutes les techniques mettent en évidence une partie des défauts de l'échantillon, alors que la technique LN $ _2 $ ne représente les défauts qu'au début; ceci peut être dû à la conductivité élevée de l'acier. Dans les résultats thermiques, la méthode de post-traitement PCT affiche de meilleurs résultats pour toutes les procédures. Plus de défauts sont exposés dans la stimulation Flash avec le traitement PCT.
% Les résultats de cette étude ont d'abord été présentés lors d'une session orale de SPIE Thermosense: Thermal Infrared Applications XXXIX 2017, aux Etats-Unis.

\section{Abstract}
Pulsed Thermography (PT) is one of the most common methods in Active Thermography procedures of the Thermography for NDT \& E (Nondestructive Testing \& Evaluation), due to the rapidity and convenience of this inspection technique. Flashes or lamps are often used to heat the samples in the traditional PT. This paper mainly explores exactly the opposite external stimulation in IR Thermography: cooling instead of heating. A steel sample with flat-bottom holes of different depths and sizes has been tested. Liquid nitrogen (LN$_2$) is sprinkled on the surface of the specimen and the whole process is captured by a thermal camera. To obtain a good comparison, two other classic NDT techniques, Pulsed Thermography and Lock-In Thermography, are also employed. In particular, the  Lock-in  method  is  implemented  with  three  different  frequencies.  In  the  image  processing  procedure,  the Principal Component Thermography (PCT) method has been performed on all thermal images. For Lock-In results, both Phase and Amplitude images are generated by Fast Fourier Transform (FFT). Results show that all techniques presented part of the defects while the LN$_2$ technique displays the flaws only at the beginning of the test. Moreover, a binary threshold post-processing is applied to the thermal images, and by comparing these images to a binary map of the location of the defects, the corresponding Receiver Operating Characteristic (ROC) curves are established and discussed. A comparison of the results indicates that the better ROC curve is obtained using the Flash technique with PCT processing method.

\newpage
\textbf{\texttt{Contributing authors:}}

\textbf{\textsf{Lei Lei}} (Ph.D candidate): developing protocol, experiment preparation and planning, data analysis,  personnel coordination and manuscript preparation.

\textbf{Giovanni Ferrarini} (Researcher of CNR-ITC): discussion in developing protocol, experiment preparation.

\textbf{Alessandro Bortolin} (Ph.D student of CNR-ITC): discussion and experiment preparation.

\textbf{Gianluca Cadelano} (Ph.D student of CNR-ITC): data collection, discussion and experiment preparation.

\textbf{Paolo Bison} (Research supervisor of CNR-ITC): student supervision, revision and correction of the manuscript.

\textbf{Xavier Maldague} (Research director of LVSN in University Laval): student supervision, revision and correction of the manuscript.

% \phantomsection\addcontentsline{lof}{table}{5.1\quad Experimental set-up in the reflection mode}
% \phantomsection\addcontentsline{lof}{table}{5.2\quad  Steel sample dimension details with Flat-Bottom Holes of different depths and sizes.}
% \phantomsection\addcontentsline{lof}{table}{5.3\quad Experimental set-up for LN$_2$ cooling}
% \phantomsection\addcontentsline{lof}{table}{5.4\quad One example of ROC analysis (LN$_2$ results) and binary map of defect locations}
% \phantomsection\addcontentsline{lof}{table}{5.5\quad Thermal Raw Images of PT and LN$_2$ stimulation techniques}
% \phantomsection\addcontentsline{lof}{table}{5.6\quad FFT in amplitude and Phase results for LIT}
% \phantomsection\addcontentsline{lof}{table}{5.7\quad PCT results of corresponding technique}
% \phantomsection\addcontentsline{lof}{table}{5.8\quad ROC curves obtained from above results}

% \includepdf[pages=-]{Thermosense2017_Lei}
\newpage
\section{Introduction}
\label{sect:intro}  % \label{} allows reference to this section
In the Nondestructive Testing \& Evaluation (NDT \& E) field, active InfraRed (IR) thermography (\citet{Maldague2001theory}) is a technique widely used in assessing the conditions of parts of material components, with an extremely broad range of applications (\citet{Vavilov2017Thermal, Cadelano2016Corrosion}). Traditionally, Pulsed Thermography (PT) deploys a thermal stimulation pulse (flash or lamp heating) to produce a thermal contrast between the feature of interest and the background, then monitors the time evolution of the surface temperature by a thermal camera. With this rapidity and convenience, numerous studies have been devoted to this technique, that is now a standard procedure for thermal testing (\citet{Maldague1993Nondestructive,Maldague1994bInfra,2007-Ibarra-Castanedo,2011-ClementeIbarra-Castanedo,duan2013quantitative,Vavilov2015Review}).
%\added [id=GF] {that is now a standard procedure for thermal testing.}

However, if the temperature of the material to inspect is already higher than the ambient temperature, it can be of interest to make use of a cold thermal source such as a line of air jets (or water jets; sudden contact with ice, snow, etc.). In fact, a thermal front propagates the same way whether being hot or cold: what is important is the temperature differential between the thermal source and the specimen. An advantage of a cold thermal source is that it does not induce spurious thermal reflections into the IR camera as in the case of a hot thermal source. The main limitations of cold stimulation sources are related to practical considerations, as it is generally easier and more efficient to heat rather then to cool a part.
% \replaced [id=GF] {, as it is generally easier and more efficient to heat rather then to cool a part.} {as for instance it is generally easier and more efficient, to heat rather then to cool a part.}
Thus, the advantage and convenience of using the cold stimulation in active infrared thermography  still remains to be investigated in detail and better understood.

Using a cold source could extend the range of application of infrared thermography to cases where the specimen under analysis could not be heated, due to safety issues or physical reasons~(\citet{Livingston2018High}). A cold source could be required while handling biological or organic materials   that could not withstand a temperature increase, such as food. Another field of investigation is the survey of concrete and composite materials in order to find cracks and delaminations due to the presence of water or ice. Also in this case, a cold source would significantly decrease the risk of altering the physical characteristics of the specimen.
% \added [id=GF] {Using a cold source could extend the range of application of infrared thermography to cases where the specimen under analysis could not be heated, due to safety issues or physical reasons. A cold source could be required while handling biological or organic materials that could not withstand a temperature increase, such as food. Another field of investigation is the survey of concrete and composite materials in order to find cracks and delaminations due to the presence of water or ice. Also in this case, a cold source would significantly decrease the risk of altering the physical characteristics of the specimen}

Nonetheless,in the past in the scientific community, only a limited number of studies investigating a cold approach (cooling as the external stimulation in active infrared thermography) have been performed on industrial product inspection~(\citet{endohdynamical2012,2012-LewisHom}). A study by Burleigh~(\citet{Burleigh1989Thermographic}) showed that the cooling method with refrigerating liquids is feasible but requires caution to ensure the safety.
% \added [id=GF] {A study by Burleigh(\citet{1989Burleigh} showed that the cooling method with refrigerating liquids is feasible but requires caution to ensure the safety.}
The work of (\citet{lei2017detection}) chose instead to use air cooling to survey refrigerated vehicles. All the available works do not give a quantitative information about the reliability of the cooling technique, especially in comparison with the traditional heating procedure.
% \replaced [id=GF] {chose instead to use air cooling to survey refrigerated vehicles.} {demonstrates an example that air cooling approach applied in detection of the truck panel.}
% \added [id=GF] {All the available works do not give a quantitative information about the reliability of the cooling technique, especially in comparison with the traditional heating procedure.}

Therefore, the aim of this paper is improving the current knowledge on cooling thermal stimulation
% \replaced [id=GF] {improving the current knowledge on cooling thermal stimulation} {then the continuous investigation of cold approach}
in IR thermography. In order to perform a reliable comparison, three methods (two traditional techniques, Pulsed Thermography and Lock-in Thermography act as the reference) will be applied on a steel slab with different sizes of flat-bottom-holes. The thermographic images of the experiments will be  treated to  eventually produce a binary map of the location of the defects. This map will be statistically evaluated in terms of sensitivity and specificity~(\citet{Fawcett2006}) by comparison with the `true' map of the defects, furnishing a rank of the three stimulation methods.

\section{Experimental setup} % (fold)
\label{sec:experimental_setup}
One side stimulation approach is often used in the Infrared Thermography NDT \& E field, which is also known as the reflection scheme: both the stimulation device and the camera stay on the same side of the sample being tested. This approach applied in reality is shown in Figure~\ref{Exp_setup}.

\begin{figure}[ht]
   % \centering
   % \begin{center}
   \hspace{-0.95cm}
   \begin{tabular}{c}
   \includegraphics[scale=0.95]{chp5/Exp_setup.png}
   \\
   \footnotesize{(a) Pulsed Thermography set-up} \hspace{4cm} \footnotesize{(b) Lock-in Thermography set-up}
   \end{tabular}
   % \end{center}
   \caption{Experimental set-up in the \textit{reflection} mode}
   \label{Exp_setup}
\end{figure}

% \begin{figure}[ht]
%    \centering
%    \subfloat[Pulsed Thermography set-up]
%    {
%       \includegraphics[scale=0.3]{chp5/Flash_Setup.png}
%    }
%    %\hspace{5pt}
%    \subfloat[Lock-in Thermography set-up]
%    {
%       \includegraphics[scale=0.3]{chp5/LIT_setup.png}
%    }
%    \caption{Experimental set-up in the \textit{reflection} mode}
%    \label{Exp_setup}
% \end{figure}

The following equipment was set up for this study:
\begin{itemize}
   \item Infrared Camera FLIR SC3000 (spatial resolution equal to 320$\times$240 pixels, frame rate up to 50Hz, GaAs sensor, spectral range 8-9 $\mu m$)
%    \replaced [id=GF] {spatial resolution equal to 320$\times$240 pixels, framerate up to 50Hz, GaAs sensor, spectral range 8-9 $\mu m$)}{320$\times$240 pixels, framerate 50Hz, GaAs , 8-9 $\mu m$)}
   \item Two pairs of flash
%    \replaced [id=GF] {}{halogen}
    lamps for a total of 10 kJ (electric) released in 5 $ms$
   \item One pair of modulated halogen lamps with 1kW each served as Lock-in stimulation
   \item An isolated bottle (500 ml)
%   ??? \added [id=LL] {need to be rechecked, thanks Giovanni~}
filled with
%    \replaced [id=GF] {(500 ml???) filled with}{full of}
   Liquid Nitrogen.
\end{itemize}

\subsection{Specimen} % (fold)
\label{sub:specimen}
In this study, a steel specimen comprising flat-bottom holes of different depths and sizes will be examined. Their dimensions are depicted in Figure~\ref{specimen}.
   \begin{figure}[ht]
   \centering
   % \begin{tabular}{c} %% tabular useful for creating an array of images
   \includegraphics[scale=0.4]{chp5/specimen_schema.pdf}
   % \end{tabular}
   \caption{Steel sample dimension details with Flat-Bottom Holes of different depths and sizes (mirror image)}
%    \added [id=LL] {Mirror image}
%>>>> use \label inside caption to get Fig. number with \ref{}
    \label{specimen}
   \end{figure}

The steel plate has seventeen holes whose diameters vary from 0.4 $cm$  to 3 $cm$, and whose depths (from bottom) vary from 0.3 $cm$ to 0.9 $cm$. The entire thickness is 1 $cm$. This specimen is painted before the test, in order to increase its emissivity and to obtain a homogeneous external stimulation.

% subsection specimen (end)
\subsection{Stimulation Techniques} % (fold)
\label{sub:stimulation_techniques}
Three external stimulations were deployed on the sample, for the sake of  obtaining a good comparison in results:
\begin{itemize}
   \item Pulse Thermography (PT)
   \item Lock-in Thermography (LIT)
   \item Liquid Nitrogen cooling (LN$_2$)
\end{itemize}
Known as the traditional and fast technique in NDT \& E, Pulse Thermography (PT) acts as the reference during this test. One reason for this popularity is the quickness of the inspection relying on a thermal  stimulation pulse, with duration going from a few ms for high thermal conductivity material inspection (such as metal specimen in this study) to a few seconds for low thermal conductivity specimens.  Such quick thermal stimulation allows direct deployment on the plant floor with convenient heating sources.

% When neglecting heat exchange with the environment, the pulse of energy $Q$, delivered on a layer of thickness $L$, characterized by a density $\rho$, a specific heat $C_p$ and a thermal conductivity $\lambda$ (or a thermal diffusivity $\alpha$) produces a temperature increment behavior on the heated surface given by:
% \begin{equation}
%    T(t) = \frac{Q}{\rho C_p L}[1+2\sum_{n=1}^{\infty} e^{-\frac{n^2 \pi ^2\alpha t}{L^2}}]
%    \label{eq_pt}
% \end{equation}

Lock-in thermography (LIT) is based on thermal waves generated inside the inspected specimen and detected remotely. Wave generation is for instance performed by periodically depositing heat on the specimen surface (e.g. through sine-modulated lamp heating) while the resulting oscillating temperature field in the stationary regime is remotely recorded through its thermal infrared emission.

In this study, 3 different frequencies of 0.0625~$Hz$ (LIT16--one period is equivalent to 16 seconds), 0.125~$Hz$ (LIT8--one period is equivalent to 8 seconds) and 0.25~$Hz$ (LIT4--one period is equivalent to 4 seconds) are performed in Lock-in Thermography.

% By the convolution integral, Eq~(\ref{eq_pt}) becomes:
% \begin{equation}
%    T(t) = \frac{W}{\lambda}\frac{\alpha}{L}\int_0^t d\tau \Big(1+\sin(\omega \tau - \frac{\pi}{2})\Big)\Big\{1+2\sum_{n=1}^{\infty} e^{-\frac{n^2 \pi ^2\alpha(t-\tau)}{L^2}}\Big\}
% \end{equation}
% where $W$ is the absorbed heating power.

The Liquid Nitrogen is applied in the test by means of pouring it out directly onto the surface in order to cool the sample. LN$_2$ was sprinkled onto the specimen center and allowed to spread out towards the edges. The whole capture duration is 500 frames with 50~$Hz$ of image frequency, ie. 10 seconds of recording. The pouring time (cooling time) is about 2 seconds, due to the bottle with a volume of 500 ml, with 100 frames in the results sequence. This way of cooling has a limit of that the central part will be first and mainly cooled, then the cold front propagates from center to the edges. This can lead to a drawback that in the final thermograms defects closer to the edges will be difficulty to detected. The experimental set-up is shown in Figure~\ref{Exp_LN2}.

\begin{figure}[ht]
   \centering
   \includegraphics[scale=0.4]{chp5/LN2_setup.png}
   \caption{Experimental set-up for LN$_2$ cooling}
   \label{Exp_LN2}
\end{figure}

% subsection stimulation_techniques (end)

% section experimental_setup (end)

\section{Processing Methods} % (fold)
\label{sec:processing_methods}
The following image-processing techniques and data-analysis methods were employed for this study:
\begin{itemize}
   \item Principal Component Thermography (PCT)
   \item Phase and Amplitude images by Fast Fourier Transform (FFT)
   \item Receiver Operating Characteristic curves (ROC Curves)
\end{itemize}

\subsection{Principal Component Thermography (PCT)}
The Principal Component Thermography technique~(\citet{Rajic2002}) uses ``singular value decomposition (SVD) to reduce the matrix of observations to a highly compact statistical representation of the spatial and temporal variations relating to contrast information associated with underlying structural flaws".

\subsection{FFT in Phase and Amplitude for LIT}
In addition to the common technique for NDT \& E, Fast Fourier Transform in LIT~(\citet{wu1998lock}) is also one of the most applied techniques in IR Thermography, which is based on the periodic heating of the object being tested. A thermal wave is likewise generated and propagates inside the material. In real experimental cases the thermal wave is composed by a principal frequency and several harmonics where the amplitude of the Fast Fourier Transform is a function of frequency. By selecting the component with the highest amplitude it is possible to produce a phase map at the corresponding frequency where the defect appears enhanced.

\subsection{ROC Curve analysis} % (fold)
\label{sub:roc_curve_analysis}
The Receiver operating characteristic (ROC) curve is a technique in statistics which helps visualize, organize and select classifiers based on their performance. The curve graph is created by plotting the the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. More details about concepts and definitions can be found in (\citet{Fawcett2006}). While being frequently chosen a standard method in several scientific fields, ROC are rarely applied in the thermographic field~(\citet{Bison2014a}).

Implemented in this study, a binary map of the defect locations is built and correlated to the post-processed images in gray scale, and the lay-out is provided in Figure~\ref{binary}.
% \deleted [id=LL]{, and the lay-out is provided in Figure~\ref{binary}}.
The main algorithm in the calculation of TPR and FPR is clarified as:
\begin{enumerate}
   \item Choose the post-processed thermogram in which most defects shown as the test image;
%    \replaced [id=LL] {Choose the post-processed thermogram in which most defects shown as the test image;}{Identify the best gray-scale result from the post-processing thermal images as the test image;}
   \item Resize the defect map to the same size as that of the test image;
   \item Choose a thresholding step number $N$($N=1000$ in this study) and establish the step value of thresholding [from $0$, $\frac{1}{N}$, $\frac{2}{N}$ till $1 (=\frac{N}{N})$];
   \item For each thresholding step, binarize the test image with the thresholding and then compare it to the defect map, in order to obtain the corresponding TPR and FPR values;
   \item Iterate the binarization and comparison so as to plot a whole curve.
\end{enumerate}

% \begin{figure}[ht]
%    \begin{center}
%    \begin{tabular}{c}
%    \includegraphics[scale=1.0]{ROC_exm.png}
%    \\\footnotesize{(a) Binary map of defects} \hspace{4cm} \footnotesize{(b) Cool map}
%    \end{tabular}
%    \end{center}
%    \caption{One example of ROC analysis (LN$_2$ results) and binary map of defect locations}
%    \label{binary}
% \end{figure}

\begin{figure}[ht]
   \centering
   \subfloat[Binary map of defects]
   {
      \includegraphics[scale=0.195]{chp5/Schema_done.png}
      \label{bin_map}
   }
   \subfloat[Cool map]
   {
      \includegraphics[scale=0.8]{chp5/Cool_ROC.png}
      \label{cool_map}
   }
   \caption{One example of ROC analysis (LN$_2$ results) and binary map of defect locations}
   \label{binary}
\end{figure}

%% subsection roc_curve (end)

% section methods (end)
\section{Results \& Discussion} % (fold)
\label{sec:results_&_discussion}
Figure~\ref{raw_results} illustrates the thermal raw images of PT and LN$_2$. The Lock-In FFT results can be found in Figure~\ref{LIT_results}.

% \begin{figure}[ht]
%    \begin{center}
%    % \begin{tabular}{c}
%    \includegraphics[scale=0.60]{chp5/Raw_results.png}
%    % \\\footnotesize{(a) Binary map of defects} \hspace{4cm} \footnotesize{(b) Cool map}
%    % \end{tabular}
%    \end{center}
%    \caption{Thermal raw images of PT and LN$_2$ stimulation techniques}
%    \label{raw_results}
% \end{figure}

\begin{figure}[htpb]
   \centering
   \subfloat[Flash raw frame 23]
   {
      \includegraphics[scale=0.55]{chp5/flash_raw23_2.png}
      \label{Flash_raw23}
   }
   \hspace{10pt}
   \subfloat[Flash raw frame 60]
   {
      \includegraphics[scale=0.55]{chp5/flash_raw60_2.png}
      \label{Flash_raw60}
   }
   \hspace{10pt}
   \subfloat[LN$_2$ raw frame 41]
   {
      \includegraphics[scale=0.55]{chp5/cool_raw_2.png}
      \label{LN2_raw}
   }
   \caption{Thermal raw images of PT and LN$_2$ stimulation techniques}
   \label{raw_results}
\end{figure}

% \begin{figure}[ht]
%    \begin{center}
%    % \begin{tabular}{c}
%    \includegraphics[scale=0.6]{chp5/LIT_results.png}
%    % \\\footnotesize{(a) Binary map of defects} \hspace{4cm} \footnotesize{(b) Cool map}
%    % \end{tabular}
%    \end{center}
%    \caption{FFT in amplitude and phase results for LIT}
%    \label{LIT_results}
% \end{figure}

\begin{figure}[htpb]
   \centering
   \subfloat[LIT4 FFT in amplitude]
   {
      \includegraphics[scale=0.57]{chp5/LIT4_AMP.png}
   }
   \hspace{10pt}
   \subfloat[LIT4 FFT in phase]
   {
      \includegraphics[scale=0.57]{chp5/LIT4_PHA.png}
   }
   \hspace{10pt}
   \subfloat[LIT8 FFT in amplitude]
   {
      \includegraphics[scale=0.57]{chp5/LIT8_AMP.png}
   }
   \hspace{10pt}
   \subfloat[LIT8 FFT in phase]
   {
      \includegraphics[scale=0.57]{chp5/LIT8_PHA.png}
      \label{LIT8_ph}
   }
   \hspace{10pt}
   \subfloat[LIT16 FFT in amplitude]
   {
      \includegraphics[scale=0.57]{chp5/LIT16_AMP.png}
      \label{LIT16_ph}
   }
   \hspace{10pt}
   \subfloat[LIT16 FFT in phase]
   {
      \includegraphics[scale=0.57]{chp5/LIT16_PHA.png}
   }
   \caption{FFT in amplitude and phase results for LIT}
   \label{LIT_results}
\end{figure}

\subsection{Thermal images comparison}

From the results above, raw images in Figure~\ref{raw_results} indicate that the most detectable flaws are the ones with a high aspect ratio (ie. diameter-to-depth). In addition, for the PT stimulation, a small hole with diameter 0.4 $cm$, depth\footnote{It should be noted that the depth values mentioned here and after are from the bottom of the sample, therefore the real corresponding depths should be these values subtracted from the thickness.} 0.9 $cm$ (left-upper in Figure~\ref{Flash_raw23}) appeared in frame 23, and disappeared after. Other holes, two with diameter 2 $cm$ and depth 0.9 $cm$, and two with diameter 3 $cm$ and depth 0.5 $cm$,
% \replaced [id=GF]{. Other holes, two with diameter 2 $cm$ and depth 0.9 $cm$, and two with diameter 3 $cm$ and depth 0.5 $cm$,} {, while another two holes with diameter 2 $cm$, depth 0.9 $cm$ and two with diameter 3 $cm$, depth 0.5 $cm$}
(center in Figure~\ref{Flash_raw60}) appeared in frame 60.  Nonetheless, in LN$_2$ raw results, same defects as the one in Flash frame 60 appeared in LN$_2$ frame 41 (however, it should be noted that the first 100 frames of cooling time has been removed from the these final raw images. The reason is because there was much noise from nitrogen gases in the thermal images).
% \added [id=LL] {The reason is because there was much noise from nitrogent gases in the thermal images})
After this, there was no more defect that emerged. Another remark is that since the LN$_2$ is sprinkled onto the center of the surface, there could be the situation that the center part of specimen was over cooled while the `cold front' (opposite to heat front) might have not be able to propagate to the edges. Due to the high conductivity of steel, defects only showed up at the beginning
% \replaced [id=LL] {end} {beginning}
of the cooling procedure. These may be the main issues of pouring-out method.

For LIT results, FFT in amplitude has a better flaw detection capability than FFT in phase, as there is some noise in all of the phase images. LIT8 and LIT16 have about four more detected flaws than LIT4. Whereas, for FFT in phase for LIT8 and LIT16, there some inverse gray-scale values occur. This might be due to the reverse image question.

Following PCT processing, figure~\ref{PCT_results} exhibits a clearer result. It can be observed that most of the flaws are visible, especially in the flash image (Figure~\ref{PCT_Flash}). Less flaws are visible in the LIT4 PCT third image.
%The corresponding PCT results are represented in Figure~\ref{PCT_results}.
% \begin{figure}[ht]
%    \begin{center}
%    % \begin{tabular}{c}
%    \includegraphics[scale=0.6]{chp5/PCT_results.png}
%    % \\\footnotesize{(a) Binary map of defects} \hspace{4cm} \footnotesize{(b) Cool map}
%    % \end{tabular}
%    \end{center}
%    \caption{PCT results of corresponding technique}
%    \label{PCT_results}
% \end{figure}

\begin{figure}[htpb]
    \centering
    \subfloat[Flash PCT 2nd Image]{
      \includegraphics[scale=0.57]{chp5/Flash_PCT_2.png}
      \label{PCT_Flash}
      }
    \hspace{10pt}
    \subfloat[LN$_2$ PCT 2nd Image]{
      \includegraphics[scale=0.57]{chp5/Cool_PCT_2.png}
      }
    \hspace{10pt}
    \subfloat[LIT4 PCT 3rd Image]{
      \includegraphics[scale=0.57]{chp5/LIT4_PCT_3.png}
      }
    \\ %\hspace{10pt}
    \subfloat[LIT8 PCT 3rd Image]{
      \includegraphics[scale=0.57]{chp5/LIT8_PCT_3.png}
      }
    \hspace{10pt}
    \subfloat[LIT16 PCT 3rd Image]{
      \includegraphics[scale=0.57]{chp5/LIT16_PCT_3.png}
      }
    % \includegraphics[scale=0.4]{graph/LIT4_PCT_3.png}
    % \includegraphics[scale=0.4]{graph/LIT8_PCT_3.png}
    % \includegraphics[scale=0.4]{graph/LIT16_PCT_3.png}
    \caption{PCT results of corresponding technique}
    \label{PCT_results}
\end{figure}

Comparing the processed thermal images, the following observations can be made:
\begin{itemize}
    \item All techniques present part of the flaws in the sample;
    \item The PCT post-processing method displays a better results for all images;
    \item More defects are exhibited in Flash stimulation with PCT processing;
    %\item For LIT8 and LIT16, there are some reverse image question.
\end{itemize}

\subsection{Corresponding ROC curves comparison}
The ROC curves obtained from comparing the binary map of defect locations to the above results are represented in Figure \ref{roc_pct} and ~\ref{ROC_curve}.
\begin{figure}[htbp]
   \centering
   \includegraphics[scale=0.7]{chp5/ROC_PCT_2017.pdf}
   \caption{ROC curves for PCT processing results}
   \label{roc_pct}
\end{figure}

% \begin{figure}[htbp]
%    % \centering
%    \begin{center}
%    \begin{tabular}{c}
%    \includegraphics[scale=0.75]{chp5/ROC_LIT_results.png}
%    \\\footnotesize{(a) ROC from LIT FFT in amplitude results} \hspace{2.1cm} \footnotesize{(b)  ROC from LIT FFT in phase results}
%    \end{tabular}
%    \end{center}
%    \caption{ROC curves for LIT processing results}
%    \label{roc_lit}
% \end{figure}

\begin{figure}[ht]
    \centering
    % \subfloat[ROC from PCT Results]
    % {
    % \includegraphics[scale=0.75]{chp5/ROC_PCT_2017.pdf}
    % }
    % %\hspace{10pt}
    % \\
    \subfloat[ROC from LIT FFT in amplitude results]
    {
    \includegraphics[scale=0.70]{chp5/ROC_LIT_AMP_2017.pdf}
    \label{roc_amp}
    }
    \hspace{10pt}
    \subfloat[ROC from LIT FFT in phase results]
    {
    \includegraphics[scale=0.70]{chp5/ROC_LIT_PHA_2017.pdf}
    \label{roc_pha}
    }
    \caption{ROC curves for LIT processing results}
    \label{ROC_curve}
\end{figure}

%\added[id=GF] {Before you called the true positive rate TPR (and the false FPR). From here you call them tp rate and fp rate, maybe it is better to choose the same nomenclature }

In the ROC curves definitions, the \textit{sensitivity} is the True Positive Rate (\textit{TPR}), and \textit{1-specificity} is the False Positive Rate (\textit{FPR}).
From the curves of the PCT results (Figure~\ref{roc_pct}), one can easily notice that the five curves have almost the same performance of classification in the beginning. When the \textit{TPR} arrives at $0.3$, the PT curve becomes the nearest to the northwest (where the \textit{TPR} is higher, the \textit{FPR} is lower or both). The second one is the LN$_2$ curve that has a slightly higher \textit{TPR} than PT after the \textit{FPR} reaches $0.4$. The LIT8 and LIT16 curves have almost the same performance before the \textit{TPR} attains $0.7$. After that the LIT16 has a higher \textit{TPR} than LIT8. The worst one is the LIT4 curve as it is the closest one to the diagonal line $y=x$, which represents the strategy of randomly guessing a class.

The situation is the same for the curves of LIT amplitude (Figure~\ref{roc_amp}), in which LIT4 has an unfavorable performance in classification. LIT16 also has a slightly higher \textit{TPR} than LIT8. However, in the curves of LIT phase results (Figure~\ref{roc_pha}), LIT16 shows a poor performance while LIT4 displays a better classification
% \added[id=GF] {while LIT4 displays a better classification}
, contrary to the former results.
% \replaced[id=GF] {} {While LIT4 displays a better classification.}
Because of the reverse image problem, the result LIT8 presents a region (top-right) which is under the equal diagonal line.

\subsection{Area under curve analysis and comparison}
In ROC analysis, a common method to compare different classifiers is to calculate the area under the ROC curve, known as AUC~(\citet{Fawcett2006}). In this way, a single scalar value will represent the expected performance of the classifiers. As in the ROC curve profile, the AUC is a part of the unit square are, therefore, its value will be between 0 and 1. Thus, random guessing classifier produces a diagonal line between (0,0) and (1,1), which has an area of 0.5. It can be then indicated that a classifier AUC value less than 0.5 is even worse than a random guessing.

The corresponding AUC obtained from the above ROC curves is found in Tab. \ref{tab_auc}. Their comparison in form of the bar chart is illustrated in Fig. \ref{fig_auc}. A straight comparison shows that the Flash method with PCT second component has the highest AUC value of 0.9. The second one is the Liquid Nitrogen method, that is very close to the flash method with an AUC value of 0.87. At the bottom of the ranking, LIT4 with PCT third component and its amplitude one have the worst values as  0.67 and 0.65 respectively, which are just a bit better and a random guessing.
On the contrary, LIT8 and LIT16 technique have a better result both in PCT processing and their amplitude images. This might be caused by the fact that during one period, 4 seconds of heating (LIT4) were not enough to penetrate the steel specimen deeply as those of 8 seconds (LIT8) and 16 seconds (LIT16). On the other hand, all the Lock-in Thermography techniques have similar AUC value in their phase images, which indicates that for phase image results, heating time does not make an effective influence in Lock-in Thermography in this study. Considering all these results, though the best approach is Flash technique with PCT processing, the LN$_2$ technique can be considered as a valid alternative method

% Figure AUC to be added.
% Table to be added.
\begin{table}[htbp]
    \centering
    \begin{tabular}{lr}%ccccccccc}
    \toprule
    \textbf{Method} & \textbf{Value} \\
    \midrule
    Flash-2nd & 0.90 \\
    LN2-2nd & 0.87 \\
    \midrule
    LIT4-3rd & 0.67 \\
    LIT8-3rd & 0.80 \\
    LIT16-3rd & 0.82 \\
    \midrule
    LIT4-AMP & 0.65 \\
    LIT8-AMP & 0.79 \\
    LIT16-AMP & 0.82 \\
    \midrule
    LIT4-PHA & 0.71 \\
    LIT8-PHA & 0.69 \\
    LIT16-PHA & 0.67 \\

%     0.90 & 0.87 & 0.67 & 0.80 & 0.82 & 0.65 & 0.79 & 0.82 & 0.71 & 0.69 & 0.67 \\
    \bottomrule
    \end{tabular}
    \caption{AUC value comparison}
    \label{tab_auc}
\end{table}

% $x = ['Flash_2nd', 'LN2_2nd', 'LIT4_3rd', 'LIT8_3rd', 'LIT16_3rd',
%      'LIT4_AMP', 'LIT8_AMP', 'LIT16_AMP', 'LIT4_PHA', 'LIT8_PHA', 'LIT16_PHA']\\
% y = np.array([0.90, 0.87, 0.67, 0.80, 0.82,
%               0.65, 0.79, 0.82, 0.71, 0.69, 0.67])$

\begin{figure}[htpb]
   \centering
   \includegraphics[scale=0.48]{chp5/AUC.png}
   \caption{AUC value comparison}
   \label{fig_auc}
\end{figure}

% section results_&_discussion (end)

\section{Conclusion} % (fold)
\label{sec:conclusion}
This study investigates an external stimulation--cooling instead of heating in IR Thermography for NDT \& E.
A steel specimen is used to test three different stimulations for thermal images and also ROC analysis comparison.
Results shows that all techniques highlight part of the flaws in the sample, whereas the LN$_2$ technique represents the defects at the end of cooling process; this maybe due to the high conductivity of steel.
In thermal results, the PCT post-processing method displays a better results for all procedures. More defects are exhibited in Flash stimulation with PCT processing.
ROC curve analysis and its AUC analysis have elucidated a straightforward classification comparison, in which the best values are obtained with the Flash technique with PCT processing, trailed narrowly by the Liquid Nitrogen method.

The LN$_2$ technique should therefore be considered as a valid option for the survey of objects that should not be heated, such as biological tissues, organic materials, dry or iced samples. With this purpose, future work will analyze different kinds of specimens.
% \replaced[id=GF] {The LN$_2$ technique should therefore be considered as a valid technique for the survey of objects that should not be heated, such as biological tissues, organic materials, dry or iced samples. With this purpose, future work will analyze different kinds of specimens.} {In future work, other common composite materials such as CFRP (Carbon fiber reinforced polymer), GFRP (Glass Fiber Reinforced Polymer) will be chosen as specimen.}
The method of Liquid Nitrogen pouring may be replaced by spraying onto the sample surface, which can reduce the inhomogeneous cooling problem. To enhance the penetration of heat inside the sample, a proposition involving heating on one side of the specimen and cooling the other side might be taken into consideration.
% \replaced[id=GF] {} {Even though LN$_2$ technique in this study has not shown enough advantages,}
% \replaced[id=GF] {The} {this}
The exploration of an opposite means of external stimulation in InfraRed Thermography might favor new ideas
% \replaced[id=GF] {} {of approaches}
for NDT \& E.

\section*{Acknowledgments}

This research was supported by the governments of Italy and Quebec (Minist\`{e}re des Relations internationales et de la Francophonie) through the Joint Subcommittee Qu\'{e}bec-Italy, project n$^{\circ}$08.0203. It was also supported by  the Natural Sciences and Engineering Research Council of Canada (NSERC). We are also thankful to our collaborative institute CNR-ITC Padova which provided expertise that greatly helped in this research.

\bibliographystyle{unsrtnat}              % style de la bibliographie
\bibliography{U:/Desktop/Bibliography/Biblio_th}


\subsection{Partially observable stochastic games}

Generalisation of reinforcement learning.


\documentclass[11pt, oneside]{article}

% Packages
\usepackage{jmlr2e}
\usepackage{xcolor}
\usepackage[super]{nth}
\usepackage{mathtools, amsfonts}

% Package config
% \sectionfont{\normalfont\sffamily\bfseries}
% \subsectionfont{\normalfont\sffamily\bfseries}
\hypersetup{colorlinks,
    linkcolor={red!50!black},
    citecolor={blue!50!black},
    urlcolor={blue!80!black}
}

% Notation and macros
\input{notation.tex}

% Preamble

% Heading arguments are
% {volume}{year}{pages}{submitted}{published}{author-full-names}
% \jmlrheading{}{}{}{}{}{}

% Short headings should be running head and authors last names
\ShortHeadings{\revrand~Technical Report}{Steinberg, Tiao, Reid, McCalman and
    O'Callaghan}
\firstpageno{1}

\title{\revrand: Technical Report}
\author{\name Daniel Steinberg \email daniel.steinberg@data61.csiro.au \\
        \name Louis Tiao \email louis.tiao@data61.csiro.au \\
        \name Alistair Reid \email alistair.reid@data61.csiro.au \\
        \name Lachlan McCalman \email lachlan.mccalman@data61.csiro.au \\
        \name Simon O'Callaghan \email simon.ocallaghan@data61.csiro.au \\
        \addr DATA61, CSIRO \\
        Sydney, Australia}

\date{}

\begin{document}

\maketitle
% \vspace{-0.5cm}
% \noindent\makebox[\linewidth]{\rule{\linewidth}{0.8pt}}
% \vspace{0.3cm}

\begin{abstract}
    This is a technical report on the \revrand software library. This
    library implements Bayesian linear models (Bayesian linear regression),
    generalized linear models and approximate Gaussian processes. These
    algorithms have been implemented such that they can be used for large-scale
    learning by using stochastic variational inference. All of the algorithms
    in \revrand use a unified feature composition framework that allows
    for easy concatenation and selective application of regression basis
    functions.
\end{abstract}

\tableofcontents

\section{Core Algorithms}

Recent developments in stochastic gradient optimisation have simplified the
application of machine learning to massive datasets, in particular, modern
stochastic optimisation algorithms are far more robust to initial learning rate
settings. Furthermore, Bayesian machine learning algorithms have a number of
well defined methods for learning model parameters and \emph{hyperparameters}
from training data, that do not involve cross validation. When used in
combination, stochastically optimized Bayesian machine learning algorithms
allow practitioners to learn probabilistic predictors from large data sets with
minimal tuning and retraining.

We make use of some of these recent developments in stochastic gradient methods
and stochastic variational inference in \revrand for supervisd regression
tasks. We outline the core algorithms implemented in \revrand in this section.

\subsection{Stochastic Gradients and Variational Objective Functions}
\label{sub:stochvar}

When a machine learning objective function factorises over data,
\begin{equation}
    \ffunc{\ins, \param}{} = \sum_{\inss \in \ins} \ffunc{\inss, \param}{},
\end{equation}
a regular gradient descent would perform the following iterations to minimise
the function w.r.t.\ the parameters $\param$,
\begin{equation}
    \param_k := \param_{k-1} - \lrate_k \sum_{\inss \in \ins}
    \nabla_{\param} \ffunc{\inss, \param}{}\!|_{\param = \param_{k-1}},
\end{equation}
where $\lrate_k$ is the learning rate (step size) at iteration $k$. Stochastic
gradients proposes the following update,
\begin{equation}
    \param_k := \param_{k-1} - \lrate_k \sum_{\inss \in \minibatch}
    \nabla_{\param} \ffunc{\inss, \param}{}\!|_{\param = \param_{k-1}},
    \label{eq:sg}
\end{equation}
where $\minibatch \subset \ins$ is a mini-batch of the original dataset, where
$|\minibatch| \ll |\ins|$.
Frequently objective functions to not entirely decompose over the data, i.e.,
\begin{equation}
    \ffunc{\ins, \param}{} = \sum_{\inss \in \ins} \ffunc{\inss, \param}{}
    + \func{g}{\param}{}.
    \label{eq:objwconst}
\end{equation}
However, it is trivial to make these objectives work in a stochastic gradient
setting. Let $M = |\minibatch|$ and $N = |\ins|$, then we divide the
contribution of the constant term amongst the mini-batches in stochastic
gradients,
\begin{equation}
    \param_k := \param_{k-1} - \lrate_k \sum_{\inss \in \minibatch}
    \nabla_{\param} \ffunc{\inss, \param}{}\!|_{\param = \param_{k-1}}
    - \frac{M}{N} \lrate_k \nabla_{\param}
    \func{g}{\param}{}\!|_{\param = \param_{k-1}}.
    \label{eq:wsg}
\end{equation}
or, equivalently, boost the contribution of the mini-batch,
\begin{equation}
    \param_k := \param_{k-1} - \frac{N}{M} \lrate_k \sum_{\inss \in \minibatch}
    \nabla_{\param} \ffunc{\inss, \param}{}\!|_{\param = \param_{k-1}}
    - \lrate_k \nabla_{\param}
    \func{g}{\param}{}\!|_{\param = \param_{k-1}}.
    \label{eq:wsg2}
\end{equation}
This is particularly relevant for variational inference where the evidence
lower bound objective has a component independent of the data. For example,
lets consider the model,
\begin{align}
    \text{Likelihood:} \quad &\prod^N_{n=1} \probC{\targs_n}{\param}, \\
    \text{prior:} \quad &\probC{\param}{\hyper},
\end{align}
where we want to learn the values of the hyper-parameters, $\hyper$. Minimising
negative log-marginal likelihood is a good objective in this instance, since we
don't care about the value(s) of $\param$,
\begin{equation}
    \argmin_\hyper - \log \int \prod^N_{n=1} \probC{\targs_n}{\param}
    \probC{\param}{\hyper} d \param.
    \label{eq:lml}
\end{equation}
There are two problems with this objective however, (1) it may not factor over
data and (2) the integral may be intractable, for instance, if the prior and
likelihood are not conjugate. In variational inference we use Jensen's
inequality to lower-bound log-marginal likelihood with a tractable objective
function called the evidence lower bound (ELBO),
\begin{align}
    \log \probC{\targ}{\hyper} =& \log \int
        \prod^N_{n=1} \probC{\targs_n}{\param}
        \probC{\param}{\hyper} d \param \nonumber \\
        =& \log \int
        \frac{\prod_n \probC{\targs_n}{\param} \probC{\param}{\hyper}}
        {\qrob{\param}} \qrob{\param} d \param \nonumber \\
        \geq& \int \qrob{\param} \log \sbrac{%
            \frac{\prod_n \probC{\targs_n}{\param}
            \probC{\param}{\hyper}}{\qrob{\param}}}
        d \param
\end{align}
where $\qrob{\param}$ is an approximation of $\probC{\param}{\hyper}$ that
makes inference easier. This can be re-written as,
\begin{equation}
    \elbo = \sum^N_{n=1} \expec{q}{\log\probC{\targs_n}{\param}} -
    \KL{\qrob{\param}}{\probC{\param}{\hyper}},
    \label{eq:elbo}
\end{equation}
which takes the form of Equation~\eqref{eq:objwconst}, and so if we use
stochastic gradients optimisation we can weight the Kullback-Leibler term like
the constant term, $\func{g}{\cdot}{}$, from Equation~\eqref{eq:wsg}, or boost
the expected log likelihood term like in Equation~\eqref{eq:wsg2}.

\subsection{Bayesian Linear Regression -- \texttt{StandardLinearModel}}

The first machine learning algorithm in \revrand is a simple Bayesian linear
regressor of the following form,
\begin{align}
    \text{Likelihood:} \quad &\prod^N_{n=1}
    \gausC{\targs_n}{\feats_n\T\weights, \var}, \\
    \text{prior:} \quad &\gausC{\weights}{\mathbf{0}, \Reg},
\end{align}
where $\feats_n := \func{\featsym}{\inss_n, \param}{}$ is a feature, or basis,
function that maps $\featsym : \real{d} \to \real{D}$, and $\Reg \in
\real{D\times D}$ is a diagonal matrix (i.e.~it could be $\reg\ident{D}$) that
has the effect of regularising the magnitude of the weights. This is the same
algorithm described in~\citet[Chapter 2]{Rasmussen2006}. We then:
\begin{itemize}
    \item Optimise $\var, \Reg$ and $\param$ w.r.t.\ log-marginal likelihood,
        \begin{equation}
            \log \probC{\targ}{\var, \Reg, \param} =
            \log \gausC{\targ}{\mathbf{0}, \var\ident{N} + \feat\T\Reg\feat},
        \end{equation}
        where $\feat \in \real{N \times D}$ is the concatenation of all the
        features, $\feats_n$. Note this results in the covariance of the
        log-marginal likelihood being $N \times N$, though we can use the
        Woodbury identity to simplify the corresponding matrix inversion.
    \item Solve analytically for the posterior over weights, $\weights | \targ
        \sim \gaus{\pomean, \pocov}$ given the above hyperparameters, where,
        \begin{align*}
            \pocov &= \sbrac{\Reg\inv + \frac{1}{\var}
                \feat\T \feat}\inv, \\
            \pomean &= \frac{1}{\var} \pocov \feat\T \targ.
        \end{align*}
    \item Use the predictive distribution
        \begin{align}
            \probC{\targs\test}{\targ, \ins, \inss\test} &= \int
            \gausC{\targs\test}{\feats\testT\weights, \var}
            \gausC{\weights}{\pomean, \pocov} d\weights, \nonumber \\
            &= \gausC{\targs\test}{\feats\testT \pomean,
                \var + \feats\testT \pocov \feats\test}
        \end{align}
        for query inputs, $\inss\test$. This gives us the useful expectations,
        \begin{align}
            \expece{}{\targs\test} &= \feats\testT\pomean, \\
            \evar{}{\targs\test} &= \var + \feats\testT\pocov\feats\testT.
        \end{align}
\end{itemize}

It is actually easier to use the ELBO form with stochastic gradients for
learning the parameters of this algorithm, rather than log-marginal likelihood
recast using the Woodbury identity.  This is because it is plainly in the same
form as Equation \eqref{eq:objwconst}, though it would give the same result as
log-marginal likelihood because the ``approximate'' posterior is the same form
as the true posterior, i.e.\ $\qrob{\weights} = \gausC{\weights}{\pomean,
    \pocov}$. The ELBO for this model is,
\begin{equation}
    \elbo = \sum^N_{n=1}
    \expec{q}{\log\gausC{\targs_n}{\feats_n\T\weights, \var}}
    - \KL{\gausC{\weights}{\pomean, \pocov}}
        {\gausC{\weights}{\mathbf{0}, \Reg}}.
    \label{eq:slmobj}
\end{equation}
More specifically,
\begin{align*}
    \expec{q}{\log\gausC{\targs_n}{\feats_n\T\weights, \var}} &=
    \log \gausC{\targs_n}{\feats_n\T\pomean, \var}
    - \frac{1}{2 \var} \trace{\feats_n\T\feats_n\pocov}, \\
    \KL{\gausC{\weights}{\pomean, \pocov}}
        {\gausC{\weights}{\mathbf{0}, \Reg}} &=
        \frac{1}{2} \sbrac{\trace{\Reg\inv\pocov} + \pomean\T\Reg\inv\pomean
        -  \log\deter{\pocov} + \log\deter{\Reg} - D}
\end{align*}
We have not implemented a stochastic gradient version of this algorithm since
it still requires a matrix solve of a $D \times D$ matrix, and so is \bigo{D^3}
in complexity, per iteration. This is true even if we optimise the posterior
covariance directly (or a triangular parameterisation). The GLM presented in
the next section circumvents this issue, and is more suited to really large $N$
and $D$ problems.

\subsection{Bayesian Generalized Linear Models --
    \texttt{GeneralizedLinearModel}}

The algorithm of primary interest in \revrand is the Bayesian generalized
linear model. The general form of the model implemented by this algorithm is,
\begin{align}
    \text{Likelihood:} \quad &\prod^N_{n=1}
        \probC{\targs_n}{\activ{\feats_n\T\weights}, \lparam},
        \label{eq:glmlike} \\
    \text{prior:} \quad &\gausC{\weights}{\mathbf{0}, \Reg},
\end{align}
for an arbitrary univariate likelihood, $\prob{\cdot}$, with an appropriate
transformation (inverse link) function, $\activ{\cdot}$, and parameter(s),
$\lparam$.

Naturally, both calculating the exact posterior over the weights,
$\probC{\weights}{\targ, \ins}$, and the log-marginal likelihood,
$\prob{\targ}$, for hyperparameter learning are intractable since we may have a
non-conjugate relationship between the likelihood and prior. Therefore we must
resort to approximating the true posterior and the log-marginal likelihood.

Firstly, we approximate the true posterior over weights with a mixture of $K$
diagonal Gaussians,
\begin{align}
    \probC{\weights}{\targ, \ins} &\approx \qrob{\weights}, \nonumber \\
    &= \frac{1}{K} \sum^K_{k=1} \gausC{\weights}{\pomean_k, \dpocov_k},
\end{align}
where $\dpocov_k = \diag{[\dpocovs_{k,1}, \ldots, \dpocovs_{k, D}]\T}$, which
is inspired from similar approximations made in \citet{gershman2012,
    nguyen2014automated}. This is a very flexible form for the approximate
posterior, and has the nice property that our algorithm no longer has a
\bigo{D^3} cost associated with the number of features.

Then we approximate the log marginal likelihood using auto-encoding variational
Bayes \citep{kingma2014auto}. The exact lower bound on log marginal likelihood
is,
\begin{equation}
    \elbo = \sum^N_{n=1}
    \expec{q}{\log\probC{\targs_n}{\activ{\feats_n\T\weights}, \lparam}}
    - \KL{\textstyle\frac{1}{K}\textstyle\sum_{k}
            \gausC{\weights}{\pomean_k, \dpocov_k}}
        {\gausC{\weights}{\mathbf{0}, \Reg}}.
    \label{eq:glmobj}
\end{equation}
This can be expanded,
\begin{multline}
    \elbo = \frac{1}{K} \sum^K_{k=1} \sum^N_{n=1}
    \expec{q_k}{\log\probC{\targs_n}
        {\activ{\feats_n\T\weights}, \lparam}}
    + \frac{1}{K} \sum^K_{k=1}
        \expec{q_k}{\log\gausC{\weights}{\mathbf{0}, \Reg}} \\
    + \entropy{\textstyle\frac{1}{K}\textstyle\sum_{k}
            \gausC{\weights}{\pomean_k, \dpocov_k}},
    \label{eq:glmobj_exp}
\end{multline}
but unfortunately there are two intractable integrals here, the expected log
likelihood, and the entropy of the Gaussian mixture. We can use the lower bound
on the entropy term also used in \citet{gershman2012, nguyen2014automated},
\begin{equation}
    \entropy{\textstyle\frac{1}{K}\textstyle\sum_{k}
        \gausC{\weights}{\pomean_k, \dpocov_k}} \geq
    - \frac{1}{K} \sum_{k=1}^K \log \sum_{j=1}^K \frac{1}{K}
    \gausC{\pomean_k}{\pomean_j, \dpocov_k + \dpocov_j}.
\end{equation}
We can then use the reparameterisation trick in auto-encoding variational Bayes
\citep{kingma2014auto} to sample the expected log likelihood,
\begin{equation}
    \sum^N_{n=1}
    \expec{q_k}{\log\probC{\targs_n}
        {\activ{\feats_n\T\weights}, \lparam}} \approx
    \frac{1}{L} \sum^{L}_{l=1} \sum^N_{n=1} \log\probC{\targs_n}
    {\activ{\feats_n\T\hat{\weights}_k^{\brac{l}}}, \lparam}
\end{equation}
where,
\begin{equation}
    \hat{\weights}_k^{\brac{l}} =
    \func{f}{\pomean_k, \dpocov_k, \resamp^{(l)}}{k} =
    \pomean_k + \sqrt{\dpocov_k} \odot \resamp^{(l)},
    \qquad \resamp^{(l)} \sim \gaus{\mathbf{0}, \ident{D}}.
\end{equation}
Here $\odot$ is the element-wise product. We can also use this trick to compute
approximate derivatives, $\frac{\partial}{\partial \alpha}
\expec{q(\alpha)}{\log \probC{\targs}{\alpha}} \approx \frac{1}{L} \sum^L_{l=1}
\frac{\partial}{\partial\alpha} \log \probC{\targs}{\func{f}{\alpha,
        \resamp^{(l)}}{}}$, which simplifies the implementation greatly! The
final auto-encoding variational Bayes objective for our GLM is,
\begin{multline}
    \elbo \approx \frac{1}{K} \sum^K_{k=1} \Bigg[
    \frac{1}{L} \sum^L_{l=1} \sum^N_{n=1}
    \log\probC{\targs_n}
    {\activ{\feats_n\T \hat{\weights}_k^{\brac{l}}}, \lparam}
    + \log\gausC{\pomean_k}{\mathbf{0}, \Reg}
    - \frac{1}{2} \trace{\Reg\inv\dpocov_k} \\
    - \log \sum_{j=1}^K \frac{1}{K}
    \gausC{\pomean_k}{\pomean_j, \dpocov_k + \dpocov_j}
    \Bigg].
    \label{eq:glmobj_exp}
\end{multline}
We can straight forwardly use this objective within a stochastic gradients
setting using with the tactic in Equations \eqref{eq:wsg} or \eqref{eq:wsg2}.

The most simple and accurate method for approximating the predictive
distribution, $\probC{\targs\test}{\targ, \ins, \inss\test}$ is to Monte-Carlo
sample the integral,
\begin{equation}
    \probC{\targs\test}{\targ, \ins, \inss\test} \approx
    \int \probC{\targ}{\activ{\feats\testT\weights}, \lparam}
    \frac{1}{K} \sum^K_{k=1} \gausC{\weights}{\pomean_k, \dpocov_k} d \weights.
    \label{eq:glppred}
\end{equation}
However, this integral is not particularly useful unless we wish to evaluate
known $\targ\test$ under the model. For prediction, it is more useful to
compute (using Monte-Carlo integration) the predictive expectation,
\begin{align}
    \expece{}{\targs\test} \approx&
    \int \frac{1}{K} \sum^K_{k=1} \gausC{\weights}{\pomean_k, \dpocov_k}
    \int \targs\test \probC{\targs\test}{\activ{\feats\testT\weights}, \lparam}
    d \targs\test d \weights
    \nonumber\\
    =& \int \expece{}{\probC{\targs\test}
        {\activ{\feats\testT\weights}, \lparam}}
    \frac{1}{K} \sum^K_{k=1} \gausC{\weights}{\pomean_k, \dpocov_k}
    d \weights.
    \label{eq:glmexpec}
\end{align}
Often we find $\expece{}{\probC{\targs\test}{\activ{\feats\testT\weights},
        \lparam}} = \activ{\feats\testT\weights}$, however this is only true
with with right choice and usage of the activation function. Furthermore, it is
useful to compute quantiles of the predictive density in order to ascertain the
predictive uncertainty. We start by sampling the predictive cumulative density
function, $\Prob{\cdot}$,
\begin{align}
    &\ProbC{\targs\test \leq \cdfAlph}{\targ, \ins, \inss\test} \nonumber\\
    &\qquad\approx \int
    \frac{1}{K} \sum^K_{k=1} \gausC{\weights}{\pomean_k, \dpocov_k}
    \int^{\cdfAlph}_{-\infty}
    \probC{\targs\test}{\activ{\feats\testT\weights}, \lparam}
    d \targs\test d \weights \nonumber\\
    &\qquad= \int
    \ProbC{\targs\test \leq \cdfAlph}{\activ{\feats\testT\weights}, \lparam}
    \frac{1}{K} \sum^K_{k=1} \gausC{\weights}{\pomean_k, \dpocov_k}
    d \weights.
    \label{eq:lapexpec}
\end{align}
Once we have obtained sufficient samples from the (mixture) posterior we can
obtain quantiles, $\cdfAlph$, for some chosen level of probability, $p$, using
root finding techniques. Specifically, we use root finding techniques to solve
the following for $\cdfAlph$,
\begin{equation}
    \ProbC{\targs\test \leq \cdfAlph}{\targ, \ins, \inss\test} - p = 0.
\end{equation}

\subsection{Large Scale Gaussian Process Approximation}

In \revrand we approximate Gaussian Processes \citep{Rasmussen2006} with
our standard and generalized linear models by using random feature functions
such as those of \citeauthor{rahimi2007} \citeyearpar{rahimi2007,rahimi2008}
and \cite{le2013fastfood}. They use Bochner's theorem regarding the
relationship between a kernel and the Fourier transform of a non-negative
measure (e.g.~a probability measure) that establishes the duality of the
covariance function of a stationary process and its spectral density,
\begin{align}
	\kernl(\tfourier) &= \int \specfourier(\ffourier)
    e^{i \ffourier\T  \tfourier} d \ffourier,  \\
	\specfourier(\ffourier) &= \int \kernl(\tfourier)
    e^{- i \ffourier\T \tfourier}  d \tfourier.
	\label{eq:fourier}
\end{align}
where $\kernl(\cdot)$ is a kernel function, and $\specfourier(\cdot)$ its
spectral density. \citeauthor{rahimi2007}'s  main insight
\citeyearpar{rahimi2007} is that we can approximate the kernel by constructing
`suitable' random features and Monte Carlo averaging over samples from
$\specfourier(\ffourier)$ for \emph{shift invariant} kernels,
\begin{equation}
    \kernl(\inss - \inssprime) = \kernl(\tfourier)
    \approx \frac{1}{D} \sum_{i=1}^{D} \singlefeatfunc{\inss}{i}\T\!
	\singlefeatfunc{\inssprime}{i},
	\label{eq:mcapprox}
\end{equation}
$\singlefeatfunc{\inss}{i}$ corresponds to the $i$th sample from the feature
map. An example of a radial basis kernel feature vector construction using the
above approximation is,
\begin{align}
	\nonumber
    \sbrac{\singlefeatfunc{\inss}{i}, \singlefeatfunc{\inss}{D+i}}&=
    \frac{1}{\sqrt{D}} \sbrac{\cos(\ffourier_i^T \inss),
    \sin(\ffourier_i^T \inss)}, \qquad \\
    \text{with}~\ffourier_i & \sim
    \gausC{\ffourier_i}{\mathbf{0}, \lenscale\inv \ident{d}},
\end{align}
for $i=1, \ldots, D$,  which in fact is a mapping into a $2 D$-dimensional
feature space. See Table~\ref{tab:randommappings} for some of the random kernel
approximations we use in \revrand.

\begin{table}[htb]

    \centering

    \caption{Kernels and the corresponding Fourier weight ($\ffourier_i$)
        sampling distributions for the \citet{rahimi2007}-style random bases in
        \revrand. Here GAL refers to a multivariate Laplace
        distribution~\cite{kozubowski2013multivariate}.}

    \begin{tabular}{r|c c}
        \textbf{Kernel} & $\kernl\!\brac{\inss - \inssprime}$
        & $\specfourier(\ffourier)$ \\
        \hline
        RBF &
        $\exp\brac{-\frac{\lnorm{}{\inss - \inssprime}^2}{2\lenscale^2}}$ &
            $\ffourier_i \sim \gaus{\mathbf{0}, \lenscale\inv\ident{d}}$, \\
        Laplace & $\exp\brac{-\frac{\lnorm{}{\inss-\inssprime}}{\lenscale}}$ &
            $\ffourier_i \sim \prod_d \text{Cauchy}\!\brac{\lenscale\inv}$ \\
        Cauchy &
            $\frac{1}{1+\brac{\lnorm{}{\inss-\inssprime}/\lenscale}^2}$ &
            $\ffourier_i \sim \text{GAL}\brac{1, \mathbf{0},
                \lenscale\inv\ident{d}}$ \\
        Matern $3/2$ &
            $\brac{1 + \frac{\sqrt{3}\lnorm{}{\inss-\inssprime}}{\lenscale}}
            \exp\brac{-\frac{\sqrt{3}\lnorm{}{\inss-\inssprime}}{\lenscale}}$ &
            $\ffourier_i \sim
            \text{t}_{\nu=3}\!\brac{\mathbf{0}, \lenscale\inv\ident{d}}$ \\
        Matern $5/2$ &
            $\brac{1 + \frac{\sqrt{5}\lnorm{}{\inss-\inssprime}}{\lenscale} +
                \frac{5\lnorm{}{\inss - \inssprime}^2}{3\lenscale^2}}
            \exp\brac{-\frac{\sqrt{5}\lnorm{}{\inss-\inssprime}}{\lenscale}}$ &
            $\ffourier_i \sim
            \text{t}_{\nu=5}\!\brac{\mathbf{0}, \lenscale\inv\ident{d}}$ \\
        \hline
    \end{tabular}

    \label{tab:randommappings}

\end{table}

All of the kernel functions in Table \ref{tab:randommappings} have length scale
hyperparameters, and in \revrand these can be isotropic or anisotropic
length scales that are learned alongside the other hyperparameters. We have
also implemented a few other variants of these random basis functions, namely
the Fastfood \citep{le2013fastfood} and A la Carte \citep{yang2014} basis
functions that improve scalability and representational flexibility
respectively. We refer the reader to our demo notebooks for further discussion
and experiments comparing these approximate basis functions and the kernels
they approximate.

TODO: a few figures of the approximate kernels?

TODO: Basis CONCATENATION and how regularizer act as amplitude parameters

An important feature of \revrand is that we can combine these random features
(and non-random features) to build even more expressive features. In
particular, we support basis \emph{concatenation} while still allowing basis
function hyperparameter learning,
\begin{equation}
    \func{\featsym}{\ins, \boldsymbol\param}{} = \sbrac{
        \func{\featsym}{\ins, \param_1}{1},
        \func{\featsym}{\ins, \param_2}{2},
        \ldots,
        \func{\featsym}{\ins, \param_P}{P}
    }
\end{equation}
This is in fact equivalent to kernel addition,
\begin{equation}
    TODO
\end{equation}
We also support \emph{partial application} of basis functions to certain
dimensions of the inputs ($\ins$), while also allowing concatentation, e.g,
\begin{equation}
    \func{\featsym}{\ins, \boldsymbol\param}{} = \sbrac{
        \func{\featsym}{\ins_{1:10}, \param_1}{1},
        \func{\featsym}{\ins_{5:D}, \param_2}{2},
        \ldots,
        \func{\featsym}{\ins, \param_P}{P}
    }
\end{equation}
Where the subscript of $\ins$ denotes (arbitrary) column slices. See \revrand's
documentation for how to use this features.

\section{Experiments}

\subsection{Boston Housing Regression}

\begin{table}[htb]

    \centering
    \begin{tabular}{r|c c}
        \textbf{Algorithm} & \textbf{R-square} & \textbf{MSLL} \\
        \hline
        \emph{SLM} & 0.9018 (0.0134) & -1.504 (0.1191) \\
        % \emph{GLM} & 0.8340 (0.0491) & -0.8763 (0.1242) \\
        \emph{GLM} & 0.8411 (0.0491) & -0.9209 (0.1530) \\
        GP & \textbf{0.9027 (0.0137)} & \textbf{-1.1792 (0.1581)} \\
        RF & 0.8467 (0.0709) & N/A \\
        SVR & 0.6295 (0.0863) & N/A \\
        \hline
    \end{tabular}

\end{table}

\subsection{Handwritten Digits Classification}

\begin{table}[htb]

    \centering
    \begin{tabular}{r|c c}
        \textbf{Algorithm} & \textbf{Log-loss} & \textbf{Error (\%)} \\
        \hline
        \emph{GLM} & 0.1138 & \textbf{2.07} \\
        Logistic & 0.1734 & 3.62 \\
        SVC & \textbf{0.1003} & 6.99 \\
        GPC &  0.1405 & 2.59 \\
        RF & 0.1368 & 2.72 \\
        \hline
    \end{tabular}

\end{table}

\subsection{SARCOS Regression}

1 million iterations

\begin{table}[htb]

    \centering
    \begin{tabular}{r|c|c c}
        \textbf{Algorithm} & \textbf{Approximation size} & \textbf{SMSE} &
        \textbf{MSLL} \\
        \hline
        \emph{GLM} & 256 & 0.0358 & -1.6644 \\
        & 512 & 0.0252 & -1.8397 \\
        & 1024 & 0.0207 & -1.9341 \\
        & 2048 & 0.0171 & -2.0243 \\
        & 8192 & 0.0152 & -1.9804 \\
        \hline
    \end{tabular}

\end{table}

\bibliography{report}

\end{document}


\chapter{Probability}

\section{Mutually exclusive and independent events}
\section{Probability}
\section{Modelling with probability}


\chapter{Independent Component Analysis on the Sphere}
\label{ch_mms}
\index{Multiscale}

\section{Introduction to ICA}

\section{The JADE Method}
\section{The SMICA Method}

\section{ICA and Wavelets}
\subsection{WSMICA}
\subsection{WJADE}

\section{Applications}

\section{WMAP Data}


% !TEX root=/home/tavant/these/manuscript/src/manuscript.tex

\section{Characteristics and performances of the LPPic simulation code }
\label{sec-lppic}

\LPPic is the electrostatic particle-in-cell simulation code developed at \ac{LPP} to study principally magnetized plasma thruster, such as the \ac{HET}.
Written in Fortran (2008), it uses the Message Passing Interface (MPI) to parallelize the computational load using a domain decomposition.
The principle of the domain decomposition is that each Central Processing Unit (CPU) knows a small chunk of the whole domain.
We use {\sc Hypre}'s multigrid solver to solve Poisson equation in \ac{2D} \citep{falgout2002}.

The main data input/output is done using the HDF5 format, which enables parallel access to a hierarchical file.
The output data does not yet use the open standard for particle-mesh data files (openPMD).
Thus, the data post processing is done using a python package.

The use of these efficient libraries allows us to obtain a good scalability.
\Cref{fig-strongperfo} shows the speed up measured for the Strong Scalability test.
The Strong scalability compares the performances of the execution of the task at fixed workload with varying the number of CPUs.
The theoretical speedup accessible is given by \citet{amdahl1967} as
\begin{equation} \label{eq-amdahl}
  S = \frac{1}{1 - \alpha + \frac{\alpha}{s}}
\end{equation}
with $\alpha$ the proportion of the task that is parallel, and $s$ the speedup of the parallelized part.
A perfectly parallel code has $\alpha=1$, hence a speedup $S=s$.

In \cref{fig-strongperfo} are shown the speedup for the overall simulation, as well as the main functions of the \ac{PIC} algorithm (Charge deposition to the mesh, Particle pusher, Monte Carlo Collision and the Poisson solver).
The scalability test is realized over two \ac{2D} radial-azimuthal cases.
The bigger case uses a mesh of $1000\times1024$ cells with $140$ particles per cell, which corresponds to a typical simulation case.
Unfortunately, this case cannot be run with less than 240 CPUs because of the maximum memory available per CPU.
Hence, we use a case four times smaller to test fewer number of CPU.
The simulation is run over 2000 time steps.
More information on the scalability is given in \cref{an-scalability}.
\begin{figure}[hbt]
  \centering
  \includegraphics[width=\textwidth]{StrongScalingOccigen_speedup.pdf}
  \caption{Strong scalability profile of \LPPic with a small and a large job on the Cluster Occigen (CINES). In addition to the overall performance, the profiles of the main sub-functions of the \acs{PIC} loops are given. }
  \label{fig-strongperfo}
\end{figure}

We can see that the overall performance of the code is very good, as it scales as if it is parallelized at 91\% from Amdahl's law.
Most of the sub-functions have higher speed-ups, but the Poisson solver does not scale at all.
It is due to the size of the chunk of domain of each CPU, which is too small.
A way to improve the performances of \LPPic would be to use an hybrid parallelization with the shared memory openMP library.
But the current performance is acceptable, as the typical simulation cases presented in this manuscript run in around one week (7 days) on 360 CPUs.

\vspace{1em}

The development of the  simulation code \LPPic started in 2015 by Vivien Croes.
I joined the development with Romain Lucken in 2016, and Thomas Charoy started in 2017.
Because of the number of contributors, and our different experiences in software development, we quickly set a Continuous Integration (CI) pipeline on a self hosted server using Teamcity.
This allows a systematic verification of the modifications by the mean of unit, mezzanine and integration tests \citep{turner2016}.

The validation of the code is done using the Benchmark of Turner et al.(2013) \citenum{turner2013}.
While it allows to validate a significant part of the \ac{PIC} code, it is not magnetized and only in \ac{1D}.
Unfortunately, this is the only Benchmark available.
Hence, the comparison of several codes on a \ac{2D} magnetized  has been conducted in 2019 by Thomas Charoy in order to create a new Benchmark to validate such simulation codes, in \citet{charoy2019}.

Since 2015, \LPPic has grown to 23000 lines of code, has been used on more than 5 different clusters of various architectures, and has been used to produce several publications \citep{croes2017a,croes2018,tavant2018,tavant2019,lucken2018,lucken2019}.


%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%
% #ifdef PTEX2TEX_EXPLANATION
%%
%% The file follows the ptex2tex extended LaTeX format, see
%% ptex2tex: http://code.google.com/p/ptex2tex/
%%
%% Run
%%      ptex2tex myfile
%% or
%%      doconce ptex2tex myfile
%%
%% to turn myfile.p.tex into an ordinary LaTeX file myfile.tex.
%% (The ptex2tex program: http://code.google.com/p/ptex2tex)
%% Many preprocess options can be added to ptex2tex or doconce ptex2tex
%%
%%      ptex2tex -DMINTED myfile
%%      doconce ptex2tex myfile envir=minted
%%
%% ptex2tex will typeset code environments according to a global or local
%% .ptex2tex.cfg configure file. doconce ptex2tex will typeset code
%% according to options on the command line (just type doconce ptex2tex to
%% see examples). If doconce ptex2tex has envir=minted, it enables the
%% minted style without needing -DMINTED.
% #endif

% #define PREAMBLE

% #ifdef PREAMBLE
%-------------------- begin preamble ----------------------

\documentclass[%
oneside,                 % oneside: electronic viewing, twoside: printing
final,                   % draft: marks overfull hboxes, figures with paths
10pt]{article}

\listfiles               %  print all files needed to compile this document

\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
\usepackage[table]{xcolor}
\usepackage{bm,ltablex,microtype}

\usepackage[pdftex]{graphicx}

\usepackage{ptex2tex}
% #ifdef MINTED
\usepackage{minted}
\usemintedstyle{default}
% #endif

\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}

\usepackage{lmodern}         % Latin Modern fonts derived from Computer Modern

% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
    breaklinks=true,
    colorlinks=true,
    linkcolor=linkcolor,
    urlcolor=linkcolor,
    citecolor=black,
    filecolor=black,
    %filecolor=blue,
    pdfmenubar=true,
    pdftoolbar=true,
    bookmarksdepth=3   % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
    }
%\hyperbaseurl{}   % hyperlinks are relative to this root

\setcounter{tocdepth}{2}  % levels in table of contents

% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000

% --- end of standard preamble for documents ---

% insert custom LaTeX commands...

\raggedbottom
\makeindex
\usepackage[totoc]{idxlayout}   % for index in the toc
\usepackage[nottoc]{tocbibind}  % for references/bibliography in the toc

%-------------------- end preamble ----------------------

\begin{document}

% matching end for #ifdef PREAMBLE
% #endif

\newcommand{\exercisesection}[1]{\subsection*{#1}}

% ------------------- main content ----------------------

% ----------------- title -------------------------

\thispagestyle{empty}

\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Day 3: Exercise set 2
\end{spacing}
}
\end{center}

% ----------------- author(s) -------------------------

\begin{center}
{\bf Data Analysis and Machine Learning for Nuclear Physics${}^{}$} \\ [0mm]
\end{center}

\begin{center}
% List of all institutions:
\end{center}

% ----------------- end author(s) -------------------------

% --- begin date ---
\begin{center}
Jun 25, 2020
\end{center}
% --- end date ---

\vspace{1cm}

\subsection{Day three exercises}

\paragraph{Exercise 1.}
This exercise is a continuation of exercise 3 from exercise set 1. We will
use the same function to generate our data set, still staying with a
simple function $y(x)$ which we want to fit using linear regression,
but now extending the analysis to include the Ridge and the Lasso
regression methods. You can use the code under the Regression as an example on how to use the Ridge and the Lasso methods, see the \href{{https://compphysics.github.io/MachineLearning/doc/pub/Regression/html/Regression-bs.html}}{regression slides}).

We will thus again generate our own dataset for a function $y(x)$ where
$x \in [0,1]$ and defined by random numbers computed with the uniform
distribution. The function $y$ is a quadratic polynomial in $x$ with
added stochastic noise according to the normal distribution $\cal{N}(0,1)$.

The following simple Python instructions define our $x$ and $y$ values (with 100 data points).
\bpycod
x = np.random.rand(100,1)
y = 5*x*x+0.1*np.random.randn(100,1)
\epycod

\begin{enumerate}
\item (1a) Write your own code for the Ridge method (see chapter 3.4 of Hastie \emph{et al.}, equations (3.43) and (3.44)) and compute the parametrization for different values of $\lambda$. Compare and analyze your results with those from exercise 2. Study the dependence on $\lambda$ while also varying the strength of the noise in your expression for $y(x)$.

\item (1b) Repeat the above but using the functionality of \textbf{Scikit-Learn}. Compare your code with the results from \textbf{Scikit-Learn}. Remember to run with the same random numbers for generating $x$ and $y$.

\item (1c) Our next step is to study the variance of the parameters $\beta_1$ and $\beta_2$ (assuming that we are parameterizing our function with a second-order polynomial). We will use standard linear regression and the Ridge regression.  You can now opt for either writing your own function or using \textbf{Scikit-Learn} to find the parameters $\beta$. From your results calculate the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix $(\hat{X}^T\hat{X})+\lambda\hat{I})^{-1}$). Discuss the results of these variances as functions of $\lambda$. In particular, try to link your discussion with the discussion in Hastie \emph{et al.} and their figures 3.10 and  3.11. \textbf{Scikit-Learn} may not provide the variance of the parameters $\beta$. This needs to be checked. With your own code you can however do so.

\item (1d) Repeat the previous step but add now the Lasso method, see equation (3.53) of Hastie \emph{et al.}. Discuss your results and compare with standard regression and the Ridge regression results. You can write your own code or use the functionality of \textbf{scikit-learn}.  We recommend the latter since we have not yet discussed how to solve the Lasso equations numerically. Also, you do not need to compute the variance of the parameters $\beta$ but you can extract their values and study their behavior as functions of the regularization parameter $\lambda$.

\item (1e) Finally, using \textbf{Scikit-Learn} or your own code, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as
\end{enumerate}

\noindent
\[ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
\]
and the $R^2$ score function.
If $\tilde{\hat{y}}_i$ is the predicted value of the $i-th$ sample and $y_i$ is the corresponding true value, then the score $R^2$ is defined as
\[
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
\]
where we have defined the mean value  of $\hat{y}$ as
\[
\bar{y} =  \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
\]
Discuss these quantities as functions of the variable $\lambda$ in the Ridge and Lasso regression methods.

\subsection{Exercise 2}

A much used approach before starting to train the data is  to preprocess our
data. Normally the data may need a rescaling and/or may be sensitive
to extreme values. Scaling the data renders our inputs much more
suitable for the algorithms we want to employ.

\textbf{Scikit-Learn} has several functions which allow us to rescale the
data, normally resulting in much better results in terms of various
accuracy scores.  The \textbf{StandardScaler} function in \textbf{Scikit-Learn}
ensures that for each feature/predictor we study the mean value is
zero and the variance is one (every column in the design/feature
matrix).  This scaling has the drawback that it does not ensure that
we have a particular maximum or minimum in our data set. Another
function included in \textbf{Scikit-Learn} is the \textbf{MinMaxScaler} which
ensures that all features are exactly between $0$ and $1$. The

The \textbf{Normalizer} scales each data
point such that the feature vector has a euclidean length of one. In other words, it
projects a data point on the circle (or sphere in the case of higher dimensions) with a
radius of 1. This means every data point is scaled by a different number (by the
inverse of it’s length).
This normalization is often used when only the direction (or angle) of the data matters,
not the length of the feature vector.

The \textbf{RobustScaler} works similarly to the StandardScaler in that it
ensures statistical properties for each feature that guarantee that
they are on the same scale. However, the RobustScaler uses the median
and quartiles, instead of mean and variance. This makes the
RobustScaler ignore data points that are very different from the rest
(like measurement errors). These odd data points are also called
outliers, and might often lead to trouble for other scaling
techniques.

It also common to split the data in a \textbf{training} set and a \textbf{testing} set. A typical split is to use $80\%$ of the data for training and the rest
for testing. This can be done as follows with our design matrix $\bm{X}$ and data $\bm{y}$ (remember to import \textbf{scikit-learn})
\bpycod
# split in training and test data
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)
\epycod
Then we can use the stadndard scale to scale our data as
\bpycod
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
\epycod

In this exercise we want you to to compute the MSE for the training
data and the test data as function of the complexity of a polynomial,
that is the degree of a given polynomial. We want you also to compute the $R2$ score as function of the complexity of the model for both training data and test data.  You should also run the calculation with and without scaling.

One of
the aims is to reproduce Figure 2.11 of \href{{https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf}}{Hastie et al}.
We will also use Ridge and Lasso regression.

Our data is defined by $x\in [-3,3]$ with a total of for example $100$ data points.
\bpycod
np.random.seed()
n = 100
maxdegree = 14
# Make data set.
x = np.linspace(-3, 3, n).reshape(-1, 1)
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
\epycod
where $y$ is the function we want to fit with a given polynomial.

\begin{enumerate}
\item (2a) Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial.  Scale your data and split it in training and test data.

\item (2b) Perform an ordinary least squares and compute the means squared error and the $R2$ factor for the training data and the test data, with and without scaling.

\item (2c) Add now a model which allows you to make polynomials up to degree $15$.  Perform a standard OLS fitting of the training data and compute the MSE and $R2$ for the training and test data and plot both test and training data MSE and $R2$ as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al.~Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?

\item (2d) Repeat part (2c) but now using Ridge regressions with various hyperparameters $\lambda$. Make the same plots for the optimal $\lambda$ value for each polynomial degree. Compare these results with those from the standard OLS approach.
\end{enumerate}

\noindent
\subsection{Example of how to solve the previous exercise}

\bpycod
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import Ridge
from sklearn.preprocessing import PolynomialFeatures
from sklearn.metrics import mean_squared_error, r2_score, mean_absolute_error

n = 100
# Make data set.
x = np.linspace(-3, 3, n).reshape(-1, 1)
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Decide which values of lambda to use
nlambdas = 500
lambdas = np.logspace(-3, 5, nlambdas)

estimated_mse_sklearn = np.zeros(nlambdas)
i = 0
for lmb in lambdas:
    clf_ridge = Ridge(alpha=lmb).fit(X_train_scaled, y_train)
    yridge = clf_ridge.predict(X_test_scaled)
    estimated_mse_sklearn[i] = mean_squared_error(y_test, yridge)
    i += 1
plt.figure()
plt.plot(np.log10(lambdas), estimated_mse_sklearn, label = 'Ridge MSE')
plt.xlabel('log10(lambda)')
plt.ylabel('MSE')
plt.legend()
plt.show()

\epycod

\subsection{And now with OLS only and Bootstrap}

\bpycod

import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.utils import resample
from sklearn.linear_model import LinearRegression, Ridge, Lasso
from sklearn.preprocessing import PolynomialFeatures
from sklearn.metrics import mean_squared_error, r2_score, mean_absolute_error

n = 100
n_boostraps = 100
maxdegree = 14

# Make data set.
x = np.linspace(-3, 3, n).reshape(-1, 1)
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
error = np.zeros(maxdegree)
bias = np.zeros(maxdegree)
variance = np.zeros(maxdegree)
polydegree = np.zeros(maxdegree)

# Make data set.
x = np.linspace(-3, 3, n).reshape(-1, 1)
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)

for degree in range(maxdegree):
    model = make_pipeline(PolynomialFeatures(degree=degree), LinearRegression(fit_intercept=False))
    y_pred = np.empty((y_test.shape[0], n_boostraps))
    for i in range(n_boostraps):
        x_, y_ = resample(X_train_scaled, y_train)
        y_pred[:, i] = model.fit(x_, y_).predict(X_test_scaled).ravel()

    polydegree[degree] = degree
    error[degree] = np.mean( np.mean((y_test - y_pred)**2, axis=1, keepdims=True) )
    bias[degree] = np.mean( (y_test - np.mean(y_pred, axis=1, keepdims=True))**2 )
    variance[degree] = np.mean( np.var(y_pred, axis=1, keepdims=True) )
    print('Polynomial degree:', degree)
    print('Error:', error[degree])
    print('Bias^2:', bias[degree])
    print('Var:', variance[degree])
    print('{} >= {} + {} = {}'.format(error[degree], bias[degree], variance[degree], bias[degree]+variance[degree]))

plt.plot(polydegree, error, label='Error')
plt.plot(polydegree, bias, label='bias')
plt.plot(polydegree, variance, label='Variance')
plt.legend()
plt.show()

\epycod

% ------------------- end of main content ---------------

% #ifdef PREAMBLE
\end{document}
% #endif