\chapter{Introduction to Artificial Neural Networks}
\chapterintrobox{Artificial neural networks are computational systems capable of finding complex functions that link an input to an output. These systems can be used for a variety of tasks such as regression and classification. They can be used in predictive maintenance and prognostics to estimate the health of equipment and predict with some uncertainty its remaining useful life. This chapter discusses neural networks, their topology, training and mathematical formulation.}

\section{Structure of artificial neural networks}
Artificial neural networks are computational systems used to find the mapping between an input and an output, they consist of several layers (input layer, output layer and an arbitrary number of hidden layers between the input and the output) and each layer contains a number of neurons where each neuron in each layer is connected to all neurons in the previous and the next layer (except for the input and output layers which are connected only to the next and the previous layer respectively).
Each neuron of each layer receives an input from the neurons of the previous layer (in the form of a vector), multiplies the vector by a few weights and sums the result and then applies a linear activation function. Each neuron ends up with a single numerical value called activation, which will be passed on to the neurons in the next layer.

Figure \ref{fig:neural-network-structure} shows a neural network with the following structure:

\begin{itemize}
    \item Input layer with 3 inputs
    \item Single hidden layer with 4 neurons
    \item Output layer with 1 neuron
\end{itemize}

\begin{figure}[h]
    \centering
	\input{figures/mlp-structure.tex}
    \caption{Structure of an artificial neural network}
    \label{fig:neural-network-structure}
\end{figure}

\section{Feedforwad networks: from input to output}
\label{section:feedforward-neural-network}
The architecture of figure \ref{fig:neural-network-structure} is called Feedforward Neural Network, it is an acyclic architecture where information flows from the first to the last layer without any internal loop, unlike other architectures such as recurrent neural networks. The first layer is the input layer, it does not perform any operation and is limited to receiving input. The input is a vector of numbers representing the different variables. The vector is multiplied by a matrix of weights that transforms it and sends it to the next layer (or to the first hidden layer). An activation function is applied to the values resulting from the multiplication of the values of the previous layer with the weight matrix, the result becomes the values of the next layer, or activations (the value of each neuron is called activation).

A general formula for moving from the $l-1$ layer to the $l$ layer is given by the equation \ref{equation:forward-step}:

\begin{equation}
    a^{[l]} = g^{[l]}(W^{[l]}a^{[l-1]}+b^{[l]})
    \label{equation:forward-step}
\end{equation}

Where $g^{[l]}$ is the activation function of the $l$ layer, $W^{[l]}$ is the weights matrix that transforms the values (activations) from the $l-1$ layer to the $l$ layer and $a^{[l]}$ represents the activations of the $l$ layer. $b$ is the value of the bias (or intercept value), it is added to the multiplication between the activations and the weights matrix, it is a parameter that can be learned like the weights. The operation is repeated for each layer up to the output layer.
The first layer can be considered as layer 0 and the inputs can be designated by the vector $a^{[0]}$.

\section{Activation functions}
The activation functions are applied to the result of multiplying the inputs of the previous layer with the corresponding weights to determine the value of each neuron. There are different types of these functions.

The use of the non-linear activation function is very important for neural networks, they allow learning the complex non-linear mapping from input to output. If the network does not use non-linear activation (for example, linear activation or the identity function), then the whole network (regardless of its depth) is equivalent to a network with only one hidden layer.

There are a variety of activation functions that can be used for the hidden and output layers. Figure \ref{fig:activation-function} shows some examples:
\begin{figure}[h]
    \centering
	\input{figures/activation-functions.tex}
    \caption{Different activation functions}
    \label{fig:activation-function}
\end{figure}

Table \ref{table:activation-functions} shows mathematical definition of several activation functions:

\begin{table}[h]
    \centering
    \begin{tabular}{c|c}
        \hline
        Activation function & Mathematical definition \\
        \hline
        Identity (no activation) & Id(x) = x \\
        Sigmoid & $\sigma(x)= \frac{1}{1+e^{-x}}$ \\
        tanh & $tanh(x)=\frac{(e^x-e^{-x})}{(e^x+e^{-x})}$\\
        Rectified Linear Unit (ReLU) & $ReLU(x)=max(0,x)$\\
        Leaky ReLU & $LeakyReLU(x)=max(0.1 x,x)$\\
    \hline

    \end{tabular}
    \caption{Mathematical definition of several activation functions}
    \label{table:activation-functions}
\end{table}

\section{Network training}
The process of training a neural network consists of determining the weights (coefficients) that map the neurons of each layer to the neurons of the next layer. This process can be formulated in more mathematical terms as an optimization problem: optimization of the network coefficients to find their values that minimize a cost function.

Neural network training uses training data that provide inputs and their corresponding outputs.

\subsection{Cost function}
The cost function is the function used to calculate the difference between the output of the neural network and the expected actual output, it quantifies the performance of the network. The goal of the training process is to minimize this function by using Gradient Descent (see next section) to find the best set of weights that gives the lowest difference between the training data and the network prediction.

There are many types of cost functions, each type corresponds to different neural networks tasks (e.g. regression, binary classification, …). The cost function usually used for regression problems is mean-squared error function (Equation \ref{equation:mse}) which calculates the sum of distances between the model predictions $\hat{y}_i$ and the actual output ($y_i$), $N$ is the number of data points available for training:
\begin{equation}
    MSE=\frac{1}{N}\sum_{i=1}^N(y_i-\hat{y}_i)
    \label{equation:mse}
\end{equation}

The other major task of neural networks and other types of models is classification. There are two main types of classification: binary classification, or classification of two classes and multiclass classification or classification of many classes. The first one uses binary cross-entropy loss function (Equation \ref{equation:logloss}) and the latter uses categorical cross-entropy.

\begin{equation}
    BCE = \sum_{i=1}^{N}\hat{y}_i log(y_i)+(1-\hat{y}_i)log(1-\hat{y}_i)
    \label{equation:logloss}
\end{equation}

The choice of activation function for the last layer of the network is directly related to the used cost function. For regression tasks, linear activation function is used. For binary classification it's sigmoid and for multiclass classification it's softmax function.

\subsection{Gradient Descent}
Gradient Descent is an iterative optimization algorithm used to optimize a differentiable function. Gradient Descent works by calculating the gradients of the objective function and then taking iterative steps in the negative direction of the gradients.

\subsection{Convex and non-convex functions}
Gradient descent in neural networks is used to optimize (minimize) the cost function by finding the best set of weights and biases that give the lowest cost possible. Objective functions can be categorized into two types: convex and non-convex functions.
A convex function is said to be convex if it has the property that every chord lies on or above the function. Any value of $x$ in the interval from $x=a$ to $x=b$ can be written in the form $\lambda a+(1-\lambda)b$ where $0\leq\lambda\leq 1$. The corresponding point on the chord is given by $\lambda f(a)+(1-\lambda)f(b)$, and the corresponding value of the function is $f(\lambda a+(1-\lambda)b)$ (Figure \ref{fig:convexity}). Convexity then implies:
\begin{equation}
    f(\lambda a+(1-\lambda)b)\leq \lambda f(a)+(1-\lambda)f(b)
    \label{equation:convexity}
\end{equation}

This is equivalent to the requirement that the second derivative of the function be everywhere positive \cite{Bishop2006}. This condition of convexity can be extended to spaces with arbitrarily large number of dimensions.

\begin{figure}[h]
    \centering
    \includegraphics{figures/convex_function.pdf}
    \caption{Condition of convexity}
    \label{fig:convexity}
\end{figure}

A non-convex function is a function that doesn't satisfy the convexity condition. Figure \ref{fig:convex-nonconvex-functions} shows an example of a convex (left) and a non-convex function (right) where the function parameters are in two-dimensional space.

\begin{figure}[h]
    \centering
    \includegraphics{figures/gradient_descent.pdf}
    \caption{Example of convex and non-convex functions}
    \label{fig:convex-nonconvex-functions}
\end{figure}

\subsection{Global and local minima}
Global minimum refers to the lowest possible value in a set (or of a function). Finding the global minimum refers to finding the set of parameters that correspond to this minimum value. When the function is convex, finding the global maximum is possible and easy, algorithms like gradient descent always converge to the global minimum in convex optimization. Examples of convex cost functions is linear regression cost function. For more complex models such as neural networks, cost function is highly non-convex with many local minima.

Figure \ref{fig:global_local_minima} shows an example of a non-convex function with a global and local minima. The problem with non-convex optimization is that the optimization algorithm can converge to the local minimum instead of the global one. The algorithm convergence is related to the random initialization of network weights.

\begin{figure}[h]
    \centering
    \includegraphics{figures/global_local_minima.pdf}
    \caption{Non-convex function with global and local minima}
    \label{fig:global_local_minima}
\end{figure}

\subsection{Saddle points}
A saddle point or minimax point is a point on the surface of the graph of a function where the slopes (derivatives) in orthogonal directions are all zero (a critical point), but which is not a local extremum of the function (Figure \ref{fig:saddle-point}).

\begin{figure}[h]
    \centering
    \includegraphics{figures/saddle_point.pdf}
    \caption{Saddle point}
    \label{fig:saddle-point}
\end{figure}

\subsection{Optimization of multilayer networks}
In  \cite{Choromanska2014} the authors showed that getting stuck in poor local minima is a major problem only for shallow networks but becomes gradually of less importance as the network size increases. This is mainly due to the fact that in large networks, local minima lie close to the global minimum so they yield similar good performance.

The following hypotheses were also verified empirically in the mentioned paper regarding learning with large-size networks:
\begin{itemize}
    \item When a network is large, there isn't a significant difference in performance on test data among most of local minima.
    \item Probability of finding a bad local minimum in small networks is higher than the probability of finding them in larger networks.
    \item Finding the global minimum on training set doesn't guarantee a better performance on testing data, rather it could cause overfitting and reduced performance.
\end{itemize}

\subsection{Backpropagation}
Backpropagation is the algorithm used to calculate the gradients of the cost function (the objective function) of a neural network. Since a neural network can be interpreted as functions composition, Backpropagation uses the derivation theorem of a composition of functions to find the gradients in relation to the network weights.

The use of Backpropagation for training neural networks was popularized by David E. Rumelhart, Geoffrey E. Hinton and Ronald J. Williams \cite{Rumelhart1986}, they described it as a procedure that repeatedly adjusts the weights of the network connections so as to minimize a measure of the difference between the actual output vector of the network and the desired output vector. As a result of the weight adjustments, "hidden" internal units that are not part of the input or output come to represent important characteristics of the task domain, and the regularities of the task are captured by the interactions of these units.

Figure \ref{fig:forward-backward-pass} represents the Forward Pass and the Backward Pass, the Forward Pass calculates the network output, the Backward Pass calculates the gradients of the cost function that measures the difference between this output and the actual desired output:

\begin{figure}[h]
    \centering
	\input{figures/forward-backward-pass.tex}
    \caption{Forward (left) and Backward (right) passes}
    \label{fig:forward-backward-pass}
\end{figure}

\section{Recurrent Neural Networks}
\acrlong{rnn} (\acrshort{rnn}) is a special architecture that is more suitable for modeling sequential data. \acrshort{rnn}s process an input sequence one element at a time, maintaining in their hidden units a 'state vector' that implicitly contains information about the history of all the past elements of the sequence \cite{LeCun2015}. Figure \ref{fig:rnn} shows an example of a \acrshort{rnn} architecture. To the left is the unfolded version with a cyclic loop in the hidden layer, to the right is shows unfolding of the network which is easier to understand. $x_1$, $x_2$, …$x_t$ represents the input vector (sequence), $y_1$, $y_2$, …$y_t$ represents the output vector (can be a sequence with length equals to the input, of different length or a single element). $h_1$, $h_2$, …$h_t$ are neurons of the hidden layer. In \acrshort{rnn} each neuron passes a vector of its hidden state to the next neuron corresponding to the next time step.

\begin{figure}[H]
        \centering
        \input{figures/rnn.tex}
        \caption{Recurrent neural networks architecture}
        \label{fig:rnn}
\end{figure}

\subsection{Long Short-Term Memory}
\label{section:lstm}
Simple \acrshort{rnn}s have problems with learning long-term dependencies (i.e. making predictions based on predictions made many steps previously in the past) and vanishing gradients. \acrlong{lstm} (\acrshort{lstm}) was first introduced by Jürgen Schmidhuber and Sepp Hochreiter \cite{Hochreiter1997}. \acrshort{lstm} overcomes problems with basic \acrshort{rnn} by using what is called a cell state which enables \acrshort{lstm} networks to learn long-term dependencies. \acrshort{lstm} cells also possess three types of gates which together control the flow of information inside the cell:
\begin{itemize}
    \item \textbf{Forget gate}: Controls which information are kept or discarded at time $t$
    \item \textbf{Input gate}: Controls which information to store in the cell state at time $t$
    \item \textbf{Output gate}: Controls the final output of the cell at time $t$
\end{itemize}

Input, forget and output gates values are calculated using Equations \ref{equation:lstm_input_gate}, \ref{equation:lstm_forget_gate} and \ref{equation:lstm_output_gate} respectively:

\begin{align}
i_t &= \sigma(W_{xi}x_t + W_{hi}h_{t-1}+W_{ci}c_{t-1}+b_i) \label{equation:lstm_input_gate}\\
f_t &= \sigma(W_{xf}x_t + W_{hf}h_{t-1}+W_{cf}c_{t-1}+b_f) \label{equation:lstm_forget_gate}\\
o_t &= \sigma(W_{xo}x_t + W_{ho}h_{t-1}+W_{co}c_t+b_o) \label{equation:lstm_output_gate}\\
\end{align}

Input, forget and output gates are calculated at time $t$ using sets of weights and biases ($W_{xi}$, $W_{hi}$, $W_{ci}$, $b_i$), ($W_{xf}$, $W_{hf}$, $W_{cf}$, $b_f$) and ($W_{xo}$, $W_{ho}$, $W_{co}$, $b_o$) that controls how each of $x_t$, $h_{t-1}$ and $c_{t-1}$ affects the value of the gate respectively. A sigmoid function is used to convert the values to the range of 0 to 1.

At each time step $t$, a candidate cell state $\tilde{c}_t$ is calculated using weights and a bias term ($W_{xc}$, $W_{hc}$, $b_c$) that maps the values of input $x_t$ and previous hidden state $h_{t-1}$ to the candidate cell state (Equation \ref{equation:lstm_candidate_cell_state}). As the name indicates, $\tilde{c}_t$ serves as a candidate to replace current cell state $c_t$ at time $t$. Cell state at time $t$ is obtained by using the forget gate to control which information are kept from the previous cell state and input gate to control which information are kept from the candidate cell state according to Equation \ref{equation:lstm_cell_state}. Final cell output is calculated using current cell state $c_t$ and output gate according to Equation \ref{equation:lstm_hidden_state}.

\begin{align}
    \tilde{c}_t &= tanh(W_{xc}x_t+W_{hc}h_{t-1}+b_c) \label{equation:lstm_candidate_cell_state} \\
    c_t &= f_tc_{t-1}+i_t\tilde{c}_t \label{equation:lstm_cell_state}\\
    h_t &= o_ttanh(c_t) \label{equation:lstm_hidden_state}
\end{align}

Gates control the flow of information inside the \acrshort{lstm} cell, their values range from 0 to 1 (sigmoid function) and control which information to keep and which to discard when updating the cell state (forget and input gates) and when calculating the \acrshort{lstm} cell output (output gate).

Figure \ref{fig:lstm} illustrates the different operations happening in a single \acrshort{lstm} cell:

\begin{figure}[h]
    \centering
    \input{figures/lstm.tex}
    \caption{Long Short-Term Memory Cell}
    \label{fig:lstm}
\end{figure}

\section{Convolutional neural networks}
\label{section:cnn}
\acrlong{cnn} (\acrshort{cnn}) have what is called convolutional layers. A convolutional layer takes an input (usually 2D images, but it can be 1D or 3D either) and apply a mathematical operation called convolution\footnote{Mathematically, the operation happening in a \acrshort{cnn} is called cross-correlation which is a bit different than the mathematical definition of a convolution. In Machine Learning literature those operations are called convolutions, this is the terminology that will be used here.}. Convolutions act like filters that extract features from raw data by recursively multiplying what is called kernel with the input data. Different kernels serve different purposes and can extract a wide variety of features from raw data. After feature extraction done by the convolutional layer, they serve as an input for a feedforward neural network (as described in section \ref{section:feedforward-neural-network}) for classification or regression.

\begin{figure}[H]
    \centering
    \begin{subfigure}{0.22\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/convolutions/no_padding_no_strides_00.pdf}
        \subcaption*{step 01}
    \end{subfigure}\hfill%
    \begin{subfigure}{0.22\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/convolutions/no_padding_no_strides_01.pdf}
        \subcaption*{step 02}
    \end{subfigure}\hfill%
    \begin{subfigure}{0.22\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/convolutions/no_padding_no_strides_02.pdf}
        \subcaption*{step 03}
    \end{subfigure}\hfill%
    \begin{subfigure}{0.22\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/convolutions/no_padding_no_strides_03.pdf}
        \subcaption*{step 04}
    \end{subfigure}
    \caption{Applying convolutions (by multiplying the input by a kernel) iteratively to different regions (dark blue) of the 2D input (whole blue square). Each iteration gives a numerical value (dark green). The result of all iterations is the green 2D output \cite{dumoulin2016guide}.}
    \label{fig:convolutions}
\end{figure}

\section{Conclusion}
Neural networks are a powerful tool to find the complex relationship between an input and an output (e.g. \acrlong{cm} data and machine degradation). They consist of different layers of interconnected neurons. the connection between each 2 layers is defined by a set of weights, which are multiplied by the values of the previous layer to find the values of the next layer, the process is repeated until the output layer is reached. the predicted output is compared to the actual output obtained from the training data, the weights are adjusted to minimize (using Backpropagation and Gradient Descent) the difference between the predictions and the reality.


\section{Matching}
\label{sec:matching}

Matching is the process of finding an assignment for a pattern such that when this pattern is instantiated with that assignment, the result equals the target value. Namely, given two terms $\phi$ and $\psi$, we are looking for a substitution $\sigma$ such that $\phi[\sigma] = \psi$. Matching can be seen as a special case of a more general problem called unification $\phi[\sigma] = \psi[\sigma]$, because the patterns appear only on one side \cite{Knight1989}.

Most programming languages that implement pattern matching, such as Scala, disallow the reuse of pattern variables. These kinds of restricted patterns are referred to as linear. On the other hand, non-linear patterns allow variables to appear multiple times. In that case the usually implied semantic is that all the occurrences of the same variable should be pairwise equal.

In this framework we relax the notion of equality for non-linear pattern variables by allowing them to be equivalent with respect the equivalence checker implemented in LISA: the orthocomplemented bisemilattices (OCBSL) equivalence checker \cite{Guilloud2022}. This leads to an ambiguity in the assignment because by construction a variable can only be assigned a single value; thus not only may the resulting assignment not lead to an instantiation that is structurally equal to the target, but it is also unclear which assigned value to choose. To address these ambiguities, we define deterministic rules that are not necessarily intended to be practical but are required to exhibit consistent behavior over different execution contexts.

\subsection{Terminology}

Formally, a matching is characterized by the following elements:

\begin{definition}[Tree]
A \textbf{tree} refers to either a term, a formula or a sequent.
\end{definition}

The type of a tree is known and indicates its nature (term, formula or sequent).

\begin{definition}[Pattern variable and constant]
A \textbf{pattern variable} is a term or a formula represented as a schema or a bound/free variable.
A \textbf{constant} is a tree that is free of pattern variables.
\end{definition}

Without context, it is not possible to tell if a schema or a variable is a pattern variable or a constant.

\begin{definition}[Pattern and value]
A \textbf{pattern} is a tree in which the nodes can be either pattern variables or constants.
A \textbf{value} is a tree in which the nodes are all constants.
\end{definition}

In a value, schemas and bound/free variables are interpreted as constants and shouldn't clash with pattern variables.

\begin{definition}[Solution of a matching]
A \textbf{solution} is an assignment mapping pattern variables to values, such that when the pattern is instantiated it becomes equivalent to the value.
\end{definition}

The matching procedure takes three arguments: a sequence of patterns, a sequence of values and a partial assignment. It returns a solution or fails. The partial assignment corresponds to a partial solution that the procedure should complete.

Patterns are referred to as the left hand side, and values as the right hand side.

\subsection{Specification}

The goal is to find an assignment for the sequence of patterns such that when it is instantiated it becomes equivalent to the sequence of values. Each pattern is matched against the value located at the same index in the sequence. The matching is successful if all the patterns can be resolved without reaching a contradiction. The matching fails if a contradiction is reached, or if the problem is under-constrained (solution space is infinite, or problem is ill-posed). If a solution is returned, then it should always be a superset of the provided partial assignment. By default the partial assignment is the empty set.

Because the procedure can also be used to rename bound variables, the pattern should not declare a bound variable more than once. Moreover, no free variable can reuse the name of a bound variable, regardless the scope. Apart from these restrictions, all pattern variables can appear any number of times in the pattern. In addition, pattern variables can appear as arguments of other pattern variables (without any limitation on the depth).

By convention, if a pattern variable can take more than one equivalent form, we should select the one that appears first with respect to the pre-order listing of the nodes of that tree.

\begin{table}[H]
  \newcounter{counter}
  \NewEnviron{example}[4]{%
    \toks0=\expandafter{\BODY}%
    \toks2={\refstepcounter{counter}\thecounter & $#1$ & $#2$ & $#3$ & $#4$}%
    \xdef\TableRowContents{\the\toks2 \the\toks0 }%
    \aftergroup\TableRowContents
  }
  \centering
  \begin{tabular}{||c | c@{\hskip 0.2cm} c@{\hskip 0cm} c | c||}
    \hline
    \# & \textbf{Patterns} & \textbf{Values} & \textbf{Partial Assignment} & \textbf{Result} \\ [0.5ex]
    \hline\hline
    \begin{example}{{?a} \land b}{c \land b}{\emptyset}{\{ {?a} \mapsto c \}}\end{example} \\ \hline
    \begin{example}{{?a} \land {?b}}{a \land b}{\{ {?a} \mapsto b \}}{\bot}\end{example} \\ \hline
    \begin{example}{{?a}}{c \land b}{\{ {?a} \mapsto {b \land c} \}}{\{ {?a} \mapsto {b \land c} \}}\end{example} \\ \hline
    \begin{example}{{?a}}{{?a} \land b}{\emptyset}{\{ {?a} \mapsto {{?a} \land b} \}}\end{example} \\ \hline
    \begin{example}{{?a} \land {?a}}{(b \lor c) \land (c \lor b)}{\emptyset}{\{ {?a} \mapsto {b \lor c} \}}\end{example} \\ \hline
    \begin{example}{{?p}(t)}{t = u}{\emptyset}{\{ {?p} \mapsto {\lambda x. x = u} \}}\end{example} \\ \hline
    \begin{example}{{?p}({?t})}{t = u}{\emptyset}{\bot}\end{example} \\ \hline
    \begin{example}{{?p}({?t})}{t = u}{\{ {?t} \mapsto t \}}{\{ {?p} \mapsto \lambda x. x = u, {?t} \mapsto t \}}\end{example} \\ \hline
    \begin{example}{{?p}({?t})}{t = u}{\{ {?p} \mapsto \lambda x. x = u \}}{\{ {?p} \mapsto \lambda x. x = u, {?t} \mapsto t \}}\end{example} \\ \hline
    \begin{example}{{?p}({?t}); q({?t})}{t = u; q(t)}{\emptyset}{\{ {?p} \mapsto \lambda x. x = u; {?t} \mapsto t \}}\end{example} \\ \hline
    \begin{example}{{?f}({?a} \land b)}{(a \land b) \lor (b \land a)}{\emptyset}{\bot}\end{example} \\ \hline
    \begin{example}{{?f}({?a} \land b); {?a}}{c \lor (b \land a); a}{\emptyset}{\{ {?a} \mapsto a, {?f} \mapsto \lambda x. c \lor x \}}\end{example} \\ \hline
    \begin{example}{\exists x. q(x)}{\exists y. q(y)}{\emptyset}{\{ x \mapsto y \}}\end{example} \\ \hline
    \begin{example}{\exists x. {?a}}{\exists y. y = u}{\emptyset}{\bot}\end{example} \\ \hline
    \begin{example}{\exists x. {?p}(x)}{\exists y. y = u}{\emptyset}{\{ x \mapsto y, {?p} \mapsto \lambda x. x = u \}}\end{example} \\ \hline
    % \begin{example}{a}{a}{a}{a}\end{example} \\ \hline
  \end{tabular}
  \captionsetup{singlelinecheck=off}
  \caption[Matching examples]{Selected examples to illustrate the matching procedure.
    Patterns and values are all formulas; types of the other symbols should be inferred consequently. Detailed explanations for each example are provided below:}
  \label{tab:matching-examples}
\end{table}

\begin{enumerate}
\itemsep-0.1cm
\item Simple pattern matching, ${?a}$ is assigned the value $c$, and $b$ is treated as a constant.
\item The provided partial assignment prevents the matching from succeeding. Note that $\{ ?a \mapsto b, ?b \mapsto a \}$ would still be a valid assignment according to OCBSL but it would not be returned by our procedure.
\item ${?a}$ must match $c \land b$ but should also satisfy the constraint $b \land c$; since both are OCBSL equivalent, the procedure succeeds.
\item ${?a}$ is treated as a constant in the value and has nothing to do with ${?a}$ in the pattern.
\item ${?a}$ is assigned $b \lor c$ (left branch), then checked for equivalence against $c \lor b$.
\item All occurrences of $t$ are factored out, resulting in $\lambda x. x = u$.
\item The problem is under-constrained; an infinite number of solutions exist.
\item By specifying the argument, the problem is reduced to (6).
\item By specifying the predicate, the problem is transformed into the matching of $t = u$ against pattern ${?x} = u$ (where ${?x}$ is a fresh variable).
\item The first pattern/value pair is the same as (7), however the second pair adds the necessary constraint to reduce the problem to (8).
\item The problem is under-constrained because the argument contains an unresolved variable.
\item Similarly to (10), the variable is resolved while handling the second pair unlocking the first one. In addition, both branches are factored because they are equivalent.
\item This example showcases the renaming of bound/free variables.
\item Here ${?a}$ is a variable that should match a constant. However the formula $y = u$ depends on the bound variable $y$, and thus cannot match that pattern.
\item This is a fix for (14): the bound variable appears as an argument of the predicate variable.
\end{enumerate}

\subsection{Algorithm\texorpdfstring{\footnote{The implementation of this algorithm can be found in the file \href{https://github.com/FlorianCassayre/master-project/blob/master/src/main/scala/me/cassayre/florian/masterproject/front/proof/unification/UnificationUtils.scala}{\code{UnificationUtils.scala}}.}}{Lg}}

\subsubsection{Preliminary checks}

We should initially check that the inputs are well-formed. For instance the pattern should satisfy the requirements described in the previous section. Since the patterns and values are matched sequentially, we should check that the lengths of the two sequences correspond. Other checks on the parameters should be performed but are not described in details here.

\subsubsection{Fresh names}

Schemas and variables have a different meaning depending whether they appear in the pattern or in the value. One of the requirements was to avoid extending the first order logic package with new datatypes. To disambiguate both meanings there are two options. The first one consists of defining a new intermediate representation that is exclusive to patterns. The alternative approach is to rename schemas and variable in such a way that we can distinguish them from each other. The second option was chosen not only because its implementation is significantly shorter, but because it also simplified some use cases as schemas were designed to be instantiated.

The first step of the algorithm is therefore to list all schemas and variables on both sides, and then to rename them by freshly generated identifiers. The mapping between the original names and the fresh ones should be kept because we will use it to restore the names at the end. The mapping also serves as a predicate to determine whether a name is a pattern variable or a constant.

\subsubsection{Constraints collection}

Pattern variables can appear as arguments of other pattern variables, which does not make it possible to solve the constraints in a single traversal. Instead, we decompose the problem into different phases, which can be mutually recursive. The first one of these being the constraints collection phase. A constraint is a triplet consisting of a pattern variable, its arguments and a target value. In this phase we traverse the pattern and value trees at the same time, in pre-order. As long as a pattern node is constant, the only liability of this procedure is to ensure that both sides are structurally equal. When a pattern variable is encountered, a constraint is returned and the traversal stops for this branch. At any point, if a contradiction is detected, it is transferred back to the first caller in order to halt the procedure; we further refer to that as a \textbf{failure}.

\subsubsection{Pattern to value matching}

At this point we have not determined to what value each pattern corresponded to. This information may be provided explicitly but may as well be omitted. This can be seen as an instance of a bipartite matching problem, however one that we cannot easily solve efficiently. The reason is because the pattern to value matching cannot be considered independent, therefore we cannot construct the bipartite graph efficiently. The solution we adopted was to enumerate all possible bipartite matching instances in a lazy manner, and then call the constraints resolution on each such matching until a valid solution is found. We argue that in practice the cost associated to this solution is not high as the number of premises and conclusions in a sequent remains low.

\subsubsection{Constraints resolution}

After collecting the constraints the objective of the next phase is to attempt to resolve them. We say that a constraint is resolved as soon as the pattern variable it represents has been assigned a value in this context, and that value is compatible with the constraint's. For that, the constraints are processed in the order they appear; by assumption these constraints were listed in pre-order. However, not all constraints can be immediately solved: sometimes the resolution of a constraint may unlock another constraint. Therefore while there remains unsolved constraints the procedure picks the first constraints that can be solved. If none exists, that means the problem is under-constrained and the procedure returns a failure. Although the current implementation of the resolution phase does not create additional constraints, other implementations that may want to return a result for under-constrained instances has the possibility to produce and return such additional constraints. In that case the new constraints are inserted at the same position in the sequence.

In this phase a constraint can be solved if and only if all the pattern variables in the pattern's arguments have been assigned a value with respect to the current context. As a consequence, all constraints represented by nullary schemas or variables can be solved immediately.

Once a constraint satisfying the above condition has been located, we can proceed with the resolution. Firstly, each argument is instantiated (thus becoming values). Then, there are two cases to consider.

If the pattern variable is already assigned a value, we instantiate this assignment with the instantiated arguments and compare it against the value for equivalence. If we cannot prove that they are equivalent we return a failure, otherwise we do not do anything and carry on.

If the pattern variable does not have an assigned value we must assign one to it. If applicable we generate fresh schemas for the arguments of this assignment. We then apply a greedy factoring algorithm to replace all occurrences of the instantiated arguments by their associated schema. Again, we traverse the tree in pre-order to satisfy the specification, and then find the first instantiated argument that is equivalent to this node. The factoring procedure cannot fail: a special case could be that no value is factored, however that is still a valid solution.

\subsubsection{Conclusion}

If all the constraints have been solved, we can return the assignment as a solution after restoring the original pattern variable names.

\subsection{Analysis}

The above procedure is polynomial but could be implemented more efficiently by avoiding recomputing substitutions. As for soundness, the weak version states that any assignment returned by the procedure should make the pattern equivalent to the value after instantiation. The stronger version states that the equivalence class is in fact OCBSL's. The argument in either case is by construction and depends on two properties of equivalence (noted $\equiv$):

\begin{gather}
  {x \equiv y \land y \equiv z} \implies {x \equiv z} \tag{\textsc{Transitivity}} \\[1em]
  {a \equiv b} \implies {x[a \mapsto b] \equiv x} \tag{\textsc{Substitution}}
\end{gather}

It is easy to see that these properties hold for logical equivalence. For the substitution property of OCBSL equivalence, since $a \equiv b$, both $a$ and $b$ reduce to a common normal form $n$. Therefore $x[a \mapsto b] \equiv x$ is the same problem as $x[a \mapsto b][\{a, b\} \mapsto n] \equiv x[\{a, b\} \mapsto n]$ which is a logical tautology. For transitivity we have that $\{x, y\}$ have a normal form $n_1$, $\{y, z\}$ a normal form $n_2$, $\{x, z\}$ a normal form $n_3$. Because $y$ has a normal form equal to $n_1$ and $n_2$ it must be the case that $n_1 = n_2$; therefore $n_1 = n_2 = n_3$.
It should be noted that in any case, $x \equiv y \implies x \Leftrightarrow y$. Therefore, even though a resolved pattern may not be checked for equivalence (e.g. if a weaker or alternative checker is used), it is still expected to be logically equivalent.

\subsection{Limitations and extensions}

The procedure is by design incomplete, i.e. there are problem instances where a matching (as we defined it) exists but would not be found. For instance, the pattern ${?a} \land b$ matches the value $b \land a$ but despite being OCBSL equivalent, it would not be detected by the algorithm. This issue could be solved relatively easily at an exponential cost by enumerating all the equivalent formulas and calling matching on them. We suspect this problem to be solvable in polynomial time, as an extension of OCBSL, but we could not come up with an algorithm to demonstrate it.

\subsection{Applications}

The first application of matching is to automatically infer arguments of rules (\autoref{sec:proof-framework-rules}). This reduces the burden of applying rules. Because rules are defined in a general way, it is particularly desirable to solve that problem using the same procedure.

While introduction rules can usually be applied backwards without ambiguity, other rules may need some parameters to be specified explicitly. This motivates the introduction of an additional parameter: an optional partial assignment that constrains the solution space. It can also be used as a hint when the framework picked the ``wrong'' solution. It is guaranteed that the partial assignment will be subset of the returned solution, if there is any. In general, if a pattern variable appears at the target side but not at the source, then it must be specified.

Because pattern variables are encoded as schemas, and since our sequent calculus provides a rule to instantiate schemas it is possible to use regular sequents as patterns. In practice we make us of this property to automatically instantiate justifications (\autoref{sec:proof-framework-justifications}).


\documentclass [12pt, executivepaper]{article}

\usepackage{mathtools}

\usepackage{outlines}

\usepackage{amsfonts}

\usepackage{booktabs}

\usepackage{graphicx}

\everymath{\displaystyle}

\usepackage{hyperref}

\usepackage{graphics}

\usepackage{commath}

\usepackage{adjustbox}

\usepackage{listings}

\begin{document}

\title {Technical Interview Prep Guide}
\author{Brendan Busey\\Wake Forest University\\ busebd12@wfu.edu}
\maketitle

\pagebreak

\vspace*{-40mm}

\section*{Complexity Analysis}

\begin{enumerate}

\item Big Oh Definitions:

\begin{enumerate}

\item $f(n)=\mathcal{O}(g(n))$ means $c \cdot g(n)$ is an upper bound on $f(n)$. Thus, there exists some constant $c$ such that $f(n)$ is always $\leq$ $c \cdot g(n)$, for large enough $n$

\item $f(n)=\Omega(g(n))$ means $c \cdot g(n)$ is a lower bound on $f(n)$. Thus, there exists some constant $c$ such that $f(n)$ is always $\geq$ $c \cdot g(n)$, for all $n$ $\geq$ $n_{0}$

\item $f(n)=\Theta(g(n))$ means $c_{1} \cdot g(n)$ is an upper bound on $f(n)$ and $c_{2} \cdot g(n)$ is a lower bound on $f(n)$, for all $n$ $\geq$ $n_{0}$. Thus, there exists constants $c_{1}$ and $c_{2}$ such that
$f(n) \leq c_{1} \cdot g(n)$ and $f(n) \geq c_{2} \cdot g(n)$. This means that $g(n)$ provides a nice, tight bound on $f(n)$.

\end{enumerate}

\item Tips on analysis:

\begin{enumerate}

\item If your algorithm is in the form ``do this, then, when you're all done, do that," then you add the runtimes

\item If your algorithm is in the form ``do this for each time you do that," then you multiply the runtimes

\item Any algorithm that repeatedly doubles or halves, you should be thinking logarithmic complexity aka $\mathcal{O}(\log{} n)$
\item Amortized time allows us to describe that, yes, this worst case happens every once in a while. But, once this happens, it won't happen again for so long that the cost is ``amortized"

\item When you have a recursive function that makes multiple calls, the runtime will often (but not always) look like $\mathcal{O}(branches^{depth})$, where branches is the number of times each recursive call branches aka
how many times the recursive function is called

\item Memoization is a very common technique used to optimize exponential time recursive algorithms

\end{enumerate}

\end{enumerate}

\pagebreak

\vspace*{-40mm}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%START OF SORTING SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Sorting}

%%%%%%%%%%START OF 	QUICKSORT%%%%%%%%
\begin{enumerate}

\item QuickSort

\begin{enumerate}

\item Complexity: $\mathcal{O}(n \log{} n)$

\item How it works:

\begin{enumerate}

\item Choose the pivot value: we take the value of the middle element as the pivot value, but it can be any value, which is in range of sorted values, even if it doesn't appear in the array.

\item Partition: Rearrange elements in a such a way that all elements which are less than the pivot go the left part of the array and all elements greater than the pivot go to the right part of the array. Values equal to pivot can stay in any part of the array. Note, that the array may be divided into non-equal parts.

\item Sort both parts: apply the Quicksort algorithm recursively to the left and right parts.

\end{enumerate}

\item Advantage(s):

\begin{enumerate}

\item Fast and efficient for large data sets

\item Can carry out sequential traversal through array elements which results in good locality of reference and cache behaviour for arrays

\end{enumerate}

\item Disadvantage(s):

\begin{enumerate}

\item Not efficient if the elements are already sorted and if each element in the array is equal (gives worst case time complexity of $\mathcal{O}(n^2)$)

\item Might be space expensive for large data sets due to the fact that it uses $\mathcal{O}(\log{} n)$ auxiliary space for recursive function calls

\end{enumerate}

\pagebreak

\vspace*{-40mm}

\item Implementation

\includegraphics[scale=0.5]{QuickSort}

\end{enumerate}
%%%%%%%%%END OF QUICKSORT%%%%%%

%%%%%%%%%START OF MERGE SORT%%%%

\item Merge Sort

\begin{enumerate}

\item Complexity: $\mathcal{O}(n \log{} n)$

\item How it works:

\begin{enumerate}

\item We partition the elements into two groups, sorting each of the smaller problems recursively, and then interleaving the two sorted lists to totally order the elements

\end{enumerate}

\item Advantage(s):

\begin{enumerate}

\item Want to use it when the data is stored in a linked list, because merging does not require random access to the list elements

\end{enumerate}

\item Disadvantage(s):

\begin{enumerate}

\item Requires twice as much memory as any other sophisticated sorting algorithm and likewise is not recommended for smaller arrays for the reason that it works recursively and it requires
$\mathcal{O}(n)$ auxiliary space for sorting

\item It is difficult to implement the merge operation

\end{enumerate}

\pagebreak

\vspace*{-40mm}

\item Implementation

\includegraphics[scale=0.5]{MergeSortPart1}

\vspace{3mm}

\includegraphics[scale=0.5]{MergeSortPart2}

\end{enumerate}
%%%%%%%%%END OF MERGE SORT%%%%%

\pagebreak

\vspace*{-40mm}

%%%%%%%%%START OF HEAP SORT%%%%%
\item Heap Sort

\begin{enumerate}

\item Complexity: $\mathcal{O}(n \log{} n)$

\item How it works:

\begin{enumerate}

\item It is similar to insertion sort where we first find the maximum element, exchange it with the last element, and then re-make the heap. We then repeat the process for the remaining elements.

\end{enumerate}

\item Advantage(s):

\begin{enumerate}

\item Used often for large data sets because it does not work recursively all the time

\end{enumerate}

\item Disadvantages(s):

\begin{enumerate}

\item Works slower than other sorting methods with the same computational complexity

\item Not efficient for parallelization

\item Not recommended for sorting data that is stored in a linked list because it is difficult to convert a linked list into a heap like structure

\end{enumerate}

\end{enumerate}
%%%%%%%%%END OF HEAP SORT%%%%%%
\end{enumerate}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%END OF SORTING SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\pagebreak

\vspace*{-40mm}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%START OF HASH TABLE SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Hash Table(s)}

\begin{enumerate}

\item How do they work?

\begin{enumerate}

\item Let's assume you want to fill up a library of books and not just stuff them in there, but you want to be able to easily find them again when you need them.\\

So, you decide that if the person that wants to read a book knows the title of the book and the exact title to boot, then that's all it should take. With the title, the person, with the aid of the librarian, should be able to find the book easily and quickly. \\

So, how can you do that? Well, obviously you can keep some kind of list of where you put each book, but then you have the same problem as searching the library, you need to search the list. Granted, the list would be smaller and easier to search, but still you don't want to search sequentially from one end of the library (or list) to the other.\\

You want something that, with the title of the book, can give you the right spot at once, so all you have to do is just stroll over to the right shelf, and pick up the book.\\

But how can that be done? Well, with a bit of forethought when you fill up the library and a lot of work when you fill up the library.\\

Instead of just starting to fill up the library from one end to the other, you devise a clever little method. You take the title of the book, run it through a small computer program, which spits out a shelf number and a slot number on that shelf. This is where you place the book.\\

\pagebreak

\vspace*{-40mm}

The beauty of this program is that later on, when a person comes back in to read the book, you feed the title through the program once more, and get back the same shelf number and slot number that you were originally given, and this is where the book is located.\\

The program is called a hash algorithm or hash computation and usually works by taking the data fed into it (the title of the book in this case) and calculates a number from it.\\

For simplicity, let's say that it just converts each letter and symbol into a number and sums them all up. In reality, it's a lot more complicated than that, but let's leave it at that for now.\\

The beauty of such an algorithm is that if you feed the same input into it again and again, it will keep spitting out the same number each time.\\

Ok, so that's basically how a hash table works.

\end{enumerate}

\item (One possible) Implementation (please, note that for brevity, I have only provided the core functions of the Hash Table class. If the reader wants to run the code, you will have to complete the template class implementation):

\begin{adjustbox}{max height=2in, max width=2in}

\lstinputlisting[language=C++]{Node.h}

\end{adjustbox}

\pagebreak

\vspace*{-40mm}

\includegraphics[scale=0.5]{HashTableHeaderfilePart1}

\includegraphics[scale=0.5]{HashTableHeaderfilePart2}

\pagebreak

\vspace*{-40mm}

\includegraphics[scale=0.5]{HashTableHeaderfilePart3}

\includegraphics[scale=0.5]{HashTableHeaderfilePart4}

\item Complexities

\begin{enumerate}

\item Insertion: $\mathcal{O}(1)$

\item Deletion: $\mathcal{O}(1)$

\item Search: $\mathcal{O}(1)$

\end{enumerate}

\end{enumerate}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%END OF HASH TABLE SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\pagebreak

\vspace*{-40mm}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%START OF TREE SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Tree(s)}

\begin{enumerate}

%%%%%%%%%START OF BINARY TREE SECTION%%%%%%%%%%
\item Binary Trees

\begin{enumerate}

\item A data structure in which each node has two children, a left and right child

\item Major Algorithms

\begin{enumerate}

\item Pre-order traversal (Type of depth first traversal/search)

\begin{enumerate}

\item Visit order: root, left child, right child

\item Complexity: $\mathcal{O}(n)$, where $n$ is the number of nodes in your tree

\end{enumerate}

\item In-order traversal (type of depth first search/traversal)

\begin{enumerate}

\item Visit order: left child, root, right child

\item Complexity: $\mathcal{O}(n)$, where $n$ is the number of nodes in your tree

\end{enumerate}

\item Post-order traversal (type of depth first search/traversal)

\begin{enumerate}

\item Visit order: left child, right child, root

\item Complexity: $\mathcal{O}(n)$, where $n$ is the number of nodes in your tree

\end{enumerate}

\item Breadth First Traversal/Search

\begin{enumerate}

\item How it works: traverses the tree one level at a time, left to right within a level

\item Time complexity: $\mathcal{O}(n)$, where $n$ is the number of nodes since you have to visit all the nodes in the tree

\item Space complexity: $\mathcal{O}(n)$, where $n$ is the number of nodes since you have to hold all of the nodes in a queue

\item Implementation:

\vspace{1mm}

\includegraphics[scale=0.5]{BreadthFirstTraversalForTree}

\end{enumerate}

\end{enumerate}

\end{enumerate}
%%%%%%%END OF BINARY TREE SECTION%%%%%%%%

\pagebreak

\vspace*{-40mm}

%%%%%%%START OF N-ARY TREE SECTION%%%%%%%
\item $N$-ary Trees

\begin{enumerate}

\item Is a data structure that consists of a root node and then $N$ subtrees

\item Major Algorithms

\begin{enumerate}

\item Pre-order traversal

\begin{enumerate}

\item Print out the current node, then loop through the children of the current node and recursively call the pre-order traversal function on each of them

\item Complexity: $\mathcal{O}(n)$, where $n$ is the number of nodes since you have to visit all the nodes in the tree

\end{enumerate}

\item Post-order traversal

\begin{enumerate}

\item Loop through the children of the current node and recursively call the post-order traversal function on each of them, then print out the current node

\item Complexity: $\mathcal{O}(n)$, where $n$ is the number of nodes since you have to visit all the nodes in the tree

\end{enumerate}

\item Breadth First Search

\vspace{1mm}

\includegraphics[scale=0.5]{BreadthFirstSearchN-aryTree}

\begin{enumerate}

\item Complexity: $\mathcal{O}(n)$, where $n$ is the number of nodes since you have to visit all the nodes in the tree

\end{enumerate}

\item Depth First Search

\vspace{1mm}

\includegraphics[scale=0.5]{DepthFirstSearchN-aryTree}

\pagebreak

\vspace*{-40mm}

\begin{enumerate}

\item Complexity: $\mathcal{O}(n)$, where $n$ is the number of nodes since you have to visit all the nodes in the tree

\end{enumerate}

\item Implementation: each node will have data associated with it as well as a vector to store it's $n$ other childen

\end{enumerate}

\end{enumerate}
%%%%%%%END OF N-ARY TREE SECTION%%%%%%%%

%%%%%%%START OF TRIE SECTION%%%%%%%%%%%

\item Trie(s)

\begin{enumerate}

\item A tree where every vertex represents either a word or prefix

\item Major Algorithms:

\begin{enumerate}

\item Similar to those for $N$-ary trees

\end{enumerate}

\item Implementation: identical to that of a $N$-ary tree except that you will be storing characters in the vector

\end{enumerate}
%%%%%%%END OF TRIE SECTION%%%%%%%%%%%%

%%%%%%%START OF AVL TREE SECTION%%%%

\item AVL Tree(s)

\begin{enumerate}

\item Is a binary search tree with the following properties:

\begin{enumerate}

\item The sub-tree(s) of every node differ in height by exactly one

\item Every sub-tree is also an AVL tree

\end{enumerate}

\item Why do we care about AVL tree(s)?

\begin{enumerate}

\item Most of the BST operations (e.g., search, max, min, insert, delete.. etc) take $\mathcal{O}(h)$ time where $h$ is the height of the BST. The cost of these operations may become $\mathcal{O}(n)$ for a skewed Binary tree.
If we make sure that height of the tree remains $\mathcal{O}(\log{} n)$ after every insertion and deletion, then we can guarantee an upper bound of $\mathcal{O}(\log{} n)$ for all these operations.

\end{enumerate}

\item Insertion process for a node $n$

\begin{enumerate}

\item Perform the standard Binary Search Tree insertion for $n$

\item Starting from $n$, travel up in the tree until we find the first unbalanced node, call it $z$

\item Re-balance the tree by performing the appropriate rotations for the sub-tree with root $z$

\pagebreak

\vspace*{-40mm}

\item There are four possible cases to consider for re-balancing:

\begin{enumerate}

\item Case $1$: Left Left

\vspace{2mm}

\includegraphics[scale=0.5]{RotationCase1AVLTree}

\item Case $2$: Left Right

\vspace{2mm}

\includegraphics[scale=0.5]{RotationCase2AVLTree}

\item Case $3$: Right Right

\vspace{2mm}

\includegraphics[scale=0.5]{RotationCase3AVLTree}

\item Case $4$: Right Left

\vspace{2mm}

\includegraphics[scale=0.5]{RotationCase4AVLTree}

\end{enumerate}

\item Implementation (in words) for insertion:

\begin{enumerate}

\item Perform the normal Binary Search Tree insertion

\item The current node must be one of the ancestors of the newly inserted node. Update the height of the current node.

\item Get the balance factor (left subtree height - right subtree height)

\item If the balance factor is greater than $1$, then the current node is unbalanced and we are either in the Left Left case or the Left Right case. We are in the Left Left case if the newly inserted key is less than the key of its left sub-tree. We are in the Left Right case if the newly inserted key is greater than the key of its left sub-tree.

\pagebreak

\vspace*{-40mm}

\item If the balance factor is less than $-1$, then the current node is unbalanced and we are either in the Right Right case or the Right Left case. We are in the Right Right case if the newly inserted key is greater than the key of its right sub-tree. We are in the Right Left case if the newly inserted key is less than the key of its right sub-tree.

\end{enumerate}

\end{enumerate}

\item Deletion process for a node $n$

\begin{enumerate}

\item Perform the normal deletion for Binary Search Tree

\item Starting from $n$, travel up in the tree until we find the first unbalanced node, call it $z$

\item Re-balance the tree by performing the appropriate rotations for the sub-tree with root $z$

\item There are four cases to consider for re-balancing:

\begin{enumerate}

\item Case $1$: Left Left

\vspace{2mm}

\includegraphics[scale=0.5]{RotationCase1AVLTree}

\item Case $2$: Left Right

\vspace{2mm}

\includegraphics[scale=0.5]{RotationCase2AVLTree}

\item Case $3$: Right Right

\vspace{2mm}

\includegraphics[scale=0.5]{RotationCase3AVLTree}

\item Case $4$: Right Left

\vspace{2mm}

\includegraphics[scale=0.5]{RotationCase4AVLTree}

\end{enumerate}

\pagebreak

\vspace*{-40mm}

\item Implementation for deletion:

\begin{enumerate}

\item Perform the normal Binary Search Tree deletion

\item The current node must be one of the ancestors of the deleted node; update the height of the current node

\item Get the balance factor (left subtree height - right subtree height) of the current node

\item If the balance factor is greater than $1$, then the current node is unbalanced and we are either in Left Left case or Left Right case. To check whether it is the Left Left case or the Left Right case, get the balance factor of the left subtree. If the balance factor of the left subtree is greater than or equal to $0$, then it is the Left Left case, else it is the Left Right case.

\item If the balance factor is less than $-1$, then the current node is unbalanced and we are either in the Right Right case or the Right Left case. To check whether it is the Right Right case or the Right Left case, get the balance factor of right subtree. If the balance factor of the right subtree is less than or equal to $0$, then it is the Right Right case, else it is the Right Left case.

\end{enumerate}

\end{enumerate}

\end{enumerate}
%%%%%%%END OF AVL TREE SECTION%%%%%

%%%%%%%START OF BREADTH FIRST AND DEPTH FIRST COMPARISON SECTION%%%%%
\item Breadth First and Depth First Comparison

\begin{enumerate}

\item When to use Breadth First

\begin{enumerate}

\item If you know what you want to search for is not far from the root

\item If the tree is very deep and solutions are not very common

\end{enumerate}

\item When to use Depth First

\begin{enumerate}

\item If the tree is wide, since breadth first might take up too much memory

\item If solutions are frequent and located deep in the tree, breadth first might take too long

\end{enumerate}

\item Complexities

\begin{enumerate}

\item Breadth First: $\mathcal{O}(n)$

\item Depth First: $\mathcal{O}(n)$

\end{enumerate}

\end{enumerate}
%%%%%%%END OF BREADTH FIRST AND DEPTH FIRST COMPARISON SECTION%%%%%%

\end{enumerate}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%END OF TREE SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\pagebreak

\vspace*{-40mm}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%START OF GRAPH SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Graphs}

\begin{enumerate}

\item Adjacency Matrix Representation

\begin{enumerate}

\item is a 2D array of size $V$ x $V$ where $V$ is the number of vertices in a graph

\item Complexity:

\begin{enumerate}

\item Adding an edge $\rightarrow \mathcal{O}(1)$

\item Deleting an edge $\rightarrow \mathcal{O}(1)$

\item Answering the question ``is there an edge between two vertices" $\rightarrow \mathcal{O}(1)$

\item Finding the successors of a given vertex $\rightarrow \mathcal{O}(n)$

\item Determining if a path exists between two vertices $\rightarrow \mathcal{O}(n^2)$

\end{enumerate}

\item Advantage(s):

\begin{enumerate}

\item Representation is easier to implement and follow

\item Removing an edge takes constant time

\item Answering the question ``is there an edge between two vertices" can be answered in constant time

\end{enumerate}

\item Disadvantage(s):

\begin{enumerate}

\item Space complexity is $\mathcal{O}(V^2)$, where $V$ is the number of vertices

\item Even if the graph is sparse, still takes up $\mathcal{O}(V^2)$ space

\item Adding a vertex is $\mathcal{O}(V^2)$ time

\end{enumerate}

\end{enumerate}

\item Adjacency List Representation

\begin{enumerate}

\item We keep a list of all vertices and each vertex within that list has its own list that contains their adjacent vertices

\item Complexity:

\begin{enumerate}

\item Adding an edge $\rightarrow \mathcal{O}(\log{} V)$

\item Deleting an edge $\rightarrow \mathcal{O}(\log{} V)$

\item Answer the questions "is there an edge between two vertices"  $\rightarrow \mathcal{O}(\log{} V)$

\item Finding the successor of a given vertex $\rightarrow \mathcal{O}(\log{} n)$, where $n$ is the length of the lists containing the successors of a given vertex

\pagebreak

\vspace*{-40mm}

\item Determining if a path exists between two vertices $\rightarrow \mathcal{O}(V+E)$, where $V$ is number of vertices and $E$ is the number of edges

\end{enumerate}

\item Advantage(s):

\begin{enumerate}

\item Saves space; average space complexity is $\mathcal{O}(V+E)$; worst case is $\mathcal{O}(V^2)$

\item Adding a vertex is easier

\end{enumerate}

\item Disadvantage(s):

\begin{enumerate}

\item Queries like whether there is an edge from vertex $u$ to vertex $v$ are not efficient and can be done $\mathcal{O}(V)$, where $V$ is the number of vertices

\end{enumerate}

\end{enumerate}

\item Major Algorithms

\begin{enumerate}

\item Breadth First Traversal

\begin{enumerate}

\item Complexity: $\mathcal{O}(V+E)$, where $V$ is the number of vertices and $E$ is the number of edges

\item Implementation:

\vspace{1mm}

\includegraphics[scale=0.5]{BreadthFirstSearchForGraph}

\end{enumerate}

\pagebreak

\vspace*{-40mm}

\item Depth First Search

\begin{enumerate}

\item Complexity: $\mathcal{O}(V+E)$, where $V$ is the number of vertices and $E$ is the number of edges

\item Implementation:

\vspace{1mm}

\includegraphics[scale=0.5]{DepthFirstSearchGraph}

\vspace{3mm}

\includegraphics[scale=0.5]{DepthFirstSearchGraphIterative}

\end{enumerate}

\item Dijkstra's Algorithm

\begin{enumerate}

\item Description:

\includegraphics[scale=0.4]{DijkstraAlgorithmDescription}

\pagebreak

\vspace*{-40mm}

\item Implementation (in pseudo code):

\vspace{1mm}

\includegraphics[scale=0.4]{DijkstraPseudoCode}

\end{enumerate}

\item A* Algorithm

\begin{enumerate}

\item A* is like Dijkstra's algorithm in that it can be used to find a shortest path

\item A* is like Greedy Best-First-Search in that it can use a heuristic to guide itself

\item The secret to its success is that it combines the pieces of information that Dijkstra's algorithm uses (favoring vertices that are close to the starting point) and information that Greedy Best-First-Search uses (favoring vertices that are close to the goal)

\item When talking about the algorithm, $g(n)$ represents the exact cost of the path from the starting point to any vertex $n$

\item When talking about the algorithm, $h(n)$ represents the heuristic estimated cost from vertex n to the goal

\end{enumerate}

\end{enumerate}

\end{enumerate}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%END OF GRAPH SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%START OF NP-COMPLETE PROBLEM(S) SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{NP-Complete Problem(s)}

\begin{enumerate}

\item NP stands for Non-deterministic Polynominal time

\item A yes-or-no problem is said to be NP if a yes can be verified in polynominal time

\item NP-complete is a family of NP problems for which you know that if one had a polynominal time solution, then everyone of them has a polynominal time solution

\item Some famous examples of NP problems:

\pagebreak

\vspace*{-40mm}

\begin{enumerate}

\item Traveling Salesman problem: finding the shortest path (on a graph) that allows you to visit each city exactly once

\item Bin packing problem: there are a number of fixed (integer) size bins and objects of varying sizes. Minimize the number of bins required to hold all of the objects

\item Knapsack problem: given objects of various sizes and values and a knapsack with a fixed integer size, choose the objects that can fit inside with the most value

\item Minimal Vertex Cover: finding the smallest set of vertices such that every edge contains at least one chosen vertex

\item Clique: finding that largest group of people who all know each other

\item Subgraph Isomorphism: does one graph contain a subgraph isomorphic to another?

\item Set packing: given a number of sets, what is the maximum number of disjoint sets that can be selected? This is related to set cover, where we are trying to choose sets so that every element is within at least one set

\item Subset sum: Given a set of integers, does some subset sum to 0?

\end{enumerate}

\end{enumerate}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%END OF NP-COMPLETE PROBLEM(S) SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\pagebreak

\vspace*{-40mm}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%START OF OPERATING SYSTEM SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Operating Systems}

\begin{enumerate}

\item Process: an instance of computer program in execution

\item Thread: a basic unit of CPU utilization, often called a ``lightweight" process

\item Concurrency issues

\begin{enumerate}

\item Race condition(s): behaviour of software or system where the output is dependent on the sequence or timing of other uncontrollable events

\item Deadlock: a specific condition when two or more processes are each waiting for another to release a resource, or more than two processes are waiting for resources in a circular chain

\item Livelock: similar to a deadlock, except that the states of the processes involved in the livelock constantly change with regard to one another, none progressing

\end{enumerate}

\item Lock(s): something that programmers annotate source code with, especially critical sections, to ensure that any such critical section executes as if it was a single atomic instruction

\item Mutex(es): used to serialise access to a section of  re-entrant code that cannot be executed concurrently by more than one thread

\begin{enumerate}

\item Real world example: Imagine a mutex as a key to a toilet. One person can have the key - occupy the toilet - at a time. When they are finished, the person gives (frees) the key to the next person in the queue

\end{enumerate}

\item Semaphore(s): something that restricts the number of simultaneous users (threads) of a shared resource up to a maximum number

\begin{enumerate}

\item Real world example: Going back to our earlier toilet example, a semaphore is the number of free identical toilet keys. Say we have four toilets with identical locks and keys. The semaphore count - the count of keys - is set to 4 at beginning (all four toilets are free), then the count value is decremented as people are coming in. If all toilets are full, ie. there are no free keys left, the semaphore count is 0.

\pagebreak

\vspace*{-40mm}

Now, when one person leaves the toilet, the semaphore count is increased to 1 (one free key), and given to the next person in the queue

\end{enumerate}

\item Monitor(s): A monitor is a set of multiple routines which are protected by a mutual exclusion lock

\begin{enumerate}

\item Four main components:

\begin{enumerate}

\item Initialization: contains the code that is used exactly once when the monitor is created

\item Private data: private data, including private procedures, that can only be used within the monitor

\item Monitor procedure(s): procedures that can be called from outside of the monitor

\item Entry queue: contains all threads that called monitor procedures but have not been granted permissions

\end{enumerate}

\end{enumerate}

\item Deadlock

\begin{enumerate}

\item Technical definition: a specific condition when two or more processes are each waiting for another to release a resource, or more than two processes are waiting for resources in a circular chain

\item Real world example: imagine two children are rummaging through a toy box because they want to play with a drum. However, one child finds the drumstick while the other finds the actual drum. Now, both children want to play with the drum but in order for this to happen, either the child with the drumstick has to give up the drumstick or the child with the drum has to give up the drum. Of course, since both children want to play with the drum, neither is going to do this, so both will be waiting forever for the other to give up what they have.

\end{enumerate}

\item Livelock

\begin{enumerate}

\item Technical definition: similar to a deadlock, except that the states of the processes involved in the livelock constantly change with regard to one another, none progressing

\item Real world example: two people meet in a narrow corridor, and each tries to be polite by moving aside to let the other pass, but they end up swaying from side to side without making any progress because they both repeatedly move the same way at the same time

\end{enumerate}

\pagebreak

\vspace*{-40mm}

\item What resources does a thread need?

\begin{enumerate}

\item Thread ID

\item Program counter

\item Function stack

\item Set of registers

\end{enumerate}

\item What resources does a process need?

\begin{enumerate}

\item Process Control Block

\begin{enumerate}

\item Process state: new, ready, running, waiting, terminating

\item Process ID and parent process ID

\item CPU registers and program counter

\item CPU scheduling information such as priority information and pointers to scheduling queues

\item Memory management information such as page tables or segment tables

\item Accounting information such as user and kernel CPU time consumed, account numbers, limits, etc

\item I/O status information such as devices allocated, open file tables, etc

\end{enumerate}

\end{enumerate}

\item Context Switching

\begin{enumerate}

\item What is it?

\begin{enumerate}

\item Context switching is the process of switching a process from working on one task to working on another even before the former task is completed

\end{enumerate}

\item What does it involve?

\begin{enumerate}

\item This involves saving the state of all volatile data like registers, program counter, memory, etc. (in other words the ``context" of the process)  to persistent storage and then loading up the context of a new process

\end{enumerate}

\item Before we get to process switching, we have to talk about the steps in thread context switching:

\pagebreak

\vspace*{-40mm}

\begin{enumerate}

\item All context switches are initiated by an 'interrupt'. This could be an actual hardware interrupt that runs a driver, (eg. from a network card, keyboard, memory-management or timer hardware), or a software call, (system call), that performs a hardware-interrupt-like call sequence to enter the OS

\item Non-trivial systems will have to initiate a hardware-protection-level change to enter a kernel-state so that the kernel code/data etc. can be accessed

\item Core state for the interrupted thread has to be saved. On a simple embedded system, this might just be pushing all registers onto the thread stack and saving the stack pointer in its Thread Control Block

\item Many systems switch to an OS-dedicated stack at this stage so that the bulk of OS-internal stack requirements are not inflicted on the stack of every thread

\item It may be necessary to mark the thread stack position where the change to interrupt-state occurred to allow for nested interrupts

\item The driver/system call runs and may change the set of ready threads by adding/removing TCB's from internal queues for the different thread priorities, eg. network card driver may have set an event or signaled a semaphore that another thread was waiting on, so that thread will be added to the ready set, or a running thread may have called sleep() and so elected to remove itself from the ready set

\item The OS scheduler algorithm is run to decide which thread to run next, typically the highest-priority ready thread that is at the front of the queue for that priority

\item The saved stack pointer from the TCB for that thread is retrieved and loaded into the hardware stack pointer

\item The core state for the selected thread is restored. On my simple system, the registers would be popped from the stack of the selected thread. More complex systems will have to handle a return to user-level protection

\item An interrupt-return is performed, so transferring execution to the selected thread

\end{enumerate}

\item Okay, now that we have talked about thread context switching, we can now talk about the steps in process context switching:

\pagebreak

\vspace*{-40mm}

\begin{enumerate}

\item Process context switches are initiated by a thread-context switch, so all of the above, 1-9, is going to need to happen

\item At step 5 in the thread context switching process, the scheduler decides to run a thread belonging to a different process from the one that owned the previously-running thread

\item The memory-management hardware has to be loaded with the address-space for the new process, i.e. whatever selectors/segments/flags/whatever that allow the thread(s) of the new process to access its memory

\item The context of any Floating Point Unit  hardware needs to be saved/restored from the Process Control Block

\item There may be other process-dedicated hardware that needs to be saved/restored

\end{enumerate}

\end{enumerate}

\end{enumerate}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%END OF OPERATING SYSTEM SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%START OF COMPLEXITIES SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Complexities}

Data structures

\vspace{3mm}

\begin{adjustbox}{max height=6in, max width=6in}

    \begin{tabular}{l*{6}{c}r}

    Data Structure & Access (average) & Search (average) & Insertion (average) & Deletion (average) & Space \\

    \hline

    Singly-Linked List & $\mathcal{O}(n)$ & $\mathcal{O}(n)$ & $\mathcal{O}(1)$ & $\mathcal{O}(1)$ & $\mathcal{O}(n)$  \\

    Doubly-Linked List & $\mathcal{O}(n)$ & $\mathcal{O}(n)$ & $\mathcal{O}(1)$ & $\mathcal{O}(1)$ & $\mathcal{O}(n)$  \\

    Array & $\mathcal{O}(1)$ & $\mathcal{O}(n)$ & $\mathcal{O}(n)$ & $\mathcal{O}(n)$ & $\mathcal{O}(n)$  \\

    Stack & $\mathcal{O}(n)$ & $\mathcal{O}(n)$ & $\mathcal{O}(1)$ & $\mathcal{O}(1)$ & $\mathcal{O}(n)$  \\

    Queue & $\mathcal{O}(1)$ & $\mathcal{O}(n)$ & $\mathcal{O}(1)$ & $\mathcal{O}(1)$ & $\mathcal{O}(n)$  \\

    Binary Tree & $\mathcal{O}(\log{} n)$ & $\mathcal{O}(\log{} n)$ & $\mathcal{O}(\log{} n)$ & $\mathcal{O}(\log{} n)$ & $\mathcal{O}(n)$  \\

    AVL Tree & $\mathcal{O}(\log{} n)$ & $\mathcal{O}(\log{} n)$ & $\mathcal{O}(\log{} n)$ & $\mathcal{O}(\log{} n)$ & $\mathcal{O}(n)$

    \end{tabular}

\end{adjustbox}

\vspace{3mm}

\hspace{-8mm} Algorithms

\vspace{3mm}

\begin{adjustbox}{max height=4in, max width=4in}

    \begin{tabular}{l*{6}{c}r}

    Algorithm & Time & Space \\

    \hline

   Quicksort & $\mathcal{O}(n \log{} n)$ & $\mathcal{O}(\log{} n)$  \\

   Mergesort & $\mathcal{O}(n \log{} n)$ & $\mathcal{O}(n)$  \\

   Heapsort & $\mathcal{O}(\log{} n)$ & $\mathcal{O}(1)$ \\

   Depth First Search (Trees) & $\mathcal{O}(n)$ & $\mathcal{O}(n)$  \\

   Breadth First Search (Trees) & $\mathcal{O}(n)$ & $\mathcal{O}(n)$  \\

   Depth First Search (Graph(s)) & $\mathcal{O}(\abs{V} + \abs{E})$ & $\mathcal{O}(\abs{V})$  \\

   Breadth First Search (Graph(s)) & $\mathcal{O}(\abs{V} + \abs{E})$ & $\mathcal{O}(\abs{V})$

    \end{tabular}

\end{adjustbox}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%END OF COMPLEXITIES SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\pagebreak

\vspace*{-40mm}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%START OF REFERENCE SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{center}

\begin{thebibliography}{9}

\bibitem{AlgorithmDesignManual}
Steven S. Skienna \\
\textit{The Algorithm Design Manual, Second Edition}.

\bibitem{CrackingTheCodingInterview}
Gayle McDowell \\
\textit{Cracking The Code Interview, 6th Edition}

\bibitem{GeeksForGeeks}
Geeks for Geeks \\
\url{http://www.geeksforgeeks.org/}

\bibitem{QuicksortImplementation}
Quicksort Implementation \\
\url{http://www.algolist.net/Algorithms/Sorting/Quicksort}

\bibitem{MergeSortImplementation}
Mergesort Implementation \\
\url{https://en.wikibooks.org/wiki/Algorithm_Implementation/Sorting/Merge_sort#C.2B.2B}

\bibitem{HashTable}
Hash Table Description \\
\url{http://stackoverflow.com/questions/730620/how-does-a-hash-table-work}

\bibitem{Trees}
Trees \\
\url{http://stackoverflow.com/questions/5262308/how-do-implement-a-breadth-first-traversal}

\vspace{1mm}

\url{http://www.cs.cmu.edu/~pattis/15-1XX/15-200/lectures/specialtrees/}

\vspace{1mm}

\url{http://www.brpreiss.com/books/opus5/html/page257.html}

\vspace{1mm}

\url{http://stackoverflow.com/questions/5987867/traversing-a-n-ary-tree-without-using-recurrsion}

\bibitem{AVLTrees}
AVL Trees \\
\url{http://www.geeksforgeeks.org/avl-tree-set-1-insertion/}

\vspace{1mm}

\url{http://www.geeksforgeeks.org/avl-tree-set-2-deletion/}

\bibitem{BFSvsDFS}
Breadth First Search vs Depth First Search \\
\url{http://stackoverflow.com/questions/3332947/when-is-it-practical-to-use-dfs-vs-bfs}

\pagebreak

\vspace*{-40mm}

\bibitem{Graphs}
Graphs \\
\url{http://stackoverflow.com/questions/3287003/three-ways-to-store-a-graph-in-memory-advantages-and-disadvantages}

\vspace{1mm}

\url{https://www.khanacademy.org/computing/computer-science/algorithms/graph-representation/a/representing-graphs}

\vspace{1mm}

\url{http://www.algorithmist.com/index.php/Graph_data_structures}

\vspace{1mm}

\url{http://www.geeksforgeeks.org/graph-and-its-representations/}

\vspace{1mm}

\url{http://www.geeksforgeeks.org/breadth-first-traversal-for-a-graph/}

\vspace{1mm}

\url{http://www.geeksforgeeks.org/depth-first-traversal-for-a-graph/}

\vspace{1mm}

\url{https://en.wikipedia.org/wiki/Depth-first_search}

\vspace{1mm}

\url{https://en.wikipedia.org/wiki/Dijkstra\%27s_algorithm}

\vspace{1mm}

\url{http://theory.stanford.edu/~amitp/GameProgramming/AStarComparison.html}

\bibitem{NPCompleteProblems}
NP Complete Stuff \\
\url{http://stackoverflow.com/questions/111307/whats-p-np-and-why-is-it-such-a-famous-question}

\vspace{1mm}

\url{http://math.stackexchange.com/questions/726/what-are-np-complete-problems-and-why-are-they-so-important}

\bibitem{OperatingSystems}
Operating Systems \\
\url{http://niclasw.mbnet.fi/MutexSemaphore.html}

\vspace{1mm}

\url{https://www.cs.uic.edu/~jbell/CourseNotes/OperatingSystems/3_Processes.html}

\vspace{1mm}

\url{https://www.cs.uic.edu/~jbell/CourseNotes/OperatingSystems/4_Threads.html}

\vspace{1mm}

\url{http://www.programmerinterview.com/index.php/operating-systems/monitors-vs-semaphores}

\vspace{1mm}

\url{https://www.cs.mtu.edu/~shene/NSF-3/e-Book/MONITOR/basics.html}

\bibitem{ContextSwitching}
Context switching \\

\url{http://stackoverflow.com/questions/7439608/steps-in-context-switching}

\pagebreak

\vspace*{-40mm}

\url{http://stackoverflow.com/questions/5440128/thread-context-switch-vs-process-context-switch?rq=1}

\bibitem{Complexity}
Complexity \\
\url{http://stackoverflow.com/questions/7294634/what-are-the-time-complexities-of-various-data-structures}

\vspace{1mm}

\url{http://bigocheatsheet.com/}

\end{thebibliography}

\end{center}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%END OF REFERENCE SECTION%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}


\subsection{Bootstrapping confidence intervals}

Need to know estimate is unbiased for this.


\chapter{Assignment-Numerical Techniques Solutions}
\begin{enumerate}
	\item $\left. \right. $
	\begin{answer}
		Taylor's series for $y(x)$ is given by
		\begin{align*}
		y(x)&=1+x y_{0}^{\prime}+\frac{x^{2}}{2} y_{0}^{\prime \prime}+\frac{x^{3}}{6} y_{0}^{\prime \prime \prime}+\frac{x^{4}}{24} y_{0}^{i v} \ldots \ldots \ldots\\
		y^{\prime \prime}(x)&=x y^{\prime}(x)+y(x) \Rightarrow y^{\prime \prime}(0)=x y^{\prime}(0)+y(0)=1\\
		y^{\prime \prime \prime}(x)&=x y^{\prime \prime}(x)+2 y^{\prime}(x) \Rightarrow y^{\prime \prime \prime}(0)=x y^{\prime \prime}(x)+2 y^{\prime}(0)=0\\
		y^{i v}(x)&=x y^{\prime \prime \prime}+3 y^{\prime \prime} \Rightarrow y^{i v}(0)=x y^{\prime \prime \prime}(0)+3 y^{\prime \prime}(0)=3\\
		y(x)&=1+\frac{x^{2}}{2}+\frac{x^{4}}{24}(3) \ldots \ldots \ldots\\
		\Rightarrow y(0.1)&=1.0050 \Rightarrow y(0.05)=0.5025
		\end{align*}
		So the correct answer is \textbf{Option (d)}
	\end{answer}
	\item $\left. \right. $
	\begin{answer}
		$h=0.5$; The value of $x$ and $y$ are tabulated below:
		\begin{align*}
		\begin{tabular}{|l|l|}
		\hline$x$ & $y$ \\
		\hline $0.0$ & $1.0000$ \\
		\hline $0.5$ & $0.6667$ \\
		\hline $1.0$ & $0.5000$ \\
		\hline
		\end{tabular}\\
		\text{Simpson's }&1 / 3\text{ rule}\\
		I=\int_{x_{0}}^{x_{n}} y d x&=\frac{h}{3}\left[y_{0}+4\left(y_{1}+y_{3}+\ldots .+y_{n-1}\right)+2\left(y_{2}+y_{4}+\ldots .+y_{n-2}\right)+y_{n}\right]\\
		\Rightarrow I&=\frac{0.5}{3}[1.0000+4(0.6667)+0.5]=0.6945
		\end{align*}
			So the correct answer is \textbf{Option (a)}
	\end{answer}
		\item $\left. \right. $
	\begin{answer}$\left. \right. $\\
		\begin{minipage}{0.45\textwidth}
		\begin{align*}
		I&=\frac{2}{3}\left[y_{0}+4\left(y_{1}+y_{3}\right)+2 y_{2}+y_{4}\right]\\
		&=\frac{2}{3}\left[\frac{1}{5}+4\left(\frac{1}{9}+\frac{1}{31}\right)+2 \times \frac{1}{21}+\frac{1}{69}\right]\\
		&=\frac{2}{3}\left[\frac{1}{5}+0.5734+0.09523+0.0145\right]\\
		&=\frac{2}{3}[0.2+0.5734+0.09523+0.0145]\\
		&=\frac{2}{3} \times 0.8831=0.5887
		\end{align*}
	\end{minipage}
	\begin{minipage}{0.45\textwidth}
		\begin{align*}
		\begin{tabular}{c|c}
		$x$ & $y=\frac{1}{x^{2}+5}$ \\
		\hline 0 & $\frac{1}{5}$ \\
		2 & $\frac{1}{9}$ \\
		4 & $\frac{1}{21}$ \\
		6 & $\frac{1}{31}$ \\
		8 & $\frac{1}{69}$
		\end{tabular}
		\end{align*}
	\end{minipage}
	\\So the correct answer is \textbf{Option (a)}
	\end{answer}
	\item $\left. \right. $
	\begin{answer}
		\begin{align*}
		 y(0.01)&=1+0.1(-1)=0.99\\
		y(0.02)&=0.99+0.01(-0.99)=0.9801
		\end{align*}
		So the correct answer is \textbf{Option (c)}
	\end{answer}
	\item  $\left. \right. $
\begin{answer}
	\begin{align*}
	y(0.01)&=1+0.1(-1)=0.99\\
	y(0.02)&=0.99+0.01(-0.99)=0.9801\\
	y(0.03)&=0.9801+0.01(-0.9801)=0.9703\\
	y(0.04)&=0.9703+0.01(-0.9703)=0.9606\\
	\text{The exact solution is }y&=e^{-x}\text{ and from this the value at }x=0.04\text{ is } 0.9606
	\end{align*}
		So the correct answer is \textbf{Option (d)}
\end{answer}
	\item  $\left. \right. $
	\begin{answer}
		\begin{align*}
	\text{	First, let }t_{0}&=0, y_{0}=1,\text{ and }h=1.\text{ Thus, we have}\\
	K_{0}&=f(0,1)=1, \quad K_{1}=f(0.5,1.5)=0.25, \quad K_{2}=\mathrm{f}(0.5,1.125)=0.4375 \\
	K_{3}&=\mathrm{f}(1,1.4375)=-0.4375 \\
	&\left(K_{0}+2 K_{1}+2 K_{2}+K_{3}\right) / 6=0.3229\\
	&\text{Therefore, approximation is }y_{1}=y_{0}+h 0.3229=1.322.
		\end{align*}
			So the correct answer is \textbf{Option (c)}
	\end{answer}
		\item  $\left. \right. $
	\begin{answer}
		For this data, we have the Forward difference table\\
		\begin{tabular}{|c|c|c|c|c|c|c|}
		\hline$x_{i}$ & $y_{i}$ & $\Delta y_{i}$ & $\Delta^{2} y_{3}$ & $\Delta^{3} y_{i}$ & $\Delta^{4} y_{i}$ & $\Delta^{5} y_{i}$ \\
		\hline 0 & $1.121$ & $0.002$ & $0.0005$ & $-0.0015$ & $0.002$ & $-.0025$ \\
		\hline $0.001$ & $1.123$ & $0.0025$ & $-0.0010$ & $0.0005$ & $-0.0005$ & \\
		\hline $0.002$ & $1.1255$ & $0.0015$ & $-0.0005$ & $0.0$ & & \\
		\hline $0.003$ & $1.127$ & $0.001$ & $-0.0005$ & & & \\
		\hline $0.004$ & $1.128$ & $0.0005$ & & & & \\
		\hline $0.005$ & $1.1285$ & & & & & \\
		\hline
		\end{tabular}
		\begin{align*}
	\text{	Thus, for }x&=x_{0}+h u,\text{ where }x_{0}=0, h=0.001\text{ and }u \frac{x-x_{0}}{h}\text{ we get}\\
		P_{5}(x)=& 1.121+u \times 0.002+\frac{u(u-1)}{2}(0.0005)+\frac{u(u-1)(u-2)}{3 !} \times(-0.0015) \\
		&+\frac{u(u-1)(u-2)(u-3)}{4 !}(0.002)+\frac{u(u-1)(u-2)(u-3)(u-4)}{5 !} \times(-0.0025) .
		\intertext{Thus}
		P_{5}(0.0045)&=P_{5}(0+0.001+4.5)\\
		&=1.121+0.002 \times 4.5+\frac{0.0005}{2}+4.5 \times 3.5-\frac{0.0015}{6} \times 4.5 \times 3.5 \times 2.5\\
		&+\frac{0.002}{24} \times 4.5 \times 3.5 \times 2.5 \times 1.5-\frac{0.0025}{120} \times 4.5 \times 3.5 \times 2.5 \times 1.5 \times 0.5\\
		&=1.12840045
		\end{align*}
			So the correct answer is \textbf{Option (b)}
	\end{answer}
		\item  $\left. \right. $
	\begin{answer}
		Taylor's series for $y(x)$ is given by
		\begin{align*}
		y(x)&=1+x y_{0}^{\prime}+\frac{x^{2}}{2} y_{0}^{\prime \prime}+\frac{x^{3}}{6} y_{0}^{\prime \prime \prime}+\frac{x^{4}}{24} y_{0}^{i v} \ldots \ldots \ldots\\
		y^{\prime \prime}(x)&=x y^{\prime}(x)+y(x) \Rightarrow y^{\prime \prime}(0)=x y^{\prime}(0)+y(0)=1\\
		y^{\prime \prime \prime}(x)&=x y^{\prime \prime}(x)+2 y^{\prime}(x) \Rightarrow y^{\prime \prime \prime}(0)=x y^{\prime \prime}(x)+2 y^{\prime}(0)=0\\
		y^{i v}(x)&=x y^{\prime \prime \prime}+3 y^{\prime \prime} \Rightarrow y^{i v}(0)=x y^{\prime \prime \prime}(0)+3 y^{\prime \prime}(0)=3\\
		y(x)&=1+\frac{x^{2}}{2}+\frac{x^{4}}{24}(3) \ldots \ldots \ldots\\
		\Rightarrow y(0.1)&=1.0050
		\end{align*}
			So the correct answer is \textbf{Option (a)}
	\end{answer}
	\item  $\left. \right. $
	\begin{answer}
		\begin{align*}
		\text{Taking origin at }0.65, h&=0.01\text{ and }x=0.6538\\
		\therefore p&=\frac{0 \cdot 6538-0 \cdot 65}{0 \cdot 01}=38\\
		\text{The difference table is}&
		\end{align*}
\begin{align*}
\renewcommand*{\arraystretch}{1.5}
\begin{tabular}{|p{1cm}| p{.5cm}|p{1.5cm}|p{1.2cm} |p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|}
\hline
x&p&$10^7\ y$&$10^7\ \Delta y$&$10^7\ \Delta^2 y$&$10^7\ \Delta^3 y$&$10^7\ \Delta^4 y$&$10^7\ \Delta^5 y$&$10^7\ \Delta^6 y$\\
\hline
& & &76349& & & & &\\
\hline
0.63&-2&6270463& &-955& & & &\\
\hline
& & &75394& &$-4$& & &  \\
\hline
0.64&-1&6345857& &-959& &1& & \\
\hline
& & &74435& &-3& &1& \\
\hline
0.65&0&6420292& &-962& &2& &-4 \\
\hline
 & & &73473& &-1& &-3&  \\
 \hline
 0.66&1&6493765& &-963& &-1& &  \\
 \hline
  & & &72510& &-2& & & \\
  \hline
  0.67&2&6566275& &-965& & & &  \\
  \hline
   & & &71545& & & & & \\
   \hline
    0.68&3&6637820& & & & & &  \\
    \hline
\end{tabular}
\end{align*}
		\begin{align*}
	\intertext{	Using Gauss's forward interpolation formula}
		\frac{p(p-1)}{2 !} \Delta^{2} y&+\frac{(p+1) p(p-1)}{3 !} \Delta^{3} y-1+\cdot \\
		10^{7} y_{0} 38=6420292+(0 \cdot 38)(73473)&+\frac{(0 \cdot 38)(0 \cdot 38-1)}{2}(-962)+\cdot \cdot=6448325 \\
		\therefore y_{0} 38&=0 \cdot 6448325
		\end{align*}
			So the correct answer is \textbf{Option (a)}
	\end{answer}
\end{enumerate}
\section{Jacobian}
\begin{enumerate}
		\item  $\left. \right. $
	\begin{answer}
		\begin{align*}
		f=y^{x}, \frac{\partial f}{\partial y}&=x y^{x-1}, \frac{\partial^{2} f}{\partial x \partial y}=\frac{\partial}{\partial x}\left(y^{x-1} x\right)=y^{x-1}+x y^{x-1} \ln y\\
		\left.\frac{\partial^{2} f}{\partial x \partial y}\right|_{x=2, y=1}&=1
		\end{align*}
		So the correct answer is \textbf{Option (c)}
	\end{answer}
		\item  $\left. \right. $
	\begin{answer}
		\begin{align*}
		\frac{\partial z}{\partial x}&=y \ln (x y)+\frac{x y}{x y} y \Rightarrow y(\ln x y+1) \Rightarrow x \frac{\partial z}{\partial x}=x y(\ln (x y+1)\\
		\frac{\partial z}{\partial y}&=x \ln (x y)+\frac{x y}{x y} x \Rightarrow x(\ln x y+1) \Rightarrow y \frac{\partial z}{\partial y}=x y(\ln (x y+1)\\
		x \frac{\partial z}{\partial x}-y \frac{\partial z}{\partial y}&=0
		\end{align*}
		So the correct answer is \textbf{Option (c)}
	\end{answer}
		\item  $\left. \right. $
	\begin{answer}
		\begin{align}
		f(x, y)&=\sum_{K=1}^{10}\left(x^{2}-y^{2}\right)^{K} \Rightarrow \frac{\partial f}{\partial x}(x, y)=\sum_{K=1}^{10} K\left(x^{2}-y^{2}\right)^{K-1}(2 x)\\
		\text{and }\frac{\partial f}{\partial y}(x, y)&=\sum_{K=1}^{10} K\left(x^{2}-y^{2}\right)^{K-1}(-2 y)\\
		\Rightarrow y \frac{\partial f}{\partial x}(x, y)&+x \frac{\partial f}{\partial y}(x, y)=0\notag
		\end{align}
			So the correct answer is \textbf{Option (d)}
	\end{answer}
		\item  $\left. \right. $
	\begin{answer}
		\begin{align*}
		\int_{z=0}^{1} \int_{y=0}^{z} \int_{x=0}^{y} x y^{2} z^{3} d x d y d z=\int_{z=0}^{1} \int_{y=0}^{z} \frac{y^{2}}{2} y^{2} z^{3} d y d z=\int_{z=0}^{1} \frac{z^{8}}{10} d z=\frac{1}{90}
		\end{align*}
		So the correct answer is \textbf{Option (a)}
	\end{answer}
		\item  $\left. \right. $
	\begin{answer}
		\begin{align*}
		\frac{\partial x}{\partial r}&=\cos \theta, \frac{\partial y}{\partial r}=\sin \theta, \frac{\partial x}{\partial \theta}=-r \sin \theta, \frac{\partial y}{\partial \theta}=r \cos \theta\text{ and}\\
		\frac{\partial z}{\partial r}&=0, \frac{\partial z}{\partial \theta}=0, \frac{\partial z}{\partial z}=1\\
		J&=\left|\begin{array}{lll}\frac{\partial x}{\partial r} & \frac{\partial x}{\partial \theta} & \frac{\partial x}{\partial z} \\ \frac{\partial y}{\partial r} & \frac{\partial y}{\partial \theta} & \frac{\partial y}{\partial z} \\ \frac{\partial z}{\partial r} & \frac{\partial z}{\partial \theta} & \frac{\partial z}{\partial \theta}\end{array}\right|=\left|\begin{array}{ccc}\cos \theta & -r \sin \theta & 0 \\ \sin \theta & \cos \theta & 0 \\ 0 & 0 & 1\end{array}\right|=r
		\end{align*}
		So the correct answer is \textbf{Option (a)}
	\end{answer}
		\item  $\left. \right. $
	\begin{answer}
		\begin{align*}
		J\left(\frac{\partial(x, y)}{\partial(u, v)}\right)&=\left(\begin{array}{ll}\frac{\partial x}{\partial u} & \frac{\partial x}{\partial v} \\ \frac{\partial y}{\partial u} & \frac{\partial y}{\partial v}\end{array}\right)=\left|\begin{array}{cc}1-v & -u \\ v & u\end{array}\right|=(u-u v)-(-u v)=u
		\end{align*}
			So the correct answer is \textbf{Option (a)}
	\end{answer}
		\item  $\left. \right. $
	\begin{answer}
		\begin{align*}
		\frac{\partial(u, v)}{\partial(r, \theta)}&=\frac{\partial(u, v)}{\partial(x, y)} \times \frac{\partial(x, y)}{\partial(r, \theta)}\\
		\frac{\partial(u, v)}{\partial(x, y)}&=\left|\begin{array}{ll}\frac{\partial u}{\partial x} & \frac{\partial u}{\partial v} \\ \frac{\partial v}{\partial x} & \frac{\partial v}{\partial y}\end{array}\right|=\left|\begin{array}{cc}2 x & -4 y \\ 4 x & -2 y\end{array}\right|=-4 x y+16 x y=12 x y \quad\text{ put }x=r \cos \theta, y=r \sin \theta\\
		J&=\frac{\partial(x, y)}{\partial(r, \theta)}=\left|\begin{array}{ll}\frac{\partial x}{\partial r} & \frac{\partial x}{\partial \theta} \\ \frac{\partial y}{\partial r} & \frac{\partial y}{\partial \theta}\end{array}\right| \Rightarrow\left|\begin{array}{cc}\cos \theta & -r \sin \theta \\ \sin \theta & r \cos \theta\end{array}\right|=r\\
		\frac{\partial(u, v)}{\partial(r, \theta)}&=\frac{\partial(u, v)}{\partial(x, y)} \times \frac{\partial(x, y)}{\partial(r, \theta)}=12 x y \cdot r=6 r^{3} \sin 2 \theta
		\end{align*}
		So the correct answer is \textbf{Option (c)}
	\end{answer}

\end{enumerate}

CSIR- Mathematical Physics/chapter/Assignments/Assignment-Numerical Techniques-Solutions.tex


\documentclass{article}
\usepackage[a4paper, total={6in, 9in}]{geometry}
\usepackage{fancyhdr}
\usepackage{amsmath}
\usepackage{algorithm}
\usepackage{algpseudocode}

\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{proof}{Proof}[section]

\pagestyle{fancy}
\fancyhf{}
\rhead{DanDoge}
\lhead{Notes on Miller-Rabin}
\rfoot{Page \thepage}
\cfoot{latest version: 2018/02/18}

\title{Notes on Miller-Rabin}
\date{2018-02-18}
\author{DanDoge}

\begin{document}
\section{density of prime numbers}
  \paragraph{prime number theorem} $\lim_{n \to \infty} \frac{\phi(n)}{n / \ln n} = 1$.
  \paragraph{} one simple approach to the problem of testing for primality is $\mathbf{trial division}$. we tyr dividing $n$ by each integer 2,3,...,[$\sqrt{n}$]. this works well only when $n$ is very small or happens to have a small prime factor.
\section{pseudoprimality testing}
  \paragraph{} let $\mathcal{Z}_n^+$ denote the nonzero elements of $\mathcal{Z}_n$, we say that $n$ is a \textit{base-a pseudoprime} if $n$ is composite and
  \begin{equation}
    a^{n - 1} \equiv 1\ (mod\ n).
  \end{equation}
  surprizingly, the conerse of Fermat's theorem $almost$ holds.
  \begin{algorithm}
    \caption{pseudoprime(n)}
    \begin{algorithmic}[1]
      \If{$2^{n - 1} \not\equiv 1\ (mod\ n).$}
      \State \Return{composite}
      \Else return prime
      \EndIf
    \end{algorithmic}
  \end{algorithm}
  \paragraph{} surprizingly, it rarely err. actually the error rate on a randomly chosen $\beta$-bit number goes to zero as $\beta \to \infty$. we cannot entirely eliminate all errors by simply checking for a second base number, because there exist composite integers $n$, \textit{carmichael numbers}, that safisty equation for all $a \in \mathcal{Z}_n^*$.
\section{the miller-rabin randomized primality test}
  \paragraph{} let $n - 1 = 2^tu$ where $t \geq 1$ and $u$ is odd, therefore $a^{n - 1} \equiv (a^u)^{2^t}$
  \begin{algorithm}
    \caption{witness($a, n$)}
    \begin{algorithmic}[1]
      \State let $t$ and $u$ as defined above
      \State $x_0$ = $a^u$ mod $n$
      \For{$i = 1 \to t$}
        \State $x_i = x_{i - 1}^2$ mod $n$
        \If{$x_i == 1$ and $x_{i - 1} \neq 1$ and $x_{i - 1} \neq n - 1$}
          \State \Return{true}
        \EndIf
      \EndFor
      \If{$x_t \neq 1$}
        \State \Return{true}
      \EndIf
      \State \Return{false}
    \end{algorithmic}
  \end{algorithm}
  \paragraph{} we detect whether a nontrival square root of 1 is discovered, We now examine the Miller-Rabin primality test based on the use of WITNESS. Again, we assume that n is an odd integer greater than 2
  \begin{algorithm}
    \caption{miller-rabin($n, s$)}
    \begin{algorithmic}[1]
      \For{$j = 1 \to s$}
        \State $a = $ random(1, $n$ - 1)
        \If{witness($a, n$)}
          \State \Return{composite}
        \EndIf
      \EndFor
      \State \Return{prime}
    \end{algorithmic}
  \end{algorithm}
\section{error rate of miller-rabin primality test}
  \paragraph{theorem} if $n$ is an odd composite number, then the number of witnesses to the compositeness of n is at least $(n - 1) / 2$.
  \paragraph{theorem} the probability that miller-rabin$(n, s)$ errs is at most $2^{-s}$.
  \paragraph{} by applying bayesian theorem, we can estimate the error rate more percisely.
\end{document}


\section{Representation}

\subsection{Nodes}
The possible nodes for each individual comprises nodes from a function set and a terminal set. Nodes in the function set take a certain number of parameters and perform a calculation whereas terminals simply represent some constant value.

Initially, the function set consisted only of elementary numerical operators and the terminal set was made up of the number COVID-19 cases, deaths and recoveries on each of the 30 days preceding the date for which a prediction was being made. Unfortunately, this led to the same problem identified by Hui \cite{hui2003using} in that the genetic program always converged to a local minimum which predominantly uses yesterday’s stock price for its prediction.

As a result, a similar approach to Hoi's \cite{hui2003using} was used to overcome the problem. Specifically - instead of using the number COVID-19 cases, deaths and recoveries - the daily changes in COVID-19 cases, deaths and recoveries was used. In addition, these were not directly used as terminal nodes; rather, the only terminal node used was the ephemeral floating random constant atom R.

The terminal node R produced constants in the range [0, 1) which were taken as arguments to 3 nodes in the function set (\emph{C}, \emph{D} and \emph{H}) that  map the value to the daily change in cases, deaths or recoveries on a specific day within the last 30 days. The rest of the function set was made up of the elementary numerical operators. The full function and terminal set are detailed in table \ref{tab:function_and_terminal_set}.

\begin{table}[H]
\resizebox{\textwidth}{!}{\begin{tabular}{|c|c|c|}
\hline
\textbf{Symbol} & \textbf{Arity} & \textbf{Description}                                                                                \\ \hline
+               & 2              & The addition operator                                                                                \\ \hline
-               & 2              & The subtraction operator                                                                                \\ \hline
*               & 2              & The multiplication operator                                                                                \\ \hline
/               & 2              & The protected division operator (division by 0 results in 1)                                                                                \\ \hline
C               & 1              & \begin{tabular}[c]{@{}c@{}}Maps a value in the range {[}0,1) to the change in confirmed cases\\ from the previous day for a particular day within the last 30 days\end{tabular} \\ \hline
D               & 1              & \begin{tabular}[c]{@{}c@{}}Maps a value in the range {[}0,1) to the change in deaths\\ from the previous day for a particular day within the last 30 days\end{tabular}          \\ \hline
H               & 1              & \begin{tabular}[c]{@{}c@{}}Maps a value in the range {[}0,1) to the change in recoveries\\ from the previous day for a particular day within the last 30 days\end{tabular}      \\ \hline
R               & 0              & The ephemeral random floating constant atom R in the range {[}0,1)                                                                                \\ \hline
\end{tabular}}
\caption{Function and Terminal Set}
\label{tab:function_and_terminal_set}
\end{table}

\subsection{Individuals}\label{sec:individuals}
Each chromosome within a population consists of 3 syntax trees. To clarify, syntax trees include nodes and links the nodes. Nodes indicate the instructions to execute while the links indicate the arguments for each instruction. As per Koza \cite{koza1992genetic}, the internal nodes are called functions while the tree’s leaves are called terminals. Given the function and terminal set that was identified, an example of one such syntax tree is depicted in figure \ref{fig:syntax_tree}.

The reason for having each chromosome comprise 3 different syntax trees is because the first tree will be optimized for predicting the number of confirmed cases, the second for predicting the number of deaths and the third for predicting the number of recoveries. The trees will be optimized in this way by means of the chromosome's fitness evaluation, described in section \ref{sec:fitness}.

% minimum distance between nodes on the same line
\setlength{\GapWidth}{1em}
% draw with a thick dashed line, very nice looking
\thicklines \drawwith{\dottedline{2}}
% draw an oval and center it with the rule.  You may want to fool with the
% rule values, though these seem to work quite well for me.  If you make the
% rule smaller than the text height, then the GP nodes may not line up with
% each other horizontally quite right, so watch out.
\newcommand{\gpbox}[1]{\Ovalbox{#1\rule[-.7ex]{0ex}{2.7ex}}}

\begin{figure}[H]
    \centering
    \resizebox{\textwidth}{!}{\begin{bundle}{\gpbox{*}}\chunk{\begin{bundle}{\gpbox{+}}\chunk{\begin{bundle}{\gpbox{-}}\chunk{\begin{bundle}{\gpbox{C}}\chunk{\gpbox{R0.39}}\end{bundle}}\chunk{\begin{bundle}{\gpbox{/}}\chunk{\begin{bundle}{\gpbox{C}}\chunk{\gpbox{R0.82}}\end{bundle}}\chunk{\begin{bundle}{\gpbox{R}}\chunk{\gpbox{R0.08}}\end{bundle}}\end{bundle}}\end{bundle}}\chunk{\begin{bundle}{\gpbox{+}}\chunk{\begin{bundle}{\gpbox{C}}\chunk{\gpbox{R0.25}}\end{bundle}}\chunk{\begin{bundle}{\gpbox{+}}\chunk{\gpbox{R0.84}}\chunk{\gpbox{R0.02}}\end{bundle}}\end{bundle}}\end{bundle}}\chunk{\begin{bundle}{\gpbox{/}}\chunk{\begin{bundle}{\gpbox{C}}\chunk{\gpbox{R0.09}}\end{bundle}}\chunk{\begin{bundle}{\gpbox{D}}\chunk{\gpbox{R0.08}}\end{bundle}}\end{bundle}}\end{bundle}}
    \caption{Syntax Tree}
    \label{fig:syntax_tree}
\end{figure}