% !TEX root = ./busty_transcription.tex
\section{Mean Gene Expression}\label{section_02_means}

As noted in the previous section, there are two broad classes of models in play
for computing the input-output functions of regulatory architectures as shown in
Figure~\ref{fig1:means_cartoons}. In both classes of model, the promoter is
imagined to exist in a discrete set of states of occupancy, with each such state
of occupancy accorded its own rate of transcription --including no
transcription for many of these states. This discretization of a potentially
continuous number of promoter states (due to effects such as supercoiling of
DNA~\cite{Chong2014, Sevier2016} or DNA looping \cite{Boedicker2013a}) is
analogous to how the Monod-Wyman-Changeux model of allostery coarse-grains
continuous molecule conformations into a finite number of
states~\cite{Martins2011}. The models are probabilistic with each state assigned
some probability and the overall rate of transcription given by
\begin{equation}
\mbox{average rate of transcription} = \sum_i r_i p_i,
\label{eq:transcrip_prop_pbound}
\end{equation}
where $i$ labels the distinct states, $p_i$ is the probability of the
$i^{\text{th}}$ state, and $r_i$ is the rate of transcription of that state.
Ultimately, the different models differ along several key aspects: what states
to consider and how to compute the probabilities of those states.

The first class of models that are the subject of the present section focus on
predicting the mean level of gene expression. These models, sometimes known as
thermodynamic models, invoke the tools of equilibrium statistical mechanics to
compute the probabilities of the promoter microstates~\cite{Ackers1982,
Shea1985, Buchler2003, Vilar2003a, Vilar2003b, Bintu2005a, Bintu2005c,
Gertz2009, Sherman2012, Saiz2013}. As seen in
Figure~\ref{fig1:means_cartoons}(B), even within the class of thermodynamic
models, we can make different commitments about the underlying microscopic
states of the promoter. Model 1 considers only two states: a state in which a
repressor (with copy number $R$) binds to an operator and a transcriptionally
active state. The free energy difference between the repressor binding the
operator, i.e. a specific binding site, and one of the $N_{NS}$ non-specific
sites is given by $\Delta\varepsilon_R$ (given in $k_BT$ units with $\beta\equiv
(k_BT)^{-1}$). Model 2 expands this model to include an empty promoter where no
transcription occurs, as well as a state in which one of the $P$ RNAPs binds to
the promoter with binding energy $\Delta\varepsilon_P$. Indeed, the list of
options considered here does not at all exhaust the suite of different
microscopic states we can assign to the promoter. The essence of thermodynamic
models is to assign a discrete set of states and to use equilibrium statistical
mechanics to compute the probabilities of occupancy of those states.

The second class of models that allow us to access the mean gene expression use
chemical master equations to compute the probabilities of the different
microscopic states ~\cite{Ko1991, Peccoud1995, Record1996, Kepler2001,
Sanchez2008, Shahrezaei2008, Sanchez2011, Michel2010}. The main differences
between both modeling approaches can be summarized as: 1) Although for both
classes of models the steps involving transcriptional events are assumed to be
strictly irreversible, thermodynamic models force the regulation, i.e., the
control over the expression exerted by the repressor, to be in equilibrium. This
does not need to be the case for kinetic models. 2) Thermodynamic models ignore
the mRNA count from the state of the Markov process, while kinetic models keep
track of both the promoter state and the mRNA count. 3) Finally, thermodynamic
and kinetic models coarse-grain to different degrees the molecular mechanisms
through which RNAP enters the transcriptional event. As seen in
Figure~\ref{fig1:means_cartoons}(C), we consider a host of different kinetic
models, each of which will have its own result for both the mean (this section)
and noise (next section) in gene expression.

\subsection{Fold-changes are indistinguishable across models}
As a first stop on our search for the ``right'' model of simple repression, let
us consider what we can learn from theory and experimental measurements on the
average level of gene expression in a population of cells. One experimental
strategy that has been particularly useful (if incomplete since it misses out on
gene expression dynamics) is to measure the fold-change in mean
expression~\cite{Garcia2011}. The fold-change $FC$ is defined as
\begin{equation}
FC(R)
= \frac{\langle \text{gene expression with }R > 0 \rangle}
        {\langle \text{gene expression with }R = 0 \rangle}
= \frac{\langle m (R) \rangle}{\langle m (0) \rangle}
= \frac{\langle p (R) \rangle}{\langle p (0) \rangle},
\label{eq:fc_def}
\end{equation}
where angle brackets $\left\langle \cdot \right\rangle$ denote the average over
a population of cells and mean mRNA $\langle m\rangle$ and mean protein $\langle
p\rangle$ are viewed as a function of repressor copy number $R$. What this means
is that the fold-change in gene expression is a relative measurement of the
effect of the transcriptional repressor ($R > 0$) on the gene expression level
compared to an unregulated promoter ($R = 0$). The third equality in
Eq.~\ref{eq:fc_def} follows from assuming that the translation efficiency, i.e.,
the number of proteins translated per mRNA, is the same in both conditions. In
other words, we assume that mean protein level is proportional to mean mRNA
level, and that the proportionality constant is the same in both conditions and
therefore cancels out in the ratio. This is reasonable since the cells in the
two conditions are identical except for the presence of the transcription
factor, and the model assumes that the transcription factor has no direct effect
on translation.

Fold-change has proven a very convenient observable in past
work~\cite{Garcia2011a, Brewster2014, Razo-Mejia2018, Chure2019}. Part of its
utility in dissecting transcriptional regulation is its ratiometric nature,
which removes many secondary effects that are present when making an absolute
gene expression measurement. Also, by measuring otherwise identical cells with
and without a transcription factor present, any biological noise common to both
conditions can be made to cancel out. Figure~\ref{fig1:means_cartoons}(B) and
(C) depicts a smorgasbord of mathematicized cartoons for simple repression using
both thermodynamic and kinetic models, respectively, that have appeared in
previous literature. For each cartoon, we calculate the fold-change in mean gene
expression as predicted by that model, deferring most algebraic details to
Appendix~\ref{sec:non_bursty}. What we will find is that for all cartoons the
fold-change can be written as a Fermi function of the form
\begin{equation}
FC(R) = \left( 1 + \exp(-\Delta F_R(R) + \log(\rho))  \right)^{-1},
 \label{eq:deltaFR_eq_noneq_equiv}
\end{equation}
where the effective free energy contains two terms: the parameters $\Delta F_R$,
an effective free energy parametrizing the repressor-DNA interaction, and
$\rho$, a term derived from the level of coarse-graining used to model all
repressor-free states. In other words, the effective free energy of the Fermi
function can be written as the additive effect of the regulation given by the
repressor via $\Delta F_R$, and the kinetic scheme used to describe the steps
that lead to a transcriptional event via $\log(\rho)$ (See
Figure~~\ref{fig1:means_cartoons}(D), left panel). This implies all models
collapse to a single master curve as shown in
Figure~\ref{fig1:means_cartoons}(D). We will offer some intuition for why this
master curve exists and discuss why at the level of the mean expression, we are
unable to discriminate ``right'' from ``wrong'' cartoons given only measurements
of fold-changes in expression.

\subsubsection{Two- and Three-state Thermodynamic Models}
We begin our analysis with models 1 and 2 in
Figure~\ref{fig1:means_cartoons}(B). In each of these models the promoter
is idealized as existing in a set of discrete states; the difference being
whether or not the RNAP bound state is included or not. Gene expression is then
assumed to be proportional to the probability of the promoter being in either
the empty state (model 1) or the RNAP-bound state (model (2)). We direct the
reader to Appendix~\ref{sec:non_bursty} for details on the derivation of the
fold-change. For our purposes here, it suffices to state that the functional
form of the fold-change for model 1 is
\begin{equation}
FC(R)
= \left(1 + \frac{R}{N_{NS}} e^{-\beta\Delta\varepsilon_R}\right)^{-1},
\end{equation}
where $R$ is the number of repressors per cell, $N_{NS}$ is the number of
non-specific binding sites where the repressor can bind, $\Delta\varepsilon_R$
is the repressor-operator binding energy, and $\beta \equiv (k_BT)^{-1}$. This
equation matches the form of the master curve in
Figure~\ref{fig1:means_cartoons}(D) with $\rho=1$ and $\Delta F_R =
\beta\Delta\varepsilon_r - \log (R / N_{NS})$. For model 2 we have a similar
situation. The fold-change takes the form
\begin{eqnarray}
FC(R)
&=& \left(
1 + \frac{\frac{R}{N_{NS}} e^{-\beta\Delta\varepsilon_R}}
        {1 + \frac{P}{N_{NS}} e^{-\beta\Delta\varepsilon_P}}
\right)^{-1}
\\
&=& (1 + \exp(-\Delta F_R + \log\rho))^{-1},
\end{eqnarray}
where $P$ is the number of RNAP per cell, and $\Delta\varepsilon_P$ is the
RNAP-promoter binding energy. For this model we have $\Delta F_R =
\beta\Delta\varepsilon_R - \log(R/N_{NS})$ and $\rho = 1 +
\frac{P}{N_{NS}}\mathrm{e}^{-\beta\Delta\varepsilon_P}$. Thus far, we see that
the two thermodynamic models, despite making different coarse-graining
commitments, result in the same functional form for the fold-change in mean gene
expression.  We now explore how kinetic models fare when faced with computing
the same observable.

\subsubsection{Kinetic models}
One of the main difference between models shown in
Figure~\ref{fig1:means_cartoons}(C), cast in the language of chemical master
equations, compared with the thermodynamic models discussed in the previous
section is the probability space over which they are built. Rather than keeping
track only of the microstate of the promoter, and assuming that gene expression
is proportional to the probability of the promoter being in a certain
microstate, chemical master equation models are built on the entire probability
state of both the promoter microstate, and the current mRNA count. Therefore, in
order to compute the fold-change, we must compute the mean mRNA count on each of
the promoter microstates, and add them all together~\cite{Sanchez2013}.

Again, we consign all details of the derivation to
Appendix~\ref{sec:non_bursty}. Here we just highlight the general findings for
all five kinetic models. As already shown in Figure~\ref{fig1:means_cartoons}(C)
and (D), all the kinetic models explored can be collapsed onto the master curve.
Given that the repressor-bound state only connects to the rest of the promoter
dynamics via its binding and unbinding rates, $k_R^+$ and $k_R^-$ respectively,
all models can effectively be separated into two categories: a single
repressor-bound state, and all other promoter states with different levels of
coarse graining. This structure then guarantees that, at steady-state, detailed
balance between these two groups is satisfied. What this implies is that the
steady-state distribution of each of the non-repressor states has the same
functional form with or without the repressor, allowing us to write the
fold-change as a product of the ratio of the binding and unbinding rates of the
promoter, and the promoter details. This results in a fold-change of the form
\begin{eqnarray}
FC &=& \left( 1 + \frac{k_R^+}{k_R^-} \rho \right)^{-1},\\
&=& (1 + \exp(-\Delta F_R + \log(\rho) ))^{-1},
\end{eqnarray}
where $\Delta F_R \equiv -\log(k_R^+/k_R^-)$, and the functional forms of $\rho$
for each model change as shown in Figure~\ref{fig1:means_cartoons}(C). Another
intuitive way to think about these two terms is as follows: in all kinetic
models shown in Figure~1(C) the repressor-bound state can only be reached from a
single repressor-free state. The ratio of these two states --repressor-bound and
adjacent repressor-free state-- must remain the same for all models, regardless
of the details included in other promoter states if $\Delta F_R$ represents an
effective free energy of the repressor binding the DNA operator. The presence of
other states then draws probability density from the promoter being in either of
these two states, making the ratio between the repressor-bound state and
\textit{all} repressor-free states different. The log difference in this ratio
is given by $\log(\rho)$. Since model 1 and model 5 of Figure~1(C) consist of a
single repressor-free state, $\rho$ is then necessarily 1 (See
Appendix~\ref{sec:non_bursty} for further details).

The key outcome of our analysis of the models in
Figure~\ref{fig1:means_cartoons} is the existence of a master curve shown in
Figure~\ref{fig1:means_cartoons}(D) to which the fold-change predictions of all
the models collapse. This master curve is parametrized by only two effective
parameters: $\Delta F_R$, which characterizes the number of repressors and their
binding strength to the DNA, and $\rho$, which characterizes all other features
of the promoter architecture. The key assumption underpinning this result is
that no transcription occurs when a repressor is bound to its operator. Given
this outcome, i.e., the degeneracy of the different models at the level of
fold-change, a mean-based metric such as the fold-change that can be readily
measured experimentally is insufficient to discern between these different
levels of coarse-graining. The natural extension that the field has followed for
the most part is to explore higher moments of the gene expression distribution
in order to establish if those contain the key insights into the mechanistic
nature of the gene transcription process~\cite{Iyer-Biswas2009,Munsky2012}.
Following a similar trend, in the next section we extend the analysis of the
models to higher moments of the mRNA distribution as we continue to examine the
discriminatory power of these different models.


\chapter{Project Design - New} \label{chapter:ProjectDesign}

This chapter outlines the design behind each component of the text classification model and the respected process each component may entail, several components will execute more than one step to achieve the desired result. The design is reflected within the final model and each component is broken down to display the functionality and theory within this project. In \autoref{section:FunctionalRequirements}, it is detailed there won’t be a GUI for the interaction and thus the design section relates to the inner workings of the model itself, that being: system architecture, logistics and theory.

\section{Classical vs Modern}

As originally intended, this project would have demonstrated two differing implementations of the same concept, one being of a classical nature implemented as a machine learning model and the other being a modern variation of the same approach, as previously mentioned this project experienced time management issues due to unexpected problems, to which resulted in only focusing on a traditional implementation within a modern model in attempt at a novel approach. \newpage

Classical NLP implementations can be described as:

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/ClassicalNLP.pdf}
    \caption[ClassicalNLP]{Classical NLP pipeline for text classification.
    \label{fig:ClassicalNLP}}
\end{figure}

Classical or "traditional" methods for corpus classification include: N-Grams, Hidden Markov Models using Markov Chains, Part-Of-Speech Tagging and Bag-Of-Words. Machine learning concepts can be classed as a “black-box” of functionality as the user does not necessarily see what is being executed within the hidden layers, a high-level abstraction for this project can be generalised into the following diagram:

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/MLNLP.pdf}
    \caption[MachineLearningNLP]{Machine Learning Model for an NLP pipeline for text classification.
    \label{fig:MLNLP}}
\end{figure}

\section{Model Approach --- Traditional (A)} \label{sub:C5ModelATraditionalA}

The traditional implementation would have been designed based on word-embeddings within a 2D space where word vector values would be used alongside the Bag-Of-Words approach. The combination of Bag-Of-Words with a machine learning model to calculate a word’s vector based on its TF-IDF value would have been the start, such that:

\begin{equation}
    tf(t,d) = \frac{f_t, _d}{\sum{t \in_d} f _t {_`}, _d}
\end{equation}

Where the TF-IDF value is how often a lexeme occurs in a given corpus and would have also used the inverse TF-IDF value as the project model covers multiple datasets, such that:

\begin{equation}
    idf(t, D) = \log \frac{N}{|d \in D : t \in d|}
\end{equation}

Where the inverse TF-IDF value is represents how rare a word is in a given corpus.

\subsection{Limitations of Bag-Of-Words}

As there is a specific domain for this project, it is important to focus on the limitations for potential methods, BOW models have demonstrated high training accuracy and is a relatively simple model to implement, it is naturally flexible as it can be trained on different datasets for a specific context; however, it does have disadvantages for classification and text prediction. Universities host students from different backgrounds and levels of education which can imply issues when training datasets, these issues can impact:

\begin{itemize}
    \item \textbf{\textit{Vocabulary:}} Students may have varying output to describe the same context, this can create confusion for training.
    \item \textbf{\textit{Frequency:}} The frequency of a word may influence its power in a dataset.
    \item \textbf{\textit{Context:}} If students describe the same situation in a different way, some words may lose semantic meaning or be interpreted incorrectly, for example isolating neologisms, synonyms, colloquialism, polysemes, or semantic change.
\end{itemize}

As a result, the Bag-Of-Words approach is not considered for this comparison for the domain of student feedback analysis.

\section{Model Approach --- Traditional (B)} \label{sub:C5ModelATraditionalB}

POS---TAGGING

\section{Model Approach --- Modern}

Modern or "contemporary" methods for corpus classification include: \ldots The modern implementation would have been based around Word2Vec implemented in a machine learning model for Sentiment Analysis which can be seen as a subcategory of text classification. The word2vec implementation would produce vector values for identified key words and plot them as such:

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/Example-Word-Vector.pdf}
    \caption[ExampleWordVector]{Example of student satisfaction vector graph in 2d space.
    \label{fig:Example-Word-Vector}}
\end{figure}

\subsection{Word2Vec Skip-Gram} \label{sub:Word2VecSkipGram}

Skip-gram rather than Continuous Bag-Of-Words (CBOW [which is the modern implementation of Bag-Of-Words]) as it yields better results with large datasets - such as student feedback - skip gram can also be context aware as it converts neighboring lexemes to vectors.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.49\textwidth]{figures/chapter-5/SkipGramModel.pdf}
    \caption[SkipGramModel]{Input flow of Skip-Gram model.
    \label{fig:SkipGramModel}}
\end{figure}

The skip-gram model can be seen as the inverse of the bag-of-words model as it attempts to vectorize neighboring words first to identify corpus context, whereas, bag-of-words takes each lexeme first, produces a vector sum and then categorises each word. \newpage

The Skip-Gram's network architecture is similar to the diagram displayed in \autoref{fig:MLNLP}, the diagram above is another level of abstraction as to how this project will make use of machine learning fundamentals.

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/SkipGramNetwork.pdf}
    \caption[SkipGramNetwork]{Skip-Gram implementation for Word2Vec network architecture.
    \label{fig:SkipGramNetwork}}
\end{figure}

\subsection{One-Hot Encoding} \label{sub:Word2VecOneHotEncoding}

Encoding each significant lexeme in a given dataset is helpful for the model to distinguish the level of importance, outlined as context, the diagram shown above in \autoref{fig:SkipGramModel} takes a 1D row vector as the input layer which allows for the model to acceptably one-hot encode each lexeme unit as a numeric value.

\begin{multicols}{2}
    \begin{equation*}
        "Teach" =
        \begin{bmatrix}
            0, & 0, & 0, & 0, & 1, & 0
        \end{bmatrix}
    \end{equation*}

    \begin{equation*}
        "Teaching" =
        \begin{bmatrix}
            0, & 0, & 0, & 0, & 1, & 0
        \end{bmatrix}
    \end{equation*}
\end{multicols}

This equates to differing versions of the same word (root + free or bound morpheme [affix]) to have the same influence when being parsed forward within the training model.

\subsection{Forward Propagation} \label{sub:Word2VecForwardPropagation}

Once the corpus text is encoded as an acceptable 1D matrix, it can then be parsed to the first hidden layer's node:

\begin{equation}
    h = x^T W
\end{equation}
`x' represents the row vector, `h' can be taken as the column height ($[x]^T$ of the vector `W' where `h' is the `$k_{th}$' column.

\begin{equation}
    h = W_{(k,:)}^{T} \coloneqq v_{w_I}^{T}
\end{equation}
As each lexeme unit parses through the hidden layer nodes, the exit value needs to be calculated:

\begin{equation} \label{eq:u_cDefinition}
    u_{c} = W'^{T}h = W'^{T} W^{T}x
\end{equation}
For simplicity, $u_{c}$ will transpose each unit through the model in its entirety, however, the value(s) of $u_{c}$ cause performance and memory issues in the model. Thus, \textbf{\textit{softmax}} will be used to slice the vector value of $u_{c}$ to [0,1].

\begin{equation*}
    y_{c} = Softmax(u)
\end{equation*}

\begin{equation}
    p(w_{c,_j} = w_{O,_c} | w_{I}) = y_{c,_j} = \frac{exp(u_{c_,j})}{\sum_{j' = 1}^{V} exp(u_{j'})}
\end{equation}
`x' represents the center word $w(T)$ of a vector where $W'$ will yield the softmax value of $u_{c}$, this ensures the vector of $w(T)$ will have exact values of input for `x', where:

\begin{multicols}{2}
    \begin{equation*}
        u_{c,_j} = u_{j} = {v'_{w_j}} ^ T \cdot h
    \end{equation*}

    \begin{equation}
        % TODO rewrite mathematically in set theory notation
        c=1 \ldots |C|
    \end{equation}
\end{multicols}
${v'_{w_j}}$ represents the exit vector for the $j^{th}$ lexeme unit in $w_{j}$ (the corpus vocabulary).

\subsection{Backward Propagation} \label{sub:Word2VecBackardPropagation}

As stated above in \autoref{eq:u_cDefinition}, the transposition of each unit $u_{c}$ is simple where node weight is not considered. To account for the weight of the model matrices $W$ and $W'$, the Stochastic Gradient Descent method is applied, which in turn will optimise the backward propagation of node errors; when training the model, errors are bound to occur, to which a loss function is needed to calculate each layers efficiency.

\begin{equation} \label{eq:C5ErrorCalc}
    \begin{split}
        Error (E) & = - \log \mathbb{P} (w_{O_,{_1}} , \ldots, w_{O_{,c}} | w_{I}) \\
                  & = - \log \prod_{c=1}^{C} \frac{exp(u_{c,j_{c}^{*}})}{\sum_{j'=1}^{V} exp(u_{j'})}  \\
                  & = - \sum_{c=1}^{C} u_{j_{c}^{*}} + C \cdot \log \sum_{j'=1} ^ {V} exp(u_{j'})
    \end{split}
\end{equation}
${j_{c}^{*}}$ represents the exit vector column's index within the vector's vocabulary. Once loss is calculated, the model can apply the \textbf{\textit{chain rule}} for weight error classification within $W$ and $W'$. This is achieved by obtaining the partial derivative of Error(E) of each exit node $u_{c,_j}$.

\begin{equation}
    \frac{\partial E}{\partial u_{c,_j}} = y_{c,_j} - t_{c,_j} \coloneqq e_{c,_j}
\end{equation}
$t_{c,_j}$ represents vector $u$'s \textbf{\textit{Ground Truth}}; [$t_{c,_j}$]'s state of hypothesis can be represented as the following definition:

\begin{equation}
    EI_j = \sum_{c=1}^{C} e_{c,_j} = \sum_{c=1}^{C} (y_{c,_j} - t_{c,_j}) = \frac{\partial E}{\partial u_{j}}
\end{equation}
$EI_{j}$ represents the \textbf{\textit{Row-Wise Sum}} as a column vector in attempt to account for word context errors for $w(T)$; \textbf{\textit{backpropagation}} can then occur by obtaining the partial derivative of Error(E) within matrix $W'$.

\begin{equation} \label{}
    \begin{split}
        \frac{\partial E}{\partial w'_{ij}} & = \sum_{c=1}^{C} \frac{\partial E}{\partial u_{c_{,j}}} \cdot \frac{\partial u_{c_{,j}}}{\partial w'_{i_{,j}}} \\
                                            & = \sum_{c=1}^{C} (y_{c,_j} - t_{c,_j}) \\
                                            & = EI_{j} \cdot h_{i}
    \end{split}
\end{equation}
When backpropagation occurs, the Stochastic Gradient Descent is redefined in terms of the matrix $W'$:

\begin{equation}
    w'^{(new)}_{i,j} = w'^{(old)}_{i,j} - \eta \cdot EI_{j} \cdot h_{i}
\end{equation}
$\eta$ represents the model's training (learning). Once the learning rate has been established, the model can update the distribution of errors between the model layers, particularly between unit input and the hidden layer whereby the partial derivative is weighted against an error from a hidden layer. This can be calculated with:

\begin{equation}
    \begin{split}
        \frac{\partial E}{\partial h_{i}} & = \sum_{j=1}^{V} \frac{\partial E}{\partial u_{j}} \cdot \frac{\partial u_{j}}{\partial h_{i}} \\
                                          & = \sum_{j=1}^{V} EI_{j} \cdot w'_{ij}
    \end{split}
\end{equation}
As errors are calculated between the input layer and the hidden layer accounting for error weight, it is possible to calculate the weighted loss of matrix $W$ by taking the partial derivatives of Error(E) against the partial derivative of an index within matrix $W$, as such:

\begin{equation} \label{}
    \begin{split}
        \frac{\partial E}{\partial W_{ki}} & = \frac{\partial E}{\partial h_{i}} \cdot \frac{\partial h_{i}}{\partial w_{ki}} \\
                                           & = \sum_{j=1}^{V} EI_{j} \cdot w'_{ij} \cdot x_{k}
    \end{split}
\end{equation}
The weight of the Stochastic Gradient Descent is redefined in terms of the matrix $W$:

\begin{equation}
    w^{(new)}_{i,j} = w^{(old)}_{i,j} - \eta \cdot \sum_{j=1}^{V} EI_{j} \cdot w'_{ij} \cdot x_{j}
\end{equation}
The fundamental functionality has been outlined in theory but in practice will be enough to train the given Skip-Gram Network in \autoref{fig:SkipGramNetwork}.

\section{Planning the Network Architecture}

Initial prototyping of the machine learning model for a new amalgamation of NLP techniques helped to indicate what the best route of development could be, the planning stage piggybacked off existing work flowchart diagrams in-order to apply the most appropriate method and technique combination. The sequence flowchart is as follows:

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/GooglePlan.pdf}
    \caption[GooglePlan]{Text Classification Flowchart \parencite{google2021TCF}.
    \label{fig:GooglePlan}}
\end{figure}

\section{Supervision}

The development of the project model is based on a supervised approach due to the datasets located, it was most appropriate to use a supervised approach due to the datasets having no labels or lexical categories to train the model on; the model has user input to account for missing labels on data which have been manually and algorithmically added. The supervision for this project can be represented as the following diagram:

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/SupervisedLearningChart.pdf}
    \caption[SupervisedLearning]{Sequence control for Supervised learning.
    \label{fig:SupervisedLearningChart}}
\end{figure}

\section{Pipeline Design}

\begin{figure}[H]
    \centering
    \includegraphics[width=\textwidth]{figures/chapter-5/Pipeline.pdf}
    \caption[MLTCPipeline]{Pipeline for model classification.
    \label{fig:MLTCPipeline}}
\end{figure}

There are five main steps for a text classification pipeline:

\begin{enumerate}
    \item \textbf{\textit{Pre-processing}}: prepare the raw dataset to be trained.
    \item \textbf{\textit{Splitting}}: split the processed dataset to be trained and validated.
    \item \textbf{\textit{Tuning}}: identity valuable parameters within trained data.
    \item \textbf{\textit{Training}}: train the current iteration of the model with updated hyper-parameters.
    \item \textbf{\textit{Testing}}: test and collect statistics for analysis to make further predictions.
\end{enumerate} \newpage

% FIXME
% \section{Data Preparation, Pre-processing, and Analytics}
\section[Data Preparation, Pre-processing, and Analytics]{\texorpdfstring{Data Preparation, Pre-processing, and \\ Analytics}{Data Preparation, Pre-processing, and Analytics}}

Preparing and pre-processing of the parsed data enables meaning to be allocated to the processed classification information, bypassing the implementation of a pre-processing step will ruin the chances of yielding valid returned data and will most likely detract meaning from any results throughout data analysis. The preparation and pre---preprocessing of data will be completed with the use of NLP libraries such as `pandas' and `sklearn'.

\subsection{Data Organisation and Binding}

The initial steps for data organisation will be to tidy the datasets procured from Kaggle as they held a lot of useless information and or columns with little weight. The first organisation step is to merge multiple CSV files and their respected columns into an easily readable format. The removal of unnecessary columns will aid the preprocessing as the dataset will be smaller.

\subsection{Pre-processing} \label{sub:C5Preprocessing}

The following methods of preprocessing will be applied to a given dataset:

\begin{itemize}
    \item \textbf{\textit{Punctuation:}} removal of all unnecessary punctuation, where punctuation includes the regular expression: \verb|["[.!?\\-]"]|.
    \item \textbf{\textit{Missing Values:}} word features with a percentage higher than the defined threshold for missing values will be sliced from the dataset as missing features will cause unreliable training data.
    \item \textbf{\textit{Collinear features:}} features in which are highly correlated with another lexical unit will be removed as these features can decrease overall performance on training data, this is due to higher variance of the similar features and lower interpretability of the model.
    \item \textbf{\textit{Stopwords:}} stopwords found within the given dataset will be removed as they do present meaning for classification as they are mostly common words such as "and" or "a", removing stopwords also has a positive affect on performance.
\end{itemize}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Probability Distributions}
\label{dist}

% Place these distributions first as they are simple and keep the interesting ones together on the next page
\begin{figure}
  \centering
  \savebox{\largestimage}{
    \includegraphics[width=0.47\textwidth]{figures/stats/dist/exp_pdf}
  }% Store largest image in a box

  \begin{subfigure}[b]{0.48\textwidth}\centering
    \raisebox{\dimexpr.5\ht\largestimage-.5\height}{% Adjust vertical height of smaller image
      \includegraphics[width=\textwidth]{figures/stats/dist/uniform_pdf}}
  \caption{Uniform Distribution PDF}
  \label{fig:dist:uniform}
  \end{subfigure}
  ~
  \begin{subfigure}[b]{\wd\largestimage}\centering
    \usebox{\largestimage}
  \caption{Exponential Distribution PDF}
  \label{fig:dist:exp}
  \end{subfigure}
\caption{
Uniform and exponential distribution PDFs,
by \href{https://en.wikipedia.org/wiki/File:Uniform_Distribution_PDF_SVG.svg}{AnonMoos}
and \href{https://en.wikipedia.org/wiki/File:Exponential_probability_density.svg}{AkanoToE}.
\label{fig:dist:uniform_exp}
}
\end{figure}

\begin{figure}
  \centering
  \savebox{\largestimage}{
    \includegraphics[width=0.47\textwidth]{figures/stats/dist/poisson_pmf}
  }% Store largest image in a box

  \begin{subfigure}[b]{\wd\largestimage}\centering
    \raisebox{\dimexpr.5\ht\largestimage-.5\height}{% Adjust vertical height of smaller image
      \includegraphics[width=\textwidth]{figures/stats/dist/binomial_pmf}}
  \caption{Binomial Distribution PMF}
  \label{fig:dist:binomial}
  \end{subfigure}
  ~
  \begin{subfigure}[b]{0.48\textwidth}\centering
    \usebox{\largestimage}
  \caption{Poisson Distribution PMF}
  \label{fig:dist:poisson}
  \end{subfigure}
\caption{
Binomial and Poisson distribution PMFs,
by \href{https://en.wikipedia.org/wiki/File:Binomial_distribution_pmf.svg}{Tayste}
and \href{https://en.wikipedia.org/wiki/File:Poisson_pmf.svg}{Skbkekas}.
Both plots have $k$ on the $x$-axis and $P$ on the $y$-axis, and curves for various $n$ and $p$, or $\lambda$.
\label{fig:dist:binomial_poisson}
}
\end{figure}

\begin{figure}
  \centering
  \savebox{\largestimage}{
    \includegraphics[width=0.47\textwidth]{figures/stats/dist/student_t_pdf}
  }% Store largest image in a box

  \begin{subfigure}[b]{0.48\textwidth}\centering
    \raisebox{\dimexpr.5\ht\largestimage-.5\height}{% Adjust vertical height of smaller image
      \includegraphics[width=\textwidth]{figures/stats/dist/gaussian_pdf}}
  \caption{Gaussian Distribution PDF}
  \label{fig:dist:gaus}
  \end{subfigure}
  ~
  \begin{subfigure}[b]{\wd\largestimage}\centering
    \usebox{\largestimage}
  \caption{Student's $t$-Distribution PDF}
  \label{fig:dist:student_t}
  \end{subfigure}
\caption{
Gaussian and Student's \tdist PDFs,
by \href{https://en.wikipedia.org/wiki/File:Normal_Distribution_PDF.svg}{Inductiveload}
and \href{https://en.wikipedia.org/wiki/File:Student_t_pdf.svg}{Skbkekas}.
Note that in the limit $\nu \to \infty$ Student's \tdist
approaches the standard normal distribution shown in red.
\label{fig:dist:gaus_student_t}
}
\end{figure}

\begin{figure}
\centering
\includegraphics[width=0.7\textwidth]{figures/stats/dist/chi2_pdf}
\caption{
\chiSqdist PDF,
by \href{https://en.wikipedia.org/wiki/File:Chi-square_pdf.svg}{Geek3}.
Here $k$ is being used in lieu of $\nu$ for the number of degrees of freedom.
}
\label{fig:dist:chi2}
\end{figure}

\begin{figure}
\centering
\includegraphics[width=0.7\textwidth,trim={1.5cm 0.1cm 0.1cm 0.1cm},clip]{figures/stats/dist/F_pdf}% trim={<left> <lower> <right> <upper>}
\caption{
\Fdist PDF,
by \href{https://en.wikipedia.org/wiki/File:F-distribution_pdf.svg}{IkamusumeFan}.
}
\label{fig:dist:F}
\end{figure}

\begin{figure}
\centering
  \begin{subfigure}[c]{0.475\textwidth}\centering
  \includegraphics[width=\textwidth]{figures/stats/dist/geometric_trials_pmf}
  \caption{Geometric Trials PMF}
  \label{fig:dist:geometric_trials}
  \end{subfigure}
  ~
  \begin{subfigure}[c]{0.48\textwidth}\centering
  \includegraphics[width=\textwidth]{figures/stats/dist/geometric_failures_pmf}
  \caption{Geometric Failures PMF}
  \label{fig:dist:geometric_failures}
  \end{subfigure}
\caption{
Geometric distribution PMFs for trials and failures,
by \href{https://en.wikipedia.org/wiki/File:Geometric_pmf.svg}{Skbkekas}.
\label{fig:dist:geometric}
}
\end{figure}

\begin{figure}
\centering
\includegraphics[width=0.7\textwidth]{figures/stats/dist/hypergeometric_pmf}
\caption{
Hypergeometric distribution PMF,
adapted from \href{https://commons.wikimedia.org/wiki/File:HypergeometricPDF.png}{Fuzzyrandom}.
}
\label{fig:dist:hypergeometric}
\end{figure}


\documentclass[twoside]{MATH77}
\usepackage{multicol}
\usepackage[fleqn,reqno,centertags]{amsmath}
\begin{document}
\begmath 12.2 Multi-Dimensional Table Look Up, Interpolation,
\hbox{and Differentiation}

\silentfootnote{$^\copyright$1997 Calif. Inst. of Technology, \thisyear \ Math \`a la Carte, Inc.}

\subsection{Purpose}

Given a multi-dimensional table of independent variable values and the
corresponding dependent variable values, this subroutine finds the points in
the table closest to a given value of the independent variable vector and
uses these points to interpolate for the corresponding value of the
dependent variable. Error estimates, different look up methods, and the
computation of derivative information are available.  Ragged tables,
a generalization of having known data defined on a grid, are
supported.
\subsection{Usage}

\subsubsection{Program Prototype, Single Precision}
\begin{description}
\item[INTEGER]  {\bf NDIM, NTAB}$(\geq $ see below){\bf ,\\
NDEG}($\geq $NDIM){\bf , LUP}($\geq $NDIM){\bf , IOPT}($\geq k$)\newline
[$k$ depends on options used ($\geq 3$).]

\item[REAL]  {\bf X}($\geq $NDIM){\bf , Y, XT}($\geq $ see below),\\ {\bf YT}%
($\geq $ see below){\bf , EOPT}($\geq $IOPT(2))
\end{description}\vspace{-5pt}
\begin{center}
\fbox{%
\begin{tabular}{@{\bf }c}
CALL SILUPM(NDIM, X, Y, NTAB, XT,\\
YT, NDEG, LUP, IOPT, EOPT)\\
\end{tabular}}
\end{center}\vspace{-5pt}
\subsubsection{Argument Definitions}

Data tables can be organized in two distinct ways -- the grid method and
the ragged table method. The specifications of NTAB(), XT(), and YT()
are given in this section for the (simpler) grid method. See Section B.3
for the ragged table method.
\vspace{-5pt}
\begin{description}
\item[NDIM]  \ [in] Number of dimensions for the independent variable, 1~$%
\leq $~NDIM~$\leq $~10.
\item[X()]  \ [in] Independent vector value, {\bf x}, where value of
the interpolant is desired, {\bf x} = $(x_1$, $x_2$, ..., $x_{\text{NDIM}}).$
\item[Y]  \ [out] Value of interpolant.
\item[NTAB()]  \ [inout] For the grid method, NTAB($i$) gives the number of
points in the table data with respect to the $i^{th}$ dimension for 1 $\leq $
i $\leq $ NDIM. NTAB must have a declared dimension $\geq 2\times $ NDIM +
1. On the initial entry NTAB(NDIM$+1)$ must be~0. On the initial entry
(signaled by NTAB(NDIM$+1)$ being~0) the subroutine will change the contents
of NTAB(NDIM+1) and store values in NTAB(NDIM+2:2$\times $NDIM+1). The user
must not alter the contents of NTAB() while continuing to do interpolations
in the same table. See Section B.3 for ragged tables.
\item[XT()]  \ [in] Array of independent variable values for the tables. For
the grid method the user may store the NTAB(1) tabular values of $x_1$ in
the first NTAB(1) locations of XT(), followed immediately by the NTAB(2)
tabular values of $x_2$ in the next NTAB(2) locations, etc. For each
variable its values must be either in nondecreasing or nonincreasing
order, but this is not checked for.  Consecutive values listed for one
variable may be equal, but this has a special meaning as described in
Chapter~12.1.

For each variable that is to be given at an equally spaced sequence of
values one can use an alternative method of specifying the values, $i.e.$,
one can just list two values giving respectively the first value and the
increment between values. If this is done for the $i^{th}$ variable it must
be signaled by setting LUP($i)=3$. When the specification of the $i^{th}$
variable is abbreviated in this way the specification values for the $%
i+1^{st}$ variable (if any) must begin in the immediately following location
in XT().

If no entries in LUP() are set to~3 the minimal dimension for XT() is
$\sum _{i=1}^{\text{NDIM}}$ NTAB($i$). For each $i$ with LUP($i)=3$,
replace NTAB($i$) by~2 in this sum for determining the dimension. See Section
B.3 for ragged tables.
\item[YT()]  \ [in] Array of dependent variable values for the tables. For
the grid method the number of values to be given in YT() and the minimal
dimension for YT() is the product of the values of NTAB($i$) for $i=1$,\ ...,\
NDIM. The ordering of the $y$ values must be such that the index of the last
independent variable is advanced most rapidly. For example, when NDIM $=2$
the order of the values must be%
\begin{equation*}
\hspace{-10pt}((y(x_{1,i},x_{2,j}),\ j\!=\!1,\ NTAB(2)),\ i\!=\!1,NTAB(1))
\end{equation*}
and when NDIM $=3$ the order of the values must be%
\begin{multline*}
(((y(x_{1,i},x_{2,j},x_{3,k}),\ k=1,\ NTAB(3)),\\
j=1,\ NTAB(2)),\ i=1,\ NTAB(1)).
\end{multline*}
See Section B.3 for ragged tables.
\item[NDEG()]  \ [in] NDEG($i$) defines the nominal degree of the polynomial
to be used in the $i^{th}$ dimension. The definition for NDEG($i$) is like
that of NDEG in the one-dimensional case; see the write-up for SILUP
in Chapter~12.1.
\item[LUP()]  \ [inout] LUP($i$) defines the type of look up method for the $%
i^{th}$ dimension exactly as LUP does for the one-dimensional case in
Chapter~12.1, except that the value 4 should not be used here..
\item[IOPT()]  \ [inout] IOPT(1) is used to return a status as follows:
\begin{itemize}
\item[0]  Normal return, no exceptional conditions.
\item[1]  X was outside the domain of the table, extrapolation used.
\item[2]  Available table values were so few that this restricted the degree
of the polynomial.
\item[$-1$]  Error estimate is greater than requested error.
\item[$-2$]  Bad value for NDIM.
\item[$-3$]  Bad value for LUP($i$).
\item[$-4$]  Bad specification for a ragged table.
\item[$-5$]  Ragged table does not start with a~0.
\item[$-6$]  Bad value inside a ragged table.
\item[$-7$]  Bad value at end of ragged table.
\item[$-8$]  Bad option index.
\item[$-9$]  In some dimension, the first and last XT values are equal.
\item[$-10$]  Too many derivatives requested.
\item[$-11$]  Bad value for number of derivatives in some dimension.
\item[$-12$]  Problem with storage use in EOPT.
\item[$<${$-20$}]  Had an error in SILUP. The value returned is (the value
set by SILUP)~$-$~20.
\end{itemize}
IOPT(2) gives the dimension of EOPT. In addition to the first location for
the error estimate, and the locations in EOPT used for options, SILUPM
requires a contiguous block of storage in EOPT of length $2\times \left[
2 \times \text{NDIM} + \sum_{i=1}^{\text{NDIM}-1}\text{NDEG}(i)\right]
+$ any additional space required for temporary derivative storage (see option~3
below). If you would like a message detailing memory requirements, run the
program with IOPT$(2)=0$, and everything else as it would ordinarily be set.

Starting with IOPT(3), options are specified by integers in the range~0 to~6,
followed in some cases by integers providing argument(s) for the option.
Each option, with its arguments if any, is followed in IOPT by the next
option or by a~0.  If an option index is specified more than once,
only the last specification is used.
\begin{itemize}
\item[0]  End of the option list; this must always be the last option
specified in IOPT.
\item[1]  An error estimate is to be returned in EOPT(1).
\item[2]  (Arguments: $K2_1$, ..., $K2_{\text{NDIM}})K2_i$ gives the
polynomial degree to use when extrapolating with respect to the $i^{th}$
dimension. Documentation for SILUP (Chapter~12.1) gives the default.

\item[3] (Arguments: K3, L3, $M3_1$, ..., $M3_{NDIM})$ This requests
  computation of derivatives. Let $D(i_1$, $i_2$, ...,
  $i_{\text{NDIM}})$ denote the result of differentiating P, the
  interpolating polynomial, $i_1$ times with respect to $x_1$, $i_2$
  times with respect to $x_2$, ..., $i_{NDIM}$ times with respect to
  $x_{NDIM}$, all divided by $i_1!i_2!...i_{NDIM}!$. The $i_j$ satisfy
  the restrictions $\Sigma i_j\leq $ L3 and $0\leq i_j\leq
  \text{M3}_j$. Require $0\leq \text{M3}_j\leq $ NDEG($j)$. Subject to
  these restrictions, the $D$'s are stored starting at EOPT(K3) in
  lexicographic order of $i_{NDIM}$, ..., $i_1$. (Note that for XT,
  YT, and NTAB the lexicographic order is in terms of putting $i_1$
  first in the list.) Thus, for example, if NDIM $=2$, L3 $=2$, and
  $M3_1=M3_2=1$, then EOPT(K3$)=\partial P/\partial x_1$,
  EOPT(K3$+1)=\partial P/\partial x_2$, and EOPT(K3$+2)=\partial
  ^2P/\partial x_1\partial x_2$. Note that if L3 had been~1, nothing
  would have been stored in EOPT(K3$+2).$ The Lexicographic ordering
  here was (0,1), (1,0), (1,1), where (0,1) indicates the $0^{th}$
  partial with respect to $x_2$, and the first partial with respect to
  $x_1$.  To clarify what we mean by a lexiographic ordering, if one
  is getting all the derivatives up to second with respect to every
  variable, in three dimensions, the order looks like (0,0,1),
  (0,0,2), (0,1,0), (0,1,1), (0,1,2), (0,2,0), (0,2,1), (0,2,2),
  (1,0,0), (1,0,1), (1,0,2), (1,1,0), \ldots (2,2,0), (2,2,1),
  (2,2,2).  If one only want derivative up to a total derivative of
  order 2, delete from the list, those where the sum of the numbers is
  $>2$.

Extra storage in EOPT is required when this option is used, as mentioned in
the description of IOPT(2) above. Let $D_j$ denote the space for the total
number of derivatives that must be computed in dimension $j$ to get the
final $D_1$ derivatives in dimension~1 (with $0^{th}$ derivatives being
counted in the space). We do not have an explicit formula for the $D_j$, but
a recurrence is given at the end of Functional Description below. There
must, of course, be $D_1-1$ locations available starting at K3 for storing
the final result. In addition, $\sum_{j=2}^{\text{NDIM}}($NDEG$%
(j-1)+2)(D_j-1)$ locations are required for temporary storage. If you would
like a suggestion on how much memory to set aside and what value to assign
to K3, set K3 $=0$ and such a suggestion will be printed. If K3 is set to $%
-1 $ and IOPT(2) is large enough, a value will be selected for K3, the
location in IOPT containing K3 will be set to this value, and computation
will be done as if this value had been assigned in the first place.
\item[4]  (Argument K4) The absolute and relative errors expected in YT
entries are specified in EOPT(K4) and EOPT(K4$+1)$ respectively. The values
provided here are used in estimating the error in the interpolation. An
error estimate is returned in EOPT(1).
\item[5]  (Argument K5) Do the interpolation to the accuracy requested by
the absolute error tolerance specified in EOPT(K5). An attempt is made to
keep the final error $<$ EOPT(K5). Standard polynomial interpolation is
done, but here NDEG() gives the maximal degree polynomial to use in the
interpolation. If EOPT(K5) $\leq $ 0, IOPT(1) is not set to $-1,$ and no
error message is generated due to an unsatisfied accuracy request. An error
estimate is returned in EOPT(1).
\item[6]  (Argument K6) Do not use a point in the interpolation if the value
of the dependent variable at that point is equal to EOPT(K6). This option
can simplify the specification of tables, if some of the points that would
naturally be included do not have good data associated with them. This
option only affects interpolations done in the last dimension since such
interpolations are the only ones that are based directly on YT. Inaccurate
results are liable to result if valid data points are too widely separated
when doing an interpolation in the last dimension. Thus for example, in a
2-dimensional grid, if bad data points tend to occur in runs that correspond
to the same value of the first variable, one would be better off
interchanging these variables.
\end{itemize}
\item[EOPT()]  \ [inout] Array used as follows.
\begin{description}
\item[EOPT(1)]  \ [out] contains an estimate of the error in the
interpolation if an error estimate has been requested.
\item[EOPT(2:IOPT(2))]  \ [in or out] for use by options~3--6, and for
temporary storage.
\end{description}
\end{description}
\subsubsection{Ragged Tables Defined, with Examples for all Possible 2 and
3-Dimensional Tables}

The description above gives all the information necessary to interpolate
data on a grid. Tables 1 and~2 below illustrate the setting of NDIM, NTAB(),
XT(), and YT() for 2-D and 3-D grid cases respectively. The values shown
following the symbol $``\Rightarrow "$ following NTAB() in these tables are
the values the subroutine computes and inserts into the NTAB() array on the
initializing call. Generally the user need not be concerned with these
values.

Here we define ragged tables and specify how to set NTAB(), XT(), and YT()
for this more general case. The capability to interpolate ragged tables
provides for more general tables than simple grids, but requires less
storage and allows a simpler interpolation algorithm than would be needed
for general scattered data. Information in this section is recommended as
preparation if one intends to understand the details in Section D.

If there is just one set of $x_i$ values where values of $y$ are specified
then $x_i$ is said to be a grid variable, whereas if the set of $x_i$ values
where values of $y$ are specified is different depending on the selection of
values of one or more of the variables with indexes $j < i$, $x_i$ is called
a ragged variable. This subroutine requires the first variable, $x_1$, to be
a grid variable and further requires that all grid variables come before all
ragged variables, if any. Thus we can define an integer $n_g \geq 1$ to
denote the number of grid variables, and the variables $x_i$ with $i \leq
n_g $ will be grid variables while the remaining variables, if any, will be
ragged variables. The table is regarded as a grid if all variables are grid
variables, and as a ragged table otherwise.

If $x_i$ is a grid variable, NTAB($i$) must indicate the number of tabular
values for $x_i$, as previously described. If $x_i$ is a ragged variable,
NTAB($i$) must be set to a negative value, indicating that value sets for $x_i$
depend on the values of the variables $x_1$ through $x_{|\text{NTAB}(i)|}$.
The value of NTAB($i$) is required to be $-(i-1)$ if $i < $ NDIM or if $i =$
NDIM and $x_{\text{NDIM}}$ is not the only ragged variable, whereas if $i =$
NDIM and $x_{\text{NDIM}}$ is the only ragged variable, NTAB($i$) can be any
value from $-$1 through $-$(NDIM~$-$~1).

As a consequence of these rules there are only two different possibilities
with NDIM $= 2$ and four different possibilities with NDIM $= 3$. Examples
of each of these six cases are given in Tables 1 to~6, whose characteristics
are summarized in the following list:

\begin{tabular}{@{}cccc@{}}
\bf Table & \bf NDIM & \bf NTAB(1:NDIM) & \bf Description\\
   1 & 2 & $n_1$, $n_2$ & 2-D grid\\
   2 & 3 & $n_1$, $n_2$, $n_3$ & 3-D grid\\
   3 & 2 & $n_1$, $-1$ & 2-D ragged\\
   4 & 3 & $n_1$, $n_2$, $-2$ & 3-D ragged\\
   5 & 3 & $n_1$, $-1,$ $-2$ & 3-D ragged\\
   6 & 3 & $n_1$, $n_2$, $-1$ & 3-D ragged\\
\end{tabular}

In the 2-D ragged table, the $x_2$ values must be specified depending on $%
x_1 $. In the 3-D tables, Tables 4 and~6 have $x_2$ values on a grid, and
thus these values need not be specified. In Table~5, the values of $x_2$
depend on the value of $x_1$. In Tables 4 and~5, the $x_3$ values depend on
both $x_1$ and $x_2$ values. In Table~6, the $x_3$ values depend only on $x_1
$.

Note that a 3-D table with $x_2$ and $x_3$ values both depending only on $%
x_1 $ is not allowed.

\paragraph{Specification of NTAB() for a ragged\newline table}

One must set NTAB(1:NDIM) as described above, and set NTAB(NDIM+1) =
NTAB(3$\times $NDIM+1) = 0. NTAB(NDIM+2:3$\times $NDIM) need not be
initialized.

Let $r$ denote the index of the first ragged variable. The sizes of the
value sets for $x_r$ must be stored in NTAB() consecutively beginning at
NTAB(3$\times $NDIM+2). There must be one size for each possible combination
of values of the variables indexed from~1 through $|$NTAB$(r)|$. These sizes
must be ordered in lexicographic order of the indexes of the values of the
variables on which they depend.

As an example, consider Table~3 where the first ragged variable is $x_2$ and
it necessarily depends only on $x_1$. The variable $x_1$ has four values, so
four sizes must be given. These are set as 3,2,3,2 in NTAB(8:11).

As a more complicated example, consider Table~4 where the first ragged
variable is $x_3$ and it depends on $x_1$ and $x_2$. Since $x_1$ has three
values and $x_2$ has two values the number of combinations of values of $x_1$
and $x_2$ is six, and therefore six sizes must be given for value sets for $%
x_3$. These are given as 2,3,4,3,2,3 in NTAB(11:16). This ordering results
from first fixing $x_1$ at its first value and running $x_2$ through its two
values, then fixing $x_1$ at its second value and again running $x_2$
through its two values, and finally fixing $x_1$ at its third value and
again running $x_2$ through its two values.

If there are no more ragged variables the next available location in NTAB()
must be set to $-$1 to signal the end of information in NTAB(). Otherwise this
next available location must be set to zero and the sizes for the next
ragged variable must be stored consecutively in the following locations.

Our only example having more than one ragged variable is Table~5. The second
ragged variable is $x_3$ and it depends on $x_1$ and $x_2$, with $x_2$
itself being ragged. To count the number of sizes needed for $x_3$ one must
make use of the sizes already set for $x_2$. Thus with the first value of $%
x_1$ there are 3~values for $x_2$, while the second and third values of $x_1$
each have 2~associated values of $x_2$. Thus there is a total of 7~possible
combinations of values of $x_1$ and $x_2$, and so seven sizes must be given
for $x_3$. These are given as 2,3,2,3,2,2,3 in NTAB(15:21).

\paragraph{Specification of XT() for a ragged table}

Value sets for the grid variables must be entered into XT() as previously
described. Immediately following these, enter the value sets for each ragged
variable. For each ragged variable the number of values entered must agree
with the value set sizes in NTAB(). For example, consider the value sets for
$x_3$ in Table~4. The sizes for these sets are given in NTAB(11:16) as
2,3,4,3,2,3. The six value sets for $x_3$ are of these respective sizes and
are stored in XT(6:22).

If $x_i$ is a ragged variable and LUP($i) = 3$, then each of the value sets
for $x_i$ must be entered in XT() as just a pair of numbers representing the
first value and the increment. No gaps are to be left in the XT() array
between these pairs and the data for $x_{i+1}$, if any.

\paragraph{Specification of YT() for a ragged table}

\begin{center}
{\bf Table 1. A 2-D Grid}\vspace{5pt}
\fbox{\begin{tabular}{|c|c|c|c|c|c}\cline{1-5}
 X1$\Rightarrow $ & $-1$ & 2 & 3 & 8\\\cline{1-5}
 X2$\Downarrow $ & \multicolumn{4}{c|}{Y}\\\cline{1-5}
 20 & 1 & 4 & 7 & 10\\\cline{1-5}
 22 & 2 & 5 & 8 & 11\\\cline{1-5}
 27 & 3 & 6 & 9 & 12\\\cline{1-5}
\multicolumn{6}{l}{\rule{0pt}{15pt}NDIM = 2}\\
\multicolumn{6}{l}{NTAB() = (4,3,\ 0,\ 0,0)}\\
\multicolumn{6}{l}{\phantom{NTAB()} $\Rightarrow $ (4,3,\ 1000,\ 1,5)}\\
\multicolumn{6}{l}{XT() = ($-$1,2,3,8,\ 20,22,27)}\\
\multicolumn{6}{l}{YT() = (1,2,3,\ 4,5,6,\ 7,8,9,\ 10,11,12)}
\end{tabular}}
\end{center}

\begin{center}
{\bf Table 2. A 3-D Grid}\vspace{5pt}
\fbox{\begin{tabular}{|c|r|r|r|r|r|r|r}\cline{1-7}
 X1$\Rightarrow $ & \multicolumn{2}{c|}{3} & \multicolumn{2}{c|}{7} &
 \multicolumn{2}{c|}{8}\\\cline{1-7}
 X2$\Rightarrow $ & 20 & 22 & 20 & 22 & 20 & 22\\\cline{1-7}
 X3$\Downarrow $ & \multicolumn{6}{c|}{Y}\\\cline{1-7}
 31 & 1 & 5 & 9 & 13 & 17 & 21\\\cline{1-7}
 33 & 2 & 6 & 10 & 14 & 18 & 22\\\cline{1-7}
 35 & 3 & 7 & 11 & 15 & 19 & 23\\\cline{1-7}
 36 & 4 & 8 & 12 & 16 & 20 & 24\\\cline{1-7}
\multicolumn{8}{l}{\rule{0pt}{15pt}NDIM = 3}\\
\multicolumn{8}{l}{NTAB() = (3,2,4,\ 0,\ 0,0,0)}\\
\multicolumn{8}{l}{\phantom{NTAB()} $\Rightarrow $ (3,2,4, 1000, 1,4,6)}\\
\multicolumn{8}{l}{XT() = (3,7,8,\ 20,22,\ 31,33,35,36)}\\
\multicolumn{8}{l}{YT() = (1,2,3,4,\ 5,6,7,8,\ 9,10,11,12,}\\
\multicolumn{8}{l}{\phantom{YT}13,14,15,16,\ 17,18,19,20,\ 21,22,23,24)}
\end{tabular}}
\end{center}

\begin{center}
{\bf Table 3. A 2-D Ragged Table with X2 depending on X1}\vspace{5pt}
\fbox{\begin{tabular}{c|r|r|r|r|r|r|r|r|r}\cline{1-9}
 \multicolumn{1}{|c|}{X1$\Rightarrow $} & \multicolumn{2}{c|}{$-$1} &
 \multicolumn{2}{c|}{2} & \multicolumn{2}{c|}{5} &
 \multicolumn{2}{c|}{8}\\\cline{1-9}
 & X2 & Y & X2 & Y & X2 & Y & X2 & Y\\\cline{2-9}
 & 20 & 1 & 21 & 4 & 20 & 6 & 21 & 9\\\cline{2-9}
 & 22 & 2 & 28 & 5 & 24 & 7 & 27 & 10\\\cline{2-9}
 & 27 & 3 & \multicolumn{2}{c|}{} & 28 & 8 & \multicolumn{2}{c}{}
\\\cline{2-3}\cline{6-7}
\multicolumn{10}{l}{\rule{0pt}{15pt}NDIM = 2}\\
\multicolumn{10}{l}{NTAB() = (4,$-$1,\ 0,\ 0,0,\ 0,\ 0,3,2,3,2,$-$1)}\\
\multicolumn{10}{l}{\phantom{NTAB()} $\Rightarrow
$ (4,$-$1,\ 1,\ 1,5,\ 7,\ 0,3,5,8,10,$-$1)}\\
\multicolumn{10}{l}{XT() = ($-$1,2,5,8,\ 20,22,27,\ 21,28,\ 20,24,28,\ 21,27)}\\
\multicolumn{10}{l}{YT() = (1,2,3,\ 4,5,\ 6,7,8,\ 9,10)}
\end{tabular}}
\end{center}

The $y$ values must be listed in YT() in lexicographic order of the
indexes of the independent variables. For example, if NDIM $= 3$,
YT(1) must be the value of $y$ associated with the first value of
$x_1$, the first value of $x_2$ allowed to occur with this value of
$x_1$, and the first value of $x_3$ allowed to occur with these
values of $x_1$ and $x_2$. If there are more values of $x_3$ allowed
to occur with these values of $x_1$ and $x_2$ then the values of $y$
associated with these combinations of values would come next in YT().
The setting of the YT() array is illustrated in Tables~1~--~6.  In
these tables both the initial and the final contents of NTAB() are
given.

\hspace{.15in}\begin{tabular}{c}
\vspace{-15pt}
\bf {Table 4. A 3-D Ragged Table with X3 depending on X1 and X2}\vspace{20pt}\\
\fbox{\begin{tabular}{l}
 \begin{tabular}{c*{4}{|r}|*{4}{|r}|*{4}{|r}|}\hline
 \multicolumn{1}{|c|}{X1$\Rightarrow $} & \multicolumn{4}{c||}{3} &
 \multicolumn{4}{c||}{7} & \multicolumn{4}{c|}{8}\\\hline
 \multicolumn{1}{|c|}{X2$\Rightarrow $} & \multicolumn{2}{c|}{20} &
 \multicolumn{2}{c||}{22} & \multicolumn{2}{c|}{20} &
 \multicolumn{2}{c||}{22} & \multicolumn{2}{c|}{20} &
 \multicolumn{2}{c|}{22}\\\hline
 & X3 & Y & X3 & Y  & X3 & Y  & X3 & Y  & X3 & Y  & X3 & Y\\\cline{2-13}
 & 30 & 1 & 31 & 3 & 31 & 6 & 30 & 10 & 32 & 13 & 30 & 15\\\cline{2-13}
 & 39 & 2 & 35 & 4 & 33 & 7 & 35 & 11 & 39 & 14 & 36 & 16\\\cline{2-13}
\multicolumn{3}{c|}{} & 38 & 5 & 36 & 8 & 39 & 12 &
\multicolumn{2}{c|}{} & 38 & 17\\\cline{4-5}\cline{4-9}\cline{12-13}
\multicolumn{4}{c}{} &
\multicolumn{1}{c|@{\protect\rule[-3.4pt]{2pt}{0.4pt}}|}{~} &
38 & 9\vspace{-.5pt}\\\cline{6-7}
\end{tabular}\\
\begin{tabular}{l}
 NDIM = 3\\
 NTAB() = (3,2,$-$2,\ 0,\ 0,0,0,\ 0,0,\ 0,2,3,\ 4,3,\ 2,3,\ $-$1)\\
 \phantom{NTAB()} $\Rightarrow $ (3,2,$-$2,\ 3,\ 1,4,6,\ 10,10,\ 0,2,5,\ 9,12,
14,17,\ $-$1)\\
 XT() = (3,7,8,\ 20,22,\ 30,39,\ 31,35,38,\ 31,33,36,38,\ 30,35,39,
32,39,\ 30,36,38)\\
 YT() = (1,2,\ 3,4,5,\ 6,7,8,9,\ 10,11,12,\ 13,14,\ 15,16,17)\\
\end{tabular}
\end{tabular}}\vspace{10pt}\\

{\bf Table 5. A 3-D Ragged Table with X2 depending on X1,
and X3 depending on X1 and X2}\vspace{5pt}\\
\fbox{\begin{tabular}{l}
 \begin{tabular}{c*{6}{|r}|*{4}{|r}|*{4}{|r}|}\hline
 \multicolumn{1}{|c|}{X1$\Rightarrow $} & \multicolumn{6}{c||}{3} &
 \multicolumn{4}{c||}{7} & \multicolumn{4}{c|}{8}\\\hline
 \multicolumn{1}{|c|}{X2$\Rightarrow $} & \multicolumn{2}{c|}{20} &
 \multicolumn{2}{c|}{25} & \multicolumn{2}{c||}{29} &
 \multicolumn{2}{c|}{21} & \multicolumn{2}{c||}{29} &
 \multicolumn{2}{c|}{20} & \multicolumn{2}{c|}{28}\\\hline
 & X3 & Y & X3 & Y  & X3 & Y  & X3 & Y  & X3 & Y  & X3 & Y  & X3 & Y
\\\cline{2-15}
 & 31 & 1 & 30 & 3 & 32 & 6 & 30 & 8 & 31 & 11 & 31 & 13 & 30 & 15
\\\cline{2-15}
 & 39 & 2 & 35 & 4 & 37 & 7 & 34 & 9 & 37 & 12 & 38 & 14 & 36 & 16
\\\cline{2-15}
\multicolumn{3}{c|}{} & 38 & 5 & \multicolumn{1}{c}{} &
\multicolumn{1}{c|@{\protect\rule[-3.4pt]{2pt}{0.4pt}}|}{~} & 38 & 10 &
\multicolumn{4}{c|}{} & 39 & 17\vspace{-.5pt}\\\cline{4-5}\cline{8-9}%
\cline{14-15}
\end{tabular}\\
\begin{tabular}{l}
 NDIM = 3\\
 NTAB() = (3,$-$1,$-$2,\ 0,\ 0,0,0,\ 0,0,\ 0,3,2,2,\ 0,2,3,2,\ 3,2,\
2,3,\ $-$1)\\
 \phantom{NTAB()} $\Rightarrow $ (3,$-$1,$-$2,\ 2,\ 1,4,11,\ 10,14,\
0,3,5,7,\ 0,2,5,7,\ 10,12,\ 14,17,\ $-$1)\\
 XT() = (3,7,8,\ 20,25,29,\ 21,29,\ 20,28,\ 31,39,\ 30,35,38,\ 32,37,
30,34,38,\ 31,37,\ 31,38,\ 30,36,39)\\
 YT() = (1,2,\ 3,4,5,\ 6,7,\ 8,9,10,\ 11,12,\ 13,14,\ 15,16,17)\\
\end{tabular}
\end{tabular}}\vspace{10pt}\\

{\bf Table 6. A 3-D Ragged Table with X3 depending only on
X1}\vspace{5pt}\\
\fbox{\begin{tabular}[b]{c*{3}{|r}|*{3}{|r}|*{3}{|r}|}\hline
 \multicolumn{1}{|c|}{X1$\Rightarrow $} & \multicolumn{3}{c||}{3} &
 \multicolumn{3}{c||}{7} & \multicolumn{3}{c|}{8}\\%
\cline{1-1}\cline{3-4}\cline{6-7}\cline{9-10}
 \multicolumn{1}{|c|}{X2$\Rightarrow $} & & 20 & 22 & & 20 & 22 & &
20 & 22\\\hline
& X3 & \multicolumn{2}{c||}{Y} & X3 & \multicolumn{2}{c||}{Y} &
X3 & \multicolumn{2}{c|}{Y}\\\cline{2-10}
 & 30 & 1 & 3 & 31 & 5 & 8 & 30 & 11 & 13\\\cline{2-10}
 & 38 & 2 & 4 & 35 & 6 & 9 & 39 & 12 & 14\\\cline{2-10}
\multicolumn{3}{c}{} &
\multicolumn{1}{c|@{\protect\rule[-3.4pt]{2pt}{0.4pt}}|}{~} & 39 & 7 & 10 &
\multicolumn{3}{r}{~}\vspace{-.5pt}\\\cline{5-7}
\end{tabular}
\begin{tabular}{l}
 NDIM = 3\\
 NTAB() = (3,2,$-$1,\ 0,\ 0,0,0,\ 0,0,\ 0,2,3,2,\ $-$1)\\
 \phantom{NTAB()} $\Rightarrow $ (3,2,$-$1,\ 1,\ 1,4,6,\ 10,10,\
0,2,5,7,\ $-$1)\\
 XT() = (3,7,8,\ 20,22,\ 30,38,\ 31,35,39,\ 30,39)\\
 YT() = (1,2,\ 3,4,\ 5,6,7,\ 8,9,10,\ 11,12,\ 13,14)\\
\end{tabular}}
\end{tabular}\vspace{8pt}

\subsubsection{Getting Output of Tables}
\label{sec:debug}
As verification that things have been set up correctly it can be
helpful with complicated ragged tables to see how things are laid
out.  This data can be seen with a call vary similar to that for
SILUPM.  Just prior to the call to SILUPM,
\begin{center}
\fbox{%
\begin{tabular}{@{\bf }c}
CALL SILUPMD(NDIM, X, Y, NTAB, XT,\\
YT, NDEG, LUP, IOPT, EOPT)\\
\end{tabular}}
\end{center}\vspace{-5pt}
This can result in a lot of output, so if possible it may work best
for you if you reduce the size of the tables while keeping the logical
structure intact.

\rule[-6.6in]{0pt}{8pt}

\subsubsection{Modifications for Double Precision}

Change SILUPM to DILUPM, SILUPMD to DILUPMD and the REAL type
statement to DOUBLE PRECISION.
\subsection{Examples and Remarks}

DRSILUPM below is a sample program that does interpolation in a table of $%
\sin (x_1x_2)$ given with a spacing of~0.08 in $x_1$ and a spacing of~0.12 in
$x_2$, degree~8 in $x_1$ and degree~10 in $x_2$, and obtains an error
estimate. ODSILUPM below gives the results of running DRSILUPM on an IBM PC.

Section C of Chapter~12.1 applies here also. In particular a
discontinuity occurring at $x_i = d_i$ can be indicated by having successive
XT() values associated with $x_i = d_i.$

\subsection{Functional Description}

Interpolations are done by doing nested one dimensional interpolations in
the coordinate directions. First the XT() values to be used depending on the
value of $x_1$ are determined. Then the values of XT() for the value of $x_2$
are determined, etc. For ragged tables, the values of some of the XT() that
are to be used depend on indexes selected for lower indexed XT()'s. After the
base pointer to XT() and YT() is determined for the last dimension, NDIM, a one
dimensional interpolation is done in dimension NDIM. This provides one
function value for the next lower dimension, and other values are obtained
in a similar way by varying the index in the next to the last dimension.
This process gives function values that can be interpolated to provide a
function value for the next lower dimension, etc.

Let $n_j =$ NTAB$(j)$, $j = 1$, 2, ..., NDIM. The $n_j$ must satisfy:

\begin{description}
\item[\rm If $n_k > 0,$]  then $n_j>0$ for $j<k$, and $n_k$ gives the number of
data points in dimension $k.$

\item[\rm If $n_k < 0,$]  then information for the current dimension depends on
dimensions with indexes from~1 to $|n_k|$. If $k\neq $ NDIM then $n_k=-k+1$,
else $1\leq |n_k|<$ NDIM.
\end{description}

Tables with an $n_k<0$ are called ragged, others are said to have data
defined on a grid. When data are defined on a grid, no extra information is
required. In the ragged case, the number of XT() values and their values
depend on the indexes from dimensions with a smaller index. The user must
provide this information. So users with data on a grid don't have to deal
with the complexities of the ragged case, we give the formula for finding XT()
data although for the case of ragged tables some of the symbols used rely on
things defined later. Let
\begin{equation}
\label{O1}\xi _k=
\begin{cases}
 2 & \text{LUP}_k=3,\quad n_k>0\\
n_k &\text{LUP}_k\neq 3,\quad n_k>0\\
2\mu _k & \text{LUP}_k=3,\quad n_k<0\\
S_{k,\mu _k} & \text{LUP}_k\neq 3,\quad n_k<0
\end{cases}
\end{equation}
then XT() data for interpolation with respect to the $k^{th}$ dimension starts
in
\begin{gather}\label{O2}
\hspace{-15pt}\text{XT}\left( 1+\eta _k+\sum_{j=1}^{k-1}\xi _j\right) ,
\quad \text{where}\\
\label{O3} \hspace{-15pt}\eta _k=
\begin{cases}
0 &\text{if }n_k>0\\
2\left(p_{-n_k}+i_{-n_k}-1\right) & \text{if }n_k<0\text{ \& LUP}_k=3\\
p_{1-n_k} & \text{if }n_k<0\text{ \& LUP}_k\neq 3
\end{cases}
\end{gather}
On the first call to the program, NTAB(NDIM+1) is set to $|n_k|$, where $k$
is the first index for which $n_k<0$ (or to~1000 if all $n_k>0)$, and
NTAB(NDIM+$k$+1) is set to $1+\sum_{j=1}^{k-1}\xi _j,k=1,2,...,$ NDIM. User's
with data on a grid need not know about $\mu $, $p$, S or anything else
beyond this point.

Ragged tables require information on the nature of the raggedness. This
information must be supplied in what we call lexicographic order. The first
data supplied are for the $r^{th}$ dimension, where $r$ is the index of the
first dimension that is ragged. Let $e=|$NTAB$(r)|$, and the indexes
selected from dimensions~1 to $e$ be $(i_1$, $i_2$, ..., $i_e)$. The data $%
K_{r,j}$ supplied for this dimension define the number of YT() values defined
for all the possible values of the indexes from dimensions~1 to $e$. $K_{r,0}
$ must be~0, and $K_{r,1}$ is the number of YT() values that correspond to
having $i_1$, ..., $i_e =1$, $K_{r,2}$ to having $i_e=2$, and all of the
rest~1, etc. That is a lexicographic ordering of the indexes $(i_1$, ..., $%
i_e)$. Following the data for dimension $r$ comes the data for dimension $r+1
$, if any, etc. To enhance error checking, the $K$'s for the last
dimension must be followed by $-$1. To describe how the program keeps
track of this information we introduce some additional notation. Let $\mu _k$
denote the index of the last $j$ for the $K_{k,j}$. Then
\begin{gather}\label{O4}
\mu _k=
\begin{cases}
-1 & \text{if }n_k>0\\
\prod_{j=1}^en_j & \text{if }k=r\\
S_{k-1,\mu _{k-1}} & \text{if }k>r,\quad \text{where}
\end{cases}\\
\label{O5}
S_{k,j}=\sum_{m=0}^jK_{k,m},\quad j=0,1,...,\mu _k.
\end{gather}
The $S_{k,j}$ replace the $K_{k,j}$ in storage. Let $L_1=3\times
\text{NDIM} + 1,$ and $L_k=L_{k-1}+\mu _k+1$, $k>1$. Initially NTAB$(L_k+j)$
contains $K_{k,j}$; these are replaced by the $S_{k,j}$ on the first call. $%
L_k$ is stored in NTAB$(2\times \text{NDIM}+k+1)$, $k=1$, 2, ..., NDIM
$-$ 1. With what has been defined so far, it is now possible to compute
the values stored in NTAB(NDIM+$k$+1) as described just below Eq.\,(3).

The notation introduced below defines the rest of the information required
to find the XT() values as well as the values of YT() to be accessed in the one
dimensional interpolations. In both cases the data are in lexicographic
order as defined above and this is all the user need know in order to save
the required information. As above, let $n_r$ be the first $n_k<0$, $e=-n_r$%
, and define $p_k$ to be a pointer for accessing information that depends on
the indexes $i_1$, $i_2$, ..., $i_{k-1}$. Recall that if $r\neq $ NDIM, then
$e=r-1$. With
\begin{equation}\label{O6}
\hspace{-15pt}\begin{split}
   p_1 &= 0\\
   p_{k+1} &= n_{k+1}\left( i_k-1+p_k\right) ,\quad k=1,2,...,e-1\\
   p_{e+1} &= S_{r,p_e+i_e-1},\ \text{ and for }k=e+1,\ ...,\text{ NDIM}-1,\\
   p_{k+1} &=
\begin{cases}
n_kp_k+\left( i_k-1\right)\left( S_{r,p_e+i_e}-S_{r,p_e+i_e-1}\right)
,\\
\hfill r=\text{NDIM,}\\
S_{k+1,p_k+i_k-1},\hfill r\neq \text{NDIM,}
\end{cases}
\end{split}
\end{equation}
the value of YT() corresponding to $(i_1$, $i_2$, ..., $i_{NDIM})
$ is in YT$(p_{\text{NDIM}}+i_{\text{NDIM}})$, and YT($p_{\text{NDIM}}+1)$
is passed to the one-dimensional interpolation routine along with the XT() as
defined by Eqs.\ (2), (3) and (6).

To compute the storage required for computing and storing derivatives, let $%
m_{i,k}$ denote the number of derivatives in dimension $i$ with the sum of
derivative orders exactly $k$, $M_{i,k}=\sum _{j=0}^k m_{i,j}$, and $t_i=$
the number of derivatives with respect
to $x_i$. Since we start by computing derivatives in the last
dimension, NDIM, $m_{NDIM,k}=1$ for $k\leq t_{\text{NDIM}}$, and is~0
otherwise. Also, $m_{i,0}\equiv 1$ since there will be exactly one
interpolated value. Differentiating the results from a higher dimension
0, 1, ..., $\min (k$, $t_i)$ times gives $m_{i,k} =
\sum _{j=max(0,k-t_i)}^k m_{i+1,j},$ and
with the convention that $M_{i,k}=0$ for $k<0,$%
\begin{equation}
\label{O7}M_{i,k}=M_{i,k-1}+M_{i+1,k}-M_{i+1,k-t_i-1}
\end{equation}
Let $\ell _i = \min \{\sum_{j=i}^{NDIM} t_j,\text{ L3 (from options)}\}.$
Then the desired recursion for computing the $M_{i,k}$ is given by
\begin{align}\label{O8}
\hspace{-15pt}M_{i,0}&=1\\
\hspace{-15pt}M_{i,k}&=M_{i,k-1}+M_{i+1,k}-M_{i+1,k-t_i-1},\
k=1,2,...,\ell_i.\notag
\end{align}
The $D_i$ referred to in the description of option~3 are given by
$D_i =M_{i,\ell_i}.$

\subsection{Error Procedures and Restrictions}

Values of IOPT$(1) < 0$ ordinarily cause an error message to be printed, and
those $< -2$ do not ordinarily result in a return to the user. One can
change the action on errors by calling the message/error routine MESS
of Chapter~19.3 before calling this routine.

\subsection{Supporting Information}

The source language is ANSI Fortran~77.

\begin{tabular}{@{\bf}l@{\hspace{5pt}}l}
\bf Entry & \hspace{.35in} {\bf Required Files}\vspace{2pt} \\
DILUPM & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, DILUP, DILUPM, DILUPMD, DMESS, MESS, OPTCHK\rule[-5pt]{0pt}{8pt}}\\
SILUPM & \parbox[t]{2.7in}{\hyphenpenalty10000 \raggedright
AMACH, MESS, OPTCHK, SILUP, SILUPM, SILUPMD, SMESS}\\
\end{tabular}

Subroutine designed and written by Fred T. Krogh, JPL, May~1991.
SILUPMD added, Fred T, Krogh Math \`a la Carte, May 2006.

\begcodenp

\medskip\

\lstset{language=[77]Fortran,showstringspaces=false}
\lstset{xleftmargin=.8in}

\centerline{\bf \large DRSILUPM}\vspace{3pt}
\lstinputlisting{\codeloc{silupm}}

\vspace{10pt}\centerline{\bf \large ODSILUPM}\vspace{5pt}
\lstset{language={}}
\lstinputlisting{\outputloc{silupm}}
\end{document}


    \filetitle{bsxfunc}{Standard BSXFUN implemented for tseries objects}{tseries/bsxfun}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
Z = bsxfun(Func,X,Y)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{Func} {[} function\_handle {]} - Function that will be applied
  to the input series, \texttt{FUN(X,Y)}.
\item
  \texttt{X} {[} tseries \textbar{} numeric {]} - Input time series or
  numeric array.
\item
  \texttt{Y} {[} tseries \textbar{} numeric {]} - Input time series or
  numeric array.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{Z} {[} tseries {]} - Result of \texttt{Func(X,Y)} with
  \texttt{X} and/or \texttt{Y} expanded properly in singleton
  dimensions.
\end{itemize}

\paragraph{Description}\label{description}

See help on built-in \texttt{bsxfun} for more help.

\paragraph{Example}\label{example}

We create a multivariate time series and subtract mean from its
individual columns.

\begin{verbatim}
x = tseries(1:10,rand(10,4));
xx = bsxfun(@minus,x,mean(x));
\end{verbatim}


\section*{Backup}
\begin{frame}
	\frametitle{Solutions to the Rarita-Schwinger equation}
	Solutions in the Rest-Frame are\footnote{For a general one see: \fullcite{shi2003solution}}:
	\begin{align*}
		\Psi^\mu(\sigma)_{\pm} = \sum_{\sigma', \sigma''} CG(\sigma; \sigma',\sigma'') \epsilon^\mu(\sigma') u_\pm(\sigma'') e^{\mp i mt}\quad
	\end{align*}\begin{align*}
		\footnotesize
		u_+(\sigma) =
		\begin{pmatrix}
			w(\sigma) \\
			0
		\end{pmatrix}\quad
		u_-(\sigma) =
		\begin{pmatrix}
			0 \\
			w(\sigma)
		\end{pmatrix}\quad
		w(\sfrac{1}{2}) = \begin{pmatrix}
			1 \\
			0
		\end{pmatrix}\quad
		w(-\sfrac{1}{2}) = \begin{pmatrix}
			0 \\
			1
		\end{pmatrix}\quad
		\epsilon^\mu(0)=i
		\begin{pmatrix}
			0 \\ 0 \\ 0 \\ 1
		\end{pmatrix}\quad
		\epsilon^\mu(\pm 1) = \mp \frac{i}{\sqrt 2}
		\begin{pmatrix}
			0\\ 1\\ \pm i\\ 0
		\end{pmatrix}
	\end{align*}
	Nonvanishing Clebsch-Gordan coefficents are:
	\begin{align*}
		CG(+\sfrac{3}{2}; 1, \sfrac{1}{2}) = 1\hspace{5em}
		CG(-\sfrac{3}{2}; -1, -\sfrac{1}{2}) = 1\quad               \\
		CG(+\sfrac{1}{2}; 1, -\sfrac{1}{2}) = \sqrt\frac{1}{3}\hspace{5em}
		CG(+\sfrac{1}{2}; 0, +\sfrac{1}{2}) = \sqrt\frac{2}{3}\quad \\
		CG(-\sfrac{1}{2}; -1, +\sfrac{1}{2}) = \sqrt\frac{1}{3}\hspace{5em}
		CG(-\sfrac{1}{2}; 0, -\sfrac{1}{2}) = \sqrt\frac{2}{3}\quad
	\end{align*}
\end{frame}
\begin{frame}
	\frametitle{Why does $\gamma_\mu\Psi^\mu$ transform like a Dirac spinor?}
	The Dirac part of the vector-spinor transforms like
	\begin{equation*}
		\Psi^{\mu'} = \Lambda^\mu_{\ \nu}M(\Lambda) \Psi^\nu
	\end{equation*}
	Also we need the property
	\begin{equation*}
		M(\Lambda)\gamma_\mu M^{-1}(\Lambda) = \Lambda^{\nu}_{\ \mu} \gamma_{\nu}
	\end{equation*}
	To show
	\begin{align*}
		(\gamma_\mu\Psi^\mu)' = \gamma_\mu  \Lambda^\mu_{\ \nu}M(\Lambda) \Psi^\nu
		= M(\Lambda)\gamma_\mu M^{-1}(\Lambda)M(\Lambda) \Psi^\nu
		= M(\Lambda) \gamma_\nu\Psi^\nu
	\end{align*}
\end{frame}
\begin{frame}
	\frametitle{Yet another formalism?}
	Very brief summary of the spinor-index formalism\footnote{Used in \fullcite[109]{landaulifschitzIV}}
	Rarita-Schwinger-Equation:
	\begin{align*}
		\hat p^{\delta \dot\gamma}\eta^{\dot \beta}_{\alpha\delta} &= m \zeta^{\dot \beta \dot \gamma}_{\alpha} \\
		\hat p_{\delta \dot\gamma}\zeta^{\dot \beta\dot \gamma}_{\alpha} &= m \eta^{\dot \beta}_{\alpha\delta}
	\end{align*}
	Dottet and undottet indices with values $1, 2$ \enquote{labeling columns and	rows of the Pauli-Matrices}.
	Lowering/rising with
	\begin{equation}
		g_{\alpha\beta}=
		\begin{pmatrix}
			0 & 1\\
			-1 & 0
		\end{pmatrix}
	\end{equation}
	This formalisim was the original one used by Fiertz and Pauli to describe spin $\sfrac{3}{2}$ and spin $2$ particles.
	\vspace{1em}
\end{frame}


\section{Green function theory}

\begin{framenologo}
  \frametitle{Green function theory}
  \tableofcontents[currentsection]
\end{framenologo}

\subsection{Introduction}

\begin{frame}
  \frametitle{Green function}
  \framesubtitle{Introduction}

  \begin{itemize}[<+->]
    \item %
    The single particle Green function may be written as:
    \begin{equation*}
      [(E+\im\eta)\ID - \HH_\kk]\G_\kk(E) = \ID
    \end{equation*}

    \item%
    This may be rewritten in terms of the eigenstates
    \begin{equation*}
      \G_\kk(E) = \sum_i \frac{|\psi_{i,\kk}\rangle\langle \psi_{i,\kk}|}{E+\im\eta-\E_{i,\kk}}
    \end{equation*}

    \item%
    Taking the imaginary part of the Green function yields
    \begin{align*}
      \Im \G_\kk(E) &= -\sum_i |\psi_{i,\kk}|^2\mathfrak{L}_{i,\kk}(E)
      \\
      \mathfrak{L}_{i,\kk}(E) &= \frac{\eta}{(E-\E_{i,\kk})^2 + \eta^2}
    \end{align*}

    \uncover<+->{
        \begin{center}
          \begin{tikzpicture}[my fill/.style={fill=good},
            font=\footnotesize]
            \begin{axis}[width=.7\textwidth,height=3.5cm,%
              xmin=-1,xmax=1,ymin=0,xlabel=Energy]
              \addplot[black,smooth,my fill] table {../data/lorentzian.dat};
              \addlegendentry{$\mathfrak{L}$}
              \draw[<->] (axis cs:-0.05, {.5/0.05}) -- node[below=5pt] {$2\eta$}
              (axis cs:0.05,{.5/0.05});
              \node[my fill] at (axis cs:0.5, 10) {$\int=\pi$};
            \end{axis}
          \end{tikzpicture}
        \end{center}
    }
  \end{itemize}

\end{frame}

\subsection{Rules of integration}

\begin{frame}[label=integration]
  \frametitle{Green function}
  \framesubtitle{Rules of integration -- Energy}

  \begin{block}{Numeric integration of Green function}
    \small
    \begin{equation*}
      \frac{-1}\pi\iint_{E'}^{E''} \dd E\dd \kk\Im \G_\kk(E) \approx \frac{-1}\pi
      \sum_\kk\delta\kk\sum_j^{(E''-E')/\delta E} \delta E \Im \G_\kk(E'+j\delta E)
    \end{equation*}

    \uncover<2->{
        Are there any problems here?
        \begin{itemize}
          \item What if $\delta E\ll \eta$?

          \uncover<3->{
              \textcolor{good}{Good!} The energy spacing is much smaller than FWHM.
          }

          \item What if $\delta E\gg \eta$?

          \uncover<3->{
              \textcolor{bad}{Bad!} The energy spacing is much larger than FWHM. Dependent on the
              initial $E'$ you will find different DOS as some eigenstates may be passed.
          }

          \item What if $\delta E\approx \eta$?

          \uncover<3->{
              \textcolor{ok}{Ok!} The energy spacing is half-width at half-maximum. This will
              typically yield a fine integration.
          }
        \end{itemize}

        \begin{center}
          \begin{tikzpicture}[my fill/.style={fill=good},
            font=\footnotesize]
            \begin{axis}[width=.7\textwidth,height=3.5cm,%
              my eta/.style={forget plot, only marks, mark=*, opacity=0.9, mark size=2pt},
              xmin=-1,xmax=1,ymin=0,xlabel=Energy]
              \addplot[black,smooth,my fill] table {../data/lorentzian.dat};
              \addlegendentry{$\mathfrak{L}$}
              \draw[<->] (axis cs:-0.05, {.5/0.05}) -- node[below=5pt] {$2\eta$}
              (axis cs:0.05,{.5/0.05});
              \node[my fill] at (axis cs:0.5, 10) {$\int=\pi$};
              \only<4->{
                  \addplot[my eta,color=good] table {../data/lorentzian_1o5eta.dat};
                  \addplot[my eta,color=bad] table {../data/lorentzian_5eta.dat};
                  \addplot[my eta,color=ok] table {../data/lorentzian_eta.dat};
              }
            \end{axis}
          \end{tikzpicture}
        \end{center}
    }

  \end{block}

  \uncover<3->{\hfill\hyperlink{DOS<3>}{\beamergotobutton{Return to DOS}}}

\end{frame}

\begin{frame}
  \frametitle{Green function}
  \framesubtitle{Rules of integration -- Brillouin Zone}

  \begin{block}{Numeric integration of Green function}
    \small
    \begin{equation*}
      \frac{-1}\pi\iint_{E'}^{E''} \dd E\dd \kk\Im \G_\kk(E) \approx \frac{-1}\pi
      \sum_\kk\delta\kk\sum_j^{(E''-E')/\delta E} \delta E \Im \G_\kk(E'+j\delta E)
    \end{equation*}

    \begin{itemize}
      \item%
      The Brillouin zone integration is just as important as the energy integration.

      \item%
      Prior understanding of the electronic structure of the system is \emph{important}!

      \item<2->%
      Choose $\delta\kk$ such that band-energies $E_\kk -
      E_{\kk+\delta\kk}\approx \eta$. Otherwise band features will not be captured.

    \end{itemize}

  \end{block}

  \begin{block}<3->{Difference between diagonalisation and Green function methods}

    \begin{columns}[t]
      \column{.4\linewidth}

      \begin{center}
        Diagonalization

        \Large 1D-sampling
      \end{center}

      $k$-points, all energy-eigenvalues

      \column{.4\linewidth}

      \begin{center}
        Green functions

        \Large 2D-sampling
      \end{center}

      $k$ and $E$-points are both required to be sampled

    \end{columns}

  \end{block}

\end{frame}

\subsection{Advancing to Non-Equilibrium Green Function}

\begin{frame}
  \frametitle{Advancing $\to$ NEGF}

  \begin{block}{Single particle Green function}
    \begin{equation*}
      [(E+\im\eta)\ID - \HH_\kk]\G_\kk(E) = \ID
    \end{equation*}
  \end{block}

  \begin{block}{Non-equilibrium Green function}
    \begin{equation*}
      [(E+\im\eta)\SO - \HH_\kk - \sum_\idxE \SE_{\idxE,\kk}(E-\mu_\idxE)]\G_\kk(E) = \ID
    \end{equation*}

    Additional terms:
    \begin{itemize}
      \item%
      $\SO$ is the \emph{overlap matrix} which is needed for non-orthogonal basis sets.

      \item%
      $\SE$ is the \emph{self-energy} which is describing semi-infinite directions
      (integrating out $k$ in that direction)

    \begin{center}

      \begin{columns}

        \column{.4\textwidth}
        \uncover<2->{

    \begin{tikzpicture}[z=.5cm,>=latex,scale=.75,font=\scriptsize]
      \drawcube[draw]
      \begin{scope}[xshift=-3cm,x=3cm]
        \begin{scope}
          \drawcube[draw,densely dotted]
        \end{scope}
      \end{scope}
      \begin{scope}[xshift=1cm,x=3cm]
        \begin{scope}
          \drawcube[draw,densely dotted]
        \end{scope}
      \end{scope}

      % Draw arrows
      \node[anchor=center] at (-1.5,.5,.5) {$-\infty=\SE_\leftarrow$};
      \node[anchor=center] at (2.5,.5,.5) {$+\infty=\SE_\rightarrow$};
      % \draw[->] (.5,.5,.5) -- (1.5,.5,.5);
      %\draw[->] (.5,.5,.5) -- (.5,1.5,.5);
      \draw[->] (.5,.5,.5) -- (.5,.5,.1);

    \end{tikzpicture}
}
        \column{.4\textwidth}

        \uncover<3->{
    \begin{tikzpicture}[z=.5cm,>=latex,scale=.75,font=\scriptsize]
      \drawcube[draw]
      \begin{scope}[xshift=-3cm,x=3cm]
        \begin{scope}[yshift=-1cm]
          \drawcube[draw,densely dotted]
        \end{scope}
        \begin{scope}
          \drawcube[draw,densely dotted]
        \end{scope}
        \begin{scope}[yshift=1cm]
          \drawcube[draw,densely dotted]
        \end{scope}
      \end{scope}
      \begin{scope}[yshift=-1cm]
        \drawcube[draw,densely dotted]
      \end{scope}
      \begin{scope}[yshift=1cm]
        \drawcube[draw,densely dotted]
      \end{scope}
      \begin{scope}[xshift=1cm,x=3cm]
        \begin{scope}[yshift=-1cm]
          \drawcube[draw,densely dotted]
        \end{scope}
        \begin{scope}
          \drawcube[draw,densely dotted]
        \end{scope}
        \begin{scope}[yshift=1cm]
          \drawcube[draw,densely dotted]
        \end{scope}
      \end{scope}

      % Draw arrows
      \foreach \y in {-.5,.5,1.5} {
          \node[anchor=center] at (-1.5,\y,.5) {$-\infty=\SE_\leftarrow$};
          \node[anchor=center] at (2.5,\y,.5) {$+\infty=\SE_\rightarrow$};
      }
      % \draw[->] (.5,.5,.5) -- (1.5,.5,.5);
      \draw[->] (.5,.5,.5) -- (.5,1.5,.5);
      \draw[->] (.5,.5,.5) -- (.5,.5,.1);

    \end{tikzpicture}

}
      \end{columns}

    \end{center}

    \item<4-> Self-energies have ``large'' imaginary components smearing the DOS for states
    coupled to the leads. The imaginary part ($\eta$) can thus often be neglected in the
    device region\footnote<4->{Not for bound states.}.

    \end{itemize}

  \end{block}

\end{frame}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "talk"
%%% End: