\section{Wavefunction optimization}
\label{sec:optimization}

Optimizing wavefunction is critical in all kinds of real-space QMC calculations
because it significantly improves both the accuracy and efficiency of computation.
However, it is very difficult to directly adopt deterministic minimization approaches because of the stochastic nature of evaluating quantities with MC.
Thanks to the algorithmic breakthrough during the first decade of this century and the tremendous computer power available,
it is now feasible to optimize tens of thousands of parameters in a wavefunction for a solid or molecule.
QMCPACK has multiple optimizers implemented based on the state-of-the-art linear method.
We are continually improving our optimizers for robustness and friendliness and are trying to provide a single solution.
Because of the large variation of wavefunction types carrying distinct characteristics, using several optimizers might be needed in some cases.
We strongly suggested reading recommendations from the experts who maintain these optimizers.

A typical optimization block looks like the following. It starts with method=``linear" and contains three blocks of parameters.
\begin{lstlisting}[style=QMCPXML]
 <loop max="10">
  <qmc method="linear" move="pbyp" gpu="yes">
    <!-- Specify the VMC options -->
    <parameter name="walkers">              256 </parameter>
    <parameter name="samples">          2867200 </parameter>
    <parameter name="stepsbetweensamples">    1 </parameter>
    <parameter name="substeps">               5 </parameter>
    <parameter name="warmupSteps">            5 </parameter>
    <parameter name="blocks">                70 </parameter>
    <parameter name="timestep">             1.0 </parameter>
    <parameter name="usedrift">              no </parameter>
    <estimator name="LocalEnergy" hdf5="no"/>
    ...
    <!-- Specify the correlated sampling options and define the cost function -->
    <parameter name="minwalkers">            0.3 </parameter>
         <cost name="energy">               0.95 </cost>
         <cost name="unreweightedvariance"> 0.00 </cost>
         <cost name="reweightedvariance">   0.05 </cost>
    ...
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">    OneShiftOnly </parameter>
    ...
  </qmc>
 </loop>
\end{lstlisting}
\begin{itemize}
\item Loop is helpful to repeatedly execute identical optimization blocks.
\item The first part is highly identical to a regular VMC block.
\item The second part is to specify the correlated sampling options and define the cost function.
\item The last part is used to specify the options of different optimizers, which can be very distinct from one to another.
\end{itemize}

\subsection{VMC run for the optimization}
The VMC calculation for the wavefunction optimization has a strict requirement
that \ixml{samples} or \ixml{samplesperthread} must be specified because of the optimizer needs for the stored \ixml{samples}.
The input parameters of this part are identical to the VMC method.

Recommendations:
\begin{itemize}
\item Run the inclusive VMC calculation correctly and efficiently because this takes a significant amount of time during optimization.
For example, make sure the derived \ixml{steps} per block is 1 and use larger \ixml{substeps} to control the correlation between \ixml{samples}.
\item A reasonable starting wavefunction is necessary. A lot of optimization fails because of a bad wavefunction starting point.
The sign of a bad initial wavefunction includes but is not limited to a very long equilibration time, low acceptance ratio, and huge variance.
The first thing to do after a failed optimization is to check the information provided by the VMC calculation via \texttt{*.scalar.dat files}.
\end{itemize}

\subsection{Correlated sampling and cost function}
After generating the samples with VMC, the derivatives of the wavefunction with respect to the parameters are computed for proposing a new set of parameters by optimizers.
And later, a correlated sampling calculation is performed to quickly evaluate values of the cost function on the old set of parameters and the new set for further decisions.
The input parameters are listed in the following table.
\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{linear} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   &   \texttt{nonlocalpp} &  text     & yes, no & no  & include non-local PP energy in the cost function\\
%   &   \texttt{GEVMethod} &  text     & mixed, H2 & mixed  & methods of generalized eigenvalue problem\\
%   &   \texttt{beta} &  real     & any value & 0.0  & a parameter for GEVMethod\\
%   &   \texttt{use\_nonlocalpp\_deriv} &  text     & yes, no & no  & include the derivatives of non-local PP\\
   &   \texttt{minwalkers} &  real     & 0--1   & 0.3 & Lower bound of the effective weight\\
   &   \texttt{maxWeight} &  real     & $>1$   & 1e6 & Maximum weight allowed in reweighting\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information:
\begin{itemize}
\item \ixml{maxWeight}: The default should be good.
\item \ixml{nonlocalpp}: The \texttt{nonlocalpp} contribution to the local energy depends on the wavefunction.
When a new set of parameters is proposed, this contribution needs to be updated if the cost function consists of local energy.
Fortunately, nonlocal contribution is chosen small when making a PP for small locality error.
We can ignore its change and avoid the expensive computational cost.
An implementation issue with GPU code is that a large amount of memory is consumed with this option.
\item \ixml{minwalkers}: This is a \textit{critical} parameter. When the ratio of effective samples to actual number of samples in a reweighting step goes lower than \ixml{minwalkers},
the proposed set of parameters is invalid. % The last set of acceptable parameters is kept.
\end{itemize}

The cost function consists of three components: energy, unreweighted variance, and reweighted variance.
\begin{lstlisting}[style=QMCPXML]
     <cost name="energy">                   0.95 </cost>
     <cost name="unreweightedvariance">     0.00 </cost>
     <cost name="reweightedvariance">       0.05 </cost>
\end{lstlisting}

\subsection{Optimizers}
QMCPACK implements a number of different optimizers each with different priorities for accuracy, convergence, memory usage, and stability.
The optimizers can be switched among  ``OneShiftOnly'' (default), ``adaptive,'' ``descent,'' ``hybrid,'' and ``quartic'' (old) using the following line in the optimization block:
\begin{lstlisting}
<parameter name="MinMethod"> THE METHOD YOU LIKE </parameter>
\end{lstlisting}

\subsection{OneShiftOnly Optimizer}
The OneShiftOnly optimizer targets a fast optimization by moving parameters more aggressively. It works with OpenMP and GPU and can be considered for large systems.
This method relies on the effective weight of correlated sampling rather than the cost function value to justify a new set of parameters.
If the effective weight is larger than \ixml{minwalkers}, the new set is taken whether or not the cost function value decreases.
If a proposed set is rejected, the standard output prints the measured ratio of effective samples to the total number of samples
and adjustment on \ixml{minwalkers} can be made if needed.

\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{linear} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   &   \texttt{shift\_i} &  real     & $>0$ & 0.01 & Direct stabilizer added to the Hamiltonian matrix\\
   &   \texttt{shift\_s} &  real     & $>0$ & 1.00 & Initial stabilizer based on the overlap matrix\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information:
\begin{itemize}
\item \ixml{shift_i}: This is the direct term added to the diagonal of the Hamiltonian matrix.
                         It provides more stable but slower optimization with a large value.
\item \ixml{shift_s}: This is the initial value of the stabilizer based on the overlap matrix added to the Hamiltonian matrix.
                         It provides more stable but slower optimization with a large value. The used value is auto-adjusted by the optimizer.
\end{itemize}

Recommendations:
\begin{itemize}
  \item Default \ixml{shift_i}, \ixml{shift_s} should be fine.
  \item For hard cases, increasing \ixml{shift_i} (by a factor of 5 or 10) can significantly stabilize the optimization by reducing the pace towards the optimal parameter set.
  \item If the VMC energy of the last optimization iterations grows significantly, increase \ixml{minwalkers} closer to 1 and make the optimization stable.
  \item If the first iterations of optimization are rejected on a reasonable initial wavefunction,
        lower the \ixml{minwalkers} value based on the measured value printed in the standard output to accept the move.
\end{itemize}

We recommended using this optimizer in two sections with a very small \ixml{minwalkers} in the first and a large value in the second, such as the following.
In the very beginning, parameters are far away from optimal values and large changes are proposed by the optimizer.
Having a small \ixml{minwalkers} makes it much easier to accept these changes.
When the energy gradually converges, we can have a large \ixml{minwalkers} to avoid risky parameter sets.
\begin{lstlisting}[style=QMCPXML]
 <loop max="6">
  <qmc method="linear" move="pbyp" gpu="yes">
    <!-- Specify the VMC options -->
    <parameter name="walkers">                1 </parameter>
    <parameter name="samples">            10000 </parameter>
    <parameter name="stepsbetweensamples">    1 </parameter>
    <parameter name="substeps">               5 </parameter>
    <parameter name="warmupSteps">            5 </parameter>
    <parameter name="blocks">                25 </parameter>
    <parameter name="timestep">             1.0 </parameter>
    <parameter name="usedrift">              no </parameter>
    <estimator name="LocalEnergy" hdf5="no"/>
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">    OneShiftOnly </parameter>
    <parameter name="minwalkers">           1e-4 </parameter>
  </qmc>
 </loop>
 <loop max="12">
  <qmc method="linear" move="pbyp" gpu="yes">
    <!-- Specify the VMC options -->
    <parameter name="walkers">                1 </parameter>
    <parameter name="samples">            20000 </parameter>
    <parameter name="stepsbetweensamples">    1 </parameter>
    <parameter name="substeps">               5 </parameter>
    <parameter name="warmupSteps">            2 </parameter>
    <parameter name="blocks">                50 </parameter>
    <parameter name="timestep">             1.0 </parameter>
    <parameter name="usedrift">              no </parameter>
    <estimator name="LocalEnergy" hdf5="no"/>
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">    OneShiftOnly </parameter>
    <parameter name="minwalkers">            0.5 </parameter>
  </qmc>
 </loop>
\end{lstlisting}

For each optimization step, you will see\\
\begin{lstlisting}
The new set of parameters is valid. Updating the trial wave function!
\end{lstlisting}\par
or\par
\begin{lstlisting}
The new set of parameters is not valid. Revert to the old set!
\end{lstlisting}
Occasional rejection is fine. Frequent rejection indicates potential problems, and users should inspect the VMC calculation or change optimization strategy.
To track the progress of optimization, use the command \texttt{qmca -q ev *.scalar.dat} to look at the VMC energy and variance for each optimization step.

\subsection{Adaptive Optimizer}

The default setting of the adaptive optimizer is to construct the linear method Hamiltonian and overlap matrices explicitly and add different shifts to the Hamiltonian matrix
as ``stabilizers.''
The generalized eigenvalue problem is solved for each shift to obtain updates to the wavefunction parameters.
Then a correlated sampling is performed for each shift's updated wavefunction and the initial trial wavefunction
using the middle shift's updated wavefunction as the guiding function.
The cost function for these wavefunctions is compared, and the update corresponding to the best cost function is selected.
In the next iteration, the median magnitude of the stabilizers is set to the magnitude that generated the best update in the current iteration, thus adapting the magnitude of
the stabilizers automatically.

When the trial wavefunction contains more than 10,000 parameters, constructing and storing the linear method matrices could become a memory bottleneck.
To avoid explicit construction of these matrices, the adaptive optimizer implements the block linear method (BLM) approach. \cite{Zhao:2017:blocked_lm}
The BLM tries to find an approximate
solution\:  $\vec{c}_{opt}$ to the standard LM generalized eigenvalue problem by dividing the variable space into a number of blocks
and making intelligent estimates for which directions within those blocks will be most important for constructing\:   $\vec{c}_{opt}$,
which is then obtained by solving a smaller, more memory-efficient
eigenproblem in the basis of these supposedly important block-wise directions.

\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{linear} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   %&   \texttt{stepsize} &  real     & 0--1 & 0.25  & Step size for moving parameters\\
   &   \texttt{max\_relative\_change} &  real     & $>0$ & 10.0 & Allowed change in cost function\\
   &   \texttt{max\_param\_change} &  real     & $>0$ & 0.3 & Allowed change in wavefunction parameter\\
   &   \texttt{shift\_i} &  real     & $>0$ & 0.01 & Initial diagonal      stabilizer added to the Hamiltonian matrix\\
   &   \texttt{shift\_s} &  real     & $>0$ & 1.00 & Initial overlap-based stabilizer added to the Hamiltonian matrix\\
   &   \texttt{target\_shift\_i} &  real     & any & -1.0 & Diagonal stabilizer value aimed for during adaptive method (disabled if $\leq$ 0)\\
   &   \texttt{cost\_increase\_tol} &  real     & $\geq 0$ & 0.0 & Tolerance for cost function increases\\
   &   \texttt{chase\_lowest} &  text   & yes, no & yes & Chase the lowest eigenvector in iterative solver\\
   &   \texttt{chase\_closest} &  text   & yes, no & no & Chase the eigenvector closest to initial guess\\
   &   \texttt{block\_lm} &  text   & yes, no & no & Use BLM\\
   &   \texttt{nblocks} &  integer   & $>0$ &  & Number of blocks in BLM\\
   &   \texttt{nolds} &  integer   & $>0$ &  & Number of old update vectors used in BLM\\
   &   \texttt{nkept} &  integer   & $>0$ &  & Number of eigenvectors to keep per block in BLM\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information:
\begin{itemize}
  \item \ixml{shift_i}:  This is the initial coefficient used to scale the diagonal stabilizer.
                            More stable but slower optimization is expected with a large value.
                            The adaptive method will automatically adjust this value after each linear method iteration.
  \item \ixml{shift_s}:  This is the initial coefficient used to scale the overlap-based stabilizer.
                            More stable but slower optimization is expected with a large value.
                            The adaptive method will automatically adjust this value after each linear method iteration.
  \item \ixml{target_shift_i}:  If set greater than zero, the adaptive method will choose the update whose shift\_i value is closest to
                              this target value so long as the associated cost is within cost\_increase\_tol of the lowest cost.
                              Disable this behavior by setting target\_shift\_i to a negative number.
  \item \ixml{cost_increase_tol}:  Tolerance for cost function increases when selecting the best shift.
  \item \ixml{nblocks}:   This is the number of blocks used in BLM. The amount of memory required to store LM matrices decreases
                            as the number of blocks increases. But the error introduced by BLM would increase as the number of blocks increases.
  \item \ixml{nolds}:     In BLM, the interblock correlation is accounted for by including a small number of wavefunction update vectors
                            outside the block. Larger \ixml{nolds} would include more interblock correlation and more accurate results but
                            also higher memory requirements.
  \item \ixml{nkept}:     This is the number of update directions retained from each block in the BLM. If all directions are retained in each block,
                            then the BLM becomes equivalent to the standard LM.  Retaining five or fewer directions per block is often sufficient.
\end{itemize}

Recommendations:
\begin{itemize}
  \item Default \ixml{shift_i}, \ixml{shift_s} should be fine.
  \item When there are fewer than about 5,000 variables being optimized, the traditional LM is preferred because it has a lower overhead than the BLM when the number of variables is small.
  \item Initial experience with the BLM suggests that a few hundred blocks and a handful of \ixml{nolds} and \ixml{nkept}
        often provide a good balance between memory use and accuracy.  In general, using fewer blocks should be more accurate but would require more memory.
\end{itemize}

\begin{lstlisting}[style=QMCPXML]
 <loop max="15">
  <qmc method="linear" move="pbyp">
    <!-- Specify the VMC options -->
    <parameter name="walkers">                1 </parameter>
    <parameter name="samples">            20000 </parameter>
    <parameter name="stepsbetweensamples">    1 </parameter>
    <parameter name="substeps">               5 </parameter>
    <parameter name="warmupSteps">            5 </parameter>
    <parameter name="blocks">                50 </parameter>
    <parameter name="timestep">             1.0 </parameter>
    <parameter name="usedrift">              no </parameter>
    <estimator name="LocalEnergy" hdf5="no"/>
    <!-- Specify the correlated sampling options and define the cost function -->
         <cost name="energy">               1.00 </cost>
         <cost name="unreweightedvariance"> 0.00 </cost>
         <cost name="reweightedvariance">   0.00 </cost>
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">adaptive</parameter>
    <parameter name="max_relative_cost_change">10.0</parameter>
    <parameter name="shift_i"> 1.00 </parameter>
    <parameter name="shift_s"> 1.00 </parameter>
    <parameter name="max_param_change"> 0.3 </parameter>
    <parameter name="chase_lowest"> yes </parameter>
    <parameter name="chase_closest"> yes </parameter>
    <parameter name="block_lm"> no </parameter>
    <!-- Specify the BLM specific options if needed
      <parameter name="nblocks"> 100 </parameter>
      <parameter name="nolds"> 5 </parameter>
      <parameter name="nkept"> 3 </parameter>
    -->
  </qmc>
 </loop>
\end{lstlisting}
%To activate this optimizer, add ``-D BUILD\_LMYENGINE\_INTERFACE=1'' in the CMake command line.

The adaptive optimizer is also able to optimize individual excited states directly. \cite{Zhao:2016:dir_tar}
In this case, it tries to minimize the following function:
\begin{equation*}
\Omega[\Psi]=\frac{\left<\Psi|\omega-H|\Psi\right>}{\left<\Psi|{\left(\omega-H\right)}^2|\Psi\right>}\:.
\end{equation*}
The global minimum of this function corresponds to the state whose energy lies immediately above the shift parameter $\omega$ in the energy spectrum.
For example, if $\omega$ were placed in between the ground state energy and the first excited state energy and the wavefunction ansatz was capable of a good
description for the first excited state, then the wavefunction would be optimized for the first excited state.
Note that if the ansatz is not capable of a good description of the excited state in question, the optimization could converge to a different
state, as is known to occur in some circumstances for traditional ground state optimizations.
Note also that the ground state can be targeted by this method by choosing $\omega$ to be below the ground state energy, although we should stress that this
is not the same thing as a traditional ground state optimization and will in general give a slightly different wavefunction.
Excited state targeting requires two additional parameters, as shown in the following table.\\

\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{Excited state targeting} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   %&   \texttt{stepsize} &  real     & 0--1 & 0.25  & Step size for moving parameters\\
   &   \texttt{targetExcited} &  text   & yes, no      & no   & Whether to use the excited state targeting optimization\\
   &   \texttt{omega}         &  real   & real numbers & none & Energy shift used to target different excited states\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Excited state recommendations:
\begin{itemize}
  \item Because of the finite variance in any approximate wavefunction, we recommended setting $\omega=\omega_0-\sigma$, where $\omega_0$ is placed just
        below the energy of the targeted state and $\sigma^2$ is the energy variance.
  \item To obtain an unbiased excitation energy, the ground state should be optimized with the excited state variational principle as well by setting
        \ixml{omega} below the ground state energy.  Note that using the ground state variational principle for the ground state and the excited state variational
        principle for the excited state creates a bias in favor of the ground state.
\end{itemize}

\subsection{Descent Optimizer}
Gradient descent algorithms are an alternative set of optimization methods to the OneShiftOnly and adaptive optimizers based on the linear method.
These methods use only first derivatives to optimize trial wave functions and convergence can be accelerated by retaining a memory of previous derivative values.
Multiple flavors of accelerated descent methods are available. They differ in details such as the schemes for adaptive adjustment of step sizes.\cite{Otis2019}
Descent algorithms avoid the construction of matrices that occurs in the linear method and consequently can be applied to larger sets of
optimizable parameters.
Currently, descent optimization is only available for ground state calculations.
Parameters for descent are shown in the table below.\\

\begin{table}[H]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{descent} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   &   \texttt{flavor} &  text     & \begin{tabular}{@{}c@{}}RMSprop,Random \\ ADAM,AMSGrad\end{tabular} & RMSprop & Particular type of descent method\\
   &   \texttt{Ramp\_eta} &  text     & yes,no & no & Whether to gradually ramp up step sizes\\
   &   \texttt{Ramp\_num} &  integer     & $>0$ & 30 & Number of steps over which to ramp up step size\\
   &   \texttt{TJF\_2Body\_eta} &  real     & $>0$ & 0.01 & Step size for two body Jastrow parameters\\
   &   \texttt{TJF\_1Body\_eta} &  real     & $>0$ & 0.01 & Step size for one body Jastrow parameters\\
   &   \texttt{F\_eta} &  real     & $>0$ & 0.001 & Step size for number counting Jastrow F matrix parameters\\
   &   \texttt{Gauss\_eta} &  real     & $>0$ & 0.001 & Step size for number counting Jastrow gaussian basis parameters\\
   &   \texttt{CI\_eta} &  real     & $>0$ & 0.01 & Step size for CI parameters\\
   &   \texttt{Orb\_eta} &  real     & $>0$ & 0.001 & Step size for orbital parameters\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information and recommendations:
\begin{itemize}

    \item It is generally advantageous to set different step sizes for different types of parameters.
	More nonlinear parameters such as those for number counting Jastrow factors or orbitals typically require
	smaller steps sizes than those for CI coefficients or traditional Jastrow parameters. There are defaults for
	several parameter types and a default of .001 has been chosen for all other parameters.
    \item The ability to gradually ramp up step sizes to their input values is useful for avoiding spikes in the average local energy
	during early iterations of descent optimization. This initial rise in the energy occurs as a memory of past gradients is being
	built up and it may be possible for the energy to recover without ramping if there are enough iterations in the optimization.
    \item The step sizes chosen can have a substantial influence on the quality of the optimization and the final variational energy achieved.
	Larger step sizes may be helpful if there is reason to think the descent optimization is not reaching the minimum energy. There are also
	additional hyperparameters in the descent algorithms with default values.\cite{Otis2019} They seem to have limited influence
	on the effectiveness of the optimization compared to step sizes, but users can adjust them within the source code of the descent engine if they wish.
    \item The sampling effort for individual descent steps can be small compared that for linear method iterations as shown in the example input below. Something in the range of 10,000 to 30,000
	seems sufficient for molecules with tens of electrons. However, descent optimizations may require anywhere from a few hundred to a few thousand iterations.
    \item In cases where a descent optimization struggles to reach the minimum and a linear method optimization is not possible or unsatisfactory, it may be useful to try
	the hybrid optimization approach described in the next subsection.
\end{itemize}

\begin{lstlisting}[style=QMCPXML]

 <loop max="100">
    <qmc method="linear" move="pbyp" checkpoint="-1" gpu="no">

    <!-- VMC inputs -->
     <parameter name="blocks">2000</parameter>
     <parameter name="steps">1</parameter>
     <parameter name="samples">20000</parameter>
     <parameter name="warmupsteps">100</parameter>
     <parameter name="timestep">0.05</parameter>

     <parameter name="MinMethod">descent</parameter>
     <estimator name="LocalEnergy" hdf5="no"/>
     <parameter name="usebuffer">yes</parameter>

     <estimator name="LocalEnergy" hdf5="no"/>

	   <!-- Descent Inputs -->
       <parameter name="flavor">RMSprop</parameter>

       <parameter name="Ramp_eta">no</parameter>
       <parameter name="Ramp_num">30</parameter>

      <parameter name="TJF_2Body_eta">.02</parameter>
       <parameter name="TJF_1Body_eta">.02</parameter>
      <parameter name="F_eta">.001</parameter>
      <parameter name="Gauss_eta">.001</parameter>
      <parameter name="CI_eta">.1</parameter>
      <parameter name="Orb_eta">.0001</parameter>

    </qmc>
</loop>
 \end{lstlisting}

\subsection{Hybrid Optimizer}

Another optimization option is to use a hybrid combination of accelerated descent and blocked linear method.
It provides a means to retain the advantages of both individual methods while scaling to large numbers of parameters beyond the traditional 10,000 parameter limit of the linear method.\cite{Otis2019}
In a hybrid optimization, alternating sections of descent and BLM optimization are used.
Gradient descent is used to identify the previous important directions in parameter space used by the BLM, the number of which is set by the \texttt{nold} input for the BLM.
Over the course of a section of descent, vectors of parameter differences are stored and then passed to the linear method engine after the optimization changes to the BLM.
One motivation for including sections of descent is to counteract noise in linear method updates due to uncertainties in its step direction and allow for a smoother movement to the minimum.
There are two additional parameters used in the hybrid optimization and it requires a slightly different format of input to specify the constituent methods as shown below in the example.

\begin{table}[H]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{descent} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   &   \texttt{num\_updates} &  integer     & $>0$ &  & Number of steps for a method\\
   &   \texttt{Stored\_Vectors} &  integer     & $>0$ & 5 & Number of vectors to transfer to BLM\\
  \hline
\end{tabularx}
\end{center}
\end{table}

\begin{lstlisting}[style=QMCPXML]

  <loop max="203">
  <qmc method="linear" move="pbyp" checkpoint="-1" gpu="no">
   <parameter name="Minmethod"> hybrid </parameter>

   <optimizer num_updates="100">

	<parameter name="blocks">1000</parameter>
       <parameter name="steps">1</parameter>
       <parameter name="samples">20000</parameter>
       <parameter name="warmupsteps">1000</parameter>
       <parameter name="timestep">0.05</parameter>

       <estimator name="LocalEnergy" hdf5="no"/>

       <parameter name="Minmethod"> descent </parameter>
       <parameter name="Stored_Vectors">5</parameter>
       <parameter name="flavor">RMSprop</parameter>
       <parameter name="TJF_2Body_eta">.01</parameter>
       <parameter name="TJF_1Body_eta">.01</parameter>
       <parameter name="CI_eta">.1</parameter>

       <parameter name="Ramp_eta">no</parameter>
       <parameter name="Ramp_num">10</parameter>
   </optimizer>

   <optimizer num_updates="3">

       <parameter name="blocks">2000</parameter>
       <parameter name="steps">1</parameter>
       <parameter name="samples">1000000</parameter>
       <parameter name="warmupsteps">1000</parameter>
       <parameter name="timestep">0.05</parameter>

       <estimator name="LocalEnergy" hdf5="no"/>

       <parameter name="Minmethod"> adaptive </parameter>
       <parameter name="max_relative_cost_change">10.0</parameter>
       <parameter name="max_param_change">3</parameter>
       <parameter name="shift_i">0.01</parameter>
       <parameter name="shift_s">1.00</parameter>

       <parameter name="block_lm">yes</parameter>
       <parameter name="nblocks">2</parameter>
       <parameter name="nolds">5</parameter>
       <parameter name="nkept">5</parameter>

   </optimizer>
  </qmc>
</loop>
 \end{lstlisting}

Additional information and recommendations:
\begin{itemize}

    \item In the example above, the input for \texttt{loop} gives the total number of steps for the full optimization while the inputs for \texttt{num\_updates} specify
	the number of steps in the constituent methods. For this case, the optimization would begin with 100 steps of descent using the parameters in the first \texttt{optimizer} block
	and then switch to the BLM for 3 steps before switching back to descent for the final 100 iterations of the total of 203.
    \item The design of the hybrid method allows for more than two \texttt{optimizer} blocks to be used and the optimization will cycle through the individual methods. However, the effectiveness
	of this in terms of the quality of optimization results is unexplored.
    \item As the descent algorithms are currently only implemented for ground state optimizations, this hybrid combination of them with the BLM is also restricted to the ground state for now.
    \item It can be useful to follow a hybrid optimization with a section of pure descent optimization and take an average energy over the last few hundred iterations as the final variational energy.
	This approach can achieve a lower statistical uncertainty on the energy for less overall sampling effort compared to what a pure linear method optimization would require.
\end{itemize}

\subsection{Quartic Optimizer}
\textit{This is an older optimizer method retained for compatibility. We recommend starting with the newest OneShiftOnly or adaptive optimizers.}
The quartic optimizer fits a quartic polynomial to 7 values of the cost function obtained using reweighting along the chosen direction and determines the optimal move.
This optimizer is very robust but is a bit conservative when accepting new steps, especially when large parameters changes are proposed.
\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{linear} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   %&   \texttt{stepsize} &  real     & 0--1 & 0.25  & Step size for moving parameters\\
   &   \texttt{bigchange} &  real     & $>0$ & 50.0  & Largest parameter change allowed\\
   &   \texttt{alloweddifference} &  real     & $>0$ & 1e-4 & Allowed increased in energy\\
   &   \texttt{exp0} &  real     & any value & -16.0 & Initial value for stabilizer\\
   &   \texttt{stabilizerscale} &  real     & $>0$ & 2.0 & Increase in value of \texttt{exp0} between iterations\\
   &   \texttt{nstabilizers} &  integer     & $>0$ & 3 & Number of stabilizers to try\\
   &   \texttt{max\_its} &  integer   & $>0$ & 1 & Number of inner loops with same samples\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information:
\begin{itemize}
\item \ixml{exp0}. This is the initial value for stabilizer (shift to diagonal of H). The actual value of stabilizer is $10^{\textrm{exp0}}$.
\end{itemize}

Recommendations:
\begin{itemize}
  \item{For hard cases (e.g., simultaneous optimization of long MSD and 3-Body J), set \ixml{exp0}
to 0 and do a single inner iteration (max its=1) per sample of configurations.}
\end{itemize}

\begin{lstlisting}[style=QMCPXML]
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">quartic</parameter>
    <parameter name="exp0">-6</parameter>
    <parameter name="alloweddifference"> 1.0e-4 </parameter>
    <parameter name="nstabilizers"> 1 </parameter>
    <parameter name="bigchange">15.0</parameter>
\end{lstlisting}

\subsection{General recommendations}
Here are a few recommendations to make wavefunction optimization easier.
\begin{itemize}
\item All electron wavefunctions are typically more difficult to optimize than pseudopotential wavefunctions because of the importance of the wavefunction near the nucleus.
\item Two-body Jastrow contributes the largest portion of correlation energy from bare Slater determinants. Consequently, the recommended order for optimizing wavefunction components is two-body, one-body, three-body Jastrow factors and MSD coefficients.
\item For two-body spline Jastrows, always start from a reasonable one. The lack of physically motivated constraints in the functional form at large distances can cause slow convergence if starting from zero.
\item One-body spline Jastrow from old calculations can be a good starting point.
\item Three-body polynomial Jastrow can start from zero. It is beneficial to first optimize one-body and two-body Jastrow factors without adding three-body terms in the calculation and then add the three-body Jastrow and optimize all the three components together.
\end{itemize}
\subsubsection{Optimization of CI coefficients}
When storing a CI wavefunction in HDF5 format, the CI coefficients and the $\alpha$ and $\beta$ components of each CI are not in the XML input file. When optimizing the CI coefficients, they will be stored in HDF5 format.
The optimization header block will have to specify that the new CI coefficients will be saved to HDF5 format. If the tag is not added coefficients will not be saved.
\begin{lstlisting}[style=QMCPXML]
  <qmc method="linear" move="pbyp" gpu="no" hdf5="yes">
\end{lstlisting}

The rest of the optimization block remains the same.

When running the optimization, the new coefficients will be stored in a *.sXXX.opt.h5 file,  where XXX coressponds to the series number. The H5 file contains only the optimized coefficients. The corresponding *.sXXX.opt.xml  will be updated for each optimization block as follow:
\begin{lstlisting}[style=QMCPXML]
<detlist size="1487" type="DETS" nca="0" ncb="0" nea="2" neb="2" nstates="85" cutoff="1e-2" href="../LiH.orbs.h5" opt_coeffs="LiH.s001.opt.h5"/>
\end{lstlisting}

The opt\_coeffs tag will then reference where the new CI coefficients are stored.\\

When restarting the run with the new optimized coeffs, you need to specify the previous hdf5 containing the basis set, orbitals,
and MSD, as well as the new optimized coefficients. The code will read the previous data but will rewrite the coefficients that
were optimized with the values found in the *.sXXX.opt.h5 file. Be careful to keep the pair of optimized CI coefficients and
Jastrow coefficients together to avoid inconsistencies.


\chapter{Non-RR Code Modules in CPPINTS}

\section{What is Non-RR Code Module?}
\label{what_is_non_rr}

Comparing with Recurrence Relation(RR) modules, non-RR modules are referring to these
that the result integrals are directly calculated from module's input. For example,
the LHS integral for derivatives calculation is directly generated from RHS integrals,
where the RHS integrals are from previous modules.

non-RR modules are divided into two types: normal non-RR module and derivative module.
Normal non-RR module right now only contain the three body kinetic integrals, where
the LHS integrals are expressed as linear combination of derivatives of three
body overlap integrals. In this sense the three body kinetic integrals is also a
derivative module, except that normal derivatives is on basis set nuclei, but the
three body kinetic integral is on electron coordinate.

non-RR modules does not have temporary integrals. It's input directly generates the
module output. Therefore it does not has optimal path searching problem(see section
\ref{optimal_path}). However, redundant integral elimination still applies to the
non-RR modules. For the given output integral results; depending on the expression
the result RHS integrals may have redundancy\footnote{please see the section of
\ref{redundancy_rr} for more information}.

\section{Derivative Calculation}

\section{Code Generation}
\label{code_generation}


\section{The Importance of Inelastic Collisions}
The treatment of the inelastic collisions will not be as detailed as for the elastic case as we will focus on the qualitative implications of allowing inelastic annihilation and creation processes during thermalization. A full treatment of all inelastic effects is nearly impossible and even today it is not yet fully understood which processes are the most relevant ones driving the system towards the equilibrium state. I will try to highlight the most important observations and refer to the respective research papers for further details. \\
\noindent
First of all we observe that the inelastic processes will modify the collision integral on the right hand side of eqn. (\ref{eqn:transport}). Interestingly it can be shown that this modification leaves the relation for the scattering time $t$ unchanged and therefore also the evolution equation for the defining scales $\Lambda$ and $\Lambda_{\mathrm{s}}$. Implications on the condensate formation can be obtained via an extensive numerical study of solutions of the complete transport equation. The elastic contributions to the collision integral provide a source term for the condensate whereas the inelastic contributions may be interpreted as sink term. The task is to understand the balancing of the source and sink contributions which may allow a condensate to exist during most of the thermalization process.\\
\noindent
One could consider for example the effect of the strong longitudinal expansion of matter involved in relativistic heavy-ion collisions, i.\,e. the flattening of an isotropic initial distribution in the direction of the collision axis, by introducing a suitable drift term on the left hand side of the transport equation  or instabilities in the isotropy of $p_z$ and the transversal momentum $p_{\mathrm{T}}$ (cf. \cite{Blaizot2012}). Another important effect is the influence of gluon radiation allowing transport of momenta over a wide range of momenta on a very short time scale. This can be understood as a certain feature of gauge theories. For more details the interested reader is referred to publication \cite{Blaizot2016}.


% !TEX root = ParticleFilter.tex
\section{Method\label{method}}

\subsection{The Agent-Based Model: StationSim}

\textit{StationSim} is a simple agent-based model that has been designed to very loosely represent the behaviour of a crowd of people moving from an entrance on one side of a rectangular environment to an exit on the other side. This is analogous to a train arriving at a train station and passengers moving across the concourse to leave. A number of agents, $N$, which varies in the later experiments, are created when the model starts. They are able to enter the environment (leave their train) at a uniform rate through one of three entrances. They move across the `concourse' and then leave by one of the two exits. The entrances and exits have a set size, such that only a limited number of agents can pass through them in any given iteration. Once all agents have entered the environment and passed through the concourse then the simulation ends. The model environment is illustrated in Figure \ref{fig:StationSim}, with the trajectories of two interacting agents for illustration.
%The model is also outlined in full as per the ODD protocol~\citep{grimm_odd_2010} in Appendix~\ref{odd}.

\begin{figure}[ht]
\centering
\includegraphics[width=0.5\textwidth]{figures/PF_ABM}
\caption{StationSim environment with 3 entrance and 2 exit doors}.\label{fig:StationSim}
\end{figure}

The model has deliberately been designed to be extremely simple and does not attempt to match the behavioural realism offered by more developed crowd models \citep{chen_multiagentbased_2017, helbing_simulating_2000, klugl_largescale_2007, vanderwal_simulating_2017}. The reason for this simplicity is so that: (1) the model can execute relatively quickly; (2) the probabilistic elements in the model are limited (we know precisely from where probabilistic behaviour arises); (3) the model can be described fully using a relatively simple state vector, as discussed in Section~\ref{state_vector}. Importantly, the model is able to capture the emergence of \textit{crowding}. This results because each agent has a different maximum speed that they can travel at. Therefore, when a fast agent approaches a slower one, they attempt to get past by making a random binary choice to move left or right around them. Depending on the agents in the vicinity, this behaviour can start to lead to the formation of crowds. To illustrate this, Figure~\ref{fig:crowding} shows the paths of the agents (\ref{fig:crowding-trails}) and the total agent density (\ref{fig:crowding-density}) during an example simulation.   The degree and location of crowding depends on the random allocation of maximum speeds to agents and their random of direction taken to avoid slower agents; these cannot be estimated \textit{a priori}. Unlike in previous work where the models did not necessarily meet the common criteria that define agent-based models \citep[e.g.]{lloyd_exploring_2016, ward_dynamic_2016} this model respects three of the most important characteristics:

\begin{figure}
    \centering
    \begin{subfigure}[b]{0.48\textwidth}
        \centering
        \includegraphics[width=.95\linewidth]{figures/crowding-trails}
        \caption{Individual trails showing the paths taken by agents}
        \label{fig:crowding-trails}
    \end{subfigure}
    \label{fig:crowding}
    \begin{subfigure}[b]{0.48\textwidth}
        \centering
        \includegraphics[width=.95\linewidth]{figures/crowding-density}
        \caption{The total crowd density over the simulation run.}
        \label{fig:crowding-density}
    \end{subfigure}
    \caption{An example of crowding in the StationSim model}
    \label{fig:crowding}
\end{figure}

\begin{itemize}
	\item individual heterogeneity -- agents have different maximum travel speeds;
	\item agent interactions -- agents are not allowed to occupy the same space and try to move around slower agents who are blocking their path;
	\item emergence -- crowding is an emergent property of the system that arises as a result of the choice of exit that each agent is heading to and their maximum speed.
\end{itemize}

The model code is relatively short and easy to understand. It is written in Python, and is available in its entirety at in the project repository \citep{stationsimgit}.
% \footnote{\url{XXXX Link to repo}}.

\subsection{Data Assimilation - Introduction and Definitions}

DA methods are built on the following assumptions:

\begin{enumerate}
	\item Although they have low uncertainty, observational data are often spatio-temporally sparse. Therefore there are typically insufficient amounts of data to to describe the system in sufficient detail and a data-driven approach would not work.
	\item Models are not sparse; they can represent the target system in great detail and hence fill in the spatio-temporal gaps in observational data by propagating data from observed to unobserved areas \citep{carrassi_data_2018}. For example, some parts of a building might be more heavily observed than others, so a model that assimilated data from the observed areas might be able to estimate the state of the unobserved areas. However, if the underlying systems are complex, a model will rapidly diverge from the real system in the absence of \textit{up to date} data \citep{ward_dynamic_2016}.
	\item The combination a model and up-to-date observational data allow ``all the available information'' to be used to determine the state of the system as accurately as possible \citep{talagrand_use_1991}.
\end{enumerate}

DA algorithms work by running a model forward in time up to the point that some new observational data become available. This is typically called the \textit{predict} step. At this point, the algorithm has an estimate of the current system state and its uncertainty (the prior). The next step, \textit{update},  involves using the new observations, and their uncertainties, to update the current state estimate to create a posterior estimate of the state. As the posterior has combined the best guess of the state from the model \textit{and} the best guess of the state from the observations, it should be a closer estimate of the true system state than that which could be estimated from the observations or the model in isolation.

\subsection{The Particle Filter\label{particle_filter}}

There are many different ways to perform data assimilation, as discussed in Section \ref{da_pf}. Here, a potentially appropriate solution to the data assimilation problem for agent-based models is the particle filter -- also known as a Bayesian bootstrap filter or a sequential Monte Carlo method -- which represents the posterior state using a collection of model samples, called particles \citep{gordon_novel_1993,carpenter_improved_1999,wang_data_2015, carrassi_data_2018}. Figure~\ref{fig:PF_flowchart} illustrates the process of running a particle filter. Note that the `pseudo-truth model' is a single instance of the agent-based model that is used as a proxy for the real system as per the identical twin experimental framework that we have adopted.

The data assimilation `window' determines how often new observations are assimilated into the particle filter. The size of the window is an important factor -- larger windows result in the particles deviating further from the real system state -- but here we fix the window at 100 iterations. The simulation terminates when all agents have left the system.

\begin{figure}[ht]
\centering
\includegraphics[width=0.8\textwidth]{figures/PF_flowchart2}
\caption{Flowchart of data assimilation process using a particle filter.\label{fig:PF_flowchart}}
\end{figure}

\subsubsection{The State Vector and Transition Function\label{state_vector}}

Here, the \textit{state vector}, at a time $t$, contains all the information that a transition function needs to iterate the model forward by one step, including all of the agent ($i = \{ 0, 1, \dots, N \} $) parameters ($\overrightarrow{p_i}$) and variables ($\overrightarrow{v_i}$) as well as global model parameters $\overrightarrow{P}$:
\begin{equation}
  S_t  = \left[ \begin{array}{cccccccc}
\overrightarrow{p_0} & \overrightarrow{v_0} & \overrightarrow{p_1} &  \overrightarrow{v_1} &  \dots &  \overrightarrow{p_N} &  \overrightarrow{v_N} & \overrightarrow{P}
\end{array} \right]
\end{equation}

A similar structure, the \textit{observation vector}, contains all of the observations made from the `real world' (in this case the pseudo-truth model) at a time $t$. Here, the particle filter is only used to estimate the state of the models variables ($\overrightarrow{v_i}$), not any of the parameters ($\overrightarrow{p_i}$ and $\overrightarrow{P}$) (although it is worth noting that parameter estimation is technically feasible and will be experimented with in future work). Also, the current speed of an agent can be calculated from its current location and the locations of the agents surrounding it, so in effect the observation vector only needs to include the positions of the agents with the addition of some Gaussian noise, $\epsilon$:
\begin{equation}
  O_t  = \left[ \begin{array}{ccccccc}
x_0 & y_0 & x_1 & y_1 & \dots & x_n & y_n
\end{array} \right]
\end{equation}

 Therefore in the experiments conducted here, all model parameters are fixed. Hence a further vector is required to map the observations to the state vector that the particles can actually manipulate. We define the partial state vector $S'$ to match the shape of $O$, i.e.:
\begin{equation}
  S'_t  = \left[ \begin{array}{ccccccc}
x_0 & y_0 & x_1 & y_1 & \dots & x_n & y_n
\end{array} \right]
\end{equation}

This has the effect of `pairing' agents in the particles to those in the pseudo-truth data, in a similar approach to that taken by \citep{wang_data_2015}. It is worth noting that, because the particle filter will not be tasked with parameter estimation, then the data assimilation is somewhat simpler than it would be in a real application. For example, one of the agent parameters is used to store the location of the exit out of the environment that the agent is moving towards. As this parameter is set \textit{a priori} for each agent, then the particle filter does not need to estimate where the agents are ultimately going, only where they currently are.

\subsubsection{Observations from the pseudo-truth data}

In a real application, the particle filter would be assimilating data in real time from sensors of the real world. This is not the case here so instead, we take `observations' from the pseudo-truth data (which are, as it happens, generated by the StationSim model). Each particle evolves in time according to the StationSim dynamics and receives new observations at regular intervals. Measurement error (i.e. noise, $\epsilon$) is added to the observations (in the real world, sensors observations will be noisy). Here, observations take the form of the $(x,y)$ locations of all agents. This is analogous to tracking all individuals in a crowd and providing snapshots of their locations at discrete points in time to the particle filter. This `synthetic observation' is probably more detailed than reality, so future work will vary the amount of detail provided to the algorithm. It will firstly reduce the number of agents who are observed (e.g. tracking only \textit{some} people) and then provide only aggregate population counts (which is analogous to using a camera or other sensor to count the number of people at a certain point).

\subsubsection{Particle Weights}

Each particle in the particle filter has a weight associated with it that quantifies how similar a particle is to an observation. The weights are calculated at the end of each data assimilation window (i.e. when observations become available). At the start of the following window the particles then evolve independently from each other \citep{fearnhead_particle_2018}.  The weights are, in effect, the average distance between agents in that particle and the corresponding agents in StationSim (recall that there is a one-to-one mapping between agents in the particles and agents in the truth model). Formally, let $x^n(i,t)$ be the location of the $i$-th agent at time $t$ in the $n$-th particle for $n \in \{1,\dots,N\}$ and let $x(i,t)$ be the location of the $i$-th agent in StationSim for $i \in \{1,\dots,I\}$. The error of the $n$-th particle $\epsilon^n(t)$ at time $t$ is then given by
\begin{equation}
\epsilon^n(t) = \frac{1}{I} \sum_{i=1}^I |x(i,t) - x^n(i,t)|,
\end{equation}
and the particle filter error $\nu(t)$ at time $t$ is given by
\begin{equation}
\label{eqn:particle_error}
\begin{split}
\nu(t) =& \frac{1}{N} \sum_{n=1}^{N} \epsilon^n(t), \\
=& \frac{1}{NI} \sum_{n=1}^{N}\sum_{i=1}^I |x(i,t) - x^n(i,t)|.
\end{split}
\end{equation}

It is worth noting that, because the agent locations are the only data stored in the partial state vector, the particle error is equivalent to the Euclidean distance ($l_2$-norm) between the particle partial state vector $S'_{t}$ and the observation vector $O_t$,
\begin{equation}
 \nu(t) = || S'_{t} - O_t  ||_2.
\end{equation}

Particles with relatively large error are likely to be removed during the sampling procedure (discussed in the following section), whereas those with low error are likely to be duplicated. In addition for their use in resampling, the population of particle weights can be used to gain insight into how well the particle filter is able to represent the `true' system state overall.

\subsubsection{Sampling Procedure\label{particle_sampling}}

Here, a bootstrap filter is implemented which uses systematic resampling \citep{doucet_introduction_2001, douc_comparison_2005, wang_data_2015, long_spatial_2017, carrassi_data_2018}. This begins by taking a random sample $U$ from the uniform distribution on the interval $[0,1/N]$ and then selecting $N$ points $U^i$ for $i \in \{1,\dots,N\}$ on the interval $[0,1]$ such that
\begin{equation}
U^i = (i-1)/N + U.
\end{equation}
Let the particles currently have locations $x_i$. Using the inversion method, we calculate the cumulative sum of the normalised particle weights $w_i$ and define the inverse function $D$ of this cumulative sum, that is:
\begin{equation}
D(u) = i \text{ for } u \in \left(\sum_{j=1}^{i-1}w_j,\sum_{j=1}^{i}w_j\right].
\end{equation}
Finally, the new locations of the resampled particles are given by $x_{D(U^i)}$.

As discussed in Section~\ref{background}, a well-studied issue that particle filters face is that of particle deprivation \citep{snyder_obstacles_2008}, which refers to the problem of particles converging to a single point such that all particles, but one, vanish \citep{kong_sequential_1994}.
% This vastly reduces the size of the state space covered by the population of particles and will make it difficult or impossible to find particles with low error in later windows. Common approaches to resolve the deprivation problem include increasing the number of particles or trying to reduce the dimensionality of the state space. Bespoke approaches also exist; for example \citep{wang_data_2015} develop a technique called `component set resampling' that samples \textit{parts} of particles (i.e. those parts that are working well) rather than whole particles in their entirety.
Here, the problem is addressed in two ways. Firstly by simply using large numbers of particles relative to the size of the state space and, secondly, by diversifying the particles \citep{vadakkepat_improved_2006} -- also known as roughening, jittering, and diffusing \citep{li_fight_2014, shephard_learning_2009, pantrigo_combining_2005}. In each iteration of the model we add Gaussian white noise to the particles' state vector to increase their variance, which increases particle diversity. This encourages a greater variety of particles to be resampled and therefore makes the algorithm more likely to represent the state of the underlying model. This method is a special case of the resample-move method presented in \citep{gilks_following_2001}. The amount of noise to add is a key hyper-parameter -- too little and it has no effect, too much and the state of the particles moves far away from the true state -- as discussed in the following section.


\documentclass[11pt]{article}
%\documentclass{article}
\usepackage[margin=1in]{geometry}
\usepackage[T1]{fontenc}
%\usepackage{ebgaramond}
%\usepackage[bitstream-charter]{mathdesign}
\usepackage[british]{babel}
\usepackage{libertine}
\usepackage{amsmath}
\usepackage{microtype}

\title{Optimal Lobotomy}
\author{Clement Lee '17\\\emph{advised by Jianxiong Xiao}}
\date{}
\begin{document}
%Title. Include your name, class, project title, and advisor.
\maketitle

%Motivation and Goal. Give a high-level introduction to your topic area and state specifically what problem you will be addressing.  You should clearly state the goal of your project with a sentence beginning “The goal of my project is …”. Explain why that goal is important and/or interesting, perhaps with a description of applications or people enabled by achieving it.
\section{\textbf Motivation and Goal}
The recent and astronomical rise of deep learning algorithms applied to a variety of datasets has been built on the foundation of significant boosts to computational power, especially with the advent of GPGPU techniques.
These developments have proceeded at a much faster rate than the speed of hardware-based technological advancements, and as such algorithms to efficiently utilize existing hardware are critically important.
The goal of this project is thus to produce an algorithm and implementation which will aid in the automatic parallelization of deep neural networks across multiple GPUs.
Our proposed algorithm would potentially greatly speed up any deep learning problem of contemporary nontrivial size, as it intends to automatically solve the many difficulties of properly creating parallelism within models with high degress of connectivity and interdependence.
This would also be beneficial for deep learning problems where the learning parameter space and architectures are not studied extensively, as it could be applied as a general self-learning mechanism.

%Problem Background and Related Work. Place your project in context of prior work.  Give some background of what has been done before to achieve your stated goal. Include citations to closely related academic papers and/or list commercial products targeted at the same goal.  Finish this section with a brief explanation of the problem unsolved by previous work that will be addressed by your project.
\section{Problem Background and Related Work}
Of primary importance to this project is the paper \emph{Optimal Brain Damage} \cite{lecun1989optimal} which describes an algorithm to reduce the size of a feedforward neural network by removing elements which contribute little to its classification ability.
It defines the concept of \emph{saliency} which is the effective importance of a weight in a network.
This should generally allow an algorithm to reduce the dimensionality of the network with minimal impact on its performance, which is a cornerstone of our algorithm.

An extensive amount of work exists in the literature regarding distributed deep learning, as it is generally a problem that resists parallelization.
Especially in increasingly distributed systems, attaining sublinear results in scaling is itself an extremely difficult problem, which is described in more detail in \emph{Large Scale Distributed Deep Networks} \cite{dean2012large}.
This gives a description of the overarching challenges of parallelization, and creates a framework for parallel learning.

Multiple forms of parallelism exist, as are described in \emph{On Model Parallelization and Scheduling Strategies for Distributed Machine Learning} \cite{lee2014model}.
This paper lays out the differences between model and data parallelism, and discusses the difficulties of applying both in varying degrees to deep learning.
It develops a language for efficient partitioning of training and updating parameters, particularly regarding scheduling of gradient calculations across a distributed network of participating servers.
In general, we aim to build on its usage of \emph{model parallelism}: that is, the splitting of model parameters across multiple computing devices.

The benefits of parallelism are further put into practice in \emph{Reducing the Training Time of Neural Networks by Partitioning} \cite{miranda2015reducing}.
This constructs a set of pretraining models which provide a natural partitioning of the parameters when they are later merged.
This allows for better usage of GPUs especially when they are not utilized to their full capacity.
A partitioning as utilized by this text is important, though it focuses on a slightly different formulation than what we intend to perform.
At the same time, much of the methodology is broadly applicable to our problem space.

A key limitation is that most current work focuses on the training phase, and is reliant on careful parameter tuning on the part of the researchers.
We aim to overcome the limitations of significant human involvement in training of deep learning, and utilize computer-based optimization procedures to determine architectural decisions of the network.
We also want to work on networks that are already pretrained and developed, and focus on efficient partitionings that allow for model parallelism with minimal loss in classification performance.

%Approach. Provide a concise description of the key idea underlying your approach to achieving the stated goal.  Provide an argument of why your approach is a good idea – i.e., why it can achieve the stated goal where others have not.
\section{Approach}
To begin, we will be taking the previously defined concept of saliency and extending it to remove significant numbers of weights from a pre-existing and trained network.
We want to gradually remove weights (or clusters of weights) with low saliency while repeatedly retraining the network.
We expect that the performance of the network will decrease with fewer weights, so an acceptable error range will be established.
Furthermore, we want to develop a metric for connectivity that we aim to generally minimize, in order to get a better partitioning of the network.
At the same time, it is not the intention of this project to produce a complete partitioning of the network into two separate networks, as the result of this is not one network but rather a boosted set of networks.

Instead, our approach is to maintain some level of communication between the nodes, and to use this interconnectivity to establish a self-optimizing network that will spread across GPUs.
As we are performing our experiments on GPUs all located within a single machine, moving data between nodes is a considerable cost but not largely unsurmountable unlike in distributed systems.
Very few papers have been published on self-optimizing runtimes of networks, as it has largely been the realm of deep learning human experts.
We believe that our proposed work represents a novel direction in the literature, and it remains a topic which deserves more investigation than it has received in the past.

%Plan. Describe the steps you plan to take and/or the issues you plan to address during the execution of your project.  What data sets will you have to acquire?  What algorithms will you have to develop?  What theorems will you have to prove?  etc.  For the steps that are non-trivial, provide a brief description of the issue, options, and planned approach.    Please indicate any particularly risky aspects of the project and discuss contingencies in case they do not go as planned.
\section{Plan}
This project is divisible into smaller relatively-contained modules which will be developed in sequence.
Firstly, we want to test the performance of the Optimal Brain Damage algorithm on the scale of modern day networks.
This concerns the application of the algorithm under different conditions than it was originally developed and tested; it is likely that modifications will have to be made to ensure the feasibility of the algorithm.

Once we have a good understanding of the basic algorithm, we will move on to developing the automatic partitioning.
This will depend on a subalgorithm to choose which weights which generally are optimal to remove when considering a splitting of the model.
The tradeoff between simply picking the least salient features and the least connected features to remove will be considered in depth, and we aim to use this in order to produce the first iteration of the algorithm.

If this is accomplished, we would also like to explore the possibility of constructing the network rather than simply trimming a preconstructed one.
The first step to accomplishing this is to experiment with an algorithm that also allows gradually adding weights in tandem with the partitioning.
This would allow us to achieve a more aggressive trimming if inter-partition connections could be established on the fly as necessary.
This could later progress towards a network that starts out in a minimal state and gradually develops its own optimal architecture to spread over the GPUs available.

As we are aiming more towards an algorithmic problem, we aim to test on a dataset that is more commonly used.
In the forefront here is MNIST, a large publicly available dataset of handwritten digits that represents a common benchmark for image classification.
We plan to take a relatively standard ConvNet architecture, and apply our algorithm to perform modifications automatically.
At the same time, MNIST is comparatively an extremely small dataset with very low resolution, so it may be necessary to perform experiments on larger networks to achieve better parallelization results.
We intend to do development on MNIST due to physical constraints, but we would like to explore larger datasets, time permitting.

\subsection{Important Milestones}
In general, we aim to fulfill the rough schedule outlined below.
The dates are approximate and are chosen according to either formal deadlines or seminar meeting times.
This also represents a relatively ambitious timeline; ample time is left towards the latter half of the schedule to allow for corrections.
\begin{description}
  \item[1 Mar] Complete implementation of Optimal Brain Damage algorithm
  \item[8 Mar] Investigate feasibility of removing large sets of weights and test out weight-picking heuristic
  \item[15 Mar] Figure out MNIST dataset and current network architectures for handling it
  \item[22 Mar] Checkpoint -- have a working prototype of the algorithm and be apply it to the MNIST dataset
  \item[29 Mar] Test on larger datasets and structures, such as ImageNet
  \item[15 Apr] Investigate feasibility of constructively rather than destructively generating partitions
  \item[29 Apr] Finish final report and polish remaining code
\end{description}

%Evaluation. Describe the methodology you plan to use to evaluate how well your project has achieved the stated goal.  Be specific.  What data will you use?  What test will you run?  What quantitative metric will you use to measure success?  etc.  This is a very important and often over-looked aspect of a project plan – please think about it before finalizing your project selection.
\section{Evaluation}
The benchmarks on which this algorithm will be evaluated are more generally well-established.
The classification performance of modern techniques on the MNIST dataset are documented in depth, so we will aim to remain in the same general accuracy as the networks we start with.
The actual speed of running the dataset is also easily testable, so as a key component of the algorithm is creating parallelism, we will want to run it against the initial network, and see the kinds of gains we can get from varying numbers of GPUs.
As noted prior, sublinear increases in performance are significant, and we aim to approach linear scaling by better parallelization.
This step will be critical as the development of our algorithm will necessarily be informed and guided by the benchmarks and performance data that we collect.

\bibliographystyle{acm}
\bibliography{ref}
\end{document}


\section{Introduction}\label{sec:intro}

The PSBLAS-EXT  library contains a set of extensions to the base
library. The extensions provide additional storage formats beyond the
 ones already contained in the base library, as well as interfaces to
 two external libraries:
\begin{description}
\item[SPGPU] \url{https://code.google.com/p/spgpu/}, for computations on
  NVIDIA GPUs;
\item[LIBRSB] \url{http://sourceforge.net/projects/librsb/}, for
  computations on multicore parallel machines.
\end{description}
The infrastructure laid out in the base library to allow for these
extensions is detailed in the references~\cite{CaFiRo:2014,Sparse03};
the GPU-specific data formats are described in~\cite{OurTechRep}.

\subsection{Application structure}
\label{sec:appstruct}
A sample application using the PSBLAS extensions will contain the
following steps:
\begin{itemize}
\item \verb|USE| the appropriat modules (\verb|psb_ext_mod|,
  \verb|psb_gpu_mod|);
\item Declare a \emph{mold} variable of the necessary type
  (e.g. \verb|psb_d_ell_sparse_mat|, \verb|psb_d_hlg_sparse_mat|,
  \verb|psb_d_vect_gpu|);
\item Pass the mold variable to the base library interface where
  needed to ensure the appropriate dynamic type.
\end{itemize}
Suppose you want to use the GPU-enabled ELLPACK data structure; you
would use a piece of code like this (and don't forget, you need
GPU-side vectors along with the matrices):
\begin{minted}[breaklines=true,bgcolor=bg,fontsize=\small]{fortran}
program my_gpu_test
  use psb_base_mod
  use psb_util_mod
  use psb_ext_mod
  use psb_gpu_mod
  type(psb_dspmat_type) :: a, agpu
  type(psb_d_vect_type) :: x, xg, bg

  real(psb_dpk_), allocatable :: xtmp(:)
  type(psb_d_vect_gpu)       :: vmold
  type(psb_d_elg_sparse_mat) :: aelg
  type(psb_ctxt_type) :: ctxt
  integer             :: iam, np

  call psb_init(ctxt)
  call psb_info(ctxt,iam,np)
  call psb_gpu_init(ctxt, iam)

  ! My own home-grown matrix generator
  call gen_matrix(ctxt,idim,desc_a,a,x,info)
  if (info /= 0) goto 9999

  call a%cscnv(agpu,info,mold=aelg)
  if (info /= 0) goto 9999
  xtmp = x%get_vect()
  call xg%bld(xtmp,mold=vmold)
  call bg%bld(size(xtmp),mold=vmold)

  ! Do sparse MV
  call psb_spmm(done,agpu,xg,dzero,bg,desc_a,info)

9999 cotinue
  if (info == 0) then
     write(*,*) '42'
  else
     write(*,*) 'Something went wrong ',info
  end if

  call psb_gpu_exit()
  call psb_exit(ctxt)
  stop
end program my_gpu_test
\end{minted}
A full example of this strategy can be seen in the
\texttt{test/ext/kernel} and \texttt{test/\-gpu/\-kernel} subdirectories,
where we provide  sample programs
to test the speed of the sparse matrix-vector product with the various
data structures included in the library.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "userguide"
%%% End: