\section{Utility Arrows}\label{utilfns}
Following are definitions of some utility Arrows used in this paper that have been left out for brevity.
We start with the |second| combinator from \citet{HughesArrows}, which is a mirrored version of |first|, which is for example used in the definition of |***|:
\begin{code}
second :: Arrow arr => arr a b -> arr (c, a) (c, b)
second f = arr swap >>> first f >>> arr swap
	where swap (x, y) = (y, x)
\end{code}

Next, we give the definition of |evalN| which also helps us to define |map|, and |zipWith| on Arrows.
The |evalN| combinator in Fig.~\ref{fig:evalN} converts a list of Arrows |[arr a b]| into an Arrow |arr [a] [b]|.

\begin{figure}[h]
\begin{code}
evalN :: (ArrowChoice arr) => [arr a b] -> arr [a] [b]
evalN (f:fs) = arr listcase >>>
         arr (const []) ||| (f *** evalN fs >>> arr (uncurry (:)))
         where listcase []     = Left ()
               listcase (x:xs) = Right (x,xs)
evalN [] = arr (const [])
\end{code}
\caption{The definition of |evalN|}
\label{fig:evalN}
\end{figure}

The |mapArr| combinator (Fig.~\ref{fig:mapArr}) lifts any Arrow |arr a
b| to an Arrow |arr [a] [b]|. The original inspiration was from \citet{Hughes2005},
but the definition as then unified with |evalN|.

\begin{figure}[h]
\begin{code}
mapArr :: ArrowChoice arr => arr a b -> arr [a] [b]
mapArr = evalN . repeat
\end{code}
\caption{The definition of |map| over Arrows.}
\label{fig:mapArr}
\end{figure}

Finally, with the help of |mapArr| (Fig.~\ref{fig:mapArr}), we can define |zipWithArr|  (Fig.~\ref{fig:zipWithArr}) that lifts any Arrow |arr (a, b) c| to an Arrow |arr ([a], [b]) [c]|.
\begin{figure}[h]
\begin{code}
zipWithArr :: ArrowChoice arr => arr (a, b) c -> arr ([a], [b]) [c]
zipWithArr f = (arr (\(as, bs) -> zipWith (,) as bs)) >>> mapArr f
\end{code}
\caption{|zipWith| over Arrows.} %$
\label{fig:zipWithArr}
\end{figure}
 %$ %% formatting

These combinators make use of the |ArrowChoice| type class which provides the |pipepipepipe| combinator. It takes two Arrows |arr a c| and |arr b c| and combines them into a new Arrow |arr (Either a b) c| which pipes all |Left a|'s to the first Arrow and all |Right b|'s to the second Arrow:
% \begin{figure}[htb]
\begin{code}
(pipepipepipe) :: ArrowChoice arr a c -> arr b c -> arr (Either a b) c
\end{code}

\section{Profunctor Arrows}
\label{app:profunctorArrows}

In Fig.~\ref{fig:profunctorArrow} we show how specific Profunctors can be turned into Arrows. This works because Arrows are strong Monads in the bicategory |Prof| of Profunctors as shown by \citet{Asada:2010:ASM:1863597.1863607}. In Standard GHC |(>>>)| has the type |(>>>) :: Category cat => cat a b -> cat b c -> cat a c| and is therefore not part of the |Arrow| type class like presented in this paper.\footnote{For additional information on the type classes used, see: \url{https://hackage.haskell.org/package/profunctors-5.2.1/docs/Data-Profunctor.html} and \url{https://hackage.haskell.org/package/base-4.9.1.0/docs/Control-Category.html}.}

\begin{figure}[h]
\begin{code}
instance (Category p, Strong p) => Arrow p where
  arr f = dimap id f id
  first = first'

instance (Category p, Strong p, Costrong p) => ArrowLoop p where
  loop = loop'

instance (Category p, Strong p, Choice p) => ArrowChoice p where
  left = left'
\end{code}
\caption{Profunctors as Arrows.}
\label{fig:profunctorArrow}
\end{figure}

\section{Additional function definitions}
\label{app:omitted}
We have omitted some function definitions in the main text for
brevity, and redeem this here.
%
%
%
We begin with warping Eden's build-in |RemoteData| to |Future| in
Figure~\ref{fig:RDFuture}

\begin{figure}[h]
\begin{code}
data RemoteData a = RD { rd :: RD a }

put' :: (Arrow arr) => arr a (BasicFuture a)
put' = arr BF

get' :: (Arrow arr) => arr (BasicFuture a) a
get' = arr (\(~(BF a)) -> a)

instance NFData (RemoteData a) where
    rnf = rnf . rd
instance Trans (RemoteData a)

instance (Trans a) => Future RemoteData a Conf where
    put _ = put'
    get _ = get'

instance (Trans a) => Future RemoteData a () where
    put _ = put'
    get _ = get'
\end{code}
\caption{|RD|-based |RemoteData| version of |Future| for the Eden backend.}
\label{fig:RDFuture}
\end{figure}

Next, we have the definition of |BasicFuture| in Fig.~\ref{fig:BasicFuture} and the corresponding |Future| instances.

\begin{figure}[h]
\begin{code}
data BasicFuture a = BF a

put' :: (Arrow arr) => arr a (BasicFuture a)
put' = arr BF

get' :: (Arrow arr) => arr (BasicFuture a) a
get' = arr (\(~(BF a)) -> a)

instance NFData a => NFData (BasicFuture a) where
    rnf (BF a) = rnf a

instance Future BasicFuture a (Conf a) where
    put _ = put'
    get _ = get'

instance Future BasicFuture a () where
    put _ = put'
    get _ = get'
\end{code}
\caption{|BasicFuture| type and its |Future| instance for the |Par| Monad and GpH.}
\label{fig:BasicFuture}
\end{figure}

% \caption{NFData and Trans instances for the RemoteData type. The Trans instance does not have any functions declared as the default implementation suffices here. See \url{https://hackage.haskell.org/package/edenmodules-1.2.0.0/docs/Control-Parallel-Eden.html\#g:5} for more information.}
% \end{figure}

Figures~\ref{fig:parMapImg}--\ref{fig:parMapStream} show the definitions and a visualisations of two parallel |map| variants, defined using |parEvalN| and its lazy counterpart.

\begin{figure}[thb]
%parMap
\includegraphics[scale=0.7]{images/parMap}
\caption{|parMap| depiction.}
\label{fig:parMapImg}

\begin{code}
parMap :: (ArrowParallel arr a b conf) => conf -> (arr a b) -> (arr [a] [b])
parMap conf f = parEvalN conf (repeat f)
\end{code}
\caption{Definition of parMap.}
\label{fig:parMap}

%parMapStream
\includegraphics[scale=0.7]{images/parMapStream}
\caption{|parMapStream| depiction.}
\label{fig:parMapStreamImg}

\begin{code}
parMapStream :: (ArrowParallel arr a b conf, ArrowChoice arr, ArrowApply arr) =>
	conf -> ChunkSize -> arr a b -> arr [a] [b]
parMapStream conf chunkSize f = parEvalNLazy conf chunkSize (repeat f)
\end{code}
\caption{|parMapStream| definition.}
\label{fig:parMapStream}
\end{figure}

Arrow versions of Eden's |shuffle|, |unshuffle| and the definition of |takeEach| are in Figure~\ref{fig:edenshuffleetc}. Similarly, Figure~\ref{fig:edenlazyrightrotate} contains the definition of Arrow versions of Eden's |lazy| and |rightRotate| utility functions. Fig.~\ref{fig:lazyzip3etc} contains Eden's definition of |lazyzip3| together with the utility functions |uncurry3| and |threetotwo|.
The full definition of |farmChunk| is in Figure~\ref{fig:farmChunk}.
Eden definition of |ring| skeleton is in Figure~\ref{fig:ringEden}. It
follows \citet{Loogen2012}.

\begin{figure}[h]
\begin{code}
shuffle :: (Arrow arr) => arr [[a]] [a]
shuffle = arr (concat . transpose)

unshuffle :: (Arrow arr) => Int -> arr [a] [[a]]
unshuffle n = arr (\xs -> [takeEach n (drop i xs) | i <- [0..n-1]])

takeEach :: Int -> [a] -> [a]
takeEach n [] = []
takeEach n (x:xs) = x : takeEach n (drop (n-1) xs)
\end{code}
\caption{|shuffle|, |unshuffle|, |takeEach| definition.}
\label{fig:edenshuffleetc}
\end{figure}

\begin{figure}[h]
\begin{code}
lazy :: (Arrow arr) => arr [a] [a]
lazy = arr (\ ~(x:xs) -> x : lazy xs)

rightRotate :: (Arrow arr) => arr [a] [a]
rightRotate = arr $ \list -> case list of
  [] -> []
  xs -> last xs : init xs
\end{code}
\caption{|lazy| and |rightRotate| definitions.}
\label{fig:edenlazyrightrotate}
\end{figure}

\begin{figure}[h]
\begin{code}
lazyzip3 :: [a] -> [b] -> [c] -> [(a, b, c)]
lazyzip3 as bs cs = zip3 as (lazy bs) (lazy cs)

uncurry3 :: (a -> b -> c -> d) -> (a, (b, c)) -> d
uncurry3 f (a, (b, c)) = f a b c

threetotwo :: (Arrow arr) => arr (a, b, c) (a, (b, c))
threetotwo = arr $ \ ~(a, b, c) -> (a, (b, c))
\end{code}
\caption{|lazyzip3|, |uncurry3| and |threetotwo| definitions.}
\label{fig:lazyzip3etc}
\end{figure}

\begin{figure}[h]
\begin{code}
farmChunk :: (ArrowParallel arr a b conf, ArrowParallel arr [a] [b] conf,
             ArrowChoice arr, ArrowApply arr) =>
	conf -> ChunkSize -> NumCores -> arr a b -> arr [a] [b]
farmChunk conf chunkSize numCores f =
	unshuffle numCores >>>
	parEvalNLazy conf chunkSize (repeat (mapArr f)) >>>
	shuffle
\end{code}
\caption{|farmChunk| definition.}
\label{fig:farmChunk}
\end{figure}

\begin{figure}[h]
\begin{code}
ringSimple :: (Trans i, Trans o, Trans r) => (i -> r -> (o,r)) -> [i] -> [o]
ringSimple f is =  os
  where (os,ringOuts) = unzip (parMap (toRD $ uncurry f) (zip is $ lazy ringIns))
        ringIns = rightRotate ringOuts

toRD :: (Trans i, Trans o, Trans r) => ((i,r) -> (o,r)) -> ((i, RD r) -> (o, RD r))
toRD  f (i, ringIn)  = (o, release ringOut)
  where (o, ringOut) = f (i, fetch ringIn)

rightRotate    :: [a] -> [a]
rightRotate [] =  []
rightRotate xs =  last xs : init xs

lazy :: [a] -> [a]
lazy ~(x:xs) = x : lazy xs
\end{code}
\caption{Eden's definition of the |ring| skeleton.}
\label{fig:ringEden}
\end{figure}

The |parEval2| skeleton is defined in Figure~\ref{fig:parEval2}.
We start by transforming the |(a, c)| input into a two-element list |[Either a c]| by first tagging the two inputs with |Left| and |Right| and wrapping the right element in a singleton list with |return| so that we can combine them with |arr (uncurry (:))|. Next, we feed this list into a parallel Arrow running on two instances of |f +++ g| as described in the paper. After the calculation is finished, we convert the resulting |[Either b d]| into |([b], [d])| with |arr partitionEithers|. The two lists in this tuple contain only one element each by construction, so we can finally just convert the tuple to |(b, d)| in the last step.
\begin{figure}[h]
\begin{code}
parEval2 :: (ArrowChoice arr,
	ArrowParallel arr (Either a c) (Either b d) conf) =>
	conf -> arr a b -> arr c d -> arr (a, c) (b, d)
parEval2 conf f g =
	arr Left *** (arr Right >>> arr return) >>>
	arr (uncurry (:)) >>>
	parEvalN conf (replicate 2 (f +++ g)) >>>
	arr partitionEithers >>>
	arr head *** arr head
\end{code}
	\caption{|parEval2| definition.}
	\label{fig:parEval2}
\end{figure}
Furthermore, Fig.~\ref{fig:torus_example_rest} contains the omitted definitions of |prMM| (sequential matrix multiplication), |splitMatrix| (which splits the a matrix into chunks), |staggerHorizontally| and |staggerVertically| (to pre-rotate the matrices), and lastly |matAdd|, that calculates $A + B$ for two matrices $A$ and $B$.
\begin{figure}[h]
\begin{code}
prMM :: Matrix -> Matrix -> Matrix
prMM m1 m2 = prMMTr m1 (transpose m2)
  where
    prMMTr m1' m2' = [[sum (zipWith (*) row col) | col <- m2' ] | row <- m1']

splitMatrix :: Int -> Matrix -> [[Matrix]]
splitMatrix size matrix = map (transpose . map (chunksOf size)) $ chunksOf size $ matrix

staggerHorizontally :: [[a]] -> [[a]]
staggerHorizontally matrix = zipWith leftRotate [0..] matrix

staggerVertically :: [[a]] -> [[a]]
staggerVertically matrix = transpose $ zipWith leftRotate [0..] (transpose matrix)

leftRotate :: Int -> [a] -> [a]
leftRotate i xs = xs2 ++ xs1 where
    (xs1,xs2) = splitAt i xs

matAdd = chunksOf (dimX x) $ zipWith (+) (concat x) (concat y)
\end{code}
	\caption{|prMMTr|, |splitMatrix|, |staggerHorizontally|, |staggerVertically| and |matAdd| definition.}
	\label{fig:torus_example_rest}
\end{figure} %$

\section{Syntactic sugar} \label{syntacticSugar}
Finally, we also give the definitions for some syntactic sugar for PArrows, namely |parstar| and |parand|.
For basic Arrows, we have the |***| combinator (Fig.~\ref{fig:syntacticSugarArrows}) which allows us to combine two Arrows |arr a b| and |arr c d| into an Arrow |arr (a, c) (b, d)| which does both computations at once. This can easily be translated into a parallel version |parstar| with the use of |parEval2|, but for this we require a backend which has an implementation that does not require any configuration (hence the |()| as the |conf| parameter% in Fig.~\ref{fig:par***}
):
% \begin{figure}[h]
\begin{code}
(|***|) :: (ArrowChoice arr, ArrowParallel arr (Either a c) (Either b d) ())) =>
	arr a b -> arr c d -> arr (a, c) (b, d)
(|***|) = parEval2 ()
\end{code}
% \caption{Definition of |parstar|, the parallel version of |***|.}
% \label{fig:par***}
% \end{figure}
% With this we can analogously to the serial |&&&|
We define the parallel |parand| % (Fig.~\ref{fig:par&&&})
in a similar manner to its sequential pendant |&&&| (Fig.~\ref{fig:syntacticSugarArrows}):
% \begin{figure}[h]
\begin{code}
(|&&&|) :: (ArrowChoice arr, ArrowParallel arr (Either a a) (Either b c) ()) =>
	arr a b -> arr a c -> arr a (b, c)
(|&&&|) f g = (arr $ \a -> (a, a)) >>> f |***| g
\end{code} % $ %% formatting
% \caption{Definition of |parand| - the parallel version of |&&&|.}
% \label{fig:par&&&}
% \end{figure}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End:


\section{Scheimpflug geometry}
\label{sec:scheimflug}
As introduced in the previous section, now, we describe the plane correction due to the application of Scheimpflug principle, tilting the lens. \\

Tilting lenses causes a variation of the image plane, which rotates to stay parallel to the lens itself. Many works, such as \cite{inproc-lens-inc}\cite{Louhichi2006}\cite{4059344}, tried to improve the calibration processes, introducing different corrections in order to balance the effects introduced by lens tilt. For the most part, they have tried to apply different rotations to the plane, but these have been shown to work only for small angles. For big angles, the rotations are not enough. \\

Consider the two image planes, the one parallel to the sensor and the tilted one: as shown in Figure \ref{fig:scheimpflug-line}, the origin of the two planes is the same, since the parallel plane can be arbitrarily placed.
  \begin{figure}[t!]
    \centering
    \includegraphics[width=\textwidth]{./images/model/scheimpflug.png}
    \caption{Intersection of the point in the real world in the tilted plane and in the perpendicular plane.}
    \label{fig:scheimpflug-line}
  \end{figure}
Similar to what is shown in the Figure \ref{fig:perspective_projection}, we can draw a line $r$ from the optical center $O$ to the point $p_w = \left( x_w, y_w, z_w \right)$ in the world. Accordingly with the sections above, $p_w$ is a point belonging to the laser plane, without considering radial compensation, and the tilted plane is the one in which we have projected $p_w$ in Section \ref{sec:wrd2cam}. Accordingly with \cite{SchCameraCalib}, the line $r$ passes through the two planes, allowing the definition of a system of equations that solve the intersection problem. The solution of this system can be written as follows:

  \begin{equation*}
      x_p = \lambda \cdot \left( x_s \cdot \cos\chi + y_s \cdot \sin\upsilon\cos\chi \right)
  \end{equation*}
  \begin{equation*}
    y_p = \lambda \cdot y_s \cdot \cos\upsilon
  \end{equation*}
where $\left( x_p, y_p\right) = p_{perp}$ in the figure, and $\left( x_s, y_s\right)$ ($p_{tilt}$ in figure) are the coordinate of the point in the sensor reference system, introduced in the section above. The parameter $\lambda$ is the plane coefficient and it is:
  \begin{equation*}
    \lambda = \frac{f}{f + x_s \sin\chi + y_s \sin\upsilon\cos\chi}
  \end{equation*}
The angles $\chi$ and $\upsilon$ are the swing and tilt angle of the lens, respectively. In literature, the term swing refers to a rotation with respect to the $x$ axis, while tilt refers to a rotation with respect to the $y$ axis. \\

The equation considered has been defined in a proposal for an algorithm for camera calibration. Despite that, the model capability to consider point projections gives to it a big advantage over simple rotations: the fact that we are able to consider also perspective distortions, as we have done in Section \ref{sec:pinhole_camera} for world to camera coordinate systems transformations. Note that also in this case, the $x$ coordinates are function of the $y$ ones. \\

From an analytical point of view, this is the most delicate step. The big number of elements to consider makes this calculation a big source of error. The error propagation model suggests to compute

  \begin{equation*}
    \sigma_{x_{s_i}} = \sqrt{
      \left( \frac{\partial x_{s_i}}{\partial y_{p_i}} \right)^2 \sigma_{y_{p_i}}^2 +
      \left( \frac{\partial x_{s_i}}{\partial x_{p_i}} \right)^2 \sigma_{x_{p_i}}^2 +
      \left( \frac{\partial x_{s_i}}{\partial \upsilon} \right)^2 \sigma_\upsilon^2 +
      \left( \frac{\partial x_{s_i}}{\partial \chi} \right)^2 \sigma_\chi^2
    }
  \end{equation*}
  \begin{equation*}
    \sigma_{y_{s_i}} = \sqrt{
      \left( \frac{\partial y_{s_i}}{\partial y_{p_i}} \right)^2 \sigma_{y_{p_i}}^2 +
      \left( \frac{\partial y_{s_i}}{\partial x_{p_i}} \right)^2 \sigma_{x_{p_i}}^2 +
      \left( \frac{\partial y_{s_i}}{\partial \upsilon} \right)^2 \sigma_\upsilon^2 +
      \left( \frac{\partial y_{s_i}}{\partial \chi} \right)^2 \sigma_\chi^2
    }
  \end{equation*} \\

\noindent
Even in this case we could ignore the contribution of the focal length, because it is a parameter estimated by the calibration process.\\

To complete this model, we should take into account also focal length and image center transformations. Reference \cite{SchCameraCalib} suggests the following set of relations:
  \begin{equation*}
    f' = \frac{f}{\cos\upsilon}
  \end{equation*}
  \begin{equation*}
    C_x' = C_x + f\cdot n_x \cdot \sin\upsilon
  \end{equation*}
  \begin{equation*}
    C_y' = C_y + f\cdot n_y \cdot \sin\chi
  \end{equation*} \\
However, our tests (described below) demonstrated that these corrections are useless. Calibration algorithms that do not implement this type of heuristics, will implement parameters (i.e focal length and optical center) optimization methods anyway, that try to evaluate these parameters as precisely as possible. The methods seem to be accurate enough to ignore the latter passages, in favour of ones suggested by the calibration.


%
%
\section{Methodology}\label{sec:method}
\subsection{Observed galaxy density}
In our methodology, we treat the mitigation of imaging systematics as a regression problem, in which we aim to model the relationship between the observed galaxy density (\textit{label}) and the imaging attributes (\textit{features}) that are the potential sources of the systematic error. Note that we do not include the positional information as \textit{input} features since we do not want the mitigation to fit the cosmological clustering pattern. The solution of the regression then would provide the predicted mean galaxy density (i.e., in the absence of clustering or shot-noise) solely based on the imaging attributes of that location. We use this predicted galaxy density as the \textit{survey} selection mask to be applied to any observed galaxy map in the attempt to eliminate the systematic effects and therefore isolate the cosmological fluctuation. Below we describe our procedure.\\

In this paper, we focus on the multiplicative systematic effects. The observed number of galaxies within pixel $i$ can be expressed in terms of the true number of galaxies $n_{i}$ and the contamination model $\mathcal{F}(\textbf{s}_{i})$ as
\begin{equation}\label{eq:ngal_fs}
    n_{i}^{o}(\textbf{s}_{i}) = n_{i} \mathcal{F}(\textbf{s}_{i}),
\end{equation}
where \textbf{s}$_{i}$ is a vector representing the imaging attributes \textbf{s} of pixel $i$, and the contamination model $\mathcal{F}(\textbf{s}_i)$ is an unknown function representing the systematic effects which could be either a linear, non-linear, or a more complex combination of the imaging attributes. Multiplicative systematics are associated with obscuration and area-loss due to foreground stellar density, Galactic extinction, etc. On the other hand, additive systematics are associated mostly with stellar contamination, as described in \citet{myers2007clustering, ross2011ameliorating, 2012ApJ...761...14H, prakash2016sdss, 2016MNRAS.455.4301C}. When averaged over many pixels, the effect of additive systematics can be absorbed into the constant term of the multiplicative model $\mathcal{F}$, assuming there is no correlation between the imaging maps and the true galaxy density field. The modeling of $\mathcal{F}(\textbf{s}_{i})$ can be approached by a wide variety of techniques, ranging from the traditional methods based on multivariate functions to non-parametric and non-linear models based on machine learning or deep learning such as random tree forests and neural networks \citep{breiman2001random, geurts2006extremely}.\\

The cosmological information is contained in the true overdensity that is given by
\begin{equation}
\delta_{i} = n_i/(f_{{\rm pix},i}\bar{n})-1,\label{eq:overden}
\end{equation}
accounting for the pixel completeness where $\bar{n}$ is the `true' average number of galaxies. Then,
\begin{equation}
n_{i}^{o}= f_{{\rm pix},i}\overline{n}~(1 + \delta_{i})~ \mathcal{F}(\textbf{s}_{i}).\label{eq:nobsi}
\end{equation}
This $n_{i}^{o}/f_{{\rm pix},i}$ is equivalent to the observed $ngal$ aforementioned.
Since we do not know the true average number density $\overline{n}$ of the data, we estimate $\bar{n}$ from the average of the observed galaxy field,
\begin{equation}\label{eq:nbar}
\hat{\overline{n}} = \frac{\displaystyle\sum_{i} n^{o}_{i}  }{\displaystyle\sum_{i} f_{{\rm pix},i}},
\end{equation}
and treat $\hat{\overline{n}} \equiv \overline{n}$.
Due to the finite volume of our sample, $\hat{\overline{n}} \neq \overline{n}$ even in the absence of systematic effects. This imposes the well-known integral constraint effect on any clustering analysis. We further ignore any systematic effect on $\hat{\overline{n}}$ due to the fact we use $n^{o}_{i}$; that is, Eq.~\ref{eq:nbar} converges to $\overline{n}$ only when $\sum_{i} f_{{\rm pix},i}= \sum_{i} f_{{\rm pix},i}\mathcal{F}_i$.
In this sense we are modeling the relative systematic effect without necessarily determining the accurate `true' $\overline{n}$. We will use simulated results to test our methodology, and the analysis applied to the simulations with a limited footprint will be subject to the similar finite-volume and systematic effects on $\hat{\overline{n}}$, thus providing a fair comparison and means to catch any obvious problem with this approximation.\\

Finally, we define the normalized galaxy density per pixel $t_{i}$,
\begin{equation}\label{eq:nnbar}
    t_{i}(\textbf{s}_{i}) \equiv \frac{n_{i}^{o}(\textbf{s}_{i})}{f_{{\rm pix},i}\hat{\overline{n}}} = (1+\delta_{i}) ~ \mathcal{F}(\textbf{s}_{i}).
\end{equation}
With this definition, we can estimate the unknown contamination model $\hat{\mathcal{F}}$ (or selection mask) by modeling the dependence of $t_{i}$ on $\textbf{s}_{i}$. When averaged over many spatial positions, the cosmological fluctuations $\delta_i$ will be averaged out and therefore the observed $t_i$ averaged across many pixels with the same imaging attribute, should only be a function of \textbf{s} and return $\mathcal{F}$: %\mro{}{Please also check the notation throughout equations 3-7, since I think there are either some discrepancies or undefined s.}
\begin{equation}\label{eq:tfs}
    <t_{i}(\textbf{s}_{i})>_{i} \simeq <{\mathcal{F}}(\textbf{s}_{i})>_{i} = {\mathcal{F}}(\textbf{s}).
\end{equation}
The inverse of the selection mask which is equivalent to the photometric weights ($wt^{{\rm sys}}$) in other studies can therefore be used to remove the systematic effects from the observed galaxy number map (cf. Eq. \ref{eq:ngal_fs}),

\begin{align}\label{eq:wsys}
\hat{n}_i = \frac{n^o_i}{\hat{\mathcal{F}}}=n^o_i wt^{{\rm sys}}_{i}.
\end{align}
In the following we describe how we obtain $\hat{\mathcal{F}}$ using different regression approaches, e.g., neural networks and multivariate linear functions. From now on, the terms \textit{features} and \textit{label} associated with each data point refer to \textbf{s} and $t$ of each HEALPix pixel, respectively.

\subsection{Mitigation with Neural Networks}\label{subsec:MethodNN}
We will apply {\it fully connected feed forward neural networks} in order to tackle our regression problem. Fig. \ref{fig:perceptron} illustrates a schematic diagram of a neuron, the building block of a neural network, which generates the output based on a linear combination of the inputs followed by a nonlinear transformation, the activation function $f$.\\

\begin{figure}
\centering
\includegraphics[width=0.4\textwidth]{figures/fig2-perceptron.pdf}
\caption{A schematic diagram of a single neuron with the activation function $f$. The neuron takes a set of inputs \textbf{x}=$(x_{1}, x_{2},...,x_{n})$, multiplies each of them by its associated weight \textbf{w}=$(w_{1}, w_{2},...,w_{n})$, and sums the weighted values and a thresh-hold (or a constant offset) which is called \textit{bias}, to form a pre-activation value, $z=\sum_{i=1}^{n}x_{i}w_{i} + bias$, which is a linear process. The neuron then transforms the pre-activation $z$ to the output using the activation function $f(z)$, which is where the nonlinear process can enter. }\label{fig:perceptron}
\end{figure}

Fig. \ref{fig:ffnn} illustrates the architecture of a fully connected feed forward neural network with the imaging attributes in the input layer, three hidden layers of six non-linear neurons in the middle, and a single neuron without any activation function in the output layer, as an example. The \textit{bias} neuron in each layer is shown in orange and is analogous to the intercept in linear regression. The output of the neural network will be an estimation of the contamination model $\hat{\mathcal{F}}$ (see Eq. \ref{eq:nnbar}). If we use the identity function as the activation function (e.g., $f(z)=z$), regardless of the number of hidden layers, the neural network is equivalent to a linear model. This means that our methodology is a generalization or an extension of the conventional linear mitigation methods. The modeling capabilities of neural networks depend on the number of hidden layers, type of non-linear activation function and the number of neurons in each hidden layer \citep[see e.g.][]{cybenko1989approximation,hornik1989multilayer,funahashi1989approximate, tamura1997capabilities, huang2003learning, lin2017does, rolnick2017power}.\\

We use the rectifier $f(z) = \text{max}(0, z)$ as the activation function for the hidden layer neurons, which alleviates the `vanishing gradient' problem
~\citep[see e.g.,][]{nair2010rectified,glorot2011deep,krizhevsky2012imagenet, dahl2013improving,montufar2014number}. \\

\begin{figure}
\centering
\includegraphics[width=0.45\textwidth]{fig3-ffnn.pdf}
\caption{A schematic illustration of a fully connected feed forward neural network with the imaging attributes in the input layer, three hidden layers of six neurons in the middle, and a single neuron on the output layer, as an example. The blue-colored neurons have non-linear activation functions, while the red-colored neuron lacks any activation function. In reality, we employ the validation procedure to choose the best number of hidden layers while keeping the total number of hidden layer neurons fixed at 40 (i.e., approximately twice the number of imaging attributes in this study).}\label{fig:ffnn}
\end{figure}

We utilize  $k$-fold cross-validation with $k=5$ folds/sub-groups to train the parameters, tune the hyper-parameters, and to estimate the predictive performance of the neural network. As illustrated in Fig. \ref{fig:5fold}, we randomly split the entire pixel data (187,257 pixels) into five folds and construct the training, validation and test data sets out of these five folds; three folds are assigned to the training set, one fold is assigned to the validation set, and the remaining one fold is assigned to the test set. A specific assignment of the five folds to these three sets forms one `partition'. We construct five different partitions such that each fold is used once as test fold. This $k$-fold cross-validation scheme ensures that a test example is never used for training or tuning.\\

\begin{figure}
\centering
 \includegraphics[width=0.35\textwidth]{figures/fig4-5fold.pdf}
 \caption{A visualization of the five-fold permutations of data-split. The data is randomly split into five equal-size folds, and by permutation of the folds we construct five partitions of data. For each partition/permutation, three folds are assigned to the training set, one fold for the validation set, and the remaining one fold for the test set: therefore, 60\% training, 20\% validation, and 20\% test sets. Each column represents a partition. The test folds are shown in red, while the training and validation folds are shown in blue. The key points are : 1) The folds are not contiguous (in RA, DEC) as may be implied by this cartoon. 2) There is no overlap between the training, validation, and test folds within a partition. 3) One can reconstruct the entire data by merging the test folds from the five partitions.}
\label{fig:5fold}
\end{figure}

We standardize (i.e., renormalize) the label and features of the training, validation, and test folds using the mean and standard deviation of the label and features of the corresponding training fold (i.e., similar to Tab.~\ref{tab:meanstats}, but for the training set of each partition). We initialize the biases to zero and sample the weights of each layer randomly from a normal distribution whose variance is inversely proportional to the number of neurons of the previous layer \citep[see e.g.,][]{he2015delving}. Using the training fold, we utilize the adaptive gradient descent with momentum \citep[\textit{Adam},][]{kingma2014adam} to update the parameters of the neural network with batches of size $N_{{\rm batch}}$. Thus, the entire training set is split into $N_{{\rm batch}}$ batches and a gradient update is applied for each batch. One training epoch corresponds to processing the $N_{{\rm batch}}$ batches once \citep[for more details on the training procedure, we refer the reader to see e.g.,][]{ruder2016overview}. The hyper-parameters of \textit{Adam}, specifically the moments decay rates and the tolerance, are fixed as follows: $\beta_{1}$ = 0.9, $\beta_{2}$ = 0.999 and $\epsilon$ = 10$^{-8}$. The default learning rate of 0.001 will be tuned using the validation data.\\

The network is trained to minimize the following cost function:
\begin{equation}\label{eq:cost}
    \text{J} = \frac{1}{N_{{\rm batch}}}\sum_{i}^{N_{{\rm batch}}}f_{\rm pix, i}~[t_{i} - \hat{\mathcal{F}_{i}}]^{2} + \frac{\lambda}{2} ~ ||w||^{2} ,
\end{equation}
%
where the first term is the Mean Squared Error (MSE) weighted with $f_{\rm pix,i}$, and the second term is the L2 regularization term, used to penalize higher weight magnitudes and a larger number of neurons~\citep{hoerl1970ridge}. The strength of the L2 penalty term is controlled via the regularization scale $\lambda$. The network is trained for a number of training epochs, $N_{{\rm epoch}}$, although to avoid unnecessary training, we implement the early stopping technique with the tolerance of 1.e-4 and patience of 10, i.e., the training terminates if the validation MSE does not improve more than the tolerance within the last 10 epochs.\\

\subsubsection{Backward feature elimination}

The input features are highly correlated as shown in Fig.~\ref{fig:eboss_dr7}, and therefore the 18 maps probably contain redundant information.
We apply \textit{backward feature elimination} (feature selection) to remove the redundant input features in order to reduce the noise in the prediction as well as to protect the cosmological information by avoiding too much freedom in modeling. We find that reducing the dimension of the input features, i.e., the imaging attributes, is an essential step to avoid over-fitting and regressing out the cosmological clustering.\\

We perform the feature selection for each partition separately. Initially, we train a linear model on all 18 input features with the following hyper-parameters: the initial learning rate of 0.001, batch-size of 1024, L2 regularization scale of zero, for 500 training epochs with early stopping. We record the validation MSE as a baseline criterion. Then we eliminate one input feature and train the linear model on the remaining 17 features. This trained linear model is applied to the validation set. The input feature whose removal has produced the highest decrease in validation MSE (i.e., the highest improvement in fitting) is permanently eliminated, leaving only 17 features for training. Note that if the feature contained useful information on the systematic effects, removing the feature would have made the fit worse. We repeat the regression using 17 input features, and so on, removing one feature at each iteration until either the validation MSE does not decrease relative to the baseline or all input features are removed. Fig. \ref{fig:dr7ablation} shows the result of the backward feature elimination procedure for the first partition of DR7, ranking the input features based on their importance from left to right. This result supports the trends seen in the exploratory analysis in \S~\ref{sec:data} which indicated strong linear correlations with the stellar density, Galactic extinction, and hydrogen column density in the data (see the correlation matrix in the bottom panel of Fig. \ref{fig:eboss_dr7}). The color gradient indicates the relative change in validation Root Mean Squared Error (RMSE) when that particular feature is removed with reference to the baseline. We note that the order of removal is not the same as, for example, the color gradient order of the attributes in the first iteration. We believe it is because, as we remove the redundant features, the relevant importance of the remaining input features changes due to the complex correlations between the removed features and the remaining ones. The remaining features for each of the five partitions are shown in Fig. \ref{fig:dr7ablation2}. The attributes $lnHI$ and $nstar$ are commonly identified as the most important features and then $ebv$, $seeing-g$, $skymag-g$, $skymag-z$, $exptime-r$, $exptime-z$, $mjd-z$ are commonly identified for all 5 partitions.\\

\begin{figure}
    \centering
    \includegraphics[width=0.4\textwidth]{figures/fig5-dr7ablation.pdf}
    \caption{Feature importance of DR7 based on backward feature elimination for the first partition, as an example. This process iteratively removes the feature whose removal produces the largest decrease in the validation RMSE (i.e., the greatest improvement in fitting) until no decrease is observed. After the first iteration of removal (the top row), the removal of $skymag-r$ decreased the validation RMSE the most and therefore $skymag-r$ is removed. In the second iteration (the second row), removing $exptime-g$ decreased the validation RMSE the most and therefore it is removed. However, in the ninth iteration, the removal of $mjd-z$ did not decrease the validation RMSE and therefore the feature selection stops here, passing the rightmost ten features to the neural network regression. As a result of the process, the importance increases from left to right, and the rightmost ten maps in the figure ($ebv$, $nstar$, $logHI$, $seeing-g$, $skymag-g$, $skymag-z$, $exptime-r$, $exptime-z$, $mjd-g$, $mjd-z$) are the ones that worsen the validation RMSE when being removed from the input layer.}
    \label{fig:dr7ablation}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=0.4\textwidth]{figures/fig6-dr7ablation2.pdf}
    \caption{Important imaging maps identified by the backward feature elimination (\textit{feature selection}) procedure for the five partitions used for DR7. A darker color of a point within each partition represents a more important attribute identified by the feature selection procedure. Note that $nstar$ is identified as the most important attribute in all partitions, i.e., across the footprint.}
    \label{fig:dr7ablation2}
\end{figure}

\subsubsection{Hyper-parameter tuning, training, and testing}\label{subsubsec:hyperparam}

We train the hyper-parameters for each partition separately. At each training epoch and for each choice of hyper-parameters, we apply the trained network on the validation fold. We adjust the hyper-parameters accordingly such that the validation MSE is minimized. Our neural network has the following five hyper-parameters: number of hidden layers; number of training epochs $N_{{\rm epoch}}$; L2 regularization scale $\lambda$; batch size $N_{{\rm batch}}$; Adam's learning rate. We tune one hyper-parameter at a time. To find the optimal learning rate, we monitor the behavior of the cost function during training. We observe that a learning rate of 0.001 leads to a smoothly decreasing cost function vs. training epochs. We train the network for up to $N_{\rm epoch} = 500$ epochs although we implement the early stopping technique with the inertia (or patience) of 10 and the tolerance of 1.e-4: this means the training will be stopped if none of the last 10 epochs achieved a smaller relative error reduction with respect to the minimum validation error, within the tolerance. For the number of hidden layers, we try the following architectures, in which the total number of hidden neurons is fixed at 40 (i.e. roughly twice the number of the features) except for the linear model:\\~\\
%
%
$[0]$  : no hidden layers \\
$[40]$ : one hidden layer of 40 neurons \\
$[20, 20]$ : two hidden layers of 20 neurons on each \\
$[20, 10, 10]$ : three hidden layers of 20, 10 and 10 neurons \\
$[10, 10, 10, 10]$ : four hidden layers of 10 neurons\\~\\
After finding the best number of layers, we proceed to tune $\lambda$ by trying powers of 10, e.g., 0.001, 0.01, ..., 1, ..., 1000. Finally, we adjust $N_{\rm batch}$ by trying powers of 2, e.g., 128, 256, ... , 4096. The optimal set of the hyper-parameters for each partition is summarized in Tab. \ref{tab:hparams}.\\

\begin{table}
    \centering
    \caption{The best hyper-parameters for each partition of DR7.}
    \label{tab:hparams}
    \begin{tabular}{lccr} % four columns, alignment for each
        \hline
        %\hline
         & number of layers & $\lambda$ & $N_{\rm batch}$ \\
        \hline
        Partition 1 & [20, 20] & 0.001 & 4096\\
        Partition 2 & [20, 10, 10] & 0.001 & 512\\
        Partition 3 & [20, 10, 10] & 0.001 & 1024 \\
        Partition 4 & [20, 10, 10] & 0.001 & 512 \\
        Partition 5 &  [20, 10, 10] & 0.001 & 2048\\
        %\hline
        %\hline
    \end{tabular}
\end{table}

Once the grid search procedure identifies the best performing hyper-parameters out of the predefined ranges introduced in Section \ref{subsubsec:hyperparam}, the network is trained with these hyper-parameters for 10 independent runs, each one with a different initialization of the weights and biases, and then applied on the test set. We compute the median of the predicted test label from the 10 runs and aggregate the results over the 5 different partitions to construct the map of the predicted label ($\hat{\mathcal{F}}$) for the entire footprint. For our default method, the backward feature elimination is conducted for each partition and reduces the number of input features before the hyper-parameter training step, as illustrated in Fig.~\ref{fig:pipeline}.  This process is performed for each partition separately, each partition using a different fold as the test set, until the entire footprint is covered through the 5 test folds. The flow of the feature selection, hyper-parameter tuning, and testing is summarized in Fig. \ref{fig:pipeline}.\\

\begin{figure*}
         \centering
         \includegraphics[ width=0.8\textwidth]{figures/fig7-pipeline.pdf}
         \caption{A flow-chart of the backward feature elimination and hyper-parameter tuning for each partition. This entire process is performed five times for each of the five partitions/permutations such that the entire footprint is covered by aggregating the different testing folds. }
         \label{fig:pipeline}
\end{figure*}

% multivariate
\subsection{Mitigation with Multivariate Linear Functions}
We use linear and quadratic polynomial functions to model the normalized galaxy density dependence on the imaging attributes (Eq.~\ref{eq:tfs}), as the benchmark approaches to be compared with the neural network. Unlike the proposed neural network method, no regularization or dimensionality reduction is performed, and all data are used to train the parameters of the regression models. Despite a lack of any deliberate machinery against over-fitting, we note that over-fitting is less likely to be an issue for this method since the size of the data is much greater than the number of the fitting parameters. Nevertheless, we tried splitting the sample into 60\% of the training data to derive the best fit linear coefficients and 20\% of the test set (i.e., the same training and test sample size to have a fair comparison with the neural network) to apply the derived coefficients and permuted five times until the test set covers the entire footprint. We find that such a split does not change the results of the linear regression. On the other hand, the limited flexibility of its parameterized form could be a weakness of this method and we believe it is responsible for the differences that the neural network method makes in the comparison presented in Section \ref{sec:mitigateDR7}.\\

The contamination model from Eqs.~\ref{eq:nnbar} and ~\ref{eq:tfs} can be estimated via a multivariate linear function in terms of the standardized imaging attributes (\textbf{s}) as,

\begin{equation}\label{eq:multvar}
\hat{\mathcal{F}}(\textbf{s}|b_{0},\alpha) = b_{0} + \sum_{m=1}^{M}\sum_{k=1}^{18} \alpha_{mk} (\frac{s_{k}-\overline{s_{k}}}{\sigma_{k}})^{m} ,
\end{equation}
where $M$ is the maximum power law index equal to 1 and 2 for the linear and quadratic polynomial model, respectively; the constants $\overline{s_{k}}$ and $\sigma_{k}$ are respectively the mean and standard deviation of the k'th imaging quantity, $s_{k}$ (cf. Tab. \ref{tab:meanstats}). The parameters $b_{0}$ and $\alpha_{mk}$ are the intercept and the corresponding coefficients for each term, respectively, which are tuned by minimizing the weighted sum of the squared errors. The output of the regression is applied as the selection mask on the observed galaxy density to eliminate the systematic effects (see Eq., \ref{eq:wsys}).\\

\subsection{Angular Clustering Statistics}\label{subsec:ang_clustering}
\subsubsection{One point statistics}
In the absence of the systematic effects, the galaxy density field should be statistically independent from the imaging attributes and will only depend on the cosmological fluctuations while an individual dataset/mock will be subject to chance correlations within the statistical error. When averaged over many spatial positions, the cosmological fluctuations will be averaged out and therefore the average density should be equal to the mean density once the survey footprint is accounted for. A deviation from the mean density as a function of the imaging attributes, therefore, is indicative of the average dependence of the observed galaxy density on the corresponding imaging attributes. To assess the level of the contamination in the data, we compute the histogram of the spatially averaged galaxy density vs the imaging attributes. For each of the system attribute, $s_k$, we prepare 20 bins. For each bin of $s_{k}$, we have:

\begin{equation}\label{eq:nnbar_stat}
    \frac{\overline{n}(s_{k})}{\overline{n}_{tot}} \equiv \frac{1}{\hat{\overline{n}}}~ \frac{\displaystyle\sum_{s_{k}\leq s_{k,i}<s_{k}+\Delta s_{k}} n^{o}_{i}}{\displaystyle\sum_{s_{k}\leq s_{k,i}<s_{k}+\Delta s_{k}} f_{\rm pix, i}},
\end{equation}
where the indices $i$ and $k$ respectively represent the pixel index, and the systematic index; $\Delta s_{k}$ is the bin width arranged for different $s_k$ bins such that each bin contains almost the same amount of effective area, in an attempt to suppress fluctuations in $\overline{n}(s_{k})/\overline{n}_{tot}$ due to small number statistics. We estimate the error bars using the Jackknife resampling of 20 non-contiguous subsamples of pixels within each imaging attribute bin ~\citep[see e.g.][]{ross2011ameliorating}:

\begin{equation}\label{eq:error_jack}
    \sigma^{2}_{{\rm Jack}}(s_{k}) = \frac{19}{20}\sum_{j=1}^{20} \left[    \frac{\overline{n}(s_{k})}{\overline{n}_{tot}} -     \frac{\overline{n}_{j}(s_{k})}{\overline{n}_{tot}}\right]^{2}  ,
\end{equation}
 where $\overline{n}_{j}(s_{k})/\overline{n}_{tot}$ is computed over the entire sample when the $j$'th Jackknife region is excluded.
As a result of the adjusted $\Delta s_{k}$, the level of $\sigma^{2}_{{\rm Jack}}(s_k)$ is almost the same for all $s_k$. After mitigation of the systematic effects, one expects that the corrected density field is independent of the imaging attributes, i.e., $\overline{n}(s_{k})/\overline{n}_{tot}$ being consistent with unity.

\subsubsection{Two-point clustering statistics}
The two-point clustering statistic measures the spatial correlation of the galaxy density and has been the main statistic for extracting the cosmological information from galaxy surveys. We use the angular auto and cross two-point clustering statistics of the galaxy density field as well as of the imaging attributes to estimate the impact of the potential systematics on the cosmological clustering signal and to examine the effectiveness of the mitigation techniques tested in this paper. For pixel $i$, we calculate the galaxy overdensity $\delta_{i}$ using Eq.~\ref{eq:overden} and the fluctuation of a given imaging attribute $\delta^{s}_{i}$ as
\begin{align}\label{eq:delta}
    \hat{\delta}^{s}_{i} &= \frac{s_{i}}{\hat{\overline{s}}} - 1 ,
\end{align}
where $\hat{\overline{s}}$ is the mean of each imaging attribute weighted with $f_{{\rm pix},i}$,
\begin{align}\label{eq:deltasysbar}
\hat{\overline{s}} = \frac{\sum_i f_{{\rm pix},i} s_i}{\sum_i f_{{\rm pix},i}}
\end{align}
following \citet{ross2011ameliorating}.\\

By definition, Eqs.~\ref{eq:nbar} and \ref{eq:delta} ensure that the following integral of the observed quantity over the entire footprint vanishes:
\begin{align}\label{eq:ic}
    \sum_{i} \hat{\delta}_{i}~f_{{\rm pix},i} = 0,
\end{align}
for both the galaxy as well as imaging attribute fluctuations. We utilize both the angular correlation function and angular power spectrum to extract the cosmological information from the galaxy density field. While our mitigation efficiency is evaluated based on the angular power spectrum, we also inspect the angular correlation function to make sure that the systematics are mitigated in that estimator as well, since both estimators are commonly used in the clustering analysis and they are complementary to each other given the limited range of data.

\begin{itemize}
    \item Angular Correlation Function: we employ the HEALPix-based estimator to compute the angular correlation function which, for a separation angle $\theta$, is defined as \citep[see e.g.][]{scranton2002analysis,ross2011ameliorating}
    \begin{align}
        \omega^{p,q} (\theta) = \frac{\displaystyle\sum_{ij} \hat{\delta}^{p}_{i} \hat{\delta}^{q}_{j} \Theta_{ij}(\theta) f_{\rm pix, i}f_{\rm pix, j}}{\displaystyle\sum_{ij} \Theta_{ij}(\theta) f_{\rm pix, i}f_{\rm pix, j}},\label{eq:ancor}
    \end{align}
    where $p=q$ gives an auto correlation function estimator, $p\neq q$ gives a cross correlation function estimator, and $\Theta_{ij}$ is one when two pixels $i$ and $j$ are separated from each other within $\theta$ and $\theta+\Delta\theta$, or zero otherwise. Note that our estimator weighs each pixel overdensity with $f_{{\rm pix}, i}$ since the pixels with a greater complete area coverage should have a higher signal to noise. Such weight is straightforwardly corrected by the denominator in Eq.~\ref{eq:ancor} unlike its conjugate estimator (i.e., the power spectrum). Since our overdensity map resolution is limited by the pixel size, we set the $\Delta\theta$ to be the resolution of a pixel ($\sim$ 0.23 deg). \\

    \item Angular Power Spectrum : one can conveniently expand a coordinate on the surface of a sphere in terms of spherical harmonics or, if azimuthally symmetric, Legendre polynomials. We define the following estimator for expanding the galaxy overdensity:
    \begin{equation}
        \hat{\delta}_{i} = \sum_{\ell=0}^{\ell_{{\rm max}}}\sum_{m=-\ell}^{\ell} a_{\ell m} Y_{\ell m}(\theta_{i}, \phi_{i}),
    \end{equation}
    where $\theta, \phi$ represent the polar and azimuthal angular coordinates of pixel \textit{i}, respectively. The cutoff at $\ell=\ell_{{\rm max}}$ assumes that the signal power is not significant for modes $\ell>\ell_{{\rm max}}$. We define the following spherical harmonic (SH) transform estimator of overdensity ($\hat{\delta}$) over the total number of non-empty pixels $N_{{\rm pix}}$:

    \begin{equation}
        \hat{a}_{\ell m} = \frac{4\pi}{N_{{\rm pix}}} \sum_{i=1}^{N_{{\rm pix}}}  \hat{\delta}_{i}~f_{\rm pix, i}~ Y^{*}_{\ell m}(\theta_{i}, \phi_{i}),
    \end{equation}
    where $^{*}$ represents the complex conjugate, and we again down-weight the overdensity in pixel \textit{i} by the completeness ($f_{\rm pix, i}$). Due to the survey window function implicit in the sum over the non-empty pixels and explicit in $f_{\rm pix, i}$, our estimator would not return unbiased estimates of the SH coefficients, unless the window function effect is corrected for, and also the expected orthogonalities between different SH modes would not hold. Nevertheless, we define the angular power spectrum estimator as the average of the magnitude of SH coefficients over $m$:
    \begin{equation}\label{eq:pusedocell}
        \hat{C}^{p,q}_{\ell} = \frac{1}{2\ell +1} \sum_{m=-\ell}^{\ell} \hat{a}^{p}_{\ell m} \hat{a}^{q*}_{\ell m},
    \end{equation}
    where $p=q$ gives an auto power spectrum, $p\neq q$ gives a cross power spectrum between the galaxy density and the imaging attributes. In order to compute the angular power spectrum, $C_{\ell}$, we make use of the ANAFAST function from HEALPix \citep{gorski2005healpix} with the third order iteration of the quadrature to increase the accuracy\footnote{We refer the reader to \url{https://healpix.sourceforge.io/pdf/subroutines.pdf}, page 104.}.
    Unlike in the angular correlation function, we do not attempt to correct for the survey window function/survey mask effect in the angular power spectrum both for DR7 and the mocks. We rather calculate the window effect on the theoretical models of power spectrum in Appendix \ref{app:windowfunction}. For the mock test, we use the angular power spectrum observed in the mocks without the contamination model, i.e., the `Null' case, as our baseline to compare with different mitigation methods.\\
\end{itemize}
%
%
We use the Jackknife resampling technique with 20 equal-area contiguous regions, as shown in Fig. \ref{fig:jackknifes}, to estimate the error-bars on $\omega (\theta)$ and $C_{\ell}$ (see Eq. \ref{eq:error_jack}).\footnote{We use the mocks without imaging systematics (null mocks in Section \ref{subsec:surveymocks}) to compare the errors from the Jackknife subsamples of one mock with the errors among 100 full  DECaLS-like mock footprints, and we find that the former is greater than the latter on ell $<$ $\sim$ 10 by a factor of 2-3, possibly due to the dispersion in the survey window function of the Jackknife samples. For a real survey the observational condition also varies across the footprint. Therefore, with Jackknife errors, the significance of any improvement on systematics treatment will be conservatively assessed.} For both the mock and real datasets, we also utilize the cross power spectra between the galaxy density and various imaging maps to evaluate the performance of the mitigation. In order to estimate the significance of the contamination in $\hat{C}^{g,g}_{\ell}$ (or $\omega^{g,g} (\theta)$) before and after mitigation, we calculate $[\hat{C}^{g,s_k}_{\ell}]^2/\hat{C}^{s_k,s_k}_{\ell}$ (or $[\omega^{g,s_k}(\theta)]^2/\omega^{s_k,s_k}(\theta)$) as a proxy.~\footnote{These quantities would be the true level of contamination to $\hat{C}^{g,g}$ if the contamination model is linear and systematics are independent of one another~\citep{ashley2012MNRAS,2012ApJ...761...14H}.}
\begin{figure}
        \centering
        \includegraphics[width=0.4\textwidth]{figures/fig8-jackknifes.pdf}
        \caption{Twenty equal-area contiguous regions used to estimate the Jackknife errorbars for the 2D clustering statistics.}
        \label{fig:jackknifes}
\end{figure}

\subsection{Survey Mocks}\label{subsec:surveymocks}
Imaging systematics tend to affect the clustering signal mainly on large scales \citep{myers2007clustering, huterer2013calibration} and the distribution of galaxies on large scales at moderately low redshift can be well-approximated by a log-normal distribution~\citep{1991MNRAS.248....1C}. We therefore believe that log-normal mocks would be sufficient for the purpose of validating our systematic mitigation techniques. We use the \textit{Nbodykit} package \citep{hand2017nbodykit} to generate one hundred log-normal cubic mocks with the box-side length of $5274 \ihMpc$ and $1024^3$ mesh cells, with the input power spectrum matched to the linear power spectrum  at $z=0.85$ based on the \textit{Planck 2015} cosmology~\citep{ade2016planck} (i.e., flat $\Lambda$CDM with $\Omega_m=0.3089 \pm 0.0062$, $H_{0}=67.74 \pm 0.46$, $\sigma_8 = 0.8159 \pm 0.0086$), with the galaxy bias of 1.5 and the volume density of $1.947\mathrm{e}{-4}\trihMpc$ \citep[see e.g.][]{Raichoor2017MNRAS.471.3955R}. Then, we use the \textit{make\_survey} package \citep{white2013mock} to sub-sample the mock galaxies based on the NGC eBOSS ELG redshift distribution in \citet{Raichoor2017MNRAS.471.3955R} with the redshift cut of  0.55 $<$ z $<$ 1.5 and to transform the cubic mocks into survey-like mocks. We do not include redshift-space distortions (RSD) in the mocks as we believe that the systematics mitigation efficiency does not depend on  the presence of RSD.\\

The survey mocks are then projected onto the two-dimensional sky using HEALPix and overlaid on the NGC footprint of DR7 to be assigned with the DR7 imaging attributes. Fig. \ref{fig:mock_on_dr} illustrates the resulting projection of a simulated survey mock and DR7. Note that the mock footprint (89,672 pixels) is smaller than DR7 (187,257 pixels) almost by a factor of 2. We only use the pixels of the mock that have the DR7 imaging attributes available. The holes (e.g., RA and DEC around 200 and 5 deg) are the pixels that do not have the imaging attributes from the real data. In order to account for the mock survey footprint, we distribute 2,500 random points per deg$^{2}$ within the mock footprint and derive the completeness map for the mocks.

\begin{figure}
    \centering
    \includegraphics[width=0.4\textwidth]{figures/fig9-mockondr.pdf}
    \caption{The projection of the mock footprint (blue) onto the North Galactic Cap of the DR7 footprint (gray). With this projection, the imaging attributes from the real data are assigned to the mocks.}
    \label{fig:mock_on_dr}
\end{figure}

\subsubsection{Null Mocks}
Our goal is to develop a systematics treatment methodology that maximally removes the systematic effects while minimally removes the true cosmological signal. The two aspects may not be simultaneously accomplished, in a way that depends on the signal, noise, and the correlation between the imaging maps and the true galaxy density. In our paper we choose to prioritize losing minimal cosmological information over maximally removing systematics. One way of ensuring this is to check if the mitigation method returns the true clustering in the presence of contaminations, which will be tested using the contaminated mocks. Another way is to check if the mitigation method correctly makes a null operation on the clustering in the absence of contaminations, returning $\hat{\mathcal{F}}\simeq 1$. To this end, we utilize the 2D projected mocks without introducing any modulation due to imaging attributes in the galaxy density fields. Henceforth, we call this set of simulations, \textit{null} mocks.\\

Fig.~\ref{fig:ngal_hist} shows the pixel distribution histograms of the number of galaxies per pixel of the mocks in comparison to that of DR7 on the common footprint. The average $ngal=7.0/{\rm pixel}$ of the null mocks is smaller than $ngal=13.3/{\rm pixel}$ of the DR7 data (even after accounting for the 5\% loss due to tiling completeness and 27\% loss due to the redshift range, as stated in \citet{Raichoor2017MNRAS.471.3955R}). We believe that the difference in $ngal$ is due to the different \textit{clean photometry} criteria applied to the ELG selection in \citet{Raichoor2017MNRAS.471.3955R} and to the targets in this paper. The standard deviation of $ngal$ of the null mocks is 3.0, which is smaller than 4.6 of the DR7 ELGs.

\subsubsection{Contaminated Mocks}
We modulate the mock galaxy density fields using imaging attributes of DR7 and generate the contaminated mocks with additional random noise. The modulation is done based on the best fit coefficients of the imaging attributes and their covariances for $\mathcal{F}$ (Eq.~\ref{eq:nnbar}) that we derived from DR7 using our fiducial linear regression model.\\

\begin{figure}
    \centering
    \includegraphics[width=0.4\textwidth]{figures/fig10-ngalhist.pdf}
    \caption{Histogram of the number of galaxies per pixel for DR7 (hatched grey), null (solid blue), and contaminated (dashed red) mocks. All distributions are corrected for the pixel completeness $f_{\rm pix}$. The DR7 distribution is scaled down to account for 5\% tiling completeness, and 27\% to 0.55 < reliable redshift <1.5. The residual difference between the mocks and  DR7 shown here might be due to differences between the \textit{clean photometry} criteria applied to eBOSS target selection and those we apply to DR7.}
    \label{fig:ngal_hist}
\end{figure}

In detail, we pick the 10 imaging attributes, i.e., $EBV$, $nstar$, $lnHI$, $seeing-g$, $skymag-g$, $skymag-z$, $exptime-r$, $exptime-z$, $mjd-g$, $mjd-z$ of DR7, which were selected from the feature selection procedure on one of the partitions, and modulate the mock density field $n$ with $\mathcal{F}$ that is derived from random deviates of the imaging attribute coefficients while accounting for their covariances. Since the measured covariance of such quantities includes both the cosmological fluctuation and the fluctuations due to the imaging attributes, we rescale the measured covariance matrix of the systematics such that the random fluctuation in $ngal$ per pixel due to contamination is at a similar level to the cosmological fluctuation from the null mocks. As a result of the random fluctuation in the contamination model we introduced, some of the pixels will be assigned a negative galaxy number. We drop these pixels from our sample. This removes 3.1\% of the mock footprint, reducing our mock footprint size from 89,672 to 86,875 pixels. We then introduce the Poisson process, i.e., another random variation step, to ensure the modulated galaxy number per pixel is an integer. These two random variation processes increase the noise in the mock datasets such that the standard deviation of $ngal$ of the contaminated mocks ($=4.4$) is almost the same as that of DR7, despite the different average $ngal$ (see Fig.~\ref{fig:ngal_hist}).\\

Therefore our mock contamination is simpler than DR7 in that we adopted a linear model, which is chosen purposely since we do not want to give a priori advantage to our neural network method and also since all methods are capable of reproducing the linear model. Meanwhile, this setup is more challenging than the DR7 data since the mitigation is conducted in the presence of a greater level of noise. Note that, while we included only 10 dominant imaging attributes in the contamination, the remaining attributes in the DR7 data are correlated with these 10 attributes and therefore with the modulated galaxy density. All of the mitigation methods in the following mock test will be challenged to deal with such indirect correlations among the 18 attributes. Note that the effect of the footprint, i.e., the survey window effect, is the same for both the null mocks and contaminated mocks since we chose to apply the selection function on the galaxies while leaving the randoms intact. Therefore the null mocks serve as the baseline for estimating the level of systematics in the contaminated mocks.


\documentclass[english]{../thermomemo/thermomemo}
% NOTE: You must pass either norsk or english as an option!
\usepackage[utf8]{inputenc}

\title{Testing the Lee-Kesler equation of state implementation in ThermoPack}
\author{Morten Hammer}

\usepackage[normalem]{ulem}
\usepackage{hyperref}
\usepackage{color}
\usepackage{amsmath}
\usepackage{graphicx}
%\usepackage[norsk]{babel}
\usepackage{setspace}
\usepackage{float}
%\newcommand{\HRule}{\rule{\linewidth}{0.5mm}}
\numberwithin{equation}{section}
\usepackage{mathtools}
\usepackage{rotating}
\usepackage{verbatim}
\usepackage{subcaption,caption}
\usepackage{todonotes}
\usepackage{xspace}

\newcommand*{\pd}[2]{\frac{\partial #1}{\partial #2}}
\newcommand*{\pdd}[2]{\frac{\partial^2 #1}{\partial #2^2}}
\newcommand*{\pder}[2]{\left(\frac{\partial #1}{\partial #2}\right)}
\newcommand*{\pdder}[2]{\left(\frac{\partial^2 #1}{\partial
      #2^2}\right)}
\newcommand*{\pddder}[2]{\left(\frac{\partial^3 #1}{\partial #2^3}\right)}
\newcommand*{\pdcross}[3]{\left(\frac{\partial^2 #1}{\partial #2 \partial #3}\right)}
\newcommand*{\reff}[1]{(\ref{#1})}

\newcommand*{\coto}{\ensuremath{\text{CO}_{\text{\scriptsize
        2}}}\xspace}
\newcommand*{\nto}{\ensuremath{\text{N}_{\text{\scriptsize
        2}}}\xspace}

\definecolor{midnightblue}{RGB}{35,35,132}
\definecolor{urlblue}{RGB}{70,130,180}

\hypersetup{
    colorlinks=true,
    linkcolor=midnightblue,
    urlcolor=urlblue,
    citecolor=midnightblue,
    linktoc=page
}

\usepackage[activate={true,nocompatibility},final,kerning=true,tracking=true,spacing=true,stretch=10,shrink=10]{microtype}
\microtypecontext{spacing=nonfrench}
\SetExtraKerning[unit=space]
    {encoding={*}, family={bch}, series={*}, size={footnotesize,small,normalsize}}
    {\textendash={400,400}, % en-dash, add more space around it
     "28={ ,150}, % left bracket, add space from right
     "29={150, }, % right bracket, add space from left
     \textquotedblleft={ ,150}, % left quotation mark, space from right
     \textquotedblright={150, }} % right quotation mark, space from left
\SetTracking{encoding={*}, shape=sc}{0}

\graphicspath{{gfx/}}

\begin{document}
\frontmatter

\tableofcontents

\section{Introduction}
Modifications of the work by J\o rgen R\o ysland Aarnes on the
Lee-Kesler equation of state (EoS) \cite{LK}.

\section{The Lee-Kesler model}
The corresponding state method weigh the properties of two separate
equations of state as follows,
\begin{equation}
\label{eq:LK_main}
M = M^{(0)} + \frac{\omega_M}{\omega^{(r)}}(M^{(r)} - M^{(0)}),
\end{equation}
where, $\omega_M=\omega_M(\textbf{n})$. $M$ is $z$, $A^R$, $S^R$,
$H^R$ or any property that can be derived from these without using
mole number differentials.

The binary interaction parameters of Plöcker et al. \cite{PKP} is
implemented, and used for the testing of the Lee-Kelser EoS.

\subsection{Residual Gibbs energy}
The residual Gibbs energy, $G^R$, or the mixture fugacity coefficient,
$\ln \Phi_M$, for the reference and simple fluid was not included in
the memo by J{\o}rgen.
\begin{align}
\label{def:G^R(T,P,n)}
\ln \Phi_M = \frac{G^R(T,P,\textbf{n})}{RT} &= \frac{A^R(T,V,\textbf{n})}{RT} + n\left(z - 1 -
  \ln(z)\right)\\
\pder{\ln \Phi_M}{T}_{P,\textbf{n}} &= -\frac{H^R}{RT^2}\\
\pder{\ln \Phi_M}{P}_{T,\textbf{n}} &= \frac{V^R}{RT} = \frac{V -
  V^{ig}}{RT} = n\frac{\left(z-1\right)}{P}\\
  \ln \phi_i = \pder{\ln \Phi_M}{n_i}_{T,P} & = \frac{1}{RT} \pder{A^R}{n_i}_{T,V} - \ln z\\
\end{align}

\subsection{The fugacities and differentials}
Knowing the residual Gibbs energy, for the simple and reference fluid,
the combined fugacities can be derived. The mixture fugacity become
\begin{equation}
\label{eq:LK_fugacity}
\ln \Phi_M = \ln \Phi_M^{(0)} + \frac{\omega_M}{\omega^{(r)}}\left(\ln \Phi_M^{(r)} - \ln \Phi_M^{(0)}\right).
\end{equation}
The component fugacities become
\begin{equation}
\label{eq:LK_comp_fugacity}
\ln \phi_i = \ln \phi_i^{(0)} + \frac{\omega_M}{\omega^{(r)}}\left(\ln
\phi_i^{(r)} - \ln \phi_i^{(0)}\right) + \frac{1}{\omega^{(r)}}\pder{\omega_M}{n_i}\left(\ln \Phi_M^{(r)} - \ln \Phi_M^{(0)}\right).
\end{equation}
Further the differentials become
\begin{align}
\label{eq:LK_comp_fugacity_diff}
\pder{\ln \phi_i}{T}_{P} =& \pder{\ln \phi_i}{T}_{P}^{(0)} +
\frac{\omega_M}{\omega^{(r)}}\left(\pder{\ln \phi_i}{T}_{P}^{(r)} -
\pder{\ln \phi_i}{T}_{P}^{(0)}\right) \\
&+ \frac{1}{\omega^{(r)}}\pder{\omega_M}{n_i}\left(\pder{\ln
  \Phi_M}{T}_{P}^{(r)} - \pder{\ln \Phi_M}{T}_{P}^{(0)}\right), \\
\pder{\ln \phi_i}{P}_{T} =& \pder{\ln \phi_i}{P}_{T}^{(0)} +
\frac{\omega_M}{\omega^{(r)}}\left(\pder{\ln \phi_i}{P}_{T}^{(r)} -
\pder{\ln \phi_i}{P}_{T}^{(0)}\right) \\
&+ \frac{1}{\omega^{(r)}}\pder{\omega_M}{n_i}\left(\pder{\ln
  \Phi_M}{P}_{T}^{(r)} - \pder{\ln \Phi_M}{P}_{T}^{(0)}\right), \\
\ln \phi_{ij} =& \ln \phi_{ij}^{(0)} +
\frac{\omega_M}{\omega^{(r)}}\left(\ln \phi_{ij}^{(r)} - \ln
\phi_{ij}^{(0)}\right) + \frac{1}{\omega^{(r)}}\pder{\omega_M}{n_i}\left(\ln
\phi_j^{(r)} - \ln \phi_j^{(0)}\right)\\
& + \frac{1}{\omega^{(r)}}\pdcross{\omega_M}{n_i}{n_j}\left(\ln \Phi_M^{(r)} - \ln
\Phi_M^{(0)}\right). \\
\end{align}

\subsection{The residual entropy, enthalpy and differentials}
The combined residual entropy and enthalpy differentials will have the
same terms, so only the entropy and differentials are given. The
combined entropy is given as follows,
\begin{equation}
\label{eq:LK_entropy}
S^R(T,P,\textbf{n}) = S^R(T,P,\textbf{n})^{(0)} + \frac{\omega_M}{\omega^{(r)}}\left(S^R(T,P,\textbf{n})^{(r)} - S^R(T,P,\textbf{n})^{(0)}\right).
\end{equation}

To avoid too many superscripts, the residual, $R$, superscript is
dropped for the differentials of the entropy:
\begin{align}
\label{eq:LK_entropy_diff}
\pder{S}{T}_{P} &= \pder{S}{T}_{P}^{(0)} +
\frac{\omega_M}{\omega^{(r)}}\left(\pder{S}{T}_{P}^{(r)} -
\pder{S}{T}_{P}^{(0)}\right), \\
\pder{S}{P}_{T} &= \pder{S}{P}_{T}^{(0)} +
\frac{\omega_M}{\omega^{(r)}}\left(\pder{S}{P}_{T}^{(r)} -
\pder{S}{P}_{T}^{(0)}\right), \\
\pder{S}{n_i}_{T,P} &= \pder{S}{n_i}_{T,P}^{(0)} +
\frac{\omega_M}{\omega^{(r)}}\left(\pder{S}{n_i}_{T,P}^{(r)} -
\pder{S}{n_i}_{T,P}^{(0)}\right) +
\frac{1}{\omega^{(r)}}\pder{\omega_M}{n_i}\left(S^{(r)} - S^{(0)}\right)\\
\end{align}

\section{Modifications to the Lee-Kesler reduced specific volume solver}
Instead of solving directly for the reduced volume, $v_r(T_r,P_r)$, a solver
for the compressibility factor, $z$, is used. In order to determine the
compressibility factor, given $T_r$ and $P_r$ the following equation
must be solved,
\begin{equation}
\label{eq:LK_z_fun}
f(z) = z - \left[1 - \frac{v_r}{n} \left( \frac{\partial F}{\partial v_r} \right)_{T_r, \textbf{n}} \right] = 0,
\end{equation}
where $z=v_rP_r/T_r$.

To get good performance a third order method should be used. A third
order method require $f(z)$, $\partial f/ \partial z$, $\partial^2
f / \partial z^2$:
\begin{align}
\label{eq:LK_z_fun_diff}
\pder{f}{z} &= 1 + \frac{T_r}{P_r n} \left[\pder{F}{v_r}_{T_r, \textbf{n}} + v_r\pdder{F}{v_r}_{T_r, \textbf{n}}\right],\\
\pdder{f}{z} &= \frac{T_r^2}{P_r^2 n} \left[2\pdder{F}{v_r}_{T_r,
    \textbf{n}} + v_r \pddder{F}{v_r}_{T_r, \textbf{n}}\right].\\
\end{align}
In order to calculate these differentials, the third differential of
$F$ with respect to reduced specific volume, $\partial^3
F/ \partial v_r^3$, is required:
\begin{align}
\label{eq:LK_FDiff3vr}
\pddder{F}{v_r}_{T_r, \textbf{n}} =& n \bigg[-\frac{6B}{v_r^4} - \frac{12C}{v_r^5} - \frac{42D}{v_r^8} \\
  & - \frac{E}{v_r^5} \left(12\beta + \frac{(30 - 18\beta)\gamma}{v_r^2} + \frac{\left(4\beta-26\right)\gamma^2}{v_r^4} + \frac{4\gamma^3}{v_r^6} \right) \exp \left(-\frac{\gamma}{v_r^2} \right)\bigg]. \\
\end{align}

A Taylor expansion of $f$ is used to calculate the next step, $\Delta
z = z_{n+1} - z_n $, of the numerical procedure,
\begin{equation}
\label{eq:LK_z_fun_taylor}
f(z) = f(z_n) + f'(z_n)\Delta z + \frac{1}{2}f''(z_n)\Delta z = 0.
\end{equation}
The root giving the smallest change in z is selected, and the
series approximation,
\begin{equation}
\label{eq:LK_sqrt_series}
1 - \sqrt{1 - \alpha} = \frac{\alpha}{2} + \frac{\alpha^2}{8} + O(\alpha^3),
\end{equation}
is used to simplify the expression, and to avoid taking the square
root. We end up with the following step,
\begin{equation}
\label{eq:LK_z_step}
\Delta z = z_{n+1} - z_{n} = - \frac{f(z_{n})}{f'(z_{n})}\left(1+\frac{f(z_{n})f''(z_{n})}{f'(z_{n})^2}\right).
\end{equation}

\subsection{Selecting the correct roots of the simple and reference fluid}
In order to get physical values for the fugacities, the roots from
$f(z)$ for the simple and reference fluid must be selected
carefully. The simple fluid and reference fluid either have 1 or two
physical roots. If both the simple and reference fluid have a liquid
root, this is used. If both the simple and reference fluid have a gas
root, this is used. If the simple fluid have two roots, but the
reference fluid only have a liquid root, then the liquid root is
selected from both fluids, etc.

In some situations, the simple fluid only have a gas root, and the
reference fluid only have a liquid root, see Figure \ref{fig:roots}. In
this situation, these roots are combined.
\begin{figure}[h]
  \begin{subfigure}[b]{0.45\linewidth}
    \centering
    \includegraphics[width=1.0\textwidth]{roots}
    \caption{Reduced pressure vs. reduced specific volume}
    \label{fig:roots_a}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.45\linewidth}
    \centering
    \includegraphics[width=1.0\textwidth]{roots_zoom}
    \caption{Reduced pressure vs. reduced specific volume. Zoomed
      version of Figure \ref{fig:roots_a}}
    \label{fig:roots_b}
  \end{subfigure}
  \caption{Reduced pressure $P_r$ as a function of reduced specific
    volume, for $T_r = 0.9973..$. For $P_r = 0.9808..$ it is seen that the
    simple fluid has a gas root, while the reference fluid has a liquid root.}
  \label{fig:roots}
\end{figure}

\section{Testing}
The entropies and enthalpies are tested for consistency, by comparing
$G^R$ with $H^R-TS^R$. The ideal contribution is calculated using the
$Cp$ polinomials already implemented in ThermoPack.
\subsection{TP-flash using the Lee-Kesler EoS}
To test the Lee-Kesler implementation, a mixture of \coto and \nto is
used. The phase diagram, as calculated by the TP-flash, is shown in
Figure \ref{fig:co2n2}.
\begin{figure}[h]
  \centering
  \includegraphics[width=0.8\textwidth]{co2n2}
  \caption{Phase diagram of \coto-\nto mixture, as calculated by the TP-flash,
  with 95 mass \% \coto. The blue area is the two-phase gas-liquid area.}
  \label{fig:co2n2}
\end{figure}

\section{Further work}
The Lee-Kesler implementation need more testing, and it is also
interesting to see how the models performes for densities of high
pressure \coto-mixtures. Some further work is listed below.
\begin{itemize}
\item Further testing to ensure robustness of the TP-flash
\item Test HP, SP and UV flash using Lee-Kesler EoS
\item Add a function to solve the EoS, returning the minimum Gibbs
  single phase solution. Test how close the solutions, $z_1$ and
  $z_2$, are. Use,
  \begin{equation}
    \frac{|z_1-z_2|}{\text{max}(z_1,z_2)} < \epsilon,
  \end{equation}
  where $\epsilon$ is the tolerance for the compressibillity solver.
\end{itemize}

\begin{thebibliography}{9}

\bibitem{LK}
	Lee, B.I., Kesler, M.G., \textit{A Generalized thermodynamic correlation based on a three-parameter corresponding states}, AlChE Journal, 21 (3), 510 (1975)

\bibitem{PKP}
	Plöcker, U., Knapp, H., Prausnitz, J., \textit{Calculations of High-Pressure Vapor-Liquid Equilibria from a Corresponding-States Correlation with Emphasis on Asymmetric Mixtures}, Ind. Eng. Chem. Process Des. Dev, 17 (3) 324 (1978)

\end{thebibliography}
\end{document}


\documentclass[11pt]{article}

\setlength{\oddsidemargin}{-0.25 in}
\setlength{\evensidemargin}{-0.25 in}
\setlength{\topmargin}{-0.9 in}
\setlength{\textwidth}{7.0 in}
\setlength{\textheight}{9.0 in}
\setlength{\headsep}{0.75 in}
\setlength{\parindent}{0.3 in}
\setlength{\parskip}{0.1 in}
\usepackage{epsf}
\usepackage{pseudocode}
\usepackage{enumitem}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{color}
\usepackage[normalem]{ulem}
\usepackage{graphicx}
\usepackage[export]{adjustbox}
\pagenumbering{arabic}
\def\O{\mathop{\smash{O}}\nolimits}
\def\o{\mathop{\smash{o}}\nolimits}
\newcommand{\e}{{\rm e}}
\newcommand{\R}{{\bf R}}
\newcommand{\Z}{{\bf Z}}

%% display solutions or not
\newif\ifsol
\soltrue % comment out to hide solutions

%% todo tracker -- overleaf v2 has better one it uses but will default to this if compiled on something that doesn't have built-in todo
\newcommand{\todo}[1]{\textbf{\textcolor{red}{#1}}}

\title{Section 4: MDPs}
\author{CS 182 - Artificial Intelligence}
\date{}
\begin{document}
\maketitle

\renewcommand{\labelenumii}{\arabic{enumii}.}
\setlength{\parindent}{0pt}

Recall that a \textbf{Markov Decision Process} is defined by
\begin{itemize}
  \item $S$: the set of possible \underline{states}

  \item $A$: the set of possible \underline{actions}, which may vary
  from state to state

  \item $T(s, a, s') \rightarrow \mathbb{R}$:
  the \underline{transition function} is the probability
  that $a$ from $s$ leads to $s'$

  \item $R(s, a, s') \rightarrow \mathbb{R}$:
  the \underline{reward function}. This could also be
  listed as $R(s')$ or $R(s,a)$

  \item The \underline{starting state}

  \item And, potentially, a \underline{terminal state}
\end{itemize}

With MDPs, we want to find the optimal policy $\pi^*: S \rightarrow A$,
which maximizes the expected utility.

Since we need to maximize the expected utility, we need that utility to be finite. We can acomplish this in one of three ways:

\begin{enumerate}
  \item \underline{Choosing a finite horizon}: By terminating the game after $T$ steps the reward must be finite. This often generates nonstationary policies, as $\pi$ depends on time remaining.

  \item \underline{Discounting over an infinite horizon}: Multiplying rewards by the discount rate $0 < \gamma < 1$ every time a new step is made will cause the solution to converge to a finite return and thus a finite solution, where a smaller discount rate ($\gamma$) leads to a shorter term focus.
  $$U([r_0, \cdots, r_\infty]=\sum_{t=0}^\infty \gamma^tr_t$$

  \item \underline{Have an absorbing state}: This will guarantee that for every policy a terminal state will eventually be reached.
\end{enumerate}

We define a \textbf{Q-state} $Q(s, a)$ as the expected utility of having taken action $a$ from state $s$ and acting optimally afterward:
$$Q^*(s,a) = \sum_{s'}T(s, a, s')[R(s, a, s') + \gamma V^*(s')]$$

And the \textbf{Value} of a state $V(s)$ as the expected utility of starting in $s$ and acting optimally. This can also be thought of as taking the best action $a$ at Q-state $Q(s,a)$:
$$V^*(s) = \max_a \sum_{s'}T(s, a, s')[R(s, a, s') + \gamma V^*(s')] = \max_a Q^*(s, a)$$

\textbf{Value Iteration} is a way to compute the optimal value for every state. It is done by first initializing the values for each state to 0:
$$V_0(S) = 0$$
Then the $k+1$ iteration will take the vector of $V_k(s)$ for all states $s$ and make a one step (Bellman) update:
$$V_{k+1}(s) = \max_a \sum_{s'} T(s, a, s') [R(s,a,s') + \gamma V_k(s')]$$

Unfortunately, this process is quite slow, as while the optimal action at each state (and thus the optimal policy) rarely changes, the optimal values can take many iterations to finally converge. Furthermore, each iteration is $O(S^2A)$ and thus the dimensionality and time complexity of the problem grows quite quickly with the size of the state or action space.

\textbf{Policy Iteration} attempts to overcome these issues by iteratively alternating between \textbf{Policy Evaluation} (updating the values) and \textbf{Policy Improvement} (updating the policy). This both reduces time complexity of the algorithm and allows it to exit earlier once the policy converges.

\textbf{Step 1: Policy Evaluation:} Starting with $V_k^{\pi_k}(s')$ and iterating (over $i$), update the values under a fixed policy $\pi_k$ until convergence:
$$V^{\pi_k}_{i+1}(s) = \sum_{s'} T(s, \pi_k(s), s') [R(s,\pi_k(s),s') + \gamma V_i^{\pi_k}(s')]$$
Call the converged values $V^{\pi_k}_{k+1}(s)$, and use them to update the policy:

\textbf{Step 2: Policy Improvement:} Extract a better policy based on these new values:
$$\pi_{k+1}(s) = \text{arg max}_a \sum_{s'} T(s, a, s') [R(s,a,s')+ \gamma V^{\pi_k}_{k+1}(s')] = \text{arg max}_a Q_{k+1}(s,a)$$

Note that Policy Evaluation is $O(S^2)$ operation and that both steps of Policy Iteration are easier to extract from the Q-values than the Values themselves.

\section*{Exercises}

\begin{enumerate}

\item For the environment shown below, calculate which squares can be reached from $(1,1)$ by the action sequence $[Up,Up,Right,Right]$ and with what probabilities. In this transition model, the probability of going in the intended direction is 0.8, but the agent travels in right angles to the intended direction with probability 0.1 each. A collision with a wall results in no movement.

\ifsol
  \includegraphics[scale=0.55, center]{img/actions_solution}
\else
  \bigskip
  \includegraphics[scale=0.9, center]{img/environment}
\fi

\item Pacman is stuck in a friendlier maze where he gets a reward of $1$ every time he takes an action from state (0,0).  This setup is a bit
different from the one you have seen before:  Pacman can get the reward multiple times; these rewards do not get ``used up'' like food pellets and there are no ``living rewards.'' As usual, Pacman cannot move through walls and may take any of the following actions: go North ($\uparrow$), South ($\downarrow$), East ($\rightarrow$), West ($\leftarrow$), or stay in place ($\circ$).  State (0,0) gives a total reward of 1 every time Pacman takes an action in that state regardless of the outcome, and all other states give no reward. Actions are deterministic: each action will take Pacman in the correct direction unless Pacman runs into a wall, in which case he will remain in the same square.

\begin{enumerate}[label=\alph*)]
\item Assume a finite horizon of $h=10$ (so Pacman takes exactly 10 steps) and no discounting ($\gamma=1$).

\ifsol
  \includegraphics[scale=0.55, center]{img/maze_solutions}
\else
  \includegraphics[scale=0.55, center]{img/maze}
\fi

\item Fill in following Q-values (with 10 steps remaining) corresponding to the value function you specified above.

The Q value of state-action (0,0), (East) is:

\ifsol
  \bigskip
  ${\color{blue} Q_{10}(S,A) = R(S,A) + V_9(S') = 1 + 8 = 9}$
  \bigskip
\else
  \bigskip \bigskip \bigskip
\fi

The Q value of state-action (1,1), (East) is:

\ifsol
  \bigskip
  ${\color{blue} Q_{10}(S,A) = R(S,A) + V_9(S') = 0 + 4 = 4}$
  \bigskip
\else
  \bigskip \bigskip \bigskip
\fi

\item Now assume an infinite horizon, with $\gamma = 0.9$. Find the value of state (0,0):

\ifsol
  \bigskip
  {\color{blue}
    Since the only reward is in state (0,0) if we start in state (0,0) we would choose to stay there forever. Therefore we will accumulate the following reward: $\sum_{i=1}^{\infty} 0.9^i*1$. This is an infinite geometric series with a ratio $<1$ and therefore the closed form solution is: $V = V_0 / (1-\gamma) = 1 / (1-0.9) = 10$.}
    \bigskip
\else
  \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
\fi

\item Now let $\gamma = 1$.
Also, now at every time step, after Pacman takes an action and collects his reward, a power outage could suddenly end the game with probability $\alpha = 0.1$. Find the value of state (0,0):

\ifsol
  \bigskip
  {\color{blue}
    Again we would like to stay in state (0,0) forever. However, there is a chance $\alpha$ that the game ends on each turn therefore we expect the game to go for $1/\alpha$ rounds where we get reward $1$. $V = 1/\alpha = 10$. \\

    Another approach to this problem would calculate the value by doing an infinite geometric sum $1 + (1 - \alpha) + (1 - \alpha^2) + ... = 10$, essentially treating the probability of ending the game as a discount.}
    \bigskip
\else
  \bigskip \bigskip
\fi

\end{enumerate}

\clearpage
\item Consider an undiscounted MDP having three states, (1, 2, 3), with rewards $-1, -2, 0,$ for ending a turn in that state respectively. State 3 is a terminal state. In states 1 and 2 there are two possible actions: $A$ and $B$. The transition model is as follows:
\begin{itemize}
  \item In state 1, action $A$ moves the agent to state 2 with probability 0.8 and makes the agent stay put with probability 0.2.
  \item  In state 2, action $A$ moves the agent to state 1 with probability 0.8 and makes the agent stay put with probability 0.2.
  \item In either state 1 or state 2, action $B$ moves the agent to state 3 with probability 0.1 and makes the agent stay put with probability 0.9.
\end{itemize}

\begin{enumerate}[label=\alph*)]
  \item Without running policy iteration, what can be determined about the optimal policy in states 1 and 2? Which actions minimize the reward lost while staying in states 1 and 2 while maximizing the probability of reaching the terminal state?

  \ifsol
    \bigskip
    {\color{blue} Intuitively, the agent wants to get to state 3 as soon as possible, because it will pay a cost for each time step it spends in states 1 and 2. However, the only action that reaches state 3 (action b) succeeds with low probability, so the agent should minimize the cost it incurs while trying to reach the terminal state. This suggests that the agent should definitely try action b in state 1; in state 2, it might be better to try action a to get to state 1 (which is the better place to wait for admission to state 3), rather than aiming directly for state 3. The decision in state 2 involves a numerical tradeoff.}
    \bigskip
  \else
    \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
    \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip \bigskip
  \fi

  \item Apply policy iteration, showing each step in full, to determine the optimal policy and the values of states 1 and 2. Assume that the initial policy has action $B$ in both states and initial values of $0$ for all states and $\gamma=1$. \\

  \ifsol
    {\color{blue}
        \textbf{Initialization} \\
        Values are V(State 1) = 0, V(State 2) = 0, V(State 3) = 0. \\
        Policy is $\pi(\text{State 1}) = B$, $\pi(\text{State 2}) = B$. \\

        \textbf{Iteration 1} \\
        Calculate new values by following current policy (remember to iterate until convergence):
        \begin{itemize}
            \item V(State 1) = $0.9 (-1 + 1 * 0) + 0.1 (0 + 1 * 0) = -0.9$  \\
                  V(State 2) = $0.9 (-2 + 1 * 0) + 0.1 (0 + 1 * 0) = -1.8$  \\
                  V(State 3) = $0$
            \item V(State 1) = $0.9 (-1 + 1 * -0.9) + 0.1 (0 + 1 * 0) = -1.71$  \\
                  V(State 2) = $0.9 (-2 + 1 * -1.8) + 0.1 (0 + 1 * 0) = -3.42$  \\
                  V(State 3) = $0$
            \item In general we would have to keep going until this converges. However, in this case we can take a shortcut by solving a system of equations since we know that V(State 3) = $0$ always and the reward is also $0$ in that state. This simplifies the other states to:\\
            V(State 1$)_\text{k+1}$ = $0.9 (-1 + $V(State 1$)_\text{k}$) \;\;\;\;
            V(State 2$)_\text{k+1}$ = $0.9 (-2 + $V(State 2$)_\text{k}$)\\
            and at convergence we need $V(\cdot)_{k+1} = V(\cdot)_{k}$. Therefore, we can solve this system of equations and get our values: \\

            V(State 1) = -9, V(State 2) = -18, V(State 3) = 0.
        \end{itemize}

        Then update the policy by doing a one-step lookahead, with the values that were just computed: \\
        Q(State 1, Action A) = $0.2 (-1 + 1 * (-9)) + 0.8 (-2 + 1 * (-18)) = -18$ \\
        Q(State 1, Action B) = $0.9 (-1 + 1 * (-9)) + 0.1 (0 + 1 * 0) = -9$  \\
        Q(State 2, Action A) = $0.2 (-2 + 1 * (-18)) + 0.8 (-1 + 1 * (-9)) = -12$  \\
        Q(State 2, Action B) = $0.9 (-2 + 1 * (-18)) + 0.1 (0 + 1 * 0) = -18$  \\

        Policy update for this iteration: $\pi(\text{State 1}) = B$, $\pi(\text{State 2}) = A$. \\

        \textbf{Iteration 2} \\
        We then update the values again jumping straight to the system of equations: \\
        For V(State 1) we can note that the first iteration (computed above as Q(State 1, Action B)) has the value remaining the same at $-9$ we can then also treat that as a constant in the State 2 equation. \\
        V(State 2) = $0.2 (-2 + 1 * $V(State 2)$) + 0.8 (-1 + 1 * (-9))$  \\

        V(State 1) = -9, V(State 2) = -10.5, V(State 3) = 0.\\

        Then update the policy by doing a one-step lookahead, with the values that were just computed: \\
        Q(State 1, Action A) = $0.2 (-1 + 1 * (-9)) + 0.8 (-2 + 1 * (-10.5)) = -12$ \\
        Q(State 1, Action B) = $0.9 (-1 + 1 * (-9)) + 0.1 (0 + 1 * 0) = -9$  \\
        Q(State 2, Action A) = $0.2 (-2 + 1 * (-10.5)) + 0.8 (-1 + 1 * (-9)) = -10.5$  \\
        Q(State 2, Action B) = $0.9 (-2 + 1 * (-10.5)) + 0.1 (0 + 1 * 0) = -11.25$  \\

        Policy update for this iteration: $\pi(\text{State 1}) = B$, $\pi(\text{State 2}) = A$. \\

        \textbf{Iteration 3} \\
        Algorithm terminates because policy has converged (it has not changed between iterations). Note that the value in State 1 has converged but the value in State 2 has \underline{not} yet converged.\\

        \textbf{Side Note:} While the standard Policy Iteration requires that the Policy Evaluation step to run to convergence in each iteration, generalized version of Policy Iteration that only run $k$ steps of Policy Evaluation in each iteration will actually still converge (although they often will follow a different path to convergence). In fact, $k = 1$ generalized Policy Iteration is equivalent to Value Iteration!
    }
  \fi

\end{enumerate}
\end{enumerate}
\clearpage
\end{document}


\documentclass{tufte-handout}

\title{Recitation 1, CIS-2166}

\author{David Dobor}

\date{January 15, 2016} % without \date command, current date is supplied

%\geometry{showframe} % display margins for debugging page layout

\usepackage{graphicx} % allow embedded images
  \setkeys{Gin}{width=\linewidth,totalheight=\textheight,keepaspectratio}
  \graphicspath{{graphics/}} % set of paths to search for images
\usepackage{amsmath}  % extended mathematics
\usepackage{booktabs} % book-quality tables
\usepackage{units}    % non-stacked fractions and better unit spacing
\usepackage{multicol} % multiple column layout facilities
\usepackage{lipsum}   % filler text
\usepackage{fancyvrb} % extended verbatim environments
  \fvset{fontsize=\normalsize}% default font size for fancy-verbatim environments

% Standardize command font styles and environments
\newcommand{\doccmd}[1]{\texttt{\textbackslash#1}}% command name -- adds backslash automatically
\newcommand{\docopt}[1]{\ensuremath{\langle}\textrm{\textit{#1}}\ensuremath{\rangle}}% optional command argument
\newcommand{\docarg}[1]{\textrm{\textit{#1}}}% (required) command argument
\newcommand{\docenv}[1]{\textsf{#1}}% environment name
\newcommand{\docpkg}[1]{\texttt{#1}}% package name
\newcommand{\doccls}[1]{\texttt{#1}}% document class name
\newcommand{\docclsopt}[1]{\texttt{#1}}% document class option name
\newenvironment{docspec}{\begin{quote}\noindent}{\end{quote}}% command specification environment

\usepackage{listings}
\usepackage{color}

\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

\lstset{frame=tb,
  language=matlab,
  aboveskip=3mm,
  belowskip=3mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\small\ttfamily},
  numbers=none,
  numberstyle=\tiny\color{gray},
  keywordstyle=\color{blue},
  commentstyle=\color{dkgreen},
  stringstyle=\color{mauve},
  breaklines=true,
  breakatwhitespace=true,
  tabsize=3
}

\begin{document}

\maketitle% this prints the handout title, author, and date

\begin{abstract}
\noindent
Section 3.2 of Rosen's textbook contains an in-depth discussion of the asymptotic notation.
This note is intended to complement the discussion in the book and may be useful
to ease your understanding of that section. Some example problems and
their solutions, similar to the ones you'll see on your homework, can
be found at the end of this note. We will see more examples in following recitations.\end{abstract}

\section{Why use Big-O?}

Suppose you're writing a program that manages inventory for some nation-wide chain
of stores and that every night your job is to process the changes that occured in the inventory
during that day.
Suppose it takes about $10, 000  \ ms$ for your program to read in the data from the disk and about
$10 \ ms$ to process each item in the inventory. If you have $n$ items in the inventory then
the running time of your program may be represented by:
$$
T(n) = 10, 000 + 10 n
$$

For large $n$, the run-time of your program will be "dominated" by the second term: the time to
read in the data (10 000 $ms$) will become negligible compared to the time it takes to process a
large amount of inventory items.

In asymptotic analysis we want to abstract away from $T(n)$ the details that do not matter
in the long run, like the time it takes to read in the data in this example, or the time it takes to process
an individual item. These will vary depending on whether you are running your program on a modern super-computer
or on a desktop from the 80-s that is still stashed away in your parents' garage. We just want to draw some general
conclusions about the running time of the program you wrote.

To help reason about $T(n)$, we introduce another
function, $f(n)$, that is simpler than $T(n)$.  We use $f(n)$ for an upper
bound on $T(n)$. This $f(n)$ will enable us to draw conclusions about how fast the inventory processing
algorithm runs, without letting the details get in the way.

\bigskip
Here's a definition of Big-O that is slightly less formal than what the book has:

\bigskip
\newthought{Definition:} We say that $T(n)$ is Big-O of $f(n)$ and write $T(n) \in O(f(n))$ if and only if
$$
T(n) \leq C f(n) \text{ \ whenever } n \text{ is big for some large constant } C
$$

\bigskip
Well, but what is "big $n$"? Answer: big enough to have the first function, $T(n)$, "fit below" the second one, $f(n)$.
We'll look at an example in a second. We'll give this "big enough $n$" a name - we'll call it $k$, as the textbook does.
And what is "large enough $C$"? The answer is the same. \emph{And} we get to choose $C$.

\bigskip
That is, we get to choose the constants $C$ and $k$ such that
$$
T(n) \leq C f(n) \ \ \ \text{ whenever } \ n > k
$$

\bigskip
If we can identify such $C$ and $k$, then we say that $T(n) \in O(f(n))$; Otherwise, if we can prove that no such $C$ and $k$ exist, then we say that $T(n) \not\in O(f(n))$

\bigskip
Whenever you see $T(n) \in O(f(n))$, it may be helpful to think: "$T(n)$ is smaller than $f(n)$" or that
"$f(n)$ dominates $T(n)$".

\bigskip
(\emph{As an aside}, note that we do not usually write that $T(n) = O(f(n))$. We
instead think of $f(n)$ as being a member - or being "like" other members - of a \emph{set} of functions
denoted by $O(f(n))$.
However, people often use the "=" sign, and that is something to be aware of. We'll stick with the "$\in$" sign in our discussions.)

\section{Examples}

\newthought{Example $0$:} Consider comparing $T(n) = 10 + 2n$ to
$f(n) = n$. We want to see if $f(n)$ will dominate $T(n)$ for large enough $n$. That is, will
the plot of $f(n)$ be above the plot of $T(n)$ if
we were to plot the two functions? In fact, let's plot them.

Let's set the constant $C$ in the above definition to $1$ and see what the plots look like.
If you are inclined to use Matlab (not required for this course), here's how to do it.

\bigskip

\begin{lstlisting}
n = linspace(0,20,100);
T = 10 + 2*n;
f = 1*n;

figure(1)
plot(n,T, 'linewidth', 2, 'color', 'g')
xlim([0 20])
ylim([0 50])
hold on
plot(n,f, 'linewidth', 2, 'color', 'b')
legend('T(n) = 10 + 2 n', 'f(n) = n', 'location', 'northwest')
\end{lstlisting}

\begin{figure}
  \includegraphics{BigO1.png}
%  \checkparity This is an \pageparity\ page.%
  \caption{Here $T(n)$ dominates $f(n)$ everywhere, meaning that the plot of $f(n)$ fits entirely under the plot of $T(n)$.}
  \label{fig:textfig}
  %\zsavepos{pos:textfig}
  \setfloatalignment{b}
\end{figure}

\pagebreak
At first it may appear to you that since $f(n)$ is below $T(n)$, we can't really claim that $T(n) \in O(f(n))$.

But recall from the definition that we get to choose $C$. In the above figure, $C$ is $1$.
Now consider setting $C$ to something like 5. We then can have $f(n)$ dominate $T(n)$.

\bigskip

 \begin{lstlisting}
f = 5*n;
plot(n,T, 'linewidth', 2, 'color', 'g')
xlim([0 20])
ylim([0 50])
hold on
plot(n,f, 'linewidth', 2, 'color', 'b')
legend('T(n) = 10 + 2 n', 'f(n) = 5 n', 'location', 'northwest')
\end{lstlisting}

\begin{figure}
  \includegraphics{BigO2.png}
%  \checkparity This is an \pageparity\ page.%
  \caption{Here $f(n)$ begins to dominate $T(n)$ after a certain point.}
  \label{fig:textfig}
  %\zsavepos{pos:textfig}
  \setfloatalignment{b}
\end{figure}

\bigskip
So now we see that after a certain point $f$ begins to dominate $T$ instead (what is that
point?). Now it should be clearer how $f$ and $T$ in this example fit the Big-O definition.

\bigskip
Now consider other examples similar to what you will find in your homework. The trick to solving
these problems is to explicitly identify the constants $C$ and $k$ that fit the above definition.
We rely on the mathematical fact that $\log(n) \leq n$ for $n \geq 1$.

\newthought{Example 1:}

$5 n^2 + 3 n \log n + 2 n + 5 $ \ is  \ $ O(n^2)$

\newthought{Justification:}  $5 n^2 + 3 n \log n + 2 n + 5 \leq (5 + 3 + 2 + 5) n^2  = C n^2 $ for
$C = 15$ when $n \geq k = 1$

\newthought{Example 2:}

$20 n^3 + 10 n \log n + 5 $ \ is  \ $ O(n^3)$

\newthought{Justification:}  $5 n^2 + 3 n \log n + 2 n + 5 \leq 35 n^2$ for $n \geq k = 1$

\newthought{Example 3:}

$3n \log n + 5 $ \ is  \ $ O(\log n)$

\newthought{Justification:}  $3n \log n + 5  \leq 8 \log n$ for $n \geq n_0 = 2$. Note that
$\log n$ is zero for $n = 1$. That is why we use $k = 2$ here.

\newthought{Example 4:}

$2^{n+2}$ \ is  \ $O(2^n)$

\newthought{Justification:} $2^{n+2} = 2^n \cdot 2^2 = 4 \cdot 2^n$, so we can take $C = 4$ and $k = 1$ in this case.

\newthought{Example 5:}

$2 n + 100 \log n$ \ is  \ $O(n)$

\newthought{Justification:} $2 n + 100 \log n \leq 102 n$ for $n \geq 1$,  so we can take $C = 102$ and $k = 1$ in this case.

\end{document}

\emph{Big-O} notation is used to represent upper bounds on functions.
We think of Big-O notation as an abstract relation between functions. A function
may be the worst case running time of a program, but it doesn't have to be.

Computer & Information Science Core courses/2166/1-Jan15/BigO.tex