%
%   results
%
\section{Results}\label{sec:results}
In this section, we present the measurements of the clustering statistics before and after correcting for the systematic effects for the real dataset as well as the simulated ones. We demonstrate that the neural network is capable of learning more structure in the observed galaxy density field due to its greater flexibility beyond a fixed functional form, and therefore it can eliminate more excess clustering which is believed to be due to the imaging systematics. We then show the performance of the neural network and multivariate linear models when applied to the mock datasets.

\subsection{Mitigating systematics from DR7}
\label{sec:mitigateDR7}
In the left panel of Fig. \ref{fig:weights}, we show the pixel distribution histograms of the selection masks from the three different regression models we consider in this paper. While all three models show fairly consistent selection masks for most of the pixels (note the logarithmic scaling of $Npix$), the neural network method (solid red curve) returns extended tails due to a higher representation flexibility associated with its nonlinear nature. We remove pixels with $\hat{\mathcal{F}} < 0.5$ or $> 2.0$ from our data to avoid too aggressive selection correction since we believe none of these methods can be accurate enough for such a long baseline extrapolation. These pixels account for 1.0\% of the original data (from 187,257 to 185,781 pixels). In the right panel, we show the spatial distribution of the removed pixels in the case of the neural network selection mask. \\

\begin{figure*}
    \centering
    \includegraphics[width=\textwidth]{figures/fig11-weights.pdf}
    \caption{The pixel distribution  of  the  selection  masks for DR7. \textit{Left}: Distribution of the selection masks (i.e., estimates of the contamination model) derived from different regression models. \textit{Right}: Spatial scatter of the pixels we remove from our data due to the extreme values of the neural network selection mask.}
    \label{fig:weights}
\end{figure*}

Fig. \ref{fig:density_selection} illustrates the spatial distribution of the observed galaxy density before (top left) and after correction (top right) using the neural network selection mask. The bottom panels show the neural network selection mask used for the correction (left) in comparison to the masks derived from the linear (middle) and the quadratic polynomial (right) models. All three masks capture a very similar large scale pattern such as the decrease in the galaxy density close to the Galactic midplane, which is consistent with the negative correlation coefficients between the galaxy density and the Galactic extinction, hydrogen column density, or stellar density. On smaller angular scales, the three selection masks show different fluctuation details. In the following analyses, we examine which method returns the least contaminated galaxy density distribution. \\

\begin{figure*}
    \centering
    \includegraphics[width=\textwidth]{figures/fig12-density_selection.pdf}
    \caption{\textit{Top}: The normalized observed galaxy density of  DR7 before and after systematics treatment using the neural network selection mask, respectively, from left to right. \textit{Bottom}: the selection masks from the Neural Network, quadratic, and linear polynomial models, respectively from left to right. All three selection masks are able to capture the behavior that the galaxy density systematically drops at the footprint boundaries i.e., high extinction regions.}
    \label{fig:density_selection}
\end{figure*}

First, once the systematic effects are corrected for, the mean galaxy density should be independent of the imaging attributes. In Fig. \ref{fig:nnbar} we show the mean number density of galaxies as a function of the imaging attributes. Again, different bins are set to include the same effective pixel area and therefore have the same sampling error. The solid black curve shows the galaxy mean density before correction and the solid red shows the result after correction with the selection mask of the neural network model. The dot-dashed curve represents the correction using the linear polynomial model and the dashed curve is for the quadratic polynomial model. The errorbars are computed using 20 Jackknife sub-samples and shown on only one case for clarity. Similar to what we found from the feature selection procedure, the stellar density, Galactic extinction, and HI density exhibit the strongest dependence before correction. After correction, all three methods return the fractional galaxy density close to unity. To quantify the deviation from unity, we report the $\chi^{2}$ statistics in Tab.~\ref{tab:chi2} while ignoring the covariance between the different bins and different imaging attributes. Overall, the neural network achieves the smallest deviation from unity which indicates its highest efficiency in reducing the systematic effects. Ideally we would like to have residual contamination less than the statistical error. Figure \ref{fig:nnbar} and Table \ref{tab:chi2} implies that we need to further improve the mitigation techniques for future cosmological analyses. In Section \ref{subsec:discussion} we provide a more detailed analysis using the same $\chi^{2}$ statistics and the mocks to quantify the remaining systematics and assess whether or not the data is clean enough.\\

\begin{table}
    \centering
    \caption[Caption]{The $\chi^{2}$ values for the measured mean density of the DR7 galaxies vs imaging systematics, presented in Fig. \ref{fig:nnbar}. This table presents the cumulative values over all bins and all imaging attributes (i.e., $N_{\rm bins}=$20 bins $\times$ 18 attributes) without accounting for the covariance both between the imaging maps and between different bins\protect\footnotemark.}
    \label{tab:chi2}
    \begin{tabular}{lccr} % four columns, alignment for each
        %\hline
        \hline
        Correction scheme & $\chi^{2}$ & $N_{\rm bins}$ & $\chi^{2}/N_{\rm bins}$ \\
        \hline
        None & 20921.633 & 360 & 58.116\\
        Linear & 2588.349 & 360 & 7.190\\
        Quadratic & 2623.006 & 360 & 7.286 \\
        Neural Network & 966.601 & 360 & 2.685\\
        %\hline
        %\hline
    \end{tabular}
\end{table}

\begin{figure*}
\centering
\includegraphics[width=0.79\textwidth]{figures/fig13-nnbardr7.pdf}
\caption{The mean number density of the DR7 galaxies as a function of the potential systematics. The solid black curve shows the result before mitigation (\textit{no correction}); the solid red curve is for the result after correcting with the neural network selection mask; the dot-dashed and dashed black curves represent mitigations with the linear and quadratic polynomial selection masks, respectively. The error bars are estimated using the Jackknife resampling of 20 non-contiguous subsamples of pixels within each imaging attribute bin (a total of 20 bins per attribute) and are shown only for one case. This plot again shows that the Galactic foregrounds such as the stellar density introduce a systematic trend in the galaxy density, which indicates a significant contamination by our own galaxy before mitigation. Such systematic trends are mostly removed with any of the three mitigation methods. \label{fig:nnbar}}
\end{figure*}

We next evaluate the performance of different mitigation techniques using the two-point statistics. We first show the cross power spectra between the DR7 observed galaxy density and various imaging attributes in the form of  $[\hat{C}^{g,s_k}_{\ell}]^2/\hat{C}^{s_k,s_k}_{\ell}$ in Fig. \ref{fig:clcross}. Again, this quantity approximately represents the level of contamination from each attribute to the auto power spectrum of galaxy density and we therefore compare this with the uncertainty in the auto as well as cross power spectrum of galaxies (light and dark gray shades) which are estimated using the Jackknife resampling of 20 equal-area contiguous regions (see Fig. \ref{fig:jackknifes}). Similarly, we plot $[\omega^{g,s_k}(\theta)]^2/\omega^{s_k,s_k}(\theta)$ in Fig.~\ref{fig:xicross} to assess the contamination in the auto-correlation function. Fig. \ref{fig:clcross} and \ref{fig:xicross} show significant contamination on large scales from $ebv$, $lnHI$, and $nstar$ compared to the statistical fluctuation estimated from the Jackknife subsampling of the data. The stellar density map shows the highest cross power spectrum with the galaxy density map, which is in agreement with the previous results. Qualitatively, all three mitigation techniques perform well and substantially reduce the cross power below $\ell \sim 30$ and over all separation scales in the cross-correlation function. The neural network method shows a slightly lower cross-power, but this appears to be merely related to the lower amplitude of the corresponding auto galaxy power spectrum compared to the other two cases. We note the spurious peak in the cross-correlation against exptime-z in Fig.~\ref{fig:xicross} near the expected angular location of the BAO feature and such feature necessitates thorough investigations of imaging systematics in analyzing the auto clustering statistics of the spectroscopic data for BAO analysis.\\

\begin{figure*}
\centering
\includegraphics[width=0.78\textwidth]{figures/fig14-dr7crosscl.pdf}
\caption{The cross power spectrum $\hat{C}^{g,s_k}_{\ell}$ between the DR7 observed galaxy density and the imaging attributes $s_k$ normalized by the auto power spectrum of the imaging attribute $\hat{C}^{s_k,s_k}_{\ell}$. The plotted quantity $[\hat{C}^{g,s_k}_{\ell}]^2/\hat{C}^{s_k,s_k}_{\ell}$ approximately represents the level of contamination to the auto power spectrum of the galaxy density $\hat{C}^{g,g}_{\ell}$. The light and dark grey shaded regions, respectively, show the Jackknife error estimate of $\hat{C}^{g,g}_{\ell}$ and $[\hat{C}^{g,s_k}_{\ell}]^2/\hat{C}^{s_k,s_k}_{\ell}$ with the galaxy density $g$ before mitigation. The black solid curve shows the result before mitigation (\textit{no correction}), while the solid red curve shows the result after correcting for the systematics with the neural network selection mask. The dot-dashed and dashed black curves show the corrected results with the linear and quadratic polynomial model selection masks, respectively. \label{fig:clcross}}
\end{figure*}

\begin{figure*}
\centering
\includegraphics[width=0.79\textwidth]{figures/fig15-dr7crossxi.pdf}
\caption{The cross correlation function $\omega^{g,s_k}(\theta)$ between the DR7 observed galaxy density and the imaging attributes $s_k$ normalized by the auto correlation function of the imaging attribute $\omega^{s_k,s_k}(\theta)$. The plotted quantity $[\omega^{g,s_k}(\theta)]^2/\omega^{s_k,s_k}(\theta)$ approximately represents the level of contamination to the auto correlation function of the galaxy density $\omega^{g,g}(\theta)$.  The grey shaded region shows the Jackknife error estimate of $\omega^{g,g}(\theta)$  before mitigation. All mitigation techniques are able to reduce the excess clustering signal which is due to the imaging systematics.  \label{fig:xicross}}
\end{figure*}

\begin{figure*}
\centering
\includegraphics[width=\textwidth]{figures/fig16-dr7clxi.pdf}
\caption{Two-point clustering statistics for DR7. \textit{Left}: the measured angular power spectrum without shot-noise subtraction. \textit{Right}: the HEALPix-based angular correlation function. Solid black curves show the measured statistics without correcting for the systematic effects (\textit{no correction}). The dashed and dot-dashed black curves show the statistics after correcting with linear and quadratic polynomial mitigation methods, respectively. The solid red curves show the results after correcting with our default neural network method. The dashed blue curves show the results mitigated with the neural network method but without the feature selection process. The errors are estimated using the Jackknife resampling with 20 contiguous sub-regions and are shown only for a few cases for clarity (see Fig. \ref{fig:jackknifes}).} \label{fig:clxi}
\end{figure*}

We finally present the effect of the imaging attributes before and after mitigation on the auto galaxy clustering statistics. In Fig. \ref{fig:clxi}, we illustrate the measured two-point clustering statistics for DR7; the measured angular power spectrum without shot-noise subtraction is shown in the left panel, and the HEALPix-based angular correlation function is shown in the right panel. The solid black curve shows the measured clustering before mitigation, while the corrected measurements using the traditional linear, quadratic polynomial, and the default neural network models are shown respectively with the black dot-dashed, black dashed, and solid red curves. In the right panel, the linear and the quadratic polynomial mitigation results are indistinguishable and overlaid.\\

% for table 4
\footnotetext{Note that the best fit neural network model was applied to the unseen data (i.e., the test set) unlike in the linear and quadratic polynomial models. Nevertheless the neural network method returns the smallest $\chi^{2}$, i.e., the highest efficiency.}

The comparison between the clustering before correction (solid black curves) and  after treatment (solid red, blue, dashed, and dot-dashed curves) suggests that the imaging systematics affect the clustering measurements mostly on large scales, e.g., large separation angles or small multipoles, as expected~\citep[see e.g.,][]{myers2007clustering, ross2007higher, huterer2013calibration}. We find that all of the mitigation methods are able to reduce such large scale contamination, while there still remains substantial excess clustering on large scales, mostly, with the two traditional linear multivariate methods. The neural network method is much more efficient in reducing such excess. When we investigate the effect of the survey window function on this data, we find that the window effect at $\ell$ < 50 is expected to be less than 5\% (more details presented in Appendix \ref{app:windowfunction}, see Fig. \ref{fig:Cellwindowratio}).\\

In comparison to our default neural network model, we also show the measurements mitigated with the neural network model without the feature selection process labeled as `plain' (blue dashed curves), which is very similar to the default case. In the next section, we test the mitigation methods using the mock datasets for which we know the true clustering signals. As will be demonstrated, our default neural network model with the feature selection process is chosen based on this mock test.

\subsection{Testing the mitigation methods on the mock data}
We treat the mocks as if the contamination model was unknown and apply the mitigation pipeline on the mocks as exactly used for the real dataset. After modeling the selection mask for each mock, we remove the pixels whose selection masks values are < 0.5 or > 2.0. This reduces the mock footprint size from 86,875 to 86,867 pixels. Again, the mocks do not include the redshift-space distortions.

\subsubsection{Feature selection of mock galaxies}
Fig.~\ref{fig:mockablation} shows the distribution of the imaging attributes selected by the feature selection process for all of the five partitions of the 100 null (left) and contaminated (right) mocks. For the null mocks, there is no contamination and the feature selection correctly removes most or all of the imaging attributes, as demonstrated by the sparse distribution of the points in the left panel. The imaging attributes that survived feature selection, probably due to a coincidental correlation with the galaxy density, are randomly distributed. On the other hand, the right panel shows that the feature selection procedure correctly identifies most of the input contamination attributes (marked by `*' on the y-axis) for the contaminated mocks and almost always selects $lnHI$ and $nstar$. Indeed, as shown in Fig.~\ref{fig:dr7ablation}, these two attributes were the two most significant input contamination.

\begin{figure*}
    \centering
    \includegraphics[width=\textwidth]{figures/fig17-mockablation.pdf}
    \caption{Important imaging maps identified in the mocks by the feature selection procedure for the five partitions of the 100 null (left) and contaminated mocks (right). The maps used in the input contamination model were marked by `*'. The right panel shows that the feature selection procedure has identified EBV, Stellar density, skymag-g, seeing-g as important in most of the contaminated realizations whereas in the left panel, no map is consistently selected as important among all of the 100 null mocks.}
    \label{fig:mockablation}
\end{figure*}

\subsubsection{Mean mock galaxy density}
In Fig.~\ref{fig:nnbarmock} we show the number density of mock galaxies, averaged over the 100 mocks, as a function of the imaging attributes. As expected, the galaxy density of the contaminated mocks shows strong or moderate dependence on $ebv$, $nstar$, $lnHI$, $seeing-g$, $skymag-g$, $skymag-z$, $exptime-r$, $exptime-z$, $mjd-g$, and $mjd-z$ which were indeed the inputs to the contamination model. Meanwhile, the galaxy density also shows strong dependencies on $mjd-r$, $depth-r$, $depth-g$, and $depth-z$ through the correlation between these and the input contamination attributes. Looking at this result alone from a real data perspective, one would not be able to single out the underlying imaging attributes that are directly responsible for the contamination. When the inputs to the mitigation procedure include all of the input contamination maps, Fig.~\ref{fig:nnbarmock} shows that all methods effectively remove the dependence. In subsection \ref{subsec:discussion}, we discuss further how well the underlying true mean density is recovered after mitigation.

\begin{figure*}
    \centering
    \includegraphics[width=0.79\textwidth]{figures/fig18-mocksnnbar.pdf}
    \caption{The number density of the mock galaxies as a function of the potential systematics averaged over the 100 mock datasets. The grey shaded region illustrates 1$-\sigma$ dispersion in the null mocks. The dotted curve shows the mean density of the null mocks. The black solid curve shows the mean density dependence on the imaging attributes for the contaminated mocks. The solid red curve shows the mean density after correcting for the systematics with the neural network selection mask. The dashed black curve shows the corrected results with the quadratic polynomial model selection mask. The result of the linear polynomial model is almost unity, and therefore is omitted for clarity.}
    \label{fig:nnbarmock}
\end{figure*}

\subsubsection{Angular power spectrum of mock galaxies}

Fig. \ref{fig:deltaclmock} shows the mean angular power spectrum of the 100 null and 100 contaminated mocks in the left and right panel, respectively, in the top row. In the middle row, we illustrate the remaining bias\footnote{Due to the two-step noise introduced during contamination, the shot noise of the contaminated power spectra is increased by almost a factor of two. We estimate the total offset noise to be around $3.05\times10^{-6}$ from $\sigma^2(ngal)/\overline{n}^2$ and subtract it from the power spectrum of the contaminated mocks. The additional shot noise is mostly originated from the Poisson process we applied, i.e., precisely 1/$\overline{n}$, where $\overline{n}$ is the average number density after contamination, while an extra $\sim$ 10\% is also due to the noise we added to the contamination model.} on clustering after mitigation as an offset from the true power spectrum (i.e., the null power spectrum from the left panel\footnote{Note that we use the clustering of the null mocks before mitigation as the ground truth model since the survey window for all of the mocks before and after systematics treatment does not change.}). One can see that the contamination substantially increased power at $\ell < 50$. Since the contamination model is based on the linear polynomial model, all three fitting methods, i.e., the linear, quadratic, and neural network, are capable of reproducing the true input contamination, while they perform differently in the presence of the two layers of noise we added and the eight additional imaging attributes that are non-trivially correlated with the ten input contamination attributes. \\
\begin{figure*}
\centering
\includegraphics[width=\textwidth]{fig19-mockscell.pdf}
\caption{\textit{Top row}: The mean angular power spectrum of the 100 contaminated (null) mocks in the right (left) panel. \textit{Middle row}: The mean power spectrum subtracted by the mean of the null mocks to better visualize the remaining bias after each mitigation. The dark grey shaded region shows the 1$-\sigma$ confidence region of the mean of 100 mocks, while the light grey area shows the 1$-\sigma$ confidence region for one mock, calculated from the dispersion of 100 mocks. To account for the increased shot noise during contamination, we remove the same constant power from all contaminated/mitigated power spectra until their small scale power matches that of the null mock power spectrum. We quantify the significance of the remaining bias by calculating $\chi^2$, the sum of the squared offset weighted with the diagonal variance of the mean $C_\ell$ of null mocks over the last six bins ($\ell~\geq 12$). The middle panel on the left illustrates the neural network without feature selection (`plain') tends to remove the cosmological clustering signal. \label{fig:deltaclmock}}
\end{figure*}

In the right top and middle panel, we find all three methods effectively remove the contamination over $\ell < 100$; in detail, the linear (black dot-dashed) and the quadratic polynomial (black dashed) methods slightly over-correct power while the neural network method (solid red) slightly under-corrects it. Note that, without the feature selection process (dashed blue), the neural network method would also over-correct the large scale power like the linear and quadratic polynomial models. The left top and middle panel show that, in the absence of contamination, both the linear and quadratic polynomial methods over-correct the large scale power since the fitting methods can always find purely coincidental consistency between the imaging attributes and the cosmic variance. The quadratic method that has a greater freedom appears more prone to such problem. On the other hand, the neural network method without the feature selection process shows a lesser degree of overfitting than the linear methods, probably due to the validation procedure. Our default neural network method, which incorporates feature selection, is the most robust mitigation methodology against overfitting.\\

\begin{figure*}
\centering
\includegraphics[width=\textwidth]{figures/fig20-chi2clmock.pdf}
\caption{Dependence of $\chi^{2}$ on the lowest bin $\ell_{\rm min}$ in the null (left) and contaminated (right) mocks. To better quantify the residual bias introduced by each method, we evaluate the dependence of the bias on the lowest bin $\ell_{{\rm min}}$ that is included in the $\chi^{2}$ computation. The default neural network method performs significantly better than the conventional methods for the null mocks, mainly because the feature selection procedure successfully prevents the method from regressing out the cosmological clustering signal. For the contaminated mocks, all methods tend to perform similarly, as expected, since all mitigation methods can reproduce the input contamination model. }\label{fig:chi2clmock}
\end{figure*}

The remaining bias can be compared to the typical error expected for such data. The dark and light grey shaded regions in the middle panels indicate the 1-$\sigma$ confidence regions for the mean and the individual mock of the 100 mocks, respectively.  We quantify the significance of such remaining bias by calculating $\chi^2$, the sum of the squared offset weighted with the diagonal variance for the mean at each $\ell$ bin. Note that we use the variance from the 100 null mocks for calculating $\chi^2$ of the contaminated mocks in order to avoid an advantage of the increased variance after contamination. We find that the default neural network with $\chi^{2}=0.74$ (reduced $\chi^{2} = 0.12 $ with $dof=6$) recovers the true underlying clustering when applied to the null mocks well within $1\sigma$ C.L. of the sample variance. We estimate the significance with taking the residual systematics as one extra degree of freedom,
\begin{equation}\label{eq:chi2sigma}
 \text{max systematics} \sim \sqrt{\chi^{2}}
\end{equation}
On the other hand, the linear and quadratic models have systematic biases with more than $4\sigma$ and $7.4\sigma$ significance. When applied to the contaminated mocks, we find that the neural network method returns$\chi^2=15.3$ from six $\ell$ bins ($\ell \geq 12$), while the linear and quadratic methods, respectively, polynomial mitigation return $27.2$ and $86.9$. The difference in $\chi^2$ among the different mitigation methods is not significant compared to the $\chi^2=17,466.8$ before mitigation. While the neural network model appears to perform the best, $\chi2$ of $15.3$ (reduced $\chi^2$ of $2.6$) indicates that the residual is much more significant than the sample variance. However, for the contaminated mocks, we added substantial statistical noises, almost doubling the noise. If we use the covariance of each contaminated/mitigated case, we get $\chi^2$ of $7.7$ (reduced $\chi2$ of $1.3$) for our default case; that is our residual is at the level of the statistical noise we added in the process of contamination. These $\chi^{2}$ values can be translated to $2.8-3.9\sigma$ uncertainties in the residual systematics (see Eq. \ref{eq:chi2sigma}) which implies any particular cosmological study requires a more thorough analysis of systematics to determine an estimate of the residual systematic uncertainty for the parameters of interest.\\

Such $\chi^2$ can depend on the lower limit of $\ell$ we consider. In Fig. \ref{fig:chi2clmock}, we investigate the behavior of the remaining bias depending on $\ell_{min}$, which shows that the neural network method consistently returns a lower remaining bias for various $\ell_{\rm min}$ choices. However, for the contaminated mocks, the difference is small and we conclude that all mitigation methods perform similarly for the contaminated mocks.\\

The bottom panels of Fig.~\ref{fig:deltaclmock} compare the noise introduced by the three different mitigation processes. The right panel shows that the contamination process (solid black) introduces additional noise on large scales relative to the variance of the null mocks (the gray shade). After mitigation, the variance is reduced, which is probably related to the decrease in the large scale power, since the variance of power is proportional to the amplitude of power itself in the Gaussian limit. The quadratic method shows the smallest fractional error on large scales due to the reduced amplitude after correction. In all cases, if we calculate the fractional variance with respect to the measured $C_\ell$ (e.g., $\sigma C_{\ell, NN}/C_{\ell, NN}$ instead of  $\sigma C_{\ell, NN}/C_{\ell, Null}$), it agrees with the fractional variance of the null mock (gray shade). Therefore, we do not observe a nontrivial increase in variance by any of the mitigation methods we tested.

\subsubsection{Cross power spectrum of mock galaxies and imaging attributes}
In Fig. \ref{fig:clcrossmock}, we show the mean cross power spectrum of the 100 mock catalogs and the imaging attributes for the different mitigation techniques. All three methods substantially reduce the cross power with the imaging attributes. The neural network method tends to show a small residual for $\ell <10$ that is greater than those of the linear and quadratic polynomial models. The dark shaded region shows the 1$-\sigma$ confidence region of the mean cross power propagated to $C_{s,g}^{2}/C_{s,s}$ and the light shaded region shows the 1$\sigma$ confidence interval of the mean auto-power spectrum of the mocks as shown in Fig. \ref{fig:clcross}. Therefore, we find that these residuals are greater than the statistical noise of $C_{s,g}^{2}/C_{s,s}$, but  the effect on the \textit{auto power spectrum} are marginal for $\ell~ >~ 10$. This excess on small $\ell$ is partly due to the greater auto galaxy power spectrum amplitude (in Fig.~\ref{fig:deltaclmock}) after mitigation than those by the other methods. Meanwhile we still find a residual correlation with skymag-z and mjd-z even after accounting for the effect of the auto power spectrum amplitude. Without the feature selection procedure (dashed blue), the neural network also returns a smaller residual. In essence, we see that once feature selection is applied, the neural network only corrects to a certain level, controlled by the specifics of the feature selection procedure. This protects against over-fitting due to random correlations between the imaging attributes and the galaxy density field.\\

\begin{figure*}
\centering
\includegraphics[width=0.78\textwidth]{figures/fig21-clcrossmock.pdf}
\caption{The mean cross power spectrum of the contaminated mock catalogs and the imaging attributes for different mitigation techniques. Our default neural network method with feature selection is shown in solid red while the performance without feature selection (`Neural Network (Plain)') is shown in dashed blue. The dark grey shaded region shows the 1$\sigma$ confidence region of the plotted quantity derived from $2\sigma(C_{g,s})*C_{g,s}/C_{s,s}$, and the light grey region shows the typical 1-$\sigma$ confidence region of the mean auto power spectrum of the 100 mocks. The mitigation with the ground truth contamination model is shown with a purple dot-dashed curve as `cont. with linear (truth)'. \label{fig:clcrossmock}}
\end{figure*}

As a sanity check, if we use the true input linear contamination model to mitigate the systematics (purple dot-dashed line in Fig. \ref{fig:clcrossmock}), the cross-correlation completely vanishes as expected\footnote{The auto-power spectrum of the contaminated mocks mitigated with the true input contamination model (in Fig. \ref{fig:mockdclextra}) returns the smallest residual bias relative to the uncontaminated clustering, as expected.}. For the null mocks, all mitigation methods return negligible cross-correlation, which is omitted from Fig. \ref{fig:clcrossmock} for clarity.\\

Overall, while the cross-correlation statistics between the galaxy density and the systematics attributes are a useful indicator for the level of contamination, we find it may be difficult to infer and discriminate the level of contamination in the density field from such cross-correlation statistics beyond what can be probed by the auto power spectrum.

\subsubsection{A case with underfitting}
It is possible that we may identify only a subset of the contamination attributes for a given data set and attempt to mitigate the contamination based on such limited information. We consider a situation where we input only five imaging attributes to the mitigation procedure: four from the true contamination inputs, i.e., $EBV$, $lnHI$, $nstar$, $skymag-g$ and one that is not among the true contamination inputs, but correlated with the contamination inputs, i.e., $depth-r$. The neural network method could be more resilient to such limited information since its nonlinear activation function may allow the mitigation procedure to better utilize the correlation between different input imaging attributes. In Fig.~\ref{fig:mockdclextra}, the linear polynomial method with `few' inputs shows a lesser degree of overfitting for the null mocks, compared to the default linear case, while showing under-fitting for the contaminated mocks. This is expected as the freedom of the linear model is now limited.  The neural network method with the `few' inputs (without feature selection) returns a very similar pattern as the linear `few' case, which implies that our current default neural network method does not have an advantage over the linear model in such a case despite its greater flexibility. This underfitting case may be worth future investigation, as apparent excess clustering remains in DR7 in Fig.~\ref{fig:clxi} when any method is applied, but especially in the case of the application of multivariate linear models.

\begin{figure*}
\centering
\includegraphics[width=\textwidth]{figures/fig22-mockscellex.pdf}
\caption{Same as Fig. \ref{fig:deltaclmock} showing the mean auto power spectrum of the 100 null (left) and contaminated (right) mocks mitigated with the fewer imaging maps `few' (to demonstrate under-correction), neural network without the feature selection `plain', and the ground truth contamination model `truth'. The mitigation with the ground truth model achieves the lowest residual bias as expected. For the null mocks, using fewer imaging maps prevents over-fitting simply by providing less freedom in the regression model while it leads to underfitting for the contaminated mocks. }
\label{fig:mockdclextra}
\end{figure*}

\subsection{Summary and Discussion}\label{subsec:discussion}
In summary, we find that our default neural network method is more robust against overfitting based on the test with the null mocks. This is due to the feature selection process that appropriately reduces the flexibility of mitigation. Based on the tests with the contaminated mocks, we find that both the linear and neural network methods perform equally well in terms of the residual bias, while the neural network method is more robust against overfitting. The quadratic polynomial method appears to be more prone to the overfitting problem than the other two methods since it has a greater flexibility than the input contamination model, but without a way to suppress the flexibility. All methods do not increase fractional variance during the mitigation process. Note again that we deliberately choose the linear model in contaminating the mocks in this test in order to prevent a disadvantage in using the linear and quadratic polynomial methods.  Therefore, the decent performance of the linear method is warranted. In real data, the contamination due to observational effects can take a more complex form as implied by the difference in the mitigation results between the data and our mocks. Therefore our mock test is a conservative estimation of the comparative mitigation capability of our default neural network method. \\

While we demonstrated qualitatively and quantitatively that our fiducial neural network method is more robust than the conventional methods, for both DR7 as well as for the mocks, Fig. \ref{fig:nnbar}-\ref{fig:clcross} show non-negligible residual contamination compared to the expectations. It is not surprising that the mitigation of imaging systematic effects in the real data is more challenging than that of the linear-model based systematics in our mock tests. In this subsection we attempt to provide a quantitative evaluation of the residual systematics of DR7.\\

We quantify the residual systematics in the mean density against the 18 imaging maps using $\chi^2$ of the mean density diagnostic as in Fig. \ref{fig:nnbar} and Table \ref{tab:chi2}~\footnote{To compare these with the mock results, we limit DR7 to the mock footprint, and therefore the $\chi^2$ results are slightly different from Fig. \ref{fig:nnbar} and Table \ref{tab:chi2}. The mock footprint is smaller than the data footprint by almost a factor of two.}. The null hypothesis is that the total residual squared error of the mean density observed in the data should be consistent with the distribution of the $\chi^{2}$ statistics constructed with the null mocks. The vertical lines in Fig. \ref{fig:chi2pdf} present the $\chi^{2}$ values observed in the data for before systematics treatment (9567.1) and after treatment with linear (2066.7), quadratic (1212.0), default Neural Network (767.3), and  Neural Network plain (744.4) approaches. As a comparison, we present the distributions of the $\chi^{2}$ observed in the null mocks (solid), contaminated mocks (dashed), and contaminated mocks after neural network mitigation (dot-dashed). Fig. \ref{fig:chi2pdf} illustrates a factor of 12 improvement in terms of the residual $\chi^2$ when using the neural network. Compared to the conventional quadratic method ($\chi^2=1212$), the NN-based method makes a factor of 1.6 improvement.\\

\begin{figure}
    \centering
    \includegraphics[width=0.45\textwidth]{figures/fig23-chi2_data_on_mock.pdf}
    \caption{\textit{Left}: $\chi^{2}$ distribution for the null mocks (solid black), contaminated mocks (dashed black), and contaminated mocks with NN mitigation (dot-dashed black). The vertical dotted lines overlay the $\chi^{2}$ values for the data with the default NN (red), quadratic (purple), and linear (grey) treatments. The $\chi^{2}$ statistics before treatment is shown on the right (dark blue).}
    \label{fig:chi2pdf}
\end{figure}

The mean and standard deviation for the distribution of $\chi^{2}$  values observed in the null mocks are $487.1 \pm 4.92$, respectively. The same quantities observed in the contaminated mocks are $2819.0 \pm 29.30$, while Neural Network mitigation decreases these statistics to the mean of $472.0\pm 6.59$. We perform Welch's t-test \citep{welch1947generalization} on the $\chi^{2}$ distributions of the null mocks and the mitigated contaminated mocks, and conclude that the two distributions have identical means with $t-statistic=1.8$, $p-value=0.06$ and the neural network recovers the underlying cosmological mean target density. Note that the mean $\chi^{2}$ observed in the contaminated mocks, 2819.0, is much smaller than the value observed in the data (the blue vertical line, 9567.1). This indicates that the systematic effects rigorously simulated in the mocks are still not as strong as the systematics in the real data or the effect of the non-diagonal terms, that we ignored in the covariance, is different in the data and in the mocks.\\

We perform a hypothesis testing given the $\chi^{2}$ value observed in the data after the neural network mitigation and the distribution of $\chi^{2}$ values observed in the 100 null mocks.
If the imaging maps were independent and normal deviates, the $\chi^{2}$ values must follow a Chi-squared distribution for $dof=360$ where 360 is the total number of bins. This assumption is not necessarily satisfied given the correlation among imaging systematics, and we indeed find $\chi^{2}$ of the null mocks is better fit with $dof=487$. Therefore, we assume that the underlying $dof$ for evaluating $\chi^2$ is 487. For DR7, the $\chi^2$ decreased from 9567 before contamination to 767.3 after our NN-based mitigation. Compared to $dof=487$ we expect for the approximate truth, this is a substantial excess, being very unlikely due to random fluctuation: for example, the top 5\% of distribution with $dof=487$ is at $\chi^2 = 539.4$, which is 40\% lower than 767.3. Assuming a mere random fluctuation is bounded at this upper 5\% and the $\chi^{2}$ values of the data are consistent with that of the mocks, this offset could imply that we underestimated the covariance at least by $\sqrt{(767.3/539.4)} = 19\%$ or that our model, when applied to DR7, missed at least 19\% of the systematic effects in the target density.\\

We further investigate the contribution of each imaging map to the residual systematics in DR7. Fig. \ref{fig:chi2breakdown} presents the $\chi^{2}$ vs each imaging map after the linear (grey), quadratic (purple), neural network with feature selection (red), and neural network plain (blue) treatments. The statistics before mitigation are not shown for clarity. We also plot the 50- and 95-th percentiles of the same quantity observed in the null mocks in orange horizontal curves. Depth-g seems to be the main source of the residual systematics which is not mitigated even by the neural network-based methods. All methods also show residual systematics against skymag-g, skymag-z, and MJD-z. The standard treatments show some additional residual systematics against E(B-V), lnHI, and exptime-z. We perform further tests by masking out high extinction and low depth-g regions. We find that applying a more rigorous cut on depth-g (e.g., depth-g $> 24.95$) yields a more cleaner mean density at the expense of losing 9\% of the data. Although our objective was to avoid applying rigorous cuts on imaging and demonstrate the gain by using non-linear methods, our tests suggest that careful masking based on imaging, particularly depth-g, seems necessary for cosmological studies. \\

In summary, our results indicate that any cosmological study requires a thorough analysis of systematics to determine an estimate of the residual systematic uncertainty for the parameters of interest. In order to use this sample for a cosmological exploitation, we would need to account for 19\%  (or more, depending on the assumption on the baseline) additional systematic errors to the statistical errors in the density field level. Our analysis also suggests that additional masking, especially based on depth-g, or improving the method to deal with depth and sky background issues would be the next steps to prepare this data for cosmological analysis. We leave this for future study, as the focus of this study is to compare our non-linear, neural network method to other map-based methods.

\begin{figure}
    \centering
    \includegraphics[width=0.45\textwidth]{figures/fig24-chi2_data_on_mock_breakdown.pdf}
    \caption{The breakdown of $\chi^{2}$ values observed in the data on the mock footprint after linear (grey), quadratic (purple), neural network with feature selection (red), and neural network plain (blue) treatments. We also plot the 50- and 95-th percentiles of the same quantity observed in the null mocks in orange curves (note that $\chi^{2}(\textbf{s)}$ is not a continuous quantity). Given the 5\% threshold, we can argue that there exists known residual systematics against E(B-V), depth-g, skymag-g, skymag-z, and MJD-z.}
    \label{fig:chi2breakdown}
\end{figure}


\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{ dsfont }

\usepackage{ amssymb }
\usepackage{amsfonts}
\usepackage{algorithm}
\usepackage{amsmath}
\usepackage{algpseudocode}
\usepackage{graphicx}
\usepackage{tikz} %Рисование автоматов
\usetikzlibrary{automata,positioning,shapes,snakes,arrows}
\usetikzlibrary{arrows}
\usepackage[english,russian]{babel}
\usepackage{hyperref}
\usepackage{ upgreek }
\usepackage{colortbl}

\definecolor{urlcolor}{HTML}{799B03}

\hypersetup{pdfstartview=FitH,  urlcolor=urlcolor, colorlinks=true}
\title{Lab work 7\\ 4D-video dimensionality reduction}

\date{}
\author{Pankratov Viktor}
\begin{document}
\maketitle
\selectlanguage{english}
\tableofcontents
\newpage
\section{Lab 7 report}

\subsection{Motivation}
Data in computer vision is often characterised by plotting into less-dimensional subspaces using algorithms such as PCA or PLS. Every element is supposed to be vector, not tensor or even 2-dimensional matrix. However, one of the data types is video and there exist videos in even more than 3-dimensional space. To be able to project such videos to more comfortable less-dimensional spaces HOPCA is proposed.
\subsection{Problem statement}
Given a sequence of images $\{\upchi_i\}_{i=1}^{T}$, every image is a 4-dimensional tensor containing integer values. Thus video is a 5-dimensional tensor with first four axes coordinates representing the position of pixel in 4d-space and the fifth representing time.
$$\upchi \in R^{D_1 \times D_2 \times D_3 \times D_4} \times T $$ The goal is to make a compact video representation by reducing its dimensions.
\subsection{Solution}
It is supposed to use PCA with tensor notation. PCA is a method which involves SVD of the data matrix $X$.
$$X = USV^T = U^{(1)}SU^{(2)} = S \times_1 U^{(1)} \times_2 U^{(2)}$$
Then the basic vectors are columns of $U^{(1)}$ and the coordinates in that basic are defined by $SU^{(2)^T}$ \\
HOPCA is a direct PCA extension to multilinear case. In particular that means that HOPCA with vector samples is equivalent to PCA. Otherwise tensor $\upchi$ is decomposed:
$$\upchi = S \times_1 U^{(1)} \times_2 \dots \times_n U^{(n)} $$
As in SVD, HOSVD keeps first columns for each $U^{(i)}$ to produce $\hat{U}^{(i)}$and input tensor $Z$ is projected to a lower dimensional tensor as
$$Y = Z \times \hat{U}^{(1)^T} \times \dots$$
\subsection{Experimental results}
For experimental simplicity every image tensor contains boolean values. Simple sequences of images were generated, one of projections of which onto 3-dimensional space are shown below.For multidimensional case they are mixed into one bicolor image:one of them is responsible for blue color and one of them for green.\\
\includegraphics[scale=0.8]{figures/1.png}
\includegraphics[scale=0.8]{figures/2.png}
\\
Cube transformation using using PCA.\\
\includegraphics[scale=0.8]{figures/4.png}\\
That experiment was just for demonstration how the 4-dimensional cube will look like after PCA transformation.

\newpage
Bicolor image transformation using HOPCA.\\
\includegraphics[scale=0.8]{figures/3.png}\\
During the generation process the original cube coordinates were taken mod 15 where 15 is the image one coordinate size. A cube dividing into several areas was expected. As we can see, there are 3 colors: blue, green and black.
\par
Considering binary images one of the blue and green colors has to disappear. After PCA one of their color values is strictly less than
background and one of them is strictly more. Either blue or green color would be combined with background.
\par
Considering gray-scale images blue and green cubes remain. Still there are black cubes which meaning is intersection of previous two. And after transformation their color values are almost equal to background. That means that in this example after projection any information about intersection is lost and HOPCA might be not fit for the problem.
\subsection{Useful links}
The MPCA implementation was taken from the following \href{https://www.mathworks.com/matlabcentral/fileexchange/26168-multilinear-principal-component-analysis-mpca?s_tid=mwa_osa_a}{link}. Other parts of the code are available on \href{https://github.com/PankratovViktor/MMP_Lab1}{Github}
\begin{itemize}
\item[1]{Tucker Tensor Decomposition on FPGA.
Kaiqi Zhang, Xiyuan Zhang, Zheng Zhang}
\item[2]{J. Yang, D. Zhang, A. F. Frangi, and J. Yang. Twodimensional pca: a new approach to appearance-based face
representation and recognition}
\item[3]{Multilinear Principal Component Analysis of Tensor Objects for Recognition.
Haiping Lu, K.N. Plataniotis and A.N. Venetsanopoulos
}
\end{itemize}
\end{document}


\section{Planning and Learning with Tabular Methods}

\subsection{Models and Planning}
A \emph{model} of the environment is anything that an agent can use to predict how the environment will respond to its actions. A \emph{distribution model} is one that characterises the distribution of possible environmental changes, whereas a \emph{sample model} is one that produces sample behaviour. Distribution models are in some sense stronger, in that they can be used to produce samples of the behaviour of the environment, but it is often easier to reproduce sample responses than to model the response distribution.\\

Models can be used to simulate the environment and hence simulate experience. We use the term \emph{planning} to refer to a computational process that uses a model for improving a policy. The kind of planning that we consider here falls under the name \emph{state-space planning}, since it is a search through the state space for an optimal policy or path to a goal. (Planning as we consider it here is essentially just learning from simulated experience.)

\subsection{Dyna: Integrated Planning, Acting and Learning}
Within a planning agent, real experience can be used to improve the model or to directly improve the value function and policy. The former we call \emph{model learning} and the latter we call \emph{direct reinforcement learning}. The use of a learned model to improve the value function and policy is sometimes called \emph{indirect reinforcement learning}. The figure below illustrates this duality.\\
\includegraphics[width=\textwidth]{\NotesImages/dual_purpose_of_real_experience.png}\\

\subsubsection*{Dyna-Q}
Dyna-Q uses one-step tabular Q-learning to learn from both real and simulated experience. (It is typical to use the same update rule for both types of experience.) The idea is that a model and a value function are learned simultaneously from real experience, and the model is then used for further planning. An algorithm is given below. Note that, although not shown in this way, the planning and direct learning can run concurrently.\\

\includegraphics[width=\textwidth]{\NotesImages/tabular_dyna_q.png}\\

\subsection{When the Model is Wrong}
Of course, the model we are learning may be incorrect, meaning that planning results in a sub-optimal policy. If the environment's dynamics are non-stationary, then this will be an issue for the agent. \\

In some cases, the suboptimal policy results in discovery and correction of model error, since if the model leads to optimistic estimates for action-values the agent will take these actions and realise its modelling error. The situation can be more difficult when values are underestimated, since in this case the agent may never choose to have experience that would correct its model.\\

\subsubsection*{Dyna-Q+}
The aforementioned issue of model error, especially in non-stationary environments, is the general problem of exploration versus exploitation. There is probably no solution that is both perfect and practical, but simple heuristics are often effective.\\

The Dyna-Q+ agent keeps track of the time elapsed since it last visited each state-action pair, then increases the reward from visiting these pairs in \emph{simulated} experience to $r + \kappa \sqrt{\tau}$, where $r$ is the modeled reward for the transition, $\tau$ is the number of time-steps since the last time the state-action pair was visited and $\kappa$ is a small constant. This increases computational complexity, but has the benefit of encouraging the agent to try actions that it hasn't taken in a long time.

\subsection{Prioritised Sweeping}
In the Dyna-Q algorithm given above, planning is done using uniform sweeps of the state-action space. This could be very wasteful, for instance because it is possible that there are many parts of the state-action space that are irrelevant to the optimal policies. It is also the case that uniformly distributed planning updates could waste effort on states whose value functions have not changed recently, which is wasted computation. \\

\emph{Prioritised sweeping} focuses updates on the state-action pairs whose estimated values are likely to change the most from the most recent experience. Q queue is maintained of every state-action pair whose estimated value would change nontrivially if updated, prioritised by the size of the change. During planning, the state-action pair that is first in the queue is updated and removed from the queue first, then it's predecessors are updated and removed (if the update would be significant), and so on. An algorithm for deterministic environments is given below.\\

\includegraphics[width=\textwidth]{\NotesImages/prioritised_sweeping.png}\\

\subsection{Expected vs. Sample Updates}
This section is about the relative benefits of expected and sample updates. Expected updates consider all possible outcomes, while sample updates use only sample experience of particular outcomes. In the absence of a distribution model, expected updates are not possible. A summary of all one-step updates considered are given below.\\

\includegraphics[width=\textwidth]{\NotesImages/summary_of_updates.png}\\

Since expected updates do not directly suffer from sampling error (error could propagate through model estimation in planning), they are more computationally intensive. However, they are not always optimal. In problems with large state-spaces or branching factors, sample updates are often much more efficient. This means that one can do many sample updates in the same computational time as an expected update, in turn meaning that the sample updates produce more accurate value estimates in the given time.

\subsection{Trajectory Sampling}
As discussed previously, distributing updates uniformly during planning is often sub-optimal. This is because for many tasks, the majority of possible updates will be on irrelevant or low-probability trajectories.\\

We could generate experience and updates in planning by interacting the current policy with the model, then only updating the simulated trajectories. We call this \emph{trajectory sampling}. Naturally, trajectory sampling generates updates according to the on-policy distribution.\\

Focusing on the on-policy distribution could be beneficial because it causes uninteresting parts of the space to be ignored, but it could be detrimental because it causes the same parts of the space to be updated repeatedly. It is often the case the distributing updates according to the on-policy distribution is preferable to using the uniform distribution for larger problems.

\subsection{Real-time Dynamic Programming}
\emph{Real-time dynamic programming} (RTDP) is a on-policy, trajectory-sampling version of value-iteration DP. This is DP value iteration, but with the updates distributed according to the on-policy distribution. As such, it is a form of asynchronous DP. \\

Due to the trajectory sampling, RTDP allows us to skip portions of the state space that are not relevant to the current policy (in terms of the prediction problem). For the control problem (finding an optimal policy) all we really need is an \emph{optimal partial policy}, which is a policy that is optimal on the relevant states and specifies arbitrary actions on the others. \\

In general, finding an optimal policy with on-policy trajectory-sampling control method (e.g. Sarsa) requires visiting all state action pairs infinitely many times in the limit. This is true for RTDP as well, but there are certain types of problems for which RTDP is guaranteed to find ann optimal partial policy without visiting all states infinitely often. This is an advantage for problems with very large state sets. \\

The particular tasks for which this is the case are \emph{stochastic optimal path problems} (which are generally framed in terms of cost minimisation rather than reward maximisation). They are undiscounted episodic tasks for MDPs with absorbing goal states that generate zero rewards. For these problems, with each episode beginning in a state randomly chosen from the set of start states and ending at a goal state, RTDP converges with probability one to a policy that is optimal for all the relevant states provided: 1) the initial value of every goal state is zero, 2) there exists at least one policy that guarantees that a goal state will be reached with probability one from any start state, 3) all rewards for transitions from non-goal states are strictly negative, and 4) all the initial values are equal to, or greater than, their optimal values (which can be satisfied by simply setting the initial values of all states to zero).

\subsection{Planning at Decision Time}
The type of planning we have considered so far is the improvement of a policy or value function based on simulated experience. This is not focussed on interaction with the environment and is called \emph{background planning}.\\

An alternative type of planning, \emph{decision time planning}, is the search (sometimes many actions deep) of possible future trajectories given the current state.\\

\subsection{Heuristic Search}
In heuristic search, for each state encountered, a large tree of possible continuations is considered. The approximate value function is applied to the leaf nodes and then backed up toward the current state at the root. The backing up within the search tree is just the same as in the expected updates with maxes discussed throughout this book. The backing up stops at the state–action nodes for the current state. Once the backed-up values of these nodes are computed, the best of them is chosen as the current action, and then all backed-up values are discarded.\\

\subsection{Rollout Algorithms}
\emph{Rollout algorithms} are decision-time planning algorithms based on Monte Carlo control applied to simulated trajectories that all begin at the current environment state. Rollout algorithms start in a given state, then estimate the value of the state by averaging simulated returns from that state after following a given policy, called the \emph{rollout policy}. The action with the highest estimated value is selected and the process is repeated. This is useful when one knows a policy but needs to average over some stochasticity in the environment. \\

\subsection{Monte Carlo Tree Search}
\emph{Monte-Carlo Tree Search} (MCTS) is a successful example of decision time planning. It is a rollout algorithm that accumulates value estimates from the Monte Carlo simulations in order to guide the search. A variant of MCTS was used in AlphaGo.\\

A basic version of MCTS follows the following steps, starting at the current state:
\begin{enumerate}
    \item {\bfseries{Selection.}} Starting at the root node, a \emph{tree policy} based on action-values attached to the edges of the tree (that balances exploration and exploitation) traverses the tree to select a leaf node.
    \item {\bfseries{Expansion.}} On some iterations (depending on the implementation), the tree is expanded from the selected leaf node by adding one of more child nodes reached from the selected node via unexplored actions.
    \item {\bfseries{Simulation.}} From the selected node, or from one if its newly added child nodes (if any), simulation of a complete episode is run with actions selected by the rollout policy. The result is a Monte Carlo trial with actions selected first by the tree policy and beyond the tree by the rollout policy.
    \item {\bfseries{Backup.}} The return generated by the simulated episode is backed up to update, or to initialise, the action values attached to the edges of the tree traversed by the tree policy in this iteration of MCTS. No values are saved for the states and actions visited by the rollout policy beyond the tree.
\end{enumerate}

The figure below illustrates this process. MCTS executes this process iteratively, starting at the current state, until no more time is left or computational resources are exhausted. An action is then taken based on some statistics in the tree (e.g. largest action-value or most visited node). After the environment transitions to a new state, MCTS is run again, sometimes starting with a tree of a single root node representing the new state, but often starting with a tree containing any descendants of this node left over from the tree constructed by the previous execution of MCTS; all the remaining nodes are discarded, along with the action values associated with them.\\

\includegraphics[width=\textwidth]{\NotesImages/monte_carlo_tree_search.png}\\

\subsection*{Summary of Part I}

\includegraphics[width=\textwidth]{\NotesImages/spectrum_of_tabular_methods.png}\\


\documentclass[]{article}

% Define page
\usepackage[top=1in, bottom=1in, left=1in, right=1in]{geometry}

% Include packages for text modification
\usepackage{amsmath,amsfonts}
\usepackage{bm}

% Include packages for citing
%\usepackage[noadjust]{cite}
\usepackage{natbib}%http://merkel.zoneo.net/Latex/natbib.php

% Include packages for graphics
\usepackage{graphicx}
\usepackage{subfigure}

% Define new commands
\newcommand{\etal}{\emph{et al.}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\title{Kappa statistic}

\author{Marius Staring}
\date{}
\maketitle

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% main text

\section{Introduction}

The kappa statistic is a measure of agreement between observations.
Several definitions of the kappa statistic exist, which are not all
equivalent. In 1960 Cohen \cite{Cohen60} introduced a measure of
nominal scale agreement, which was defined for two observers and
where disagreement was weighted without taking into account the
distance between categories. Cohen \cite{Cohen68} extended this
measure in 1968 to a weighted kappa statistic $\kappa_w$ for two
observers. Fleiss \cite{Fleiss71} introduced an unweighted kappa
statistic $\kappa$ for 2 or more observers. The Fleiss kappa
statistic for two observers is not equivalent to the Cohen weighted
kappa statistic with an identity weighting. No weighted kappa
statistic exists for more than two observers.

In Section \ref{sec:Fleiss} a mathematical definition of the Fleiss
kappa is given, followed by definitions of the Cohen unweighted and
weighted kappa (Section \ref{sec:Cohen} and \ref{sec:CohenWeighted},
respectively).

The following notation is used:

\begin{table}[h]
\begin{tabular}{cl}
$N$ & the number of cases / observations \\
$n$ & the numbers of raters / observers \\
$k$ & the number of categories in which a case can be rated \\
\end{tabular}
\end{table}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Fleiss' kappa}\label{sec:Fleiss}

The definition of the Fleiss kappa is based on an $N$ by $k$
observation table or matrix in which the elements $n_{ij}$ represent
the number of observers who assigned the $i$-th case in the $j$-th
category. Then (from \cite{Fleiss71}):
\begin{align}
p_j &= \frac{1}{N n} \sum_{i = 1}^N n_{ij}, \\
P_i &= \frac{1}{n(n-1)} \sum_{j = 1}^k n_{ij} (n_{ij} - 1) =
\frac{1}{n(n-1)} \left( \sum_{j = 1}^k n_{ij}^2 - n \right), \\
P_o &= \frac{1}{N} \sum_{i = 1}^N P_i, \\
P_e &= \sum_{j = 1}^k p_j^2,
\end{align}
where $p_j$ is the proportion of all assignments to the $j$-th
category, $P_i$ is the extent of agreement among the $n$ observers
for the $i$-th subject / case, $P_o$ the observed overall agreement,
and $P_e$ the expected mean proportion of agreement due to chance.
The degree of actually attained agreement in excess of chance $P_o -
P_e$, normalised by the maximum agreement attainable above chance $1
- P_e$, defines the kappa statistic:
\begin{align}
\kappa &= \frac{P_o - P_e}{1 - P_e}.\label{eq:kappa}
\end{align}

The approximate standard deviation of $\kappa$ is:
\begin{align}
\mathit{std}(\kappa) &= \sqrt{ \frac{2}{Nn(n-1)} \frac{\sum_{j = 1}^k
p_j^2 - (2n-3)\left( \sum_{j=1}^k p_j^2 \right)^2 + 2(n-2)
\sum_{j=1}^k p_j^3}{ \left( 1 - \sum_{j=1}^k p_j^2 \right)^2} }
\end{align}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Cohens kappa}\label{sec:Cohen}

Cohen starts by defining a $k$ by $k$ confusion matrix, in which an
element $f_{ij}$ defines the number of cases that the first observer
assigned a particular case to category $i$ and the second to $j$. So,
$f_{jj}$ is the number of agreements for category $j$. Then (from
\cite{Altman91}):
\begin{align}
P_o &= \frac{1}{N} \sum_{j = 1}^k f_{jj}, \\
r_i &= \sum_{j = 1}^k f_{ij}, \forall i, \text{ and }
c_j = \sum_{i = 1}^k f_{ij}, \forall j, \\
P_e &= \frac{1}{N^2} \sum_{i = 1}^k r_i c_i,
\end{align}
where $P_o$ the observed proportional agreement, $r_i$ and $c_j$ the
row and column totals for category $i$ and $j$, and $P_e$ the
expected proportion of agreement. The final measure of agreement is
again given by Equation (\ref{eq:kappa}).

The approximate standard deviation of $\kappa$ is:
\begin{align}
\mathit{std}(\kappa) &= \sqrt{ \frac{P_o ( 1 - P_o )}{N ( 1 - P_e
)^2} } = \sqrt{ \frac{P_e + P_e^2 - \frac{1}{N^3} \sum_{i=1}^k r_i
c_i (r_i + c_i)}{N ( 1 - P_e )^2}}.
\end{align}
The left root was found in \cite{Altman91} and the right root was
adapted from \cite{LandisEA77} (using $p_{ij} = f_{ij} / N$).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Cohens weighted kappa}\label{sec:CohenWeighted}

Cohens kappa as defined above rates assignments with a one category
difference equally bad as two or more category different. This is
sometimes undesired. Cohens weighted kappa $\kappa_w$ is derived from
the normal kappa by including a weight function $w$. If $w$ is chosen
the identity matrix $I_k$, then Cohens $\kappa_w$ is identical to
Cohens $\kappa$. A linear weight is commonly chosen, which is
calculated as:
\begin{align}
w_{ij} &= 1 - \frac{|i-j|}{k-1}.
\end{align}
Alternatively a quadratic weight could be used: $w_{ij} = 1 - (i-j)^2
/ (k-1)^2$. Then:
\begin{align}
P_{o(w)} &= \frac{1}{N} \sum_{i = 1}^k \sum_{j = 1}^k w_{ij} f_{ij}, \\
P_{e(w)} &= \frac{1}{N^2} \sum_{i = 1}^k \sum_{j = 1}^k w_{ij} r_i
c_j,
\end{align}
with $r_i$ and $c_j$ again the row and column sums. The weighted
kappa statistic is now given by:
\begin{align}
\kappa_w &= \frac{P_{o(w)} - P_{e(w)}}{1 -
P_{e(w)}}.\label{eq:kappa_w}
\end{align}

The approximate standard deviation of $\kappa_w$ is (from
\cite{Fleiss81}):
\begin{align}
\mathit{std}(\kappa_w) &= \sqrt{ \frac{ \frac{1}{N^2} \sum_{i=1}^k
\sum_{j=1}^k r_i c_j (w_{ij} - \bar w_{i.} - \bar w_{.j})^2 -
P_{e(w)}^2}{N ( 1 - P_{e(w)} )^2}},\label{eq:stdCohenkw}
\end{align}
with $\bar w_{i.} = \frac{1}{N} \sum_{j=1}^k r_{.j} w_{ij}$ and $\bar
w_{.j} = \frac{1}{N} \sum_{i=1}^k c_{i.} w_{ij}$, and $p_{ij} =
f_{ij} / N$.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Comparing kappas}

From \cite{Fleiss81}. In order to compare two kappa's, i.e. for
testing the hypothesis that the underlying value of weighted kappa is
equal to a prespecified $\hat \kappa$ or $\hat \kappa_w$ other than
zero, the appropriate formula for the standard error of $\kappa$ or
$\kappa_w$ are given below.

\textbf{For Cohens kappa} this is given by:
\begin{align}
\mathit{std}(\kappa) &= \frac{1}{\sqrt{N} ( 1 - P_{e(w)} )} \sqrt{
\frac{1}{N} \sum_{i=1}^k f_{ii} \left[ 1 - \frac{1}{N} ( r_i + c_i )
( 1 - \kappa ) \right]^2 + ( 1 - \kappa )^2 \sum_{i = 1}^k \sum_{j =
1, i \neq j}^k f_{ij} ( r_i + c_j )^2 - \left[ \kappa - P_e ( 1 -
\kappa ) \right]^2}.
\end{align}

\textbf{For Cohens weighted kappa} this is given by:
\begin{align}
\mathit{std}(\kappa_w) &= \frac{1}{\sqrt{N} ( 1 - P_{e(w)} )} \sqrt{
\frac{1}{N} \sum_{i=1}^k \sum_{j=1}^k f_{ij} \left[ w_{ij} - (\bar
w_{i.} + \bar w_{.j} ) ( 1 - \kappa_w ) \right]^2 - \left[ \kappa_w -
P_{e(w)} ( 1 - \kappa_w ) \right]^2}.
\end{align}
Compare with Equation (\ref{eq:stdCohenkw}). The hypothesis may be
tested by referring the value of the critical ratio:
\begin{align}
z &= \frac{|\kappa_w - \hat \kappa_w|}{\mathit{std}(\kappa_w)}
\end{align}
to tables of the standard normal distribution and rejecting the
hypothesis if the critical ratio is too large.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\bibliographystyle{unsrt}
\bibliography{kappa}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\end{document}


\section{Na{\"i}ve Bayes}

\subsection{Independency Assumptions}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes and Statistical Independency}

  \structure{Na{\"i}ve Bayes is}

   \begin{itemize}
    \item still widely (and successfully) used \\[.5cm]
    \item often outperforming much more advanced classifiers \\[.5cm]
    \item appropriate in the presence of high dimensional features\\ (curse of dimensionality) \\[.5cm]
    \item also called \structure{``Idiot's Bayes''}
   \end{itemize}
 \end{frame}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes and Statistical Independency \cont}

  For the class dependent pdf we can do the following factorization:
%
  \begin{eqnarray*}
    p(\vec x| y) &=& p(x_1, x_2, \dots, x_d |y) \\[.3cm] \pause
                 &=& p(x_1|y)p(x_2, x_3,\dots, x_d | y, x_1) \\[.3cm] \pause
                 &=& p(x_1|y)p(x_2|y,x_1)p(x_3,x_4,\dots, x_d | y, x_1, x_2) \\[.3cm] \pause
                 &=& p(x_1|y)\prod_{i=2}^d p(x_i|y, x_1,\dots,x_{i-1})
  \end{eqnarray*}
\end{frame}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes and Statistical Independency \cont}

  \begin{itemize}
    \item The Na{\"i}ve Bayes classifier makes a very strong -- so to call \structure{na{\"i}ve} -- \structure{independency assumption}. \\[.5cm]
    \item All $d$ components of the feature vector $\vec x$ are assumed to be mutually independent. \\[.5cm] \pause
    \item This independency assumption implies:
      \begin{eqnarray*}
        p(\vec x| y) &=& \prod_{i=1}^d p(x_i|y)
      \end{eqnarray*}
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes and Statistical Independency \cont}

  The decision rule of na{\"i}ve Bayes reads as follows:

  \begin{eqnarray*}
     y^*  &=& \argmax\limits_{y} p(y|\vec x) \\ \pause
          &=& \argmax\limits_{y} p(y) p(\vec x| y) \\ \pause
          &=& \argmax\limits_{y} p(y) \prod_{i=1}^d p(x_i | y)
   \end{eqnarray*}
\end{frame}

\subsection{An Example: Gaussians}

\begin{frame}
  \frametitle{An Example: Na{\"i}ve Bayes and Gaussians}

  \begin{ovalblock}{Example}
    Assume the $100$--dimensional feature vector $\vec x\in \real^{100}$ belonging to class $y$ is normally distributed and all components are {\em mutually dependent}:

    \begin{eqnarray*}
      \vec\mu_y &\in& \real^{100} \\
      \mat\Sigma &=&  \mat\Sigma^T \in \real^{100\times 100}
    \end{eqnarray*}

    The total number of parameters to be estimated for each class is \pause $$100+100\cdot (100+1)/2= 5150.$$
  \end{ovalblock}
\end{frame}

\begin{frame}
  \frametitle{An Example: Na{\"i}ve Bayes and Gaussians \cont}

  \begin{ovalblock}{Example cont.}
    Assume the $100$--dimensional feature vector $\vec x\in \real^{100}$ belonging to class $y$ is normally distributed and all components are \emph{mutually independent}.
    \begin{displaymath}
      p(\vec x| y) = \prod_{i=1}^{100} p(x_i | y) \quad = \quad \prod_{i=1}^{100} {\cal N}(x_i;\mu_i, \sigma^2_i).
    \end{displaymath}

    For each component $i=\{1,2,3,\dots, 100\}$ we have to estimate mean $\mu_i\in \real$ and variance $\sigma_i^2\in \real$.
    The total number of parameters to be estimated for each class is \pause $$100+100= 200.$$
  \end{ovalblock}
\end{frame}

\begin{frame}
   \frametitle{An Example: Na{\"i}ve Bayes and Gaussians \cont}

  \begin{ovalblock}{Example cont.}
    \begin{figure}
      \resizebox{.75\linewidth}{!}{
        \input{\texfigdir/naive_bayes_parameters.pstex_t}
      }
   \end{figure}
 \end{ovalblock}
\end{frame}

\begin{frame}
  \frametitle{An Example: Na{\"i}ve Bayes and Gaussians \cont}

  \begin{ovalblock}{Example cont.}
    \begin{figure}
      \resizebox{.5\linewidth}{!}{
        \input{\texfigdir/gaussian1.pstex_t}
      }
      \caption{Quadratic decision boundary that considers statistical dependency}
   \end{figure}
   \end{ovalblock}
 \end{frame}

\begin{frame}
  \frametitle{An Example: Na{\"i}ve Bayes and Gaussians \cont}

  \begin{ovalblock} {Example cont.}
    \begin{figure}
      \resizebox{.5\linewidth}{!}{
        \input{\texfigdir/gaussian4.pstex_t}
      }
      \caption{Quadratic decision boundary assuming independency of $x_1$ and $x_2$}
    \end{figure}
  \end{ovalblock}
\end{frame}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes}

  Let us consider the \structure{logit transform}

  \footnotesize
  \begin{eqnarray*}
    \log \frac{p(y=0|\vec x)}{p(y=1|\vec x)}
      &=& \pause \log\frac{p(y=0) p(\vec x | y=0)}{p(y=1) p(\vec x | y=1)}\\[.3cm] \pause
      &=& \log\frac{p(y=0)}{p(y=1)} + \log\frac{p(\vec x | y=0)}{p(\vec x | y=1)}\\[.3cm] \pause
      &=& \log\frac{p(y=0)}{p(y=1)} + \log\frac{\prod_{i=1}^d p(x_i | y=0)}{\prod_{i=1}^d p(x_i | y=1)} \\[.3cm] \pause
      &=& \underbrace{\alpha_0 + \sum_{i=1}^d \alpha_{0,i}( x_i)}_{\mbox {\structure{generalized additive model}}}
   \end{eqnarray*}
\end{frame}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes \cont}

  \begin{center}
    Is there anything between Bayes and Na{\"i}ve Bayes?
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes \cont}

  There are multiple techniques to beat the curse of dimensionality, \\
  for example:

  \begin{itemize}
     \item Reduction of the parameter space\\
       \begin{itemize}
         \item Introduction of independency assumptions \\
           (from complete dependency to mutual independency)
         \item Parameter tying
       \end{itemize}
    \item Reduction of the dimension of the feature vectors
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes \cont}

  First order dependency
  \begin{eqnarray*}
    p(\vec x| y) &=& p(x_1, x_2, \dots, x_d |y) \\[.3cm] \pause
                 &=& p(x_1|y)p(x_2, x_3,\dots, x_d | y, x_1) \\[.3cm] \pause
                 &=& p(x_1|y)p(x_2|y,x_1)p(x_3,x_4,\dots, x_d | y, x_1, x_2) \\[.3cm] \pause
                 &=& p(x_1|y)\prod_{i=2}^d p(x_i|y, x_{i-1})
  \end{eqnarray*}
\end{frame}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes \cont}

  \begin{ovalblock}{Example}
    \structure{First order dependency} in a Gaussian random vector can be identified through the covariance matrix $\mat\Sigma$. It has the following structure:

    \begin{eqnarray*}
      \mat\Sigma &=& \left(
        \begin{array}{ccccccc}
          \sigma_{1,1} & \sigma_{2,1} & 0            & 0            & \cdots & 0              & 0 \\
          \sigma_{2,1} & \sigma_{2,2} & \sigma_{3,2} & 0            & \cdots & 0              & 0 \\
          0            & \sigma_{3,2} & \sigma_{3,3} & \sigma_{4,3} & \cdots & 0              & 0 \\
          0            & 0            & \sigma_{4,3} & \sigma_{4,4} & \cdots & 0              & 0 \\
          \vdots       & \vdots       & \vdots       & \vdots       & \ddots & \vdots         & \sigma_{d,d-1} \\
          0            & 0            & 0            & 0            & \cdots & \sigma_{d,d-1} & \sigma_{d,d} \\
        \end{array}
      \right)
    \end{eqnarray*}
  \end{ovalblock}
\end{frame}

\begin{frame}
  \frametitle{Na{\"i}ve Bayes \cont}

  \begin{ovalblock}{Example}
    First order dependency in Gaussian random vector with \\
    \structure{tied diagonal elements}, i.\,e.\ $\sigma_{i,i}=\sigma$:

   \begin{eqnarray*}
     \mat\Sigma &=& \left(
       \begin{array}{ccccccc}
         \sigma       & \sigma_{2,1} & 0            & 0            & \cdots & 0              & 0 \\
         \sigma_{2,1} & \sigma       & \sigma_{3,2} & 0            & \cdots & 0              & 0 \\
         0            & \sigma_{3,2} & \sigma       & \sigma_{4,3} & \cdots & 0              & 0 \\
         0            & 0            & \sigma_{4,3} & \sigma       & \cdots & 0              & 0 \\
         \vdots       & \vdots       & \vdots       & \vdots       & \ddots & \vdots         & \sigma_{d,d-1} \\
         0            & 0            & 0            & 0            & \cdots & \sigma_{d,d-1} & \sigma \\
       \end{array}
     \right)
    \end{eqnarray*}
  \end{ovalblock}
\end{frame}

\subsection{Lessons Learned}

\begin{frame}
  \frametitle{Lessons Learned}

  \begin{itemize}
    \item Na{\"i}ve Bayes is rather successful. \\[.5cm]
    \item Na{\"i}ve Bayes does not require a huge set of training data. \\[.5cm]
    \item Statistical dependency vs. dimension of the search space. \\[.5cm]
    \item Na{\"i}ve Bayes: give it a try!
  \end{itemize}
\end{frame}

\input{nextTime.tex}

\subsection{Further Readings}

\begin{frame}
  \frametitle{Further Readings}

  \begin{itemize}
    \item Brian D. Ripley: \\
      \structure{Pattern Recognition and Neural Networks}, \\
      Cambridge University Press, Cambridge, 1996.\\[.3cm]
    \item Christopher M.\ Bishop: \\
      \structure{Pattern Recognition and Machine Learning}, \\
      Springer, New York, 2006
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Comprehensive Questions}

  \begin{itemize}
    \item What is the assumption of Na{\"i}ve Bayes? \\[1cm]
    \item How does the assumption affect the class dependent pdf? \\[1cm]
    \item What is the structure of the covariance matrix of normal-distributed classes in Na{\"i}ve Bayes? \\[1cm]
    \item How can Na{\"i}ve Bayes be extended to first-order statistical dependencies?
  \end{itemize}
\end{frame}