\section{Econometric Theory}
\subsection{Model and Estimation}
<<<<<<< HEAD
The econometric model used to forecast and its estimation method are closely related to Hansen \cite{hansen2009averaging} and Andrews \cite{andrews93}.\footnote{Andrews considers GMM as the primary estimation method.} The model we are interested in is a linear time series regression with a possible structural break in the conditional mean. The observations we have are time series $\{y_t,x_t\}$ for $t = 1,...,T$, where $y_t$\footnote{Since we are interested in forecasting, $y_t$ can be thought of as the variable to be predicted for the next period using currently available information $x_t$.} is the scalar dependent variable and $x_t$ is a $k\times 1$ vector of related predictors and possibly lagged values of $y_t$, $k$ is the total number of regressors or predictors included. Parameters are estimated by ordinary least squares. The forecasting model allowing for structural break is:
\begin{equation} \label{mod:1}
	y_t = x_t'\beta_1 I_{[t<m]} + x_t'\beta_2 I_{[t \geq m]} + e_t
\end{equation}
where $I_{[\bullet]}$ is an indicator function, $m$ is the time index of the break and $E(e_t|x_t) = 0$. The break date is restricted to the interval $[m_1,m_2]$ which is bounded away from the ends of the sample on both sides, $1 < m_{1} < m_{2} < T$. In practice, a popular choice is to use the middle $70\%$ portion of the sample. We assume that all information relevant for forecasting is included in the regressors $x_t$, and the source of model misspecification comes solely from the uncertainty about parameter stability. This is in contrast to many applied econometric models where model misspecification bias comes from the wrong choice of regressors but the parameters are assumed stable.

We can also use a stable linear model to forecast:
\begin{equation} \label{mod:2}
	y_t = x_t'\beta + e_t
\end{equation}
The traditional pre-test procedure starts with performing a test for structural breaks\footnote{This can be done in various ways. One is to treat various possible number of breaks as different models, then select one according to some information criterion, e.g., AIC, SIC or Mallow's. Another way is hypothesis testing, following the relevant testing procedures outlined in Andrews \cite{andrews93}, Bai and Perron \cite{bai_perron98} and Elliot and Muller \cite{elliott_muller_RES2006}.}, either by using Andrews' SupF or SupW test, or Bai and Perron's multiple-break test, and then decide to keep the stable or unstable model.

As an alternative to model selection, we can combine these two models by assigning weight $w$ to model~\ref{mod:1} and $1 - w$ to model~\ref{mod:2}, where $w \geq 0$. So the combined predictive model is
\begin{equation} \label{mod:3}
    y_{t} = w \left\{ x_t'\beta_1 I_{[t<m]} + x_t'\beta_2 I_{[t \geq m]} \right\} + (1 - w) \left\{ x_t'\beta \right\} + e_t
\end{equation}
With the forecasting model ready, next, we are going to present the cross-validation criterion in detail which is crucial in determining the optimal weight $w$ in equation~\ref{mod:3}.
\subsection{Cross-Validation Criterion}
There are several popular information criteria for model selection: for example, Akaike information criterion (\textbf{AIC}), corrected AIC (\textbf{AIC\textsuperscript{c}}), Schwarz Bayesian information criterion (\textbf{SIC}), Hannan-Quinn (\textbf{HQ}) and Mallows' C\textsubscript{p} (\textbf{C\textsubscript{p}}). Most criteria have two components in their formulas: the first part measures model fit while the second penalizes overfitting. The quantity measuring in-sample fit are the same for most criteria, but they differ in the degree of penalization. For instance, AIC penalizes each additional parameter by $2$ while SIC penalizes overfitting by the logarithm of sample size, so SIC tends to select a more parsimonious model than AIC if the sample size is large.

For the forecasting analysis, what we care about is the test error rate assessing the model predictive ability, not the training error rate produced in the model estimation stage, so selecting a information criterion which gives a good estimate of the expected test error rate is crucial. Cross-validation is such a criterion. Specifically, we focus on the use of leave-one-out cross-validation for this paper, though other CV variants, such as K--fold cross-validation, may be considered. Cross-validation is computationally simple for one-step ahead predictive model selection and is shown robust to conditional heteroscedasticity in the econometrics and statistics literature. For forecast combination, researchers have applied CV to the quadratic programming based model averaging analysis, but its setting does not include structural break.
=======
The econometric model to forecast and its estimation method are closely related to Hansen \cite{hansen2009averaging} and Andrews\footnote{Andrews considers GMM as the primary estimation method.} \cite{andrews93}. The model we are interested in is a linear time series regression with a possible structural break. The observations we have are time series $\{y_t,x_t\}$ for $t = 1,...T$, where $y_t$\footnote{Since we are interested in forecasting, $y_t$ can be thought of as the variable to be predicted for the next period using currently available information $x_t$.} is the scalar dependent variable and $x_t$ is a $k\times 1$ vector of related predictors\footnote{It includes the constant term.} and possibly lagged values of $y_t$, $k$ is the number of regressors or predictors included. Parameters are estimated by least squares. The forecasting model\footnote{We assume that all information relevant for forecasting is included in the regressors $x_t$, and the source of model misspecification comes from the uncertainty about the parameter change. This is in contrast to most applied econometric models introduced in textbooks where model misspecification bias comes from the wrong choice of regressors, but the parameters are assumed constant.} allowing for structural break is:
\begin{equation}
	y_t = x_t'\beta_1 I_{[t<m]} + x_t'\beta_2 I_{[t \geq m]} + e_t
\end{equation}
where $I_{[\bullet]}$ is an indicator function and $E(e_t|x_t) = 0$. The break date is restricted to the interval $[m_1,m_2]$ which is bounded away from the ends of the sample on both sides, $1 < m_{1} < m_{2} < T$. In practice, a popular choice is to use the middle $70\%$ portion of the sample. If the break happens very close to the end of the sample, we may not be able to identify the break or the regression parameters.

We can also use a stable linear model to forecast:
\begin{equation}
	y_t = x_t'\beta + e_t
\end{equation}
Traditionally, we start the process by performing a test for structural breaks\footnote{This can be done in two ways. The first is to treat various possible number of breaks as different models, then select one according to some information criterion, e.g., AIC, SIC or Mallow's. The other way is hypothesis testing, following the relevant testing procedures outlined in Andrews\cite{andrews93}, Bai and Perron\cite{bai_perron98} and Elliot and Muller\cite{elliott_muller_RES2006}. }, either by using Andrews' SupF or SupW test, or Bai and Perron's multiple-break test, and then decide to keep the stable or unstable model. This is the pre-test procedure outlined in Hansen \cite{hansen2009averaging}. Under the assumption of conditional homoscedasticity, the Andrews' test statistic is of SupF type. If we relax this assumption, the appropriate test statistic should be of Sup-Wald type since the F test does not allow for heteroscedastic errors. We use $\pi = m/T$ to denote the break fraction, where $m$ is the time index of the break date in the sample. Under the assumptions listed in Andrews \cite{andrews93} and under the null of no break, the SupF or SupW test statistic converges in distribution to the supremum of the square of a standardized tied-down Bessel process of order $k$, where $k \geq 1$ is the number of regressors in the model.\footnote{This distribution is non-standard so its critical values are obtained through simulation. Readers, especially applied economics researchers, should be aware of the fact that critical values in Andrews' 1993 paper are incorrect and should not be used to conduct inference. Andrews' 2003 paper,\emph{Tests for Parameter Instability and Structural Change with Unknown Change Point: A Corrigendum}, corrects these values. Hansen \cite{hansen2009averaging} has done the same simulation in order to calculate his $\bar{p}$ value for model averaging weights. The author has replicated Andrews' critical value simulation during the writing of this paper and has obtained the same numerical results as Hansen but they are still different from Andrews' 2003 values. The difference may be caused by using different number of repetition in the simulation.}.
\subsection{Cross-Validation Criterion}
There are several popular information criteria for model selection: for example, Akaike information criterion (\textbf{AIC}), corrected AIC (\textbf{AIC\textsuperscript{c}}), Schwarz Bayesian information criterion (\textbf{SIC}), Hannan-Quinn (\textbf{HQ}) and Mallows criterion (\textbf{C\textsubscript{p}}). Their formulas are listed in table \ref{tb:1}.
\begin{table}
\centering
\caption{Information Criterion} \label{tb:1}
\begin{threeparttable}
\begin{tabular}{ll}
\toprule
AIC                 & $\mathrm{AIC}(k) = \log{(\tilde{\sigma}^{2}_{k})} + \frac{2k}{T}$ \\[0.4em]
$\mathrm{AIC}^{c}$  & $\mathrm{AIC}^{c}(k) = \mathrm{AIC}(k) + \frac{2k(k+1)}{T-k-1}$ \\[0.4em]
SIC                 & $\mathrm{SIC}(k) = \log{(\tilde{\sigma}^{2}_{k})} + \frac{k\log{(T)}}{T}$ \\[0.4em]
HQ                  & $\mathrm{HQ}(k) = \log{(\tilde{\sigma}^{2}_{k})} + \frac{2k\log{\log(T)}}{T}$ \\[0.4em]
Mallows            & $\mathrm{C}_{p}(k) = \hat{\sigma}^{2}_{k} + \frac{2k\tilde{\sigma}^{2}_{k}}{T}$\\
\bottomrule
\end{tabular}
\begin{tablenotes}[para, flushleft] \footnotesize
Notes: We use the number of regressors $k$ to index model. If there is a full structural break, the model index is $2k$.
\end{tablenotes}
\end{threeparttable}
\end{table}

Cross-validation is computationally simple for one-step ahead forecasting model selection and is robust to conditional heteroscedasticity. Researchers have derived cross-validation weights for averaging stable models, which can be translated into a quadratic programming problem. Hansen and Racine \cite{hansen2011jackknife} argue that for heteroscedastic data, cross-validation is a valid estimate of the one-step ahead MSFE. They show that the CV weights are asymptotically optimal for cross-section data under heteroscedasticity.
>>>>>>> 2966618e04912eba46807909c13546d2481a77e9

The sample leave-one-out cross-validation criterion can be computed by the following procedure:
\begin{equation} \label{cv}
	CV_T(k) = \frac{1}{T}\sum_{t=1}^{T}\tilde{e}_{t}(k)^{2}
\end{equation}
<<<<<<< HEAD
where $\tilde{e}_{t}(k) = y_t - \tilde{\beta}_{-t}(k)'x_t(k)$ are the residuals from the regression with the $t^{\mathrm{th}}$ observation dropped and $\tilde{\beta}_{-t}(k) = (\sum_{i\not= t}x_i(k) x_i(k)')^{-1}(\sum_{i\not= t}x_i(k) y_i)$ is the associated vector of parameter estimates. Intuitively, this procedure is trying to estimate the expected test error rate based on the model training data. Though equation~\ref{cv} implies that we need to run regression $T$ times for given sample size $T$, luckily, for linear models, we can calculate sample CV value by running regression only once. Formally, the leave-one-out cross validation residuals can be computed from the full sample least squares residuals, $\tilde{e_{t}} = \frac{\hat{e_{t}}}{1 - h_{t}}$, where $h_{t} = x_{t}'(X_{t}'X_{t})^{-1}x_{t}$ is the leverage associated with observation $t$, $\hat{e_{t}}$ is the full sample least squares residual and $\tilde{e_{t}}$ is the cross-validation residual. So we can rewrite equation~\ref{cv} as
\begin{equation}
    CV_T(k) = \frac{1}{T}\sum_{t=1}^{T}\left( \frac{\hat{e_{t}}(k)}{1 - h_{t}} \right)^{2}
\end{equation}
In the next section we are going to show how model averaging weights are derived from the cross-validation criterion.
\subsection{Theoretical Results}
We assume that the true data generating process for $y_t$ takes the general parameter variation form and the change is of small magnitude so that the asymptotic distributions are asymptotically continuous. Also the change of the parameter value is proportional to the unconditional standard deviation of the error term. This makes sure that the impact of the local parameter instability is not dominated by that of the volatility. Under this assumption we can tell that the empirical model outlined in the previous section could be potentially misspecified: the single break model is only a special case of the general parameter variation model, with the absolute change of parameter values positive in one period while zero in others. In almost all relevant empirical work, researchers are interested in finding the best approximating model among all candidates available instead of looking for and estimating the true data generating process\footnote{There is research focusing on fitting data with complex models, such as nonlinear continuous break models. Though this approach may achieve good in-sample fit, its forecasting performance generally does not do well.}. Based on assumptions listed below, our method should give better MSFE under model averaging forecast for possibly misspecified models than other related methods.
\begin{assumption}\label{asump:1}
Assume the following holds:
\begin{enumerate}
	\item The true data generating process satisfies the linear model $y_t = x_t'\beta_t + e_t$, $t=1,...,T,\beta_t \in \mathbb{R}^k$, where $\beta_t = \beta + T^{-1/2}\eta(t/T)\delta\sigma_t$. $\eta(\bullet)$ is a $\mathbb{R}^k$ valued Riemann integrable function on $[0,1]$ and $\delta \in \mathbb{R}\backslash\{0\}$ is a scalar indexing the magnitude of parameter variation.
	\item $\{(x_t',e_t)\}$ is $\alpha$-mixing of size $-r/(r-2),r > 2$ or $\phi$-mixing of size $-r/(2r-2),r \geq 2$.
    \item $E(x_t e_t) = 0, \forall t$, and the process $\{x_t e_t\}$ is uniformly $L_r$-bounded, i.e., $||x_t e_t||_{r} < B$, where $B$ is a constant and $B < \infty$.
=======
where $\tilde{e}_{t}(k) = y_t - \tilde{\beta}_{-t}(k)'x_t(k)$ and $\tilde{\beta}_{-t}(k) = (\sum_{i\not= t}x_i(k) x_i(k)')^{-1}(\sum_{i\not= t}x_i(k) y_i)$. That is, at each time period $t$, the parameters are estimated by using all data available but omitting current observation $t$, and then computing its error. It may look tedious since we need to run $T$ times of regression to get the value of CV, but in next section we show that this can be achieved efficiently by running regression only once. The cross-validation criterion is asymptotically an unbiased estimator of MSFE and is robust to heteroscedasticity compared with Mallows' criterion. This is especially important when we are working with macroeconomic and financial time series.

Since we only have two candidate models in this study, namely the stable model and the break model, to implement model averaging, we assign weight $w \in [0,1]$ to the break model and $1 - w$ to the stable model. The optimal weights are those minimizing the CV criterion.
\subsection{Theoretical Results}
We assume that the true data generating process for $y_t$ takes the general parameter variation form and the change is of small magnitude so that the asymptotic distributions are asymptotically continuous. Also the change of the parameter value is proportional to the unconditional standard deviation of the error term. This makes sure that the impact of the local parameter instability is not dominated by that of the volatility. Under this assumption we can tell that the empirical model outlined in the previous section could be potentially misspecified: the single break model is only a special case of the general parameter variation model, with the absolute change of parameter values positive in one period while zero in others. In almost all relevant empirical work, researchers are interested in finding the best approximating model among all candidates available instead of looking for and estimating the true data generating process\footnote{There is research focusing on fitting data with complex models, such as nonlinear continuous break models. Though this approach may achieve good in-sample fit, its forecasting performance generally does not do well.}. Based on assumptions listed below, our method should give better MSFE under model averaging forecast for possibly misspecified models than other related methods.
\begin{Assumption}\label{asump:1}
Assume the following holds:
\begin{enumerate}
	\item The true data generating process satisfies the linear model $y_t = x_t'\beta_t + e_t$, $t=1,...,T,\beta_t \in \mathbb{R}^k$, where $\beta_t = \beta + T^{-1/2}\eta(t/T)\delta\sigma_t$. $\eta(\bullet)$ is a $\mathbb{R}^k$ valued Riemann integrable function on $[0,1]$ and $\delta \in \mathbb{R}$ is a scalar indexing the magnitude of parameter variation.
	\item $\{(x_t',e_t)\}$ is $\alpha$-mixing of size $-r/(r-2),r > 2$ or $\phi$-mixing of size $-r/(2r-2),r \geq 2$.
    \item $E(x_t e_t) = 0, \forall t$, and the empirical process $\{x_t e_t\}$ is uniformly $L_r$-bounded, i.e., $||x_t e_t||_{r} < B$, where $B$ is a constant and $B < \infty$.
>>>>>>> 2966618e04912eba46807909c13546d2481a77e9
    \item $T^{-1/2}\sum_{t=1}^{[\pi T]} x_t e_t \Rightarrow W(\pi)$ where $W(\pi)$ is a $k \times 1$ Wiener process with symmetric, positive definite long-run covariance matrix $\Sigma \equiv \lim\limits_{T\to \infty}\mathrm{VAR}(T^{-1/2}\sum_{t=1}^{[\pi T]} x_t e_t)$, for $0 \leq \pi \leq 1$. ` $\Rightarrow$' denotes the weak convergence of the underlying probability measure as $T \to \infty$.
	\item $T^{-1}\sum_{t=1}^{[\pi T]}x_t x_t'$ converges uniformly to $\pi Q$ for all $\pi \in [0,1]$, $Q = E(x_t x_t')$ and all eigenvalues of $Q$ are uniformly bounded away from zero. $[\pi T]$ denotes the integer part of the product $\pi T$.
	\item $E(e_t^{2}|x_t) = \sigma_t^{2}$.
\end{enumerate}
<<<<<<< HEAD
\end{assumption}
Assumption 1.1 says that the true data generating process for $y_t$ takes a general parameter variation form and structural break occurs in all parameters. This assumption follows the theory proposed by Andrews and Hansen but is different from Bai and Perron who consider partial structural breaks. For Bai and Perron's \cite{bai_perron98} partial structural break model, there is no clear guidance on how to determine which set of regressors are subject to parameter instability. Without any prior information on the stability of parameters, it is natural to assume that all coefficients are subject to break. This full-break assumption is less restrictive and is used in many empirical applications, see Rapach and Wohar \cite{rapach_wohar_JFE2006} and Paye and Timmermann \cite{paye_timmermann_JEF2006}. This type of data generating process includes several commonly used cases, such as a single break, multiple breaks or continuous break process. Notice that our forecasting model outlined in the previous section only allows for one possible break in the parameters, so it could potentially be misspecified. We make this assumption and allow for the gap between the true DGP and the forecasting model primarily for two reasons. First, as argued by several researchers, for example, L\"{u}tkepohl \cite{lutkepohl_textbook}, the true data generating process for many macroeconomic and financial variables may be a complicated process possibly involving past values of infinite order. The main focus is not to search for the true DGP but to find the best approximating model, or the best local linear predictor based on available information. This assumption tries to capture the difficulty faced by many practitioners in forecasting. Second, a simple model is more likely to forecast better than many complex methods on average. Complicated methods may achieve good in-sample fit at the expense of compromising the forecasting model's adaptability to environment change\footnote{In an earlier version of this paper, through Monte Carlo simulation we provide an example to show that even the true DGP follows a continuous break process, the stable model forecasts best among all popular methods considered, such as Granger-Ramanathan combination.}.

Assumption 1.2 -- 1.5 ensure that we can apply all relevant mixing laws of large numbers, functional central limit theorem or Donsker's invariance principle when proving our results. See Davidson \cite{davidson_textbook} for more details on advanced asymptotic theory. Assumption 1.6 says that the error term could be conditionally homoscedastic or heteroscedastic, depending on the application of interest. This assumption contrasts our paper with Hansen's \cite{hansen2009averaging}, as Hansen only considers the conditionally homoscedastic error case.

Next, to obtain the main theoretical results, we need to derive the asymptotic distribution of the penalty term in the CV criterion since its first moment is crucial in determining the optimal weights in model averaging. We provide two theorems for the model averaging weights. Their differences depend on whether the assumption of conditional homoscedasticity holds. The proofs of all theoretical results are provided in the appendix.
\begin{proposition} \label{thm:1}
If assumption~\ref{asump:1} holds but $E(e_t^{2}|x_t) = \sigma^{2}$, the leave-one-out cross--validation criterion is asymptotically equivalent to Mallows' criterion, that is, $E(CV(T)) \stackrel{p}{\rightarrow} E(Cp(T))$.
\end{proposition}
\begin{corollary} \label{corollary:1}
The infeasible population optimal weight for the break model is $w = 1 - \frac{E(p_{\delta})}{\Sup_{\pi \in [\pi_{1}, \pi_{2}]}J_{0}(\pi)}$ for $\Sup_{\pi \in [\pi_{1}, \pi_{2}]}J_{0}(\pi) \geq E(p_{\delta}$ and $0$ otherwise.
\end{corollary}
\begin{corollary} \label{corollary:2}
The feasible sample CV weight for the break model is:
=======
\end{Assumption}
Assumption 1.1 says that the true data generating process for $y_t$ takes a general parameter variation form and structural break occurs in all parameters. This assumption follows the theory proposed by Andrews and Hansen but is different from Bai and Perron who consider partial structural breaks. For Bai and Perron's \cite{bai_perron98} partial structural break model, there is no clear guidance on how to determine which set of regressors are subject to parameter instability. Without any prior information on the stability of parameters, it is natural to assume that all coefficients are subject to break. This full-break assumption is less restrictive and is used in many empirical applications, see Rapach and Wohar \cite{rapach_wohar_JFE2006} and Paye and Timmermann \cite{paye_timmermann_JEF2006}. This type of data generating process includes several commonly used cases, such as a single break, multiple breaks or continuous break process. Notice that our forecasting model outlined in the previous section only allows for one possible break in the parameters, so it could potentially be misspecified. We make this assumption and allow for the gap between the true DGP and the forecasting model primarily for two reasons. First, as argued by several researchers, for example, L\"{u}tkepohl \cite{lutkepohl_textbook}, the true data generating process for many macroeconomic and financial variables may be a complicated process possibly involving past values of infinite order. The main focus is not to search for the true DGP but to find the best approximating model, or the best local linear predictor based on available information. This assumption tries to capture the difficulty faced by many practitioners in forecasting. Second, a simple model is more likely to forecast better than many complex methods on average. Complicated methods may achieve good in-sample fit at the expense of compromising the forecasting model's adaptability to environment change\footnote{In an earlier version of this paper, through Monte Carlo simulation we show that even the true DGP follows a continuous break process, the stable model forecasts best among all popular methods considered, such as Granger-Ramanathan combination.}.

Assumption 1.2 -- 1.5 ensure that we can apply all relevant mixing law of large numbers, functional central limit theorem or Donsker's invariance principle to proving our results. See Davidson \cite{davidson_textbook} for more details on advanced asymptotic theory. Assumption 1.6 says that the error term could be conditionally homoscedastic or heteroscedastic, depending on the application of interest. This assumption contrasts our paper with Hansen's \cite{hansen2009averaging}, as Hansen only considers the conditionally homoscedastic error case.

We start by providing a lemma showing that the cross-validation residuals can be efficiently computed by using the leverage of each observation without actually having to run least squares regression $T$ times.
\begin{lemma} \label{lem:1}
The leave--one--out cross--validation estimation residuals can be computed from the full sample least squares residuals, $\tilde{e_t} = \frac{\hat{e_t}}{1-h_t}$, where $h_t = x_t' (X'X)^{-1} x_t$ is the leverage, $\hat{e_t}$ is the least squares residual and $\tilde{e_t}$ is the cross-validation residual.
\end{lemma}

Next, to obtain the main theoretical results, we need to derive the asymptotic distribution of the penalty term in the CV criterion since its first moment is crucial in determining the optimal weights in model averaging. We provide two theorems for the model averaging weights. Their differences depend on whether the assumption of conditional homoscedasticity holds. The proofs of all theoretical results are provided in the appendix.
\begin{theorem} \label{thm:1}
If assumption~\ref{asump:1} holds under conditional homoscedastic errors, the leave-one-out cross--validation criterion is approximately equivalent to Mallows' criterion, so CV weights are equivalent to Mallows' weights. Specifically, the weight for the break model is
>>>>>>> 2966618e04912eba46807909c13546d2481a77e9
\begin{equation}
\hat{w} = \frac{(T - 2k)(\sum_{t=1}^{T}\tilde{e}_{t}^{2} - \sum_{t=1}^{T}\hat{e}_{t}^{2}) - \bar{p}\sum_{t=1}^{T}\hat{e}_{t}^{2}}{(T - 2k)(\sum_{t=1}^{T}\tilde{e}_{t}^{2} - \sum_{t=1}^{T}\hat{e}_{t}^{2})}
\end{equation}
where $T$ is the sample size, $k$ is the number of regressors, $\hat{e}_t$s are the OLS residuals from the break model, $\tilde{e}_t$s are residuals from the stable model, $\bar{p}$ is the penalty coefficient whose value depends on the asymptotic distribution of the SupW test statistic.
<<<<<<< HEAD
\end{corollary}
This theorem shows that if we assume conditional homoscedasticity in the model, then the cross-validation information criterion is approximately identical to the Mallows' criterion\footnote{This is quite intuitive because the major advantage of CV over Mallows is that CV is robust to heteroscedasticity. If we assume away conditional heteroscedasticity, then these two criteria are basically equivalent.}, so all the model averaging results in Hansen \cite{hansen2009averaging}, i.e., optimal weights, hold here. See Hansen \cite{hansen2009averaging} for the table for $\bar{p}$ values.

It is widely known in the model selection literature that the CV criterion is superior to Mallows' and other information criteria because of its robustness to heteroscedasticity \cite{andrews_JE1991}. Our next lemma establishes the asymptotic distribution of the CV penalty term if we relax the assumption of conditional homoscedasticity.
\begin{lemma} \label{thm:2}
If Assumption~\ref{asump:1} holds under conditionally heteroscedastic errors, the penalty term in the cross-validation criterion converges in distribution to a weighted sum of independent $\chi^2$ distribution with degree of freedom one plus a term whose distribution is a function of Brownian bridge,
=======
\end{theorem}
This theorem shows that if we assume conditional homoscedasticity in the model, then the cross-validation information criterion is approximately identical to the Mallows' criterion\footnote{This is quite intuitive because the major advantage of CV over Mallows is that CV is robust to heteroscedasticity. If we assume away conditional heteroscedasticity, then these two criteria are basically equivalent.}, so all the model averaging results in Hansen \cite{hansen2009averaging}, i.e., optimal weights, hold here. See Hansen \cite{hansen2009averaging} for the table for $\bar{p}$ values.

It is widely known in the model selection literature that the CV criterion is superior to Mallows' and other information criteria because of its robustness to heteroscedasticity \cite{andrews_JE1991}. Our next lemma shows the asymptotic distribution of the CV penalty term if we relax the assumption of conditional homoscedasticity.
\begin{lemma} \label{thm:2}
If assumption~\ref{asump:1} holds under conditionally heteroscedastic errors, the penalty term in the cross-validation criterion converges in distribution to a weighted sum of independent $\chi^2$ distribution with degree of freedom one plus a term whose distribution is a function of non-standard Brownian bridge,
>>>>>>> 2966618e04912eba46807909c13546d2481a77e9
\begin{equation} \label{eq:4}
	e'P(\hat{k})e \stackrel{d}{\rightarrow} \sum_{j=1}^{k} \lambda_j \chi^2(1) + J_0(\xi_{\delta})
\end{equation}
where $\lambda_j$s are the eigenvalues of the matrix $L'Q^{-1}L$, $L$ comes from the decomposition of the positive definite matrix $\Sigma = LL'$, $\Sigma$ is the long-run variance of $\rn\jian X_t e_t$, $Q = E(x_t x_t')$ and $J_0(\xi_{\delta})$ is the asymptotic distribution of the Sup-Wald type statistic as in Andrews \cite{andrews93} under the true data generating process.
\end{lemma}
<<<<<<< HEAD
Now the asymptotic distribution of the penalty term involves a weighted sum of $\chi^2$ distributions and a Sup-Wald type distribution under the true data generating process. Comparing this result with the previous one, we can see that the distribution under conditional homoscedasticity is just a special case of the new one. That is, the weights for the $\chi^2$ random variables are identical and they take the value of one, which results in a $\chi^2$ distribution with degrees of freedom equal to the total number of regressors.

Notice that the new asymptotic distribution of the penalty term is non-standard, but its expectation is relatively easy to compute, which is the only moment we need to compute the optimal model averaging weights. The expectation of $\sum_{j=1}^{k} \lambda_j \chi^2(1)$ is simply $\sum_{j=1}^{k} \lambda_j$ which is the trace of the matrix $Q^{-1} \Sigma$, where $\Sigma$ is the long-run variance of $\rn\jian X_t e_t$ and $Q = E(x_t x_t')$. Empirically, $\Sigma$ can be estimated by HAC or HAR estimators and $Q$ can be consistently estimated by its sample analogue $\frac{1}{T}\sum_{t=1}^{T}x_t x_t'$. The following theorem provides the optimal weight for the break model under conditional heteroscedasticity.
=======
Now the asymptotic distribution of the penalty term involves a weighted sum of $\chi^2$ distributions and a Sup-Wald type distribution under the true data generating process. Comparing this result with the previous one, we can see that the distribution under conditional homoscedasticity is just a special case of the new one. That is, the weights for the $\chi^2$ random variables are identical and they take the value of one, which results in a $\chi^2$ distribution with degrees of freedom equal to the total number of regressors.

Notice that the new asymptotic distribution of the penalty term is non-standard, but its expectation is relatively easy to compute, which is the only moment we care about in computing the optimal model averaging weights. The expectation of $\sum_{j=1}^{k} \lambda_j \chi^2(1)$ is simply $\sum_{j=1}^{k} \lambda_j$ which is the trace of the matrix $Q^{-1} \Sigma$, where $\Sigma$ is the long-run variance of $\rn\jian X_t e_t$ and $Q = E(x_t x_t')$. Empirically, $\Sigma$ can be estimated by HAC or HAR estimators and $Q$ can be consistently estimated by its sample analogue $\frac{1}{T}\sum_{t=1}^{T}x_t x_t'$. The following theorem provides the optimal weight for the break model under conditionally heteroscedasticity.
>>>>>>> 2966618e04912eba46807909c13546d2481a77e9

\begin{theorem} \label{lem:2}
Suppose that $\hat{e}_t$s are the OLS residuals from the break model and $\tilde{e}_t$s are residuals from the stable model, then the optimal weight for the break model under conditional heteroscedasticity takes the form \footnote{$\mathrm{E(SupW(\pi_1,k))}$ is the expectation of the SupW distribution under the null of no break in Andrews' paper, evaluated at given values of $\pi$ and $k$. Note that the expectation of the SupW distribution under the true data generating process depends on unknown parameters, so Hansen suggests that in practice we can approximate its value by taking the averaging of two extreme cases: $\delta = 0$ and $\delta = \infty$. $\delta = 0$ implies that the distribution is identical to the SupW distribution under the null of no break. However, $\delta = \infty$ indicates that when the break size goes to infinity, essentially it is like we know exactly when the break happens, so its distribution becomes the $\chi^2$ type distribution whose first moment can be easily obtained without simulation.}:
\begin{equation} \label{eq:5}
	\hat{w} = 1 - \frac{\frac{1}{2}(\sum_{j=1}^{k}\lambda_j + \mathrm{E(SupW(\pi_1,k))})}{\sum_{t=1}^{T}\hat{e}_t^2 - \sum_{t=1}^{T}\tilde{e}_t^2}
\end{equation}
where the expectation of the SupW distribution depends on the number of regressors $k$ and the value of the trimming parameter, $\pi_1$\footnote{In practice, a popular choice of $\pi_1$ is $0.15$.}.
\end{theorem}
<<<<<<< HEAD
Note that the value of $\mathrm{E(SupW(\pi_1,k))}$ can be obtained from the $\bar{p}$ value\footnote{$\bar{p} = \frac{1}{2}(\mathrm{E(SupW(\pi_1,k))} + k)$} in Hansen \cite{hansen2009averaging}. The important part is the first component $\sum_{j=1}^{k}\lambda_j$ in the parentheses of equation~\ref{eq:5}. Intuitively, it captures the impact brought to the weight by allowing for conditional heteroscedasticity. Theorem~\ref{lem:2} implies that when there is conditional heteroscedasticity, cross-validation weights are optimal in the sense of minimizing the mean squared forecast error.
=======
Note that the value of $\mathrm{E(SupW(\pi_1,k))}$ can be obtained from the $\bar{p}$ value\footnote{$\bar{p} = \frac{1}{2}(\mathrm{E(SupW(\pi_1,k))} + k)$} in Hansen \cite{hansen2009averaging}. The important part is the first component $\sum_{j=1}^{k}\lambda_j$ in the parentheses of equation~\ref{eq:5}. Intuitively, it captures the impact brought to the weight by allowing for conditional heteroscedasticity. Theorem~\ref{lem:2} implies that when there is conditional heteroscedasticity, cross-validation weights are optimal in the sense of minimizing the mean squared forecast error.
>>>>>>> 2966618e04912eba46807909c13546d2481a77e9


\section{Thacker's Planar Oscillations on a Parabolic Basin}
This test simulates planar oscillations of water in a parabolic basin. The analytical solution was derived by Thacker~\cite{Thacker1981}, and is periodic. At any instant in time, the free surface elevation is planar, and the velocity is constant (in wet regions). The scenario includes regular wetting and drying, as the flow oscillates back and forth in the basin. As well as testing the ability of the code to do wetting and drying, it will highlight any numerical energy loss or gain, and manifest as an increase or decrease in the magnitude of the flow oscillations over long time periods (compared with the analytical solution).

Consider the topography
\begin{equation}
z(x) = D_0 \left(\frac{x}{L}\right)^2
\end{equation}
where $D_0$ is the largest depth when water is still and $L$ is the distance between the centre of water surface and the shore when water is still.
The analytical solution is
\begin{equation}
u(x,t) = -A \omega \sin(\omega t),
\end{equation}
\begin{equation}
w(x,t) = D_0 + \frac{2 A D_0}{L^2} \cos(\omega t) \left( x - \frac{A}{2}\cos(\omega t) \right).
\end{equation}
Here $\omega=\frac{\sqrt{2 g D_0}}{L}$.
The initial condition is set by taking $t=0$ in the analytical solution.

\subsection{Results}
For our test, we consider $D_0=4$, $L=10$, and $A=2$. After running the simulation for some time, we have Figures~\ref{fig:cross_section_stage}--\ref{fig:cross_section_xvel} showing the stage, $x$-momentum, and $x$-velocity respectively. There should be a good agreement between numerical and analytical solutions. Small velocity spikes may appear at the moving wet-dry edge in some \anuga{} algorithms. As time goes on, some small deviations may also appear. These are shown in Figures~\ref{fig:Stage_centre}--\ref{fig:Xvel_centre}, which illustrate the stage, $x$-momentum, and $x$-velocity at the centroid of the domain.

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{cross_section_stage.png}
\caption{Stage on a cross section of the basin at time $t=10$.}
\label{fig:cross_section_stage}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{cross_section_xmom.png}
\caption{Xmomentum on a cross section of the basin at time $t=10$.}
\label{fig:cross_section_xmom}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{cross_section_xvel.png}
\caption{Xvelocity on a cross section of the basin at time $t=10$.}
\label{fig:cross_section_xvel}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{Stage_centre.png}
\caption{Stage over time in the centre of the parabolic basin.}
\label{fig:Stage_centre}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{Xmom_centre.png}
\caption{Xmomentum over time in the centre of the parabolic basin.}
\label{fig:Xmom_centre}
\end{center}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=0.9\textwidth]{Xvel_centre.png}
\caption{Xvelocity over time in the centre of the parabolic basin.}
\label{fig:Xvel_centre}
\end{center}
\end{figure}

\endinput


\documentclass{article}

\title{MetaGen Inference}
\author{Zhangir Azerbayev}
\date{2022}

\input{preamble}

\begin{document}
\maketitle

\section{Inference Framework}
In our baseline model, we have $N$ observations $X = \cbr{x^n}$, where each $x^n$ is a coordinate in 3-space. In the
full model, we have $N$ observations $X = \cbr{(x^n, c^n)}$, where the $x^n$ are locations and the $c_n$ are object
categories. We are trying to infer the world-state $\Theta = \cbr{(\mu_k, c_k)}_{k=1}^K$, where the $\mu_k$ are object
locations and the $c_k$ are object categories.

Suppose we augment are model with a latent variable $Z = \{z_k^n\}$, where $z^n$ is a one-hot vector that describes
which compnent of the GMM generated $X^n$. Thus $P(X|\Theta)$ factors as $P(X|\Theta) = P(Z)P(X|Z, \Theta)$.

The {\it expectation-maximization} (EM) algorithm hinges on two criteria:
\begin{enumerate}
    \item It is easy to optimize $P(X, Z|\Theta)$ wrt $\Theta$ (see later section)
    \item Although we have poor ``global" information about $Z$, i.e we don't know $P(Z|X)$, we have ``local"
        information about $Z$, i.e we know $P(Z|X, \Theta)$ for a fixed $\Theta$.
\end{enumerate}
The idea of the algorithm is this: instead of maximizing log-likelihood, maximize the expectation of log-likelihood
under the posterior of $Z$. More precisely, we iterate the two following steps:
\begin{enumerate}
    \item E-step: Using $\Theta^{\mathrm{old}}$ from the previous iteration, evaluate $P(Z|X, \Theta^{\mathrm{old}})$.
    \item M-Step: Solve for $\Theta = \mathrm{argmax}_{\Theta} \sum_Z P(Z|X, \Theta)\log P(X, Z|\Theta^{\mathrm{old}})$, which
        is the expectation of log-likelihood under the posterior of $Z$.
\end{enumerate}
\section{Inference for MetaGen}
Define the {\it responsibility} of $z_k^n$ for $x^n$ as
\[\gamma(z_k^n) = P(z_k^n=1|x^n, \Theta) = \frac{P(x^n|z_k^n=1, \Theta)}{\sum_{j=1}^KP(x^n|z_j^n, \Theta)}. \]
Note that the E-step is equivalent to computing the responsibilities for each $n$ and $k$. The M-step then becomes
\begin{equation}\mathrm{argmax}_\Theta\sum_{n=1}^N\sum_{k=1}^K\gamma(z_k^n)\log P(x^n, z_k^n=1| \Theta^{\mathrm{old}}).
\end{equation}
In the case of the baseline model, this is a continuous optimization problem. In the full MetaGen case, we can simply
sample category assignments and condition on them, yielding multiple continuous optimization problems.

\section{Optimization for M-step}
I currently do not know whether (1) has a closed form solution (my guess is that it doesn't). Worst case, we do
gradient-based optimizaton.
\end{document}


% !TEX root = index.tex

\section{Gluing it Back Together}
\epigraph{The only way to learn math is to do math.}{Paul Halmos}
\begin{align*}
	\xymatrix@R-2pc{
	\U \ar@{~>}[r]          & \L^\bullet \ar@{~>}[r]     & \check H^*(X)                \\
	\mbox{Good cover of } X & \mbox{Cech Complex of } \U & \mbox{Cech Cohomology of } X
	}
\end{align*}
We now know enough theory to define the {Cech Cohomology} of a space. We'll start by defining it in an ad hoc manner and later try and understand it better.

Let $\U$ be a good cover of $X$ consisting of $n$ sets. In order to define the Cech Complex $\L^\bullet$ we need two things:
\begin{alignat*}{4}
	 & \mbox{vector spaces } &  & \qquad \L^k                          \\
	 & \mbox{linear maps }   &  & \qquad d^k: \L^k\rightarrow \L^{k+1}
\end{alignat*}

\begin{definition}
	The vector space $\L^k$ has dimension
	\begin{align*}
		\dim \L^k = \sum \limits_{|I| = k+1} \mbox{ number of connected components of } U_I
	\end{align*}
	where $I$ varies over the non-empty subsets of $[n]$.\footnote{Recall that to every subset $ I \subseteq [n]$ we can associate a set $U_I $ defined as $U_I := \bigcap_{i \in I} U_i$}
\end{definition}
\begin{definition}
	The linear map $d^k$ is a matrix whose \\
  \begin{tabular}{l l}
  rows & correspond to the connected components of all the $U_I$ with $|I|=k+2$ \\
  columns & correspond to the connected components of all the $U_J$ with $|J|=k+1$ \\
  $(i,j)^{th}$ entry & equals 1 if and only if the $i^{th}$ connected component in $U_I$ is a subset of \\
  & the $j^{th}$ connected component in $U_J$ (see examples).
  \end{tabular}
\end{definition}
\begin{definition}
	Define the \textbf{Cech Cohomology} $\check H^k(X)$ of $X$ to be the cohomology of the above cochain complex $\check H^k(\L)$.
\end{definition}

\begin{example}
	\label{ex:triangle_2}
	For the triangle with cover being the three sides $ \: \U = \{U_1, U_2, U_3\}$
	\begin{figure}[H]
		\centering
		\begin{tikzpicture}[scale=0.6]
			%% vertices
			\draw[fill=black] (0,0) circle (3pt);
			\draw[fill=black] (4,0) circle (3pt);
			\draw[fill=black] (2,3) circle (3pt);
			%% vertex labels
			\node at (2,-0.5) {$U_1$};
			\node at (0.5,1.75) {$U_2$};
			\node at (3.5,1.75) {$U_3$};
			\node at (2,-0.5) {$U_1$};
			\node at (-1,0) {$U_{\{1,2\}}$};
			\node at (5,0) {$U_{\{1,3\}}$};
			\node at (2,3.5) {$U_{\{2,3\}}$};
			%%% edges
			\draw[thick] (0,0) -- (4,0) -- (2,3) -- (0,0);
		\end{tikzpicture}
		\caption{$\U = \{U_1, U_2, U_3\}$ is a good cover of the triangle, $U_{\{1,2\}}, U_{\{2,3\}}, U_{\{1,3\}}$ are the vertices, and $U_{\{1,2,3\}}$ is empty.}
		\label{fig:triangle_cech}
	\end{figure}
	\begin{align*}
		 & |I| = 0 &  & U_1, U_2, U_3                         &  & \L^0 = \F^3 \\
		 & |I| = 1 &  & U_{\{1,2\}}, U_{\{2,3\}}, U_{\{1,3\}} &  & \L^1 = \F^3
	\end{align*}
	We have inclusions $U_{\{1,2\}} \subseteq U_1$, $U_{\{1,2\}} \subseteq U_2$ etc. Hence the differential $d^0$ looks like
	\begin{center}
		\begin{tabular}{ l | c c c  }
			              & $U_1$ & $U_2$ & $U_3$ \\\hline
			$U_{\{1,2\}}$ & 1     & 1     & 0     \\
			$U_{\{2,3\}}$ & 0     & 1     & 1     \\
			$U_{\{1,3\}}$ & 1     & 0     & 1
		\end{tabular} $ = d^0$
	\end{center}
	So that
	\begin{align*}
		\L^\bullet
		 & =
		0 \rightarrow \F^3 \xrightarrow{\begin{bmatrix}1 & 1 & 0 \\0 & 1 & 1 \\ 1 & 0 & 1 \end{bmatrix}} \F^3 \rightarrow 0 \\
	\end{align*}
\end{example}

\begin{example}
	For the circle $S^1$ we can find a good cover consisting of semicircles $\: \U = \{U_1, U_2 \}$.
	\label{ex:circle_cech}
	\begin{figure}[H]
		\centering
		\begin{tikzpicture}[scale=0.5]
			%% vertices
			\draw (2,2) circle (2cm);
			\draw[fill=black] (0,2) circle (3pt);
			\draw[fill=black] (4,2) circle (3pt);
			\node at (2,4.5) {$U_1$};
			\node at (2,-0.5) {$U_2$};
			\node at (-0.5,1.5) {$a$};
			\node at (4.5,1.5) {$b$};
		\end{tikzpicture}
		\caption{$\{U_1, U_2\}$ is a good cover of the circle with $U_{\{1,2\}} = \{ a, b \}$}
		\label{fig:triangle_cech}
	\end{figure}
	\begin{align*}
		 & |I| = 0 &  & U_1, U_2               &  & \L^0 = \F^2 \\
		 & |I| = 1 &  & U_{\{1,2\}} = \{a, b\} &  & \L^1 = \F^2
	\end{align*}
	We have inclusions $\{ a \} \subseteq U_1$, $\{ a \} \subseteq U_2$ etc. Hence the differential $d^0$ looks like
	\begin{center}
		\begin{tabular}{ l | c c  }
			    & $U_1$ & $U_2$ \\\hline
			$a$ & 1     & 1     \\
			$b$ & 1     & 1
		\end{tabular} $ = d^0$
	\end{center}
	So that
	\begin{align*}
		\L^\bullet
		 & =
		0 \rightarrow \F^2 \xrightarrow{\begin{bmatrix}1 & 1  \\1 & 1 \end{bmatrix}} \F^2 \rightarrow 0
	\end{align*}
\end{example}
\newpage

\begin{ques}
	Compute the dimensions of the cohomologies in the above examples.
\end{ques}
Compute the dimensions of the cohomologies for the following spaces. Interpret your results topologically.

\begin{ques} $ $
	\begin{multicols}{2}
		\begin{enumerate}
			\item $ \R^n$
			\item A tree
			\item $d$ points in $\R^2$
			\item The bipartite graph $ K_{2,3}$
			\item $ \R^2 \setminus \{(0,0) \}$
			\item $ \R^2 \setminus \{(0,0), (1,0), \dots, (d,0) \}$ for some positive integer $ d$
			\item $ S^2 $ minus a point
			\item $ S^2 $ minus 2 points
		\end{enumerate}
	\end{multicols}
\end{ques}

\begin{ques} $ $
	\begin{enumerate}
		\item $S^1 \vee S^1$
		\item* Bouquet of $n$-circles\\
		      \begin{tikzpicture}[scale=0.6]
			      \begin{polaraxis}[grid=none, axis lines=none]
				      \addplot[mark=none,domain=0:360,samples=300] { abs(cos(8*x/2))};
			      \end{polaraxis}
		      \end{tikzpicture}
	\end{enumerate}
\end{ques}

\begin{ques} $ $
	\begin{enumerate}
		\item* $S^2$
		\item* $S^1 \vee S^2$
		\item** $S^1 \times S^1$ = torus (try to find a cover with 3 sets)
		\item*** $g$-holed torus
		\item Solid torus
	\end{enumerate}
\end{ques}

\newpage
The \emph{basis independent} way of defining the Cech Cohomology is as follows. Let $(U_{I_1}, U_{I_2}, \dots, U_{I_m})$ denote the non-empty sets $U_I$ with $|I| = k+1$. Then $L^k$ is the direct sum
\begin{align*}
	\L^k = \L(U_{I_1}) \oplus \L(U_{I_2}) \oplus \dots \oplus \L(U_{I_m})
\end{align*}
The differential maps $d^k$ are direct sums of the restriction maps $\res_{U_I \rightarrow U_J}$ for $|I|=k+1 , |J| = k+2$.
\begin{ques}*
	Prove this.
\end{ques}

\begin{ques}***
	Following is the proof of the fact that the Cech complex is indeed a cochain complex i.e. it satisfies $d^{k+1} \circ d^{k} = 0$:\\

  $\quad$ Consider subsets $J \subset I \subseteq [n]$ such that $k+1 = |J| = |I| - 2$ and $I = J \cup \{\alpha, \beta \}$. Let $J_{\alpha} := J \cup \{ \alpha \}$ and $J_{\beta} := J \cup \{ \beta \}$. Show that we have the following induced maps:
		\begin{align*}
	    J \subset J_\alpha \subset I \\
	      J \subset J_\beta \subset I
	  \end{align*}
		\begin{align*}
	    U_{I} \subset U_{J_\alpha} \subset U_{J} \\
	    U_{I} \subset U_{J_\beta} \subset U_{J}
	  \end{align*}
		\begin{align*}
			\phi_\alpha: \L(U_J) \xrightarrow{\res_{U_J \rightarrow U_{J_\alpha}}} \L(U_{J_\alpha}) \xrightarrow{\res_{U_{J_\alpha} \rightarrow U_{I}}} \L(U_I) \\
			\phi_\beta: 	\L(U_J) \xrightarrow{\res_{U_J \rightarrow U_{J_\beta}}} \L(U_{J_\beta}) \xrightarrow{\res_{U_{J_\beta} \rightarrow U_{I}}} \L(U_I)
		\end{align*}
		and $U_{I} = U_{J_\alpha} \cap U_{J_\beta}$, $U_{J} = U_{J_\alpha} \cup U_{J_\beta}$.
		Show that $d^{k+1} \circ d^{k}|_{\L(U_J)} = \phi_{\alpha} + \phi_{\beta}$. Argue that this implies $d^{k+1} \circ d^{k}|_{\L(U_J)} = 0$ and hence $d^{k+1} \circ d^{k} = 0$.
\end{ques}

Finally, notice that we're saying that for finding the Cech cohomology of $X$ we needed to choose a good open cover $\U$. It is a deep theorem that this choice does not matter. The only proof I know of this uses heavy homological algebra machinery.

\begin{thm}
	\label{thm:good_covers_theorem}
	If $ \U$, $\U'$ are \textbf{good covers} of $ X$ with associated Cech complexes $ \L^\bullet(\U)$ and $ \L^\bullet(\U')$ then
	\begin{align}
		H^i(\L(\U)) \cong  H^i(\L(\U'))
	\end{align}
	for all $ i \in \Z$ i.e. the cohomology of the Cech complex is independent of the good cover and hence is an invariant of the underlying space. Hence, we can define the Cech cohomology of $ X$ as
	\begin{align}
		\check H^i(X; \F) :=  H^i(\L(\U))
	\end{align}
	for any good cover $ \U$ of $ X$.
\end{thm}


%% -*- coding:utf-8 -*-
\documentclass{llncs}

\begin{document}

\title{Analyze of Quantum Fourier Transform Circuit Implementation}
\author{Ivan Murashko, Constantine Korikov}
\institute{Peter the Great St. Petersburg Polytechnic University,
  St. Petersburg, 195251. Russia,\\
  \email{ivan.murashko@gmail.com},
  \email{korikov.constantine@spbstu.ru}
}
\maketitle

\begin{abstract}
  Quantum Fourier transform circuit is the key element of quantum computation.
  Originally it was introduced in the Shor's paper containing a classical
  proof of correctness. We propose another corroboration that shows
  the role of each element of the circuit and can be used for the scheme analyze.
  \keywords{
    quantum Fourier transform,
    Shor's algorithm,
    quantum computing
  }
\end{abstract}

\section{Introduction}
Discrete Fourier transform (DFT) can be used for periodic sequence
(functions) analysis. The transformation is defined by the following
equation:
\begin{equation}
\tilde{X}_k = \sum^{M - 1}_{m = 0} x_m e^{-i \frac{2 \pi}{M} k\cdot m},
\label{eqPart4QuantCompShorFourierDiscretFourier}
\end{equation}
where the input sequence $\left\{x_m\right\}$ has $M$
members.

Shor \cite{shor} suggested a quantum circuit
(see Fig. \ref{figQuantCompQuantFourier}d)
that can be used for DFT
in quantum algorithms \cite{shor,kitaev,lanyon,ettinger} especially in the
well known Shor's algorithm for factorization problem solving
\cite{shor}.

Quantum Fourier transform \cite{shor} operates with the following states
\begin{equation}
\left|x\right> = \sum_{k = 0}^{M - 1}x_k \left|k\right>,
\label{eqPart4QuantCompShorQuantFourierSeries}
\end{equation}
where the sequence of amplitudes $\left\{x_k\right\}$ forms initial
sequence for the Fourier transform
(\ref{eqPart4QuantCompShorFourierDiscretFourier}).
The basis vector $\left|k\right>$ keeps the number of a sequence's element.

The state (\ref{eqPart4QuantCompShorQuantFourierSeries})
should be normalized:
\[
\sum_k\left|x_k\right|^2 = 1.
\]

Lets an operator $\hat{F}^{M}$ (quantum Fourier operator) transforms
basis vector  $\left|k\right>$ by means of a rule defined by equation
(\ref{eqPart4QuantCompShorFourierDiscretFourier}):
\begin{equation}
\hat{F}^{M}\left|k\right> = \frac{1}{\sqrt{M}}\sum_{j = 0}^{M -1}
e^{-i \omega k j}\left|j\right>_{inv},
\label{eqPart4QuantCompShorQuantFourierBasis}
\end{equation}
where $\omega = \frac{2 \pi}{M}$.
Basis vectors $\left\{\left|k\right>\right\}$ and
$\left\{\left|k\right>_{inv}\right\}$ are the same set of vectors
which are enumerated in a different order
(see Fig. \ref{figQuantCompQuantFourier}a and Example
\ref{exInputData}).

From (\ref{eqPart4QuantCompShorQuantFourierSeries}) and
(\ref{eqPart4QuantCompShorQuantFourierBasis}) one can get
\begin{eqnarray}
\hat{F}^{M}\left|x\right> = \sum_{j = 0}^{M - 1}x_k \hat{F}^{M}
\left|k\right> =
\nonumber \\
= \frac{1}{\sqrt{M}}\sum_{k = 0}^{M -1}\sum_{j = 0}^{M - 1}
e^{-i \omega k j}x_k\left|j\right>_{inv} =
\nonumber \\
= \sum_{j = 0}^{M - 1} \left\{\frac{1}{\sqrt{M}}\left(
\sum_{k = 0}^{M - 1}e^{-i \omega k j} x_k
\right)\right\}\left|j\right>_{inv} =
\nonumber \\
= \sum_{j = 0}^{M - 1}\tilde{X}_j\left|j\right>_{inv} = \left|\tilde{X}\right>_{inv},
\label{eqPart4QuantCompShorQuantFourierEnd1}
\end{eqnarray}
where
\begin{equation}
\tilde{X}_j = \tilde{X}_j^{M} =
\frac{1}{\sqrt{M}}\sum_{k = 0}^{M - 1}e^{-i \omega k j} x_k.
\label{eqPart4QuantCompShorQuantFourierEnd}
\end{equation}
The equation (\ref{eqPart4QuantCompShorQuantFourierEnd}) conforms the
classical ones that is used for discrete Fourier transform
(\ref{eqPart4QuantCompShorFourierDiscretFourier}), i.e. one can write
\[
 \left|x\right> \longleftrightarrow \left|\tilde{X}\right>_{inv}.
\]

We are going to show that the circuit shown on
Fig. \ref{figQuantCompQuantFourier}d does the transformation
defined by (\ref{eqPart4QuantCompShorQuantFourierEnd}). The subject of
analyze consists of several elements: quantum Fourier transform gate for higher
bits ($\hat{F}^{\frac{M}{2}}$), phase shift and Hadamard
gate (see Fig. \ref{figQuantCompQuantFourier}). If the elements
are added to the circuit step by step then the different
transformations of initial data are performed. The transformations are
described below in details.

\section{Input data}

Lets the input data for the system is a quantum state
(\ref{eqPart4QuantCompShorQuantFourierSeries}) that is a superposition
of $M$ basis states $\left\{\left|k\right>\right\}$
(see Fig. \ref{figQuantCompQuantFourier}a).
Lets the number of basis states is a power of 2, i.e. a basis state
can be present as a tensor product of
$n = \log_2{M}$ qbits:
\begin{equation}
\left|k\right> = \left|a^{(k)}_0\right> \otimes  \left|a^{(k)}_1\right>
\otimes \cdots \otimes \left|a^{(k)}_{n-1}\right>,
\nonumber
\end{equation}
where
\begin{eqnarray}
k = a^{(k)}_0 + 2^1 a^{(k)}_1 + \dots + 2^{n-1} a^{(k)}_{n-1},
\nonumber \\
a^{(k)}_i \in \left\{0, 1\right\}.
\nonumber
\end{eqnarray}

There is a superposition of $M$ basis states
$\left\{\left|j\right>_{inv}\right\}$ on the input
(see Fig. \ref{figQuantCompQuantFourier}a).
For the state
$\left|j\right>_{inv}$ one can get
\begin{equation}
\left|j\right>_{inv} = \left|b^{(j)}_{n-1}\right> \otimes
\left|b^{(j)}_{n-2}\right>
\otimes \cdots \otimes \left|b^{(j)}_{0}\right>,
\nonumber
\end{equation}
where
\begin{eqnarray}
j = b^{(j)}_0 + 2^1 b^{(j)}_1 + \dots + 2^{n-1} b^{(j)}_{n-1},
\nonumber \\
b^{(j)}_i \in \left\{0, 1\right\}.
\nonumber
\end{eqnarray}

\begin{example}[Input data]
  \label{exInputData}
  Let $M=8$ i.e. $n=3$. In the case the number 3 is represented as
  follows
  \[
  \left|3\right> = \left|1\right> \otimes  \left|1\right>
  \otimes  \left|0\right>.
  \]
  and
  \[
  \left|3\right>_{inv} = \left|0\right> \otimes  \left|1\right>
  \otimes  \left|1\right>.
  \]
\end{example}

Using well known Cooley–Tukey fast Fourier transform algorithm
\cite{fft} the (\ref{eqPart4QuantCompShorFourierDiscretFourier}) can be
rewritten in the following form
\begin{equation}
\tilde{X}_k = \sum^{\frac{M}{2} - 1}_{m = 0} F_{k,m}^{\frac{M}{2}} x_{2m} +
\exp{\left(-2\pi i \frac{k}{M}\right)}
\sum^{\frac{M}{2} - 1}_{m = 0}  F_{k,m}^{\frac{M}{2}} x_{2m + 1}.
\label{eqAddFourierDiscretFourierFFT}
\end{equation}
We can see (\ref{eqAddFourierDiscretFourierFFT}) that if an
input signal $x$ consists of $n = \log_2{M}$
bits when bit $a^{(k)}_0$ can be used to choose even
(the first part of sum (\ref{eqAddFourierDiscretFourierFFT}))
or odd
(the second part of sum (\ref{eqAddFourierDiscretFourierFFT}))
members.

Thus the state
(\ref{eqPart4QuantCompShorQuantFourierSeries})
can be presented in the form of sum of even and odd components:
\begin{eqnarray}
\left|x\right> = \sum_{k = 0}^{M - 1}x_k \left|k\right> =
\sum_{k = 0}^{M - 1}x_k \left|a^{(k)}_0\right> \otimes  \left|a^{(k)}_1\right>
\otimes \cdots \otimes \left|a^{(k)}_{n-1}\right> =
\nonumber \\
 = \sum_{m = 0}^{\frac{M}{2} - 1}x_{k=2m} \left|0\right> \otimes  \left|a^{(k)}_1\right>
\otimes \cdots \otimes \left|a^{(k)}_{n-1}\right> +
\nonumber \\
+
\sum_{m = 0}^{\frac{M}{2} - 1}x_{k=2m + 1} \left|1\right> \otimes  \left|a^{(k)}_1\right>
\otimes \cdots \otimes \left|a^{(k)}_{n-1}\right> =
\nonumber \\
 = \sum_{m = 0}^{\frac{M}{2} - 1}x_{k=2m} \left|0\right> \otimes  \left|m\right> +
\sum_{m = 0}^{\frac{M}{2} - 1}x_{k=2m + 1} \left|1\right> \otimes  \left|m\right> =
\nonumber \\
= \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m} \left|2m\right> +
\sum_{m = 0}^{\frac{M}{2} - 1}x_{2m + 1} \left|2m+1\right>,
\nonumber
\end{eqnarray}
where
\begin{equation}
m = a^{(k)}_1 + 2^1 a^{(k)}_2 + \dots + 2^{n-2} a^{(k)}_{n-1}.
\nonumber
\end{equation}

\section{Fourier transform for high bits}

With Fourier transform for high bits only in $\hat{F}^{\frac{M}{2}}$,
i.e. with excluding $a^{(k)}_0$ we can obtain (see
Fig. \ref{figQuantCompQuantFourier}b):
\begin{eqnarray}
\left|x\right> \rightarrow
\hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m} \left|2m\right> +
\hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m + 1}
\left|2m+1\right> =
\nonumber \\
=
\hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m}
\left|0\right> \otimes  \left|m\right> +
\hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m + 1}
\left|1\right> \otimes  \left|m\right>
=
\nonumber \\
=
\sum_{m = 0}^{\frac{M}{2} - 1}x_{2m}
\left|0\right> \otimes \hat{F}^{\frac{M}{2}} \left|m\right> +
\sum_{m = 0}^{\frac{M}{2} - 1}x_{2m + 1}
\left|1\right> \otimes \hat{F}^{\frac{M}{2}} \left|m\right>.
\label{eqPart4QuantCompShorQuantFourierStep1}
\end{eqnarray}
Using (\ref{eqPart4QuantCompShorQuantFourierBasis}) it can be got
\begin{equation}
\hat{F}^{\frac{M}{2}} \left|m\right> = \sqrt{\frac{2}{M}}
\sum_{j= 0}^{\frac{M}{2} - 1} e^{-i \frac{4 \pi}{M} m j}\left|j\right>_{inv}.
\nonumber
\end{equation}
Thus for (\ref{eqPart4QuantCompShorQuantFourierStep1}) it holds
\begin{eqnarray}
\left|x\right> \rightarrow
\sum_{m = 0}^{\frac{M}{2} - 1}x_{2m}
\left|0\right> \otimes \hat{F}^{\frac{M}{2}} \left|m\right> +
\sum_{m = 0}^{\frac{M}{2} - 1}x_{2m + 1}
\left|1\right> \otimes \hat{F}^{\frac{M}{2}} \left|m\right> =
\nonumber \\
=
\sqrt{\frac{2}{M}} \sum_{j = 0}^{\frac{M}{2} - 1} e^{-i \frac{4 \pi}{M} m j}
\sum_{m = 0}^{\frac{M}{2} - 1}x_{2m} \left|0\right> \otimes
\left|j\right>_{inv}
+
\nonumber \\
+
\sqrt{\frac{2}{M}} \sum_{j = 0}^{\frac{M}{2} - 1} e^{-i \frac{4 \pi}{M} m j}
\sum_{m = 0}^{\frac{M}{2} - 1}x_{2m+1} \left|1\right> \otimes
\left|j\right>_{inv}
=
\nonumber \\
=
\sum_{j = 0}^{\frac{M}{2} - 1}
\left( \sqrt{\frac{2}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1} e^{-i \frac{4 \pi}{M} m j} x_{2m}
\right) \left|j\right>_{inv}
+
\nonumber \\
+
\sum_{j = 0}^{\frac{M}{2} - 1}
\left( \sqrt{\frac{2}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1}e^{-i \frac{4 \pi}{M} m j} x_{2m+1}
\right)
\left|\frac{M}{2} + j\right>_{inv}
=
\nonumber \\
= \sum^{\frac{M}{2} - 1}_{j = 0}  \tilde{A}_{j} \left|j\right>_{inv} +
\sum^{\frac{M}{2} - 1}_{j = 0}  \tilde{B}_{j} \left|\frac{M}{2} + j\right>_{inv},
\label{eqStep1}
\end{eqnarray}
where
\begin{eqnarray}
\tilde{A}_{j} =
\sqrt{\frac{2}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1} e^{-i \frac{4 \pi}{M} m j} x_{2m}
\nonumber \\
\tilde{B}_{j} =
\sqrt{\frac{2}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1} e^{-i \frac{4 \pi}{M} m j} x_{2m+1}
\label{eqPart4QuantCompShorAB}
\end{eqnarray}

\section{Phase shift}

As we can see the equations (\ref{eqStep1}, \ref{eqPart4QuantCompShorAB}) do
not correspond to the required Fourier transform
(\ref{eqPart4QuantCompShorQuantFourierEnd1}).
One of the problem is that components $x_{2m}$ and $x_{2m+1}$ in
(\ref{eqPart4QuantCompShorAB}) have the same phase but it should
differ. The problem can be fixed if a phase shift is added for odd
components i.e.  for elements with $a_0^k = 1$. As result the circuit
shown on Fig. \ref{figQuantCompQuantFourier}c is got:
\begin{eqnarray}
\left|x\right> \rightarrow
\hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m} \left|2m\right> +
\hat{R}\hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1} x_{2m + 1}
\left|2m+1\right> =
\nonumber \\
=
\sum^{\frac{M}{2} - 1}_{j = 0} \tilde{A}_{j} \left|j\right>_{inv} +
\sum^{\frac{M}{2} - 1}_{j = 0}
\tilde{B}_{j} \hat{R}\left|\frac{M}{2} + j\right>_{inv},
\nonumber \\
=
\sum^{\frac{M}{2} - 1}_{j = 0}  \tilde{A}_{j} \left|j\right>_{inv} +
\sum^{\frac{M}{2} - 1}_{j = 0}
\tilde{C}_{j} \left|\frac{M}{2} + j\right>_{inv}.
\label{eqPart4QuantCompShorFourierStep2}
\end{eqnarray}
With equation
\begin{equation}
\hat{R}_l \left|b^{(j)}_l\right> =
exp{\left(- 2 \pi i \frac{b^{(j)}_l}{2^{n-l}}\right)}
\left|b^{(j)}_l\right>
\nonumber
\end{equation}
we can get that operator $\hat{R}$ applied to state
$\left|\frac{M}{2} + j\right>_{inv}$ will produce the following result:
\begin{eqnarray}
\hat{R}\left|\frac{M}{2} + j\right>_{inv} =
\hat{R}\left|1\right> \otimes  \left|j\right>_{inv} =
\nonumber \\
=
\left|1\right> \otimes \hat{R}_0 \left|b^{(j)}_0\right>
\otimes \cdots \otimes \hat{R}_{n-2} \left|b^{(j)}_{n-2}\right> =
\nonumber \\
=
\prod_{l = 0}^{n-2}exp{\left(- 2 \pi i \frac{2^lb^{(j)}_l}{2^n}\right)}
\left|1\right> \otimes \left|j\right>_{inv} =
\nonumber \\
=
exp{\left(-2 \pi i \frac{j}{M}\right)}
\left|\frac{M}{2} + j\right>_{inv}
\label{eqPart4QuantCompShorFourierStep2Pre}
\end{eqnarray}
We used the following fact
\[
j = b^{(j)}_0 + 2^1 b^{(j)}_1 + \dots + 2^{n-2} b^{(j)}_{n-2}
\]
for (\ref{eqPart4QuantCompShorFourierStep2Pre}) derivation.

Thus for $\tilde{C}_{j}$ in
(\ref{eqPart4QuantCompShorFourierStep2}) it can be got
\begin{eqnarray}
\tilde{C}_{j} =
\sqrt{\frac{2}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1}
e^{- 2 \pi i \frac{j}{M}}
e^{-i \frac{4 \pi}{M} m j} x_{2m+1} =
\nonumber \\
=
\sqrt{\frac{2}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1}
e^{-i \frac{2 \pi}{M} \left(2m+1\right) j} x_{2m+1}
\label{eqPart4QuantCompShorC}
\end{eqnarray}

\section{Hadamard transformation for the lowest bit}
The result after phase shift still does not produce the required
(Fourier) transform and additional manipulations are
required. Especially the result of sum for even components is that
required. Another situation with odd ones where sum should be done
with additional phase shift. The required transformation is provided
by Hadamard gate.

If the Hadamard transformation is applied for the qbit
$\left|a_0\right>$ then the circuit shown on
Fig. \ref{figQuantCompQuantFourier}d is got.
The initial state is transformed by the following law:
\begin{eqnarray}
\left|x\right> \rightarrow
\hat{H}\hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m} \left|2m\right> +
\hat{H}\hat{R}\hat{F}^{\frac{M}{2}}\sum_{m = 0}^{\frac{M}{2} - 1} x_{2m + 1} =
\nonumber \\
=
\sum_{j = 0}^{\frac{M}{2} - 1}
\tilde{A}_{j}
\hat{H}\left|0\right> \otimes \left|j\right>_{inv}
+
\sum_{j = 0}^{\frac{M}{2} - 1}
\tilde{C}_{j}
\hat{H}\left|1\right> \otimes \left|j\right>_{inv}
=
\nonumber \\
=
\frac{1}{\sqrt{2}}\sum_{j = 0}^{\frac{M}{2} - 1}
\tilde{A}_{j}
\left(\left|0\right> + \left|1\right> \right) \otimes
\left|j\right>_{inv}
+
\frac{1}{\sqrt{2}}\sum_{j = 0}^{\frac{M}{2} - 1}
\tilde{C}_{j}
\left(\left|0\right> - \left|1\right> \right) \otimes
\left|j\right>_{inv}
=
\nonumber \\
=
\sum_{j = 0}^{\frac{M}{2} - 1}
\frac{\tilde{A}_{j} + \tilde{C}_{j} }{\sqrt{2}}
\left|0\right> \otimes \left|j\right>_{inv} +
\sum_{j = 0}^{\frac{M}{2} - 1}
\frac{ \tilde{A}_{j} - \tilde{C}_{j}}{\sqrt{2}}
\left|1\right> \otimes \left|j\right>_{inv}
=
\nonumber \\
=
\sum_{j = 0}^{\frac{M}{2} - 1}
\frac{\tilde{A}_{j} + \tilde{C}_{j} }{\sqrt{2}} \left|j\right>_{inv} +
\sum_{j = 0}^{\frac{M}{2} - 1}
\frac{ \tilde{A}_{j} - \tilde{C}_{j}}{\sqrt{2}}
\left|\frac{M}{2} + j \right>_{inv}.
\label{eqPart4QuantCompShorFourierStep3}
\end{eqnarray}
For (\ref{eqPart4QuantCompShorFourierStep3}) members using
(\ref{eqPart4QuantCompShorAB}) and (\ref{eqPart4QuantCompShorC}) one
can conclude:
\begin{eqnarray}
\frac{\tilde{A}_{j} + \tilde{C}_{j} }{\sqrt{2}} =
\sqrt{\frac{1}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1} e^{-i \frac{4 \pi}{M} m j} x_{2m}  +
\sqrt{\frac{1}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1}
e^{-i \frac{2 \pi}{M} \left(2m+1\right) j} x_{2m+1} =
\nonumber \\
=
\sqrt{\frac{1}{M}} \sum_{m = 0}^{M - 1}
e^{-i \frac{2 \pi}{M} m j} x_{m}
\label{eqPart4QuantCompShorFourierStep3_1}
\end{eqnarray}
and
\begin{eqnarray}
\frac{\tilde{A}_{j} - \tilde{C}_{j} }{\sqrt{2}} =
\sqrt{\frac{1}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1} e^{-i \frac{4 \pi}{M} m j} x_{2m}  -
\sqrt{\frac{1}{M}}
\sum_{m = 0}^{\frac{M}{2} - 1}
e^{-i \frac{2 \pi}{M} \left(2m+1\right) j} x_{2m+1}
=
\nonumber \\
=
\sqrt{\frac{1}{M}} \sum_{m = 0}^{M - 1}
e^{-i \frac{2 \pi}{M} m j} x_{m} \frac{1 + e^{-i \pi m}}{2}
-
\sqrt{\frac{1}{M}} \sum_{m = 0}^{M - 1}
e^{-i \frac{2 \pi}{M} m j} x_{m} \frac{1 - e^{-i \pi m}}{2}
=
\nonumber \\
=
\sqrt{\frac{1}{M}} \sum_{m = 0}^{M - 1}
e^{-i \frac{2 \pi}{M} m j} e^{-i \pi m } x_{m}
=
\sqrt{\frac{1}{M}} \sum_{m = 0}^{M - 1}
e^{-i \frac{2 \pi}{M} m j} e^{-i \frac{2 \pi}{M} m \frac{M}{2} } x_{m}
=
\nonumber \\
=
\sqrt{\frac{1}{M}} \sum_{m = 0}^{M - 1}
e^{-i \frac{2 \pi}{M} m \left(\frac{M}{2} + j\right)} x_{m}
\label{eqPart4QuantCompShorFourierStep3_2}
\end{eqnarray}

Combine (\ref{eqPart4QuantCompShorFourierStep3}),
(\ref{eqPart4QuantCompShorFourierStep3_1}) and
(\ref{eqPart4QuantCompShorFourierStep3_2}) finally it can be got
\begin{eqnarray}
\left|x\right> \rightarrow
\sum_{j = 0}^{\frac{M}{2} - 1} \sqrt{\frac{1}{M}} \sum_{m = 0}^{M - 1}
e^{-i \frac{2 \pi}{M} m j} x_{m} \left|j\right>_{inv} +
\nonumber \\
+
\sum_{j = 0}^{\frac{M}{2} - 1} \sqrt{\frac{1}{M}} \sum_{m = 0}^{M - 1}
e^{-i \frac{2 \pi}{M} m \left(\frac{M}{2} + j\right)} x_{m}
\left|\frac{M}{2} + j\right>_{inv} =
\nonumber \\
= \sum_{j = 0}^{M - 1} \tilde{X}_j^{M} \left|j\right>_{inv}
\nonumber
\end{eqnarray}

Thus the final scheme (Fig. \ref{figQuantCompQuantFourier}d)
provides us the required Fourier transformation
(\ref{eqPart4QuantCompShorQuantFourierEnd1}).

\section{Conclusion}
As a result of the scheme analyze the proof of correctness was
got. The scheme analyze was divided into several steps. The first one
included Fourier transform for highest bits only. Unfortunately the
result state did not conform the requested one. Especially a phase is
not correct. The issue can be fixed with phase shift correction
elements that were added in the second step. As result the correct
Fourier transform was got for even components but not for odd
ones. The problem can be resolved by means of the Hadamard gate that was
added on the last step. It was shown that the final circuit performs
the requested Fourier transform.

\begin{thebibliography}{}
   \bibitem{shor} Shor, P.: Algorithms for Quantum Computation:
     Discrete Logarithms and Factoring. FOCS, 124--134 (1994)
   \bibitem{kitaev} Kitaev A. Yu: Quantum measurements and the
     Abelian Stabilizer Problem, arXiv:quant-ph/9511026.
   \bibitem{lanyon} Lanyon, B. P.; Weinhold, T. J.; Langford,
     N. K.; Barbieri, M.; James, D. F. V.; Gilchrist, A.; White,
     A. G. (2007): Experimental Demonstration of a Compiled Version
     of Shor's Algorithm with Quantum Entanglement. Physical Review
     Letters 99 (25): 250505
   \bibitem{ettinger} Mark Ettinger; Peter Høyer: A quantum
     observable for the graph isomorphism problem, arXiv:quant-ph/9901029
   \bibitem{fft} Cooley, James W.; Tukey, John W. (1965): An
     algorithm for the machine calculation of complex Fourier
     series. Math. Comput. 19: 297-–301. doi:10.2307/2003354
\end{thebibliography}

\newpage
\begin{figure}
\ifx\JPicScale\undefined\def\JPicScale{1}\fi
\unitlength \JPicScale mm
\begin{picture}(127.57,94.5)(0,0)
\linethickness{0.3mm}
\put(4.61,79.5){\line(1,0){11.52}}
\linethickness{0.3mm}
\put(4.61,67){\line(1,0){11.52}}
\linethickness{0.3mm}
\put(4.61,62){\line(1,0){11.52}}
\linethickness{0.3mm}
\put(16.13,79.5){\line(1,0){11.53}}
\linethickness{0.3mm}
\put(27.66,79.5){\line(1,0){26.9}}
\linethickness{0.3mm}
\put(16.13,67){\line(1,0){23.06}}
\linethickness{0.3mm}
\put(39.19,67){\line(1,0){15.36}}
\linethickness{0.3mm}
\put(16.13,62){\line(1,0){28.82}}
\linethickness{0.3mm}
\put(44.95,62){\line(1,0){9.61}}
\linethickness{0.3mm}
\put(50.72,89.5){\line(1,0){3.84}}
\put(0.77,62){\makebox(0,0)[cc]{$\left| a_{n-1} \right>$}}

\put(0.77,67){\makebox(0,0)[cc]{$\left| a_{n -2} \right>$}}

\put(0.77,89.5){\makebox(0,0)[cc]{$\left| a_0 \right>$}}
\put(0.77,97.0){\makebox(0,0)[cc]{(a)}}

\put(58.4,62){\makebox(0,0)[cc]{$\left| b_{0} \right>$}}

\put(58.4,67){\makebox(0,0)[cc]{$\left| b_{1} \right>$}}

\put(59.4,89.5){\makebox(0,0)[cc]{$\left| b_{n-1} \right>$}}

\linethickness{0.3mm}
\put(6.53,94.5){\line(1,0){46.11}}
\put(6.53,57){\line(0,1){37.5}}
\put(52.64,57){\line(0,1){37.5}}
\put(6.53,57){\line(1,0){46.11}}
\put(10.0,92){\makebox(0,0)[cc]{$\hat{F}^{M}$}}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}
\put(4.61,89.5){\line(1,0){15.37}}
\linethickness{0.3mm}
\put(31.5,89.5){\line(1,0){19.21}}
\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(0.77,79.5){\makebox(0,0)[cc]{$\left| a_k \right>$}}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(61.0,79.5){\makebox(0,0)[cc]{$\left| b_{n - 1 - k} \right>$}}

\linethickness{0.3mm}
\put(19.98,89.5){\line(1,0){11.52}}
\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}
\put(104.52,24.5){\line(1,0){3.84}}
\put(104.52,19.5){\line(0,1){5}}
\put(108.36,19.5){\line(0,1){5}}
\put(104.52,19.5){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(110.27,19.5){\line(1,0){3.84}}
\put(110.27,14.5){\line(0,1){5}}
\put(114.11,14.5){\line(0,1){5}}
\put(110.27,14.5){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(73.77,34.5){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(73.77,22){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(73.77,17){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(85.3,34.5){\line(1,0){7.68}}
\linethickness{0.3mm}
\put(96.82,34.5){\line(1,0){26.9}}
\linethickness{0.3mm}
\put(85.3,22){\line(1,0){19.21}}
\linethickness{0.3mm}
\put(108.36,22){\line(1,0){15.36}}
\linethickness{0.3mm}
\put(85.3,17){\line(1,0){24.97}}
\linethickness{0.3mm}
\put(114.11,17){\line(1,0){9.61}}
\linethickness{0.3mm}
\put(94.91,37){\line(0,1){7.5}}
\linethickness{0.3mm}
\put(106.43,24.5){\line(0,1){20}}
\linethickness{0.3mm}
\put(112.2,19.5){\line(0,1){25}}
\linethickness{0.3mm}
\put(95.35,44.27){\line(0,1){0.48}}
\multiput(95.09,43.93)(0.13,0.17){2}{\line(0,1){0.17}}
\put(94.72,43.93){\line(1,0){0.37}}
\multiput(94.46,44.27)(0.13,-0.17){2}{\line(0,-1){0.17}}
\put(94.46,44.27){\line(0,1){0.48}}
\multiput(94.46,44.74)(0.13,0.17){2}{\line(0,1){0.17}}
\put(94.72,45.08){\line(1,0){0.37}}
\multiput(95.09,45.08)(0.13,-0.17){2}{\line(0,-1){0.17}}

\linethickness{0.3mm}
\put(106.87,44.27){\line(0,1){0.48}}
\multiput(106.61,43.93)(0.13,0.17){2}{\line(0,1){0.17}}
\put(106.25,43.93){\line(1,0){0.37}}
\multiput(105.99,44.27)(0.13,-0.17){2}{\line(0,-1){0.17}}
\put(105.99,44.27){\line(0,1){0.48}}
\multiput(105.99,44.74)(0.13,0.17){2}{\line(0,1){0.17}}
\put(106.25,45.08){\line(1,0){0.37}}
\multiput(106.61,45.08)(0.13,-0.17){2}{\line(0,-1){0.17}}

\linethickness{0.3mm}
\put(112.64,44.27){\line(0,1){0.48}}
\multiput(112.38,43.93)(0.13,0.17){2}{\line(0,1){0.17}}
\put(112.01,43.93){\line(1,0){0.37}}
\multiput(111.75,44.27)(0.13,-0.17){2}{\line(0,-1){0.17}}
\put(111.75,44.27){\line(0,1){0.48}}
\multiput(111.75,44.74)(0.13,0.17){2}{\line(0,1){0.17}}
\put(112.01,45.08){\line(1,0){0.37}}
\multiput(112.38,45.08)(0.13,-0.17){2}{\line(0,-1){0.17}}

\linethickness{0.3mm}
\put(116.04,47){\line(1,0){3.84}}
\put(116.04,42){\line(0,1){5}}
\put(119.88,42){\line(0,1){5}}
\put(116.04,42){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(119.88,44.5){\line(1,0){3.84}}
\put(117.96,44.5){\makebox(0,0)[cc]{$\hat{H}$}}

\put(69.93,17){\makebox(0,0)[cc]{$\left| a_{n-1} \right>$}}

\put(69.93,22){\makebox(0,0)[cc]{$\left| a_{n -2} \right>$}}

\put(69.93,44.5){\makebox(0,0)[cc]{$\left| a_0 \right>$}}
\put(69.93,52.5){\makebox(0,0)[cc]{(d)}}

\put(127.57,17){\makebox(0,0)[cc]{$\left| b_{0} \right>$}}

\put(127.57,22){\makebox(0,0)[cc]{$\left| b_{1} \right>$}}

\put(128.57,44.5){\makebox(0,0)[cc]{$\left| b_{n-1} \right>$}}

\put(81.0,37){\makebox(0,0)[cc]{$\hat{F}^{\frac{M}{2}}$}}

\linethickness{0.3mm}
\put(75.7,49.5){\line(1,0){46.11}}
\put(75.7,12){\line(0,1){37.5}}
\put(121.8,12){\line(0,1){37.5}}
\put(75.7,12){\line(1,0){46.11}}
\put(79.0,47){\makebox(0,0)[cc]{$\hat{F}^{M}$}}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}
\put(73.77,44.5){\line(1,0){15.37}}
\linethickness{0.3mm}
\put(100.67,44.5){\line(1,0){11.53}}
\linethickness{0.3mm}
\put(112.2,44.5){\line(1,0){3.84}}
\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(112.2,17){\makebox(0,0)[cc]{$\hat{R}_{0}$}}

\put(106.43,22){\makebox(0,0)[cc]{$\hat{R}_{1}$}}

\linethickness{0.3mm}
\put(77.61,40.75){\line(1,0){7.69}}
\put(77.61,13.25){\line(0,1){27.5}}
\put(85.3,13.25){\line(0,1){27.5}}
\put(77.61,13.25){\line(1,0){7.69}}
\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(69.93,34.5){\makebox(0,0)[cc]{$\left| a_{k} \right>$}}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(130.0,34.5){\makebox(0,0)[cc]{$\left| b_{n - 1 - k} \right>$}}

\linethickness{0.3mm}
\put(92.98,37){\line(1,0){3.84}}
\put(92.98,32){\line(0,1){5}}
\put(96.82,32){\line(0,1){5}}
\put(92.98,32){\line(1,0){3.84}}
\linethickness{0.3mm}

%\put(94.91,34.5){\makebox(0,0)[cc]{$\hat{R}_k$}}

\linethickness{0.3mm}
\put(92.98,44.5){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(73.77,79.5){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(73.77,67){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(73.77,62){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(85.3,79.5){\line(1,0){11.52}}
\linethickness{0.3mm}
\put(96.82,79.5){\line(1,0){26.9}}
\linethickness{0.3mm}
\put(85.3,67){\line(1,0){23.05}}
\linethickness{0.3mm}
\put(108.36,67){\line(1,0){15.36}}
\linethickness{0.3mm}
\put(85.3,62){\line(1,0){28.81}}
\linethickness{0.3mm}
\put(114.11,62){\line(1,0){9.61}}
\linethickness{0.3mm}
\put(119.88,89.5){\line(1,0){3.84}}
\put(69.93,62){\makebox(0,0)[cc]{$\left| a_{n-1} \right>$}}

\put(69.93,67){\makebox(0,0)[cc]{$\left| a_{n -2} \right>$}}

\put(69.93,89.5){\makebox(0,0)[cc]{$\left| a_0 \right>$}}
\put(69.93,97.0){\makebox(0,0)[cc]{(b)}}

\put(127.57,62){\makebox(0,0)[cc]{$\left| b_{0} \right>$}}

\put(127.57,67){\makebox(0,0)[cc]{$\left| b_{1} \right>$}}

\put(128.57,89.5){\makebox(0,0)[cc]{$\left| b_{n-1} \right>$}}

\put(81.0,82){\makebox(0,0)[cc]{$\hat{F}^{\frac{M}{2}}$}}

\linethickness{0.3mm}
\put(75.7,94.5){\line(1,0){46.11}}
\put(75.7,57){\line(0,1){37.5}}
\put(121.8,57){\line(0,1){37.5}}
\put(75.7,57){\line(1,0){46.11}}
\put(79.0,92){\makebox(0,0)[cc]{$\hat{F}^{M}$}}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}
\put(73.77,89.5){\line(1,0){15.37}}
\linethickness{0.3mm}
\put(100.67,89.5){\line(1,0){19.21}}
\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}
\put(77.61,85.75){\line(1,0){7.69}}
\put(77.61,58.25){\line(0,1){27.5}}
\put(85.3,58.25){\line(0,1){27.5}}
\put(77.61,58.25){\line(1,0){7.69}}
\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(69.93,79.5){\makebox(0,0)[cc]{$\left| a_k \right>$}}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(130.0,79.5){\makebox(0,0)[cc]{$\left| b_{n - 1 - k} \right>$}}

\linethickness{0.3mm}

\linethickness{0.3mm}
\put(92.98,89.5){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(35.34,24.5){\line(1,0){3.85}}
\put(35.34,19.5){\line(0,1){5}}
\put(39.19,19.5){\line(0,1){5}}
\put(35.34,19.5){\line(1,0){3.85}}
\linethickness{0.3mm}
\put(41.11,19.5){\line(1,0){3.84}}
\put(41.11,14.5){\line(0,1){5}}
\put(44.95,14.5){\line(0,1){5}}
\put(41.11,14.5){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(4.61,34.5){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(4.61,22){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(4.61,17){\line(1,0){3.84}}
\linethickness{0.3mm}
\put(16.13,34.5){\line(1,0){7.69}}
\linethickness{0.3mm}
\put(27.66,34.5){\line(1,0){26.9}}
\linethickness{0.3mm}
\put(16.13,22){\line(1,0){19.21}}
\linethickness{0.3mm}
\put(39.19,22){\line(1,0){15.36}}
\linethickness{0.3mm}
\put(16.13,17){\line(1,0){24.98}}
\linethickness{0.3mm}
\put(44.95,17){\line(1,0){9.61}}
\linethickness{0.3mm}
\put(25.74,37){\line(0,1){7.5}}
\linethickness{0.3mm}
\put(37.27,24.5){\line(0,1){20}}
\linethickness{0.3mm}
\put(43.03,19.5){\line(0,1){25}}
\linethickness{0.3mm}
\put(26.19,44.27){\line(0,1){0.48}}
\multiput(25.93,43.93)(0.13,0.17){2}{\line(0,1){0.17}}
\put(25.56,43.93){\line(1,0){0.37}}
\multiput(25.29,44.27)(0.13,-0.17){2}{\line(0,-1){0.17}}
\put(25.29,44.27){\line(0,1){0.48}}
\multiput(25.29,44.74)(0.13,0.17){2}{\line(0,1){0.17}}
\put(25.56,45.08){\line(1,0){0.37}}
\multiput(25.93,45.08)(0.13,-0.17){2}{\line(0,-1){0.17}}

\linethickness{0.3mm}
\put(37.71,44.27){\line(0,1){0.48}}
\multiput(37.45,43.93)(0.13,0.17){2}{\line(0,1){0.17}}
\put(37.08,43.93){\line(1,0){0.37}}
\multiput(36.83,44.27)(0.13,-0.17){2}{\line(0,-1){0.17}}
\put(36.83,44.27){\line(0,1){0.48}}
\multiput(36.83,44.74)(0.13,0.17){2}{\line(0,1){0.17}}
\put(37.08,45.08){\line(1,0){0.37}}
\multiput(37.45,45.08)(0.13,-0.17){2}{\line(0,-1){0.17}}

\linethickness{0.3mm}
\put(43.48,44.27){\line(0,1){0.48}}
\multiput(43.22,43.93)(0.13,0.17){2}{\line(0,1){0.17}}
\put(42.85,43.93){\line(1,0){0.37}}
\multiput(42.59,44.27)(0.13,-0.17){2}{\line(0,-1){0.17}}
\put(42.59,44.27){\line(0,1){0.48}}
\multiput(42.59,44.74)(0.13,0.17){2}{\line(0,1){0.17}}
\put(42.85,45.08){\line(1,0){0.37}}
\multiput(43.22,45.08)(0.13,-0.17){2}{\line(0,-1){0.17}}

\linethickness{0.3mm}
\put(50.72,44.5){\line(1,0){3.84}}
\put(0.77,17){\makebox(0,0)[cc]{$\left| a_{n-1} \right>$}}

\put(0.77,22){\makebox(0,0)[cc]{$\left| a_{n -2} \right>$}}

\put(0.77,44.5){\makebox(0,0)[cc]{$\left| a_0 \right>$}}
\put(0.77,52.5){\makebox(0,0)[cc]{(c)}}

\put(58.4,17){\makebox(0,0)[cc]{$\left| b_{0} \right>$}}

\put(58.4,22){\makebox(0,0)[cc]{$\left| b_{1} \right>$}}

\put(59.4,44.5){\makebox(0,0)[cc]{$\left| b_{n - 1} \right>$}}

\put(12.0,37){\makebox(0,0)[cc]{$\hat{F}^{\frac{M}{2}}$}}

\linethickness{0.3mm}
\put(6.53,49.5){\line(1,0){46.11}}
\put(6.53,12){\line(0,1){37.5}}
\put(52.64,12){\line(0,1){37.5}}
\put(6.53,12){\line(1,0){46.11}}
\put(10.0,47){\makebox(0,0)[cc]{$\hat{F}^{M}$}}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}
\put(4.61,44.5){\line(1,0){15.37}}
\linethickness{0.3mm}
\put(31.5,44.5){\line(1,0){11.53}}
\linethickness{0.3mm}
\put(43.03,44.5){\line(1,0){7.68}}
\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(43.03,17){\makebox(0,0)[cc]{$\hat{R}_{0}$}}

\put(37.27,22){\makebox(0,0)[cc]{$\hat{R}_{1}$}}

\linethickness{0.3mm}
\put(8.45,40.75){\line(1,0){7.68}}
\put(8.45,13.25){\line(0,1){27.5}}
\put(16.13,13.25){\line(0,1){27.5}}
\put(8.45,13.25){\line(1,0){7.68}}
\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(0.77,34.5){\makebox(0,0)[cc]{$\left| a_{k} \right>$}}

\linethickness{0.3mm}

\linethickness{0.3mm}

\linethickness{0.3mm}

\put(61.0,34.5){\makebox(0,0)[cc]{$\left| b_{n-1-k} \right>$}}

\linethickness{0.3mm}
\put(23.82,37){\line(1,0){3.84}}
\put(23.82,32){\line(0,1){5}}
\put(27.66,32){\line(0,1){5}}
\put(23.82,32){\line(1,0){3.84}}
\linethickness{0.3mm}

%\put(25.74,34.5){\makebox(0,0)[cc]{$\hat{R}_{n-k-1}$}}

\linethickness{0.3mm}
\put(23.82,44.5){\line(1,0){3.84}}
\end{picture}
\caption{
  The quantum Fourier transform circuit based on the fast
  Fourier transform algorithm.
  Initial data on the input (a).
  Step 1 (b):
  $\left|x\right> \rightarrow
  \hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m} \left|2m\right> +
  \hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m + 1}
  \left|2m+1\right>$.
  Step 2 (c):
  $\left|x\right> \rightarrow
  \hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1}x_{2m} \left|2m\right> +
  \hat{R}\hat{F}^{\frac{M}{2}} \sum_{m = 0}^{\frac{M}{2} - 1} x_{2m + 1}$.
  Final circuit (d).
}
\label{figQuantCompQuantFourier}
\end{figure}

\end{document}


\documentclass[]{article}
\usepackage{indentfirst}
\usepackage{amsmath}
\usepackage{geometry}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{graphicx}
\geometry{a4paper,scale=0.7}

\title{GAM Zero-Inflated N-Mixture Model}
\author{Yunyi SHEN}
\begin{document}
\maketitle
\section{Model Setting}
\subsection{Combined Distribution of Latent Population Size and Imperfect Detections}
Distribution of counting data follows N-Mixture Models in every sample period number of individuals encountered follows binomial distribution given population size at site i. Assume population size follows Poisson distribution. Data's distribution is given by total probability theorem:
\begin{equation}
	\begin{aligned}
	P(\vec{d})&=\sum_{n}P(d|n,\theta)P(n|\theta)\\
	&=\sum_{n}Binom(d|n,p)Pois(n|\lambda)
	\end{aligned}
\end{equation}

\subsection{Zero Inflation}
Zero inflate is common in ecology data. For instance sites not occupied can cause zero inflation. One common way to deal with this is to use zero inflated distribution to model data. Here we generally follow occupancy modeling, assume for probability $psi$ a site were occupied and $1-\psi$ it is empty and cause zero inflation. Thus the full distribution of data $\vec{d}$ is given by:
\begin{equation}
	\begin{matrix}
	\sum_{n}Binom(d|n,p)Pois(n|\lambda) & \psi\\
	0 & 1-\psi
	\end{matrix}
\end{equation}

\subsection{Proof the Distribution of Latent n and Detections d is in Exponential Family}
\begin{proof}
First, set $(n,\vec{d})$ is the latent population and detection vectors at site i. I intend to prove that the distribution of this vector is with in exponential family witch has form:
\begin{equation}
	f(y|\theta)=b(y)exp(\eta^{T}T(y)-a(\eta))
\end{equation}
Assume detections given latent population n were binomial distributed with parameter $p_{j}$, thus:
\begin{equation}
	\begin{aligned}
		P(\vec{d}|n)&=\prod_{j=1}^{w}\binom{n}{d_{j}}p_{j}^{d_{j}}(1-p_{j})^{n-d_{j}}\\
		&=\prod_{j=1}^{w}\binom{n}{d_{j}}exp(\sum_{j=1}^{w}d_{j}log\frac{p_{j}}{1-p_{j}}+n\sum_{j=1}^{w}log(1-p_{j}))
	\end{aligned}
\end{equation}
which showed that binomial distribution given n belongs to exponential family.

Then assume latent population was Poisson distributed with rate $\lambda$
\begin{equation}
	\begin{aligned}
		P(n|\theta)&=e^{-\lambda}\frac{\lambda^{n}}{n!}\\
		&=\frac{1}{n!}exp(nlog(\lambda)-\lambda)
	\end{aligned}
\end{equation}
Thus the total probability contains the latent and detections is given by:
\begin{equation}
	\begin{aligned}
		P(n,\vec{d}|\theta)&=P(d|n,\theta)P(n|\theta)\\
		&=\frac{1}{n!}\prod_{j=1}^{w}\binom{n}{d_{j}}exp(\sum_{j=1}^{w}d_{j}log\frac{p_{j}}{1-p_{j}}+n\sum_{j=1}^{w}log(1-p_{j}))exp(nlog(\lambda)-\lambda)\\
		&=\frac{1}{n!}\prod_{j=1}^{w}\binom{n}{d_{j}}exp[\sum_{j=1}^{w}d_{j}log\frac{p_{j}}{1-p_{j}}+n(\sum_{j=1}^{w}log(1-p_{j})+log(\lambda))-\lambda]\\
		&=\frac{1}{n!}\prod_{j=1}^{w}\binom{n}{d_{j}}exp(\eta^{T}(n,\vec{d})-\lambda)
	\end{aligned}
\end{equation}
in which
\[
\eta^{T}=(\sum_{j=1}^{w}log(1-p_{j})+log(\lambda),log\frac{\vec{p}}{1-\vec{p}})
\]

\[
T(y)=y
\]
$\lambda$ can be calculated using $\eta$ since it contains all $p_{j}$ and $\lambda$ itself.
Later on, we note this function as
\[
f(n,d|\theta)
\]
\end{proof}

\section{Model Estimation}
\subsection{EM Algorithm to Deal with the Missing Population Size and Occupancy}
Since we are missing observation of Latent Population Size $n$ and Occupancy status $z$, we can not directly obtain the MLE for GAMs. Thus here we proposed an EM algorithm to deal with missing observations.

Instead of maximize log likelihood directly EM algorithm maximize the lower bound of the log likelihood every iteration. In each iteration, this algorithm contains two steps, Expectation(E) step and Maximization(M) step. In the E step, the algorithm will take expected value of log likelihood under the posterior distribution of missing observations, and in M step, maximize this expected value. The algorithm will iterate until estimation converges.

\subsubsection{E-step}
We first write down the total probability assuming knowing occupancy status $z$ and latent population size $n$:
\begin{equation}
	\begin{aligned}
	P(\vec{d},n,z)&=[\psi Pois(n|\lambda)\prod_{j=1}^{w}Bin(d_{j}|n,p_{j})]^{z}(I_{\vec{d}=0}(1-\psi))^{1-z}\\
	&=(\psi f(n,\vec{d}|\theta))^{z}(I_{\vec{d}=0}(1-\psi))^{1-z}
	\end{aligned}
\end{equation}
In E step of EM algorithm, we need $P(n,z|\vec{d})$, which need to sum all n up, here, we truncated it with some large N.
\begin{equation}
	\begin{aligned}
	P(n,z|\vec{d})&=\frac{P(n,z,\vec{d})}{\sum_{n=max(d_{j})}^{zN}\sum_{z=0}^{1}P(n,z,\vec{d})}\\
	&=\frac{(\psi f(n,\vec{d}|\theta))^{z}(I_{\vec{d}=0}(1-\psi))^{1-z}}{\psi \sum_{n=max(d_{j})}^{N}f(n,d|\theta)+(1-\psi)I_{\vec{d}=0}}
	\end{aligned}
\end{equation}
Take expectation of logL of total likelihood under $r^{th}$ $\theta$ given by summing every z and n up:
\begin{equation}
	\begin{aligned}
	\mathbb{E}(l_{p}|\vec{d},\theta^{[r]})=\sum_{sites}\frac{\sum_{n}^{N}\psi^{[r]}f(n,\vec{d}|\theta^{[r]})log[\psi f(n,\vec{d}|\theta)]+(1-\psi^{[r]})I_{\vec{d}=0}log[(1-\psi)I_{\vec{d}=0}]}{\sum_{n}^{N}\psi^{[r]}f(n,\vec{d}|\theta^{[r]})+(1-\psi^{[r]})I_{\vec{d}=0}}
	\end{aligned}
\end{equation}
This involves 3 of GAMs to be maximized during M step.

Later on, we note the a normalizing constant given $\theta^{[r]}$ to be:
\[
\sum_{n}^{N}\psi^{[r]}f(n,\vec{d}|\theta^{[r]})+(1-\psi^{[r]})I_{\vec{d}=0}=Z^{[r]}
\]

and another constant:
\[
\sum_{n}^{N}f(n,\vec{d}|\theta^{[r]})=g^{[r]}
\]

and note:
\[
f(n,\vec{d}|\theta^{[r]})=f^{[r]}_{\vec{d}}(n)
\]

\subsubsection{M step, RILS Algorithm}

\section{Model Estimation}
We get derivative of $E_{l}$ and $\psi_{i}$ at site i
\begin{equation}
	\frac{\partial E_{l}}{\partial \psi_{i}}=\frac{\psi_{i}^{[r]}g_{i}^{[r]}}{\psi_{i}^{[r]}g_{i}^{[r]}+(1-\psi_{i}^{[r]})I_{\vec{d_{i}}=0}}-\psi_{i}
\end{equation}
Involves the first weighted GAM regarding occupancy rate $\psi$

Then get derivative of $f_{\vec{d}}(n)$ at site i
\begin{equation}
\frac{\partial E_{l}}{\partial f_{\vec{d_{i}}}(n)}=\frac{\psi_{i}^{[r]}}{Z^{[r]}}\sum_{n=0}^{N}\frac{f^{[r]}_{\vec{d_{i}}}(n)}{f_{\vec{d_{i}}}(n)}
\end{equation}
Now we can get the total derivative of the expected value of logL:
\begin{equation}
	\begin{aligned}
	\frac{\partial E_{l}}{\partial \beta_{k}}&=\sum_{i}\frac{\partial E_{l}}{\partial f_{\vec{d_{i}}}(n)}\frac{\partial f_{\vec{d_{i}}}(n)}{\partial \beta_{k}}+\sum_{i}\frac{\partial E_{l}}{\partial \psi_{i}}\frac{\partial \psi_{i}}{\partial \beta_{k}}\\
	&=\sum_{i}\frac{\psi_{i}^{[r]}}{Z^{[r]}}\sum_{n=0}^{N}f^{[r]}_{\vec{d_{i}}}(n)\frac{\partial log(f_{\vec{d_{i}}}(n))}{\partial \beta_{k}}+\sum_{i}\frac{\partial E_{l}}{\partial \psi_{i}}\frac{\partial \psi_{i}}{\partial \beta_{k}}\\
	&=\sum_{n=0}^{N}\sum_{i}\frac{\psi_{i}^{[r]}}{Z^{[r]}}f^{[r]}_{\vec{d_{i}}}(n)\frac{\partial log(f_{\vec{d_{i}}}(n))}{\partial \beta_{k}}+\sum_{i}\frac{\partial E_{l}}{\partial \psi_{i}}\frac{\partial \psi_{i}}{\partial \beta_{k}}\\
	\end{aligned}
\end{equation}
Further, we derive the setting of RILS for n and p:
\begin{equation}
	\begin{aligned}
	&\sum_{n=0}^{N}\sum_{i}\frac{\psi_{i}^{[r]}}{Z^{[r]}}f^{[r]}_{\vec{d_{i}}}(n)\frac{\partial log(f_{\vec{d_{i}}}(n))}{\partial \beta_{k}}\\
	=&\sum_{n=0}^{N}\sum_{i}[\frac{\psi_{i}^{[r]}}{Z^{[r]}}f^{[r]}_{\vec{d_{i}}}(n)\frac{\partial logPois(n,\lambda_{i})}{\partial \beta_{k}}+\sum_{j=1}^{w}\frac{\psi_{i}^{[r]}}{Z^{[r]}}f^{[r]}_{\vec{d_{i}}}(n)\frac{\partial logBin(p_{ij},n)}{\partial \beta_{k}}]\\
	\end{aligned}
\end{equation}
Note that:
\[
w_{i}^{r}(n)=\frac{\psi_{i}^{[r]}}{Z^{[r]}}f^{[r]}_{\vec{d_{i}}}(n)
\]
First deal with $\lambda$ which is easier:
\begin{equation}
	\begin{aligned}
	\sum_{n=0}^{N}\sum_{i}w_{i}^{[r]}(n)\frac{\partial logPois(n,\lambda_{i})}{\partial \beta_{k}}&=\sum_{i}\frac{\sum_{n=0}^{N}w_{i}^{[r]}(n)(n-\mu_{\lambda i})}{\phi V(\mu_{\lambda i})}\frac{\partial \mu_{\lambda i}}{\partial \beta_{k}}\\
	&=\sum_{i}[\sum_{n=0}^{N}w_{i}^{[r]}(n)](\frac{\sum_{n=0}^{N}w_{i}^{[r]}(n)n}{\sum_{n=0}^{N}w_{i}^{[r]}(n)}-\mu_{\lambda i})\frac{1}{\phi V(\mu_{\lambda i})}\frac{\partial \mu_{\lambda i}}{\partial \beta_{k}}
	\end{aligned}
\end{equation}

Which is a single quasi-Poisson GAM with weight $\sum_{n=0}^{N}w_{i}^{[r]}(n)$ and pseudo data as weighted average of every possible n.

Then deal with single $p_{ij}$:
\begin{equation}
	\begin{aligned}
	\frac{\partial logBin(p_{ij},n)}{\partial \beta_{k}}&=\frac{d_{ij}-np_{ij}}{np_{ij}q_{ij}}\frac{\partial(np_{ij})}{\partial \beta_{k}}\\
	&=\frac{d_{ij}-np_{ij}}{p_{ij}q_{ij}}\frac{\partial(p_{ij})}{\partial \beta_{k}}\\
	&=\frac{d_{ij}-n\mu_{p_{ij}}}{V(\mu_{p_{ij}})}\frac{\partial(\mu_{p_{ij}})}{\partial \beta_{k}}
	\end{aligned}
\end{equation}

Now it is related to a quasi-Binomial regression.
\begin{equation}
	\begin{aligned}
	\sum_{n}w_{i}^{[r]}(n)\frac{\partial logBin(p_{ij},n)}{\partial \beta_{k}}
	&=\sum_{n}w_{i}^{[r]}(n)\frac{d_{ij}-n\mu_{p_{ij}}}{V(\mu_{p_{ij}})}\frac{\partial(\mu_{p_{ij}})}{\partial \beta_{k}}\\
	&=\frac{1}{V(\mu_{p_{ij}})}\frac{\partial(\mu_{p_{ij}})}{\partial \beta_{k}}\sum_{n}w_{i}^{[r]}(n)(d_{ij}-n\mu_{p_{ij}})\\
	&=\frac{1}{V(\mu_{p_{ij}})}\frac{\partial(\mu_{p_{ij}})}{\partial \beta_{k}}\sum_{n=0}^{N}nw_{i}^{[r]}(n)(\frac{\sum_{n=0}^{N}w_{i}^{[r]}(n)}{\sum_{n=0}^{N}nw_{i}^{[r]}(n)}d_{ij}-\mu_{p_{ij}})
	\end{aligned}
\end{equation}

Single $p_{ij}$ GAM uses pseudo-data $\frac{\sum_{n=0}^{N}w_{i}^{[r]}(n)}{\sum_{n=0}^{N}nw_{i}^{[r]}(n)}d_{ij}$ and weight $\sum_{n=0}^{N}nw_{i}^{[r]}(n)$ to fit a quasi-Binomial GAM.

Each iteration's M-step will fit total 3 weighted GAMs and use the modified PIRLS algorithm proposed by Hai Liu and Kung-Sik Chan 2009.

\section{Model Selection}
Follow Hai Liu and Kung-Sik Chan 2009, we use BIC and its Laplacian approximation for model selection.

\end{document}