text:
\section{Introduction}
\label{sec:introduction}
\import{sections/}{introduction}

\section{Related Work}
\label{sec:relatedwork}
\import{sections/}{related_work}

\section{Preliminaries}
\label{sec:background}
\import{sections/}{preliminaries}

\section{Proposed Method}
\label{sec:method}
\import{sections/}{method}

\section{Experiments}
\label{sec:experiment}
\import{sections/}{experiments}

\section{Conclusion}
\label{sec:discussion}
\import{sections/}{discussion}

\noindent\textbf{Acknowledgements.}
We thank Tom Minka and Darren Cosker for comments that greatly improved the manuscript.

\newpage
{\small
\bibliographystyle{ieee_fullname}

\subsection{Comparison to Existing Approaches}
We evaluate our approach in generating a plausible pose given sparse observations and compare it with existing methods. Table~\ref{tab:comparison} summarizes this evaluation\footnote{The MPJPE for VAE-HMD on the standard AMASS test set is relatively high. We analyze this in the supplementary material, demonstrating that this is due to imperfect utilization of the latent space resulting from the two-stage training used in VAE-HMD approach.}. We do not use optimization for this comparison. Flow-based approaches, ProHMR-HMD and FLAG (Ours), generally yield lower full body error, but  approaches that have conditional latent variable sampling tend to generate better upper bodies. This is the case with HuMoR and our approach, where the latent variable is sampled given head and hands information while for other techniques, a latent variable is sampled independent of the conditioning signal. The superiority of our approach is also evident in the qualitative results in Fig.~\ref{fig:qualitative}, where FLAG yields least error compared to other techniques, with HuMoR producing relatively good upper body. We provide more qualitative results in the supplementary material.

\subsection{Ablation Study}

\noindent\textbf{Effect of Intermediate Supervision.}
Building on Section~\ref{subsec:flow}, here we evaluate the effect of the proposed intermediate supervision. Such supervision provides an additional signal to intermediate transformation blocks of $f_\theta$, allowing better convergence to a plausible pose throughout transformations when starting from the base distribution. This was visible in Fig.~\ref{fig:intermediate_supervision} and is also evident in our quantitative results in Table~\ref{tab:intermediate_supervision} wherein we show considerable improvement in the quality of generated poses.

\begin{table}
\setlength{\tabcolsep}{5pt}
\scriptsize
    \centering
    \begin{tabular}{l c c}
    \toprule
    Setting & Upper Body MPJPE ($\downarrow$) & Full Body MPJPE  ($\downarrow$) \\
    \midrule
    w/o Intermediate Sup. & 1.64 cm & 5.22 cm \\
    w/ Intermediate Sup. & \textbf{1.39 cm} & \textbf{5.11 cm} \\
    \bottomrule
    \end{tabular}
\vspace{-5pt}
    \caption{Evaluating the effect of intermediate supervision.}\vspace{-10pt}
    \label{tab:intermediate_supervision}
\end{table}

\begin{table}
\scriptsize
    \centering
    \begin{tabular}{l c c c}
    \toprule
     Method & GT Pose & Manip. Pose (RD $\uparrow$) & Noise (RD $\uparrow$)\\
     \midrule
     CVAE * (true ELBO)  & 29.68 & 29.68 (0.0) & 32.40 (0.08) \\
     VPoser-HMD * & 34.79 & 35.56 ( 0.02) & 2.39$\times 1e3$ (0.98)\\
     HuMoR-HMD * & 46.02 & 49.21 (0.06) & 2.37$\times 1e4$ (0.99)\\
     ProHMR-HMD \dag & 110.72 & 282.01 (0.61) & 6.63$\times 1e7$ \textbf{(1.0)}\\
     \midrule
     FLAG (Ours) \dag & 98.54 & 489.66 \textbf{(0.80)} & 3.04$\times 1e13$ \textbf{(1.0)}\\
    \bottomrule
    \end{tabular}
\vspace{-5pt}
\caption{Evaluating the generalizability of learned latent representation by examining the NLL of in- and out-of-distribution samples. VAE-based methods are denoted with * while the NF-based ones are denoted with \dag.
    }\vspace{-10pt}
    \label{tab:ood}
\end{table}

\noindent\textbf{Generalizability of Latent Representations.}
As various models are trained differently, with various training tricks such as KL term annealing or modifying the ELBO for mitigating posterior collapse, we define an auxiliary task to evaluate the quality of the learned latent space. To this end, we use the negative log-likelihood (NLL) metric to identify out-of-distribution (OOD) samples. We define in-distribution samples to be the poses from the ground truth test set, whereas the OOD samples are defined in two ways (1) manipulating ground truth poses by adding a small amount of noise to a subset of joints (2) creating pose-like random noise (random values within the range of natural poses). Table~\ref{tab:ood} summarizes how different models perform when detecting OOD samples. For a clearer comparison, we also report the relative difference ($\text{RD}=\frac{|\text{NLL}_{\text{OOD}}-\text{NLL}_{\text{GT}}|}{\max(\text{NLL}_{\text{OOD}}, \text{NLL}_{\text{GT}})}$) between the NLL of the models for OOD samples and that of the ground truth poses, which higher is better. It can be seen that flow-based models are typically better at detecting OOD samples, demonstrating a richer learned latent representation, whereas VAE-based ones are less effective despite utilizing various techniques to avoid posterior collapse. For reference, we also provide the results of a CVAE trained with true ELBO.

\begin{table}
\setlength{\tabcolsep}{5pt}
\scriptsize
    \centering
    \begin{tabular}{l c c}
    \toprule
    Latent Variable Sampling & Upper Body MPJPE ($\downarrow$) & Full Body MPJPE  ($\downarrow$) \\
    \midrule
    Zeros ($z=\mathbf{0}$) & 1.39 cm & 5.11 cm \\
    MLP ($z=\text{MLP}_\mathbb{H}$) & 1.36 cm& 5.05 cm \\
    Ours ($z=\mu_\mathbb{H}$) & \textbf{1.29 cm} & \textbf{4.96 cm} \\
    \bottomrule
    \end{tabular}
\vspace{-5pt}
\caption{Evaluating the effect of latent variable sampling. Comparing $z=\mathbf{0}$~\cite{kolotouros2021probabilistic}, estimating $z$ with an MLP, and our approach.}\vspace{-10pt}
    \label{tab:sampling}
\end{table}

\begin{table}
\setlength{\tabcolsep}{12pt}
\scriptsize
    \centering
    \begin{tabular}{l c c}
    \toprule
    Latent Variable Sampling & Cosine Dist.($\downarrow$) & Sinkhorn Dist.($\downarrow$) \\
    \midrule
    Random ($z\sim\mathcal{N}(0, \mathbf{I})$) & 1.0 & 0.29\\
    Zeros ($z=\mathbf{0}$) & 1.0 & 0.22 \\
    Ours ($z=\mu_\mathbb{H}$) & \textbf{0.81} & \textbf{0.18} \\
    \bottomrule
    \end{tabular}
\vspace{-5pt}
\caption{Distance to oracle latent code $z^*=f_\theta^{-1}(x_\theta)$.}\vspace{-10pt}
\label{tab:oracle_z_distance}
\end{table}

\begin{figure}
    \centering
    \includegraphics[width=0.35\textwidth]{figure/SMPL_small_annotated.png}\vspace{-5pt}
    \caption{Qualitative evaluation of latent variable sampling, comparing our prediction from $z=\textbf{0}$ and from $z=\mu_\mathbb{H}$. Generated poses are color-coded to show large vertex errors in yellow.}
    \label{fig:init_z}
    \vspace{-10pt}
\end{figure}

\noindent\textbf{Effect of Initial Latent Code.}
A key contribution of this work is the probabilistic mapping from the condition to a sub-region in the latent space that leads to a highly plausible pose. In Table~\ref{tab:sampling}, we compare our approach, $z=\mu_\mathbb{H}$ with the proposal of ProHMR-HMD~\cite{kolotouros2021probabilistic} which claims $z=\mathbf{0}$ yields the most plausible pose. While $z=\mathbf{0}$ yields \emph{a} plausible pose, this experiment shows the existence of a better latent code, $z_\mathbb{H}$ that leads to a more plausible pose that has a high likelihood under the model. This is also shown in Table~\ref{tab:oracle_z_distance}, where we compute the distance between the oracle latent code $z^*=f_\theta^{-1}(x_\theta, [x_\mathbb{H}, \beta])$ to the latent code from our approach as well as that of ~\cite{kolotouros2021probabilistic}. For the sake of completeness, we also compare our method with an MLP that learns to find a good latent code given the condition, which we refer to as $z=\text{MLP}_\mathbb{H}$ in Table~\ref{tab:sampling}. In addition to quantitative evaluation, Fig.~\ref{fig:init_z} shows the effect of a proper initial latent code in generating pose from sparse observation.

We also observed that initial latent variable affects the quality of predicted poses refined via optimization in either the pose space or the latent space, as described in Section~\ref{subsec:optimization}. We evaluate this in Fig.~\ref{fig:optimization_result}, where we use flow-based approaches as a pose prior in the optimization process and report the MPJPE. Consistent with Table~\ref{tab:sampling}, the results demonstrate that a proper initialization leads to a better performance. Given a fixed optimization budget, our method reaches a desired error threshold quicker owing to (a) a better initialization and (b) more reliable likelihood estimates (supported by results in~\ref{tab:ood}). For instance, even after 50 iterations of optimization, ProHMR-HMD~\cite{kolotouros2021probabilistic} does not outperform the solution reached by our approach after 2 optimization iterations regardless of the (pose or latent) space we optimize in. Finally, we also demonstrate that optimization in the latent space generally yields lower error compared to optimization in the pose space, for either model designs.

\begin{figure}
    \centering
    \includegraphics[width=\columnwidth]{figure/optimization_result.PNG}\vspace{-5pt}
    \caption{MPJPE as a function of optimization iteration. As shown here, ProHMR-HMD requires 50 iterations of optimization in the latent space to achieve an error on par with our approach with a proper initial latent code $z=\mu_\mathbb{H}$ without any optimization. Optimizing in the latent space yields lower error compared to optimizing in the pose space.}
    \label{fig:optimization_result}
    \vspace{-15pt}
\end{figure}

\noindent\textbf{Partial Hand Visibility.}
Our method, as well as all other baselines, work on the assumption that head and hand signals are always visible. While this is the case with the head signal in practice, both hands may not always be visible, if they go out of the field of view. In real-world applications, robustness to hands going out of field of view is important, so we evaluate the performance of our method under partial or no hand visibility. The use of aggressive joint masking in our model allows us to fine-tune our model for another 10 epochs and randomly mask hands with probability of $p=0.2$. In Fig.~\ref{fig:partial_visibility}, we demonstrate that FLAG can generate highly plausible poses under partial or no hand observations\footnote{To get the uncertainty maps in Fig.~\ref{fig:partial_visibility}, we generate $K$ poses from $z\sim\mathcal{N}(\mu_\mathbb{H},\Sigma_\mathbb{H})$ and compute the vertices' distance of these sampled poses from the one generated with $z=\mu_\mathbb{H}$.}.

\noindent\textbf{Limitations and Future work.}
While FLAG is capable of generating highly plausible poses given extremely sparse observations in the majority of scenarios, it may fail to generate complex, less common lower-body poses, e.g., martial arts (examples are provided in the supp.\ mat.), potentially because these poses are not very common in the training dataset.
FLAG uses only static pose information; extending FLAG to consume temporal data is a natural research direction.
We use only HMD signal as the input to the model, whereas in some AR/VR scenarios, other modalities such as audio or environment scans may also be available.
Although FLAG aims to find a better latent code to generate a plausible pose, there may still be a considerable gap between our estimated latent code and the oracle one (see Table~\ref{tab:oracle_z_distance}).
Further exploration in this area may lead to more faithful and accurate avatar poses.

\begin{figure}
    \centering
    \includegraphics[width=0.85\columnwidth]{figure/visibility_diverse_uncert.png}\vspace{-5pt}
    \caption{Qualitative results of FLAG when dealing with partially visible hands. From left to right, we illustrate the GT, the avatar's hand visibility status (black box is visible, gray box is invisible), uncertainty map on the pose from $z_\mathbb{H}=\mu_\mathbb{H}$ colorized based on the uncertainty (white is certain, red is uncertain), the pose from $\mu_\mathbb{H}$, followed by generated poses starting with samples from $\mathcal{N}(\mu_\mathbb{H},\Sigma_\mathbb{H})$.
    }
    \label{fig:partial_visibility}
    \vspace{-10pt}
\end{figure}

\noindent\textbf{Societal impact.}
While current datasets such as AMASS have a large number of poses, the data comes from 346 subjects who may not represent the true diversity of the global population.
We have more work to do as a community to represent people of all age groups, and people with disabilities (\eg wheelchair users, amputees).
For anyone with a body morphology outside of the distribution represented in the datasets, we should ask: 1) Does the technology work for them? 2) Can they choose how they want to be represented?
There could be negative outcomes from representing an individual in a way that removes a disability from view, for example. Mixed reality applications bring the promise of enhanced remote collaboration and communication, but there may also be potential negative societal impacts: misrepresentation including impersonation,
further marginalization of socio-economically disadvantaged groups caused  unknowingly or intentionally.
Even so, with mindful deployment of technology and appropriate governance, we remain positive that realistic human representations can help the world grow closer without the harmful environmental impact of long-distance travel.

\subsection{Model Overview}
\label{subsec:problem_definition}
\label{subsec:overview}
Our task is to generate a full body pose $x_\theta$ given a sparse observation $x_{\mathbb{H}}$ and the shape parameters $\beta$.
$x_\theta\in\mathbb{R}^{3\times J}$ represents joint rotations as axis-angle vectors for $J$ body joints, and $x_{\mathbb{H}}\in\mathbb{R}^{9\times K}$ represents the global 6D joint rotation ~\cite{zhou2019continuity}
and a 3D joint location for each of the $K = 3$ observations (head and hands).
This information can be obtained from a parametric model of human body, e.g., SMPL~\cite{loper2015smpl}.

\begin{figure}
    \centering
    \includegraphics[width=\columnwidth]{figure/SimplifiedMethod.PNG}\vspace{-5pt}
    \caption{Overview of FLAG, consisting of a flow-based model $f_\theta$ and a latent region approximator $f_{\text{LRA}}$. During training $f_\theta$ aims to learn the distribution of $x_\theta$ and $f_{\text{LRA}}$ aims to learn a mapping from the condition to the latent representation of $x_\theta$.
    At test time, we sample a latent variable $z_\mathbb{H}$ via $f_{\text{LRA}}$ and use that to generate a new pose via $\hat{x}_\theta=f_\theta(z_\mathbb{H},[x_\mathbb{H},\beta])$.
    }
    \vspace{-10pt}
    \label{fig:method}
\end{figure}

One valid way to generate $x_\theta$ from $x_{\mathbb{H}}$ is to learn the distribution of the body pose given the observed $x_{\mathbb{H}}$ and $\beta$ via a conditional flow-based model $f_\theta$.
While this approach can effectively provide the likelihood of a given pose, the generation process remains incomplete; for generating a novel pose given $x_{\mathbb{H}}$ and $\beta$, one needs to sample a latent variable.
However, the sampling process is completely independent of the observations.
While~\cite{kolotouros2021probabilistic} rely on the mean of the latent space $z=\mathbf{0}$ (a vector of all zeros) as the latent code to generate the full pose, we argue that there exists a latent code that represents $x_\theta$ better than $z=\mathbf{0}$. In fact, while $z=\mathbf{0}$ is the most likely latent code in the base distribution, it may not necessarily translate to the most likely pose in the pose space since there can be changes in the volume of the distribution (the second term in Eq.~\ref{eq:flow}) through $f_\theta$'s transformations.
To obtain such a latent code, our model estimates a sub-region in the normalizing flow base distribution, $\mathcal{N}(\mu_\mathbb{H}, \Sigma_\mathbb{H})$, given $x_{\mathbb{H}}$ and $\beta$, from which a latent variable can be sampled to generate the full body pose.

At test time, to generate a full body pose given $x_{\mathbb{H}}$ and $\beta$, we sample a latent code from $z_\mathbb{H}\sim\mathcal{N}(\mu_\mathbb{H}, \Sigma_\mathbb{H})$ and use that as an approximation of $z_\theta$, the latent code that generates a full body pose.
We use this latent estimate to generate a full body pose via $\hat{x_\theta} = f_\theta(z_\mathbb{H},[x_{\mathbb{H}},\beta])$.
Next, we define $f_\theta$ and describe how we model $\mathcal{N}(\mu_\mathbb{H}, \Sigma_\mathbb{H})$.

\subsection{Flow over Full Body Pose}
\label{subsec:flow}
We model the distribution of $x_\theta$ with a normalizing flow model.
Our model $f_\theta$ is a conditional RealNVP~\cite{dinh2016density} conditioned on $x_{\mathbb{H}}$ and $\beta$.
This can be achieved by mapping $x_\theta$ from the pose distribution to the base distribution (and vice versa) via a composition of simple invertible transformations, where each transformation can stretch or shrink its input distribution.

While it is not straightforward to investigate the contribution of each invertible transformation in generating a human pose given $z_\theta$ sampled from the base distribution, we expect each successive transformation to add expressivity to the incoming distribution of human poses it acts upon.
To intuitively understand the role of each transformation, we visualize how a human pose is formed through all transformations in a model.
Fig.~\ref{fig:wo_intermediate}
illustrates how $z_\theta$ from the base distribution evolves through invertible transformations of $f_\theta$, up to the last transformation which produces a pose from the pose distribution.
As shown, most of the observable modifications to the intermediate distributions are happening in later stages, in which one observes a human-like pose being formed.
We argue that this is because the only source of supervision is the ground truth (GT) pose that explicitly guides the last transformation block.
To ease the training and get the most out of each transformation block in $f_\theta$, we propose to introduce \emph{intermediate supervision} to $f_\theta$.
In addition to having GT pose as the input to the last transformation block, we provide the GT pose as the input
to the intermediate transformation blocks, as if they are the last block of a sub-network.
This is possible because the transformations in $f_\theta$ do not modify the data dimension.
As a result, intermediate transformation blocks are encouraged to produce reasonable human poses and their capacity is exercised fully.
We illustrate the pose evolution through transformations with and without intermediate supervision in Fig.~\ref{fig:w_intermediate}) and also show in \ref{tab:intermediate_supervision} that intermediate supervision leads to improved plausibility of the generated poses.

\begin{figure}
\scriptsize
    \centering

    \begin{subfigure}[b]{\columnwidth}
         \centering
         \includegraphics[width=\columnwidth]{figure/DualFlow_basic.png}
         \caption{Without intermediate supervision}
         \label{fig:wo_intermediate}
     \end{subfigure}

    \begin{subfigure}[b]{\columnwidth}
         \centering
         \includegraphics[width=\columnwidth]{figure/DualFlow_intermediateSup.png}
         \caption{With intermediate supervision}
         \label{fig:w_intermediate}
     \end{subfigure}
         \vspace{-5pt}
    \caption{Pose progression through transformations from the base distribution to the pose distribution ($T_K \leftarrow T_1$) with and without intermediate supervision. First column shows the GT pose. The poses are color-coded to show large errors from GT in yellow, with dark blue showing zero error.}
    \vspace{-10pt}
    \label{fig:intermediate_supervision}
\end{figure}

\subsection{Latent Variable Sampling}
\label{subsec:transformer}
To generate a novel pose given $x_{\mathbb{H}}$ and $\beta$, we need to sample a latent variable $z$ from the base distribution and use that to generate a pose $\hat{x}_\theta=f_\theta(z,[x_{\mathbb{H}},\beta])$.
In a standard conditional flow-based model, one randomly samples $z~\sim\mathcal{N}(0, \mathbf{I})$, hoping for a plausible pose to be generated by the model, or consider $z=\mathbf{0}$~\cite{kolotouros2021probabilistic}.
Although these approaches yield valid solutions, we argue and empirically show that these do not constitute the best solution.
This can be examined explicitly thanks to the invertibility of normalizing flows, where one can obtain the oracle latent code $z^*=f_\theta^{-1}(x_\theta,[x_{\mathbb{H}},\beta])$.
Since an oracle latent code is known during training, we train our model such that it learns to map the condition ($x_{\mathbb{H}}$ and $\beta$) into a region in the base distribution where $z^*$ has a high likelihood. Utilizing $z^*$ during training allows us to take into account the changes in the volume of the base distribution, and thus the changes in the probability mass around the latent code, when transformed from the base distribution to the pose space via $f_\theta$.
We model the region of interest with a Gaussian and learn its parameters $\mu_\mathbb{H}$ and $\Sigma_\mathbb{H}=\text{diag}(\sigma_\mathbb{H})^2$.
Such a mapping should have two desirable properties:
\textbf{(i)} It should be expressive, so that it can produce a representation of full body given the sparse observation. This is necessary to estimate a sub-region of the base distribution that represents the full body.
\textbf{(ii)} It should account for uncertainty in human body representation given sparse observation. When only the head and hands are observed, there exist multiple plausible full-body poses. For each plausible pose, we need to know the corresponding sub-region in the base distribution.
With these key properties in mind, we design a transformer-based mapping function with a discrete latent space.

\noindent\textbf{Attention-based Latent Region Estimation.}
We propose a transformer-based model (with a transformer encoder) to model the mapping function, taking advantage of the self-attention mechanism which learns the relationships between different joints in the body during training.
Briefly, the transformer encoder receives as input $x_{\mathbb{H}}$ and $\beta$, and estimates $\mathcal{N}(\mu_\mathbb{H}, \Sigma_\mathbb{H})$ wherein $\mu_\mathbb{H}$ is trained to be a good approximation of the oracle latent code $z^*$.

For such a distribution to be representative of the full body, we make several design choices to come up with the model illustrated in Fig.~\ref{fig:transformer}.
First, training such a model using sparse inputs directly proves challenging.
To make it easier for the model, we define an auxiliary task of generating $x_\theta$ from the output of the transformer encoder (via \texttt{ToPoseSpace} block in Fig.~\ref{fig:transformer}), initially aiming to reconstruct $x_\theta$ from full body joints and gradually decreasing the joint visibility (through masking) in the encoding until we provide only head and hands\footnote{While in principle, masking can be done randomly, we follow the kinematic tree of the SMPL skeleton and start by masking the lower body joints, then the spine joints, followed by arm joints, and finally the pelvis (the root of the kinematic tree).}.
To further help the transformer learn the representation of the body, we introduce another auxiliary task of predicting the masked joints given the observed ones.
Such gradual masking-and-prediction (\texttt{MaskedJointPredictor}) lets the model infer the full body representation through attention (layer) on the available joints in the input.
To get a compact representation out of the transformer encoder, we apply  pooling ($\texttt{Pool}_\mathbb{H}$) over output joints and take only the head and hand representation, as they are always unmasked.

Next, we make the output of the transformer encoder stochastic, to obtain uncertainty estimates for the predicted poses.
We propose using a categorical latent space~\cite{jang2016categorical,richard2021meshtalk,saleh2021probabilistic} over human pose from the output of the transformer encoder (via \texttt{ToLatentSpace}).
One can sample a discrete latent variable from this distribution (via Gumbel-Softmax~\cite{jang2016categorical} for differentiablity) to generate $x_\theta$ with the defined auxiliary task or use the entire latent representation to estimate $\mathcal{N}(\mu_\mathbb{H}, \Sigma_\mathbb{H})$ (via \texttt{LatentRegionApproximator}) which contains information about a plausible pose and the associated uncertainty.
To efficiently model the complex distribution of human motion, we need a relatively large latent space, leading to a large number of latent categories.
To remedy this, we use a 2D categorical latent space, as shown in Fig.~\ref{fig:transformer}. We model a $G$-dimensional latent variable each responsible for $M$ modes, giving us a capacity to use $M^G$ one-hot latent codes.
\begin{figure}
\scriptsize
    \centering
    \includegraphics[width=\columnwidth]{figure/TransformerModelDetails.pdf}\vspace{-10pt}
    \caption{Transformer-based $f_{\text{LRA}}$. The attention-based encoder aims to learn the relationships between $x_\mathbb{H}$ and the rest of the body to generate an expressive representation of the body. A categorical latent space
    from the output of the transformer encoder allows us to predict a plausible pose and the associated uncertainty.}
    \vspace{-10pt}
    \label{fig:transformer}
\end{figure}

\subsection{Learning}
We use a dataset
of diverse 3D human models, where each sample is a triplet $(x_\theta, x_\mathbb{H}, \beta)$, where $\beta$ are the SMPL shape parameters.
Our loss function $\mathcal{L}$ is given by
\begin{align}
\label{eq:loss}
    \mathcal{L} = & \lambda_{\text{nll}}\mathcal{L}_{\text{nll}} +
                    \lambda_{\text{mjp}}\mathcal{L}_{\text{mjp}} +
                    \lambda_{\text{rec}}\mathcal{L}_{\text{rec}} +
                    \lambda_{\text{lra}}\mathcal{L}_{\text{lra}}
\end{align}
where $\lambda_.$s are the weights associated with each term.

\noindent\textbf{$\mathcal{L}_{\text{nll}}$:}
This term encourages the model to minimize the negative log-likelihood of $x_\theta$ under the model $f_\theta$, following Eq.~\ref{eq:flow}.
Additionally, we take into account the log-likelihoods produced by the sub-networks of $f_\theta$ as the result of intermediate supervision discussed in Section.~\ref{subsec:flow}.
\begin{align}
    \mathcal{L}_{\text{nll}} = - \big(\log p_\theta(x_\theta) +
                                \sum_{s\in S} w_s \log p_\theta^s(x_\theta)\big)
\end{align}
where $S$ is the set of sub-networks of $f_\theta$ (e.g., from block $T_i$ to $T_1$ for a pre-defined set of $i$s), $p_\theta^s(x_\theta)$ is the likelihood of $x_\theta$ under sub-network $s$, and $w_s$ is the weight associated to the sub-network that is proportional to the number of transformation blocks in each sub-network.

\noindent\textbf{$\mathcal{L}_{\text{mjp}}$:}
To train the auxiliary task of masked joint prediction, we employ the term
\begin{align}
    \mathcal{L}_{\text{mjp}} = \sum_{j\in J_{\text{masked}}}\left\|\hat{x}_P^j - x_P^j\right\|_2^2
\end{align}
where $J_{\text{masked}}$ is the list of masked joints,
$x_P^j$ is the representation of the $j^{th}$ joint in $\mathbb{R}^9$ (6D rotation and 3D location), and $\hat{x}_P^j$ is the corresponding prediction from the network.

\noindent\textbf{$\mathcal{L}_{\text{rec}}$:}
This term acts on the output of the auxiliary task of decoding the full body pose from a discrete latent variable sampled transformer's categorical latent space, aiming to guide to build a meaningful discrete latent space.
\begin{align}
    \mathcal{L}_{\text{rec}} = \left\|\hat{x}_\theta^{\text{tps}} - x_\theta\right\|_2^2
\end{align}
where $\hat{x}_\theta^{\text{tps}}$ is the output of \texttt{ToPoseSpace}.

\noindent\textbf{$\mathcal{L}_{\text{lra}}$:}
Finally, this term encourages learning a Gaussian distribution $\mathcal{N}(\mu_\mathbb{H},\Sigma_\mathbb{H})$ under which the oracle latent variable $z^*$ has high likelihood.
\begin{align}
    \mathcal{L}_{\text{lra}} = -\alpha_{\text{nll}}\log p_\mathbb{H}(z^*) + \alpha_{\text{rec}} \left\|\mu_\mathbb{H} - z^*\right\|_2^2 \nonumber \\ - \alpha_{\text{reg}} (1 + \ln \sigma_\mathbb{H} - \sigma_\mathbb{H})
    \label{eq:lra}
\end{align}
where $p_\mathbb{H}$ is the estimated sub-region of the base distribution.
While the first term in Eq.~\ref{eq:lra} is enough to achieve this goal, we add the second term to implicitly encourage $\mu_\mathbb{H}$ to be similar to $z^*$ and the third term discourages $\sigma_\mathbb{H}$ to be zero and thus avoids a deterministic mapping. Note that $\alpha_{\text{reg}}$ and $\alpha_{\text{rec}}$ can be relative small, but need to be present.

Although the entire model can be trained in an end-to-end fashion, we observed training $f_\theta$ first followed by training the latent region approximator is quite effective since we have access to a valid $z^*$ from the beginning.
The second training stage is quick, $~4$ GPU-hours.
This two-stage training may also be useful in cases where one wishes to use a previously trained $f_\theta$ as a foundation model~\cite{bommasani2021opportunities} and only train mapping functions for other data modalities, e.g., body markers or environment scans.

\begin{figure*}[!t]
    \centering
    \includegraphics[width=0.9\textwidth]{figure/qualitative_small.png}\vspace{-5pt}
    \caption{Qualitative results. First column (orange) shows the GT. Generated poses are color-coded to show large vertex errors in yellow.
    }
    \label{fig:qualitative}
    \vspace{-10pt}
\end{figure*}

\subsection{Conditional Generation}
\label{subsec:optimization}
We can generate a full body pose given $x_\mathbb{H}$ and $\beta$ by first computing $\mu_\mathbb{H}$ given the observation, then use $\mu_\mathbb{H}$ as an approximation of $z_\theta$ to generate a pose $\hat{x_\theta}=f_\theta(\mu_\mathbb{H}, [x_\mathbb{H}, \beta])$. To further enhance our quality of the generated pose, one can also use our flow-based model as a pose prior in optimization to minimize a cost function over the prior and the data. The optimization can be done either in pose space or in latent space. We use the LBFGS optimizer~\cite{liu1989limited} throughout (see supp.\ mat. for further details).

\import{sections/}{optimization}

redpajama_set_name: RedPajamaArXiv


text:
\section{Introduction}

Precision measurements of the cosmic microwave background (CMB) by the WMAP
and {\it Planck} collaborations are in very good agreement with what has
become the standard model of cosmology, namely inflation plus
$\Lambda$CDM \cite{Hinshaw:2012aka, Ade:2013zuv, Ade:2013uln}. There
are, however, some anomalies, such as a 5--10\% deficit in the
temperature power spectrum on large scales ($l\lesssim 40$)
\cite{Ade:2013kta}.  Whilst the statistical significance of this anomaly
is currently only $2.5$--$3\sigma$, the tension will be exacerbated if
the recent findings of the BICEP2 team -- who have reported a
tensor-to-scalar ratio of $r \sim \mathcal O(0.1)$ \cite{Ade:2014xna} --
are confirmed \cite{Bousso:2014jca,Smith:2014kka}.  The reason for this increased
tension is easy to understand: the temperature power spectrum on large
scales ($l\lesssim 100$) receives contributions from both scalar and
tensor perturbations, meaning that for a given observed power the scalar
contribution must be suppressed if there exists a non-negligible tensor
contribution.

Perhaps the simplest way in which a suppression on large scales can be
accommodated is to allow for a negative running of the spectral index,
defined as $\alpha_s\equiv dn_s/d\ln k$.  What one finds, however, is
that the required value of $\alpha_s$ is $\sim \mathcal O(0.01)$,
\cite{Ade:2014xna, Abazajian:2014tqa}, which is one or two orders of
magnitude larger than the running predicted in standard slow-roll models
of inflation.  Moreover, a large running also tends to spoil the fit to
data on small scales.  As such, it appears that a non-power-law scalar
spectrum may be preferred \cite{Abazajian:2014tqa,Hazra:2014aea}, namely
one that is able to give a localised suppression on large scales.  Note
that one can also try to alleviate tensions between the BICEP2 results
and those of {\it Planck} by allowing for a modified tensor spectrum
\cite{Smith:2014kka,Gong:2014qga,Gerbino:2014eqa}, but here we focus on
modifying the scalar spectrum.

In canonical single-field models of inflation the power spectrum takes
the simple form
\begin{equation}
 \mathcal{P}_{\mathcal R}(k) = \left.\frac{H^4}{4\pi^2\dot\phi^2}\right|_{k=aH},
\end{equation}
where the subscript $k=aH$ denotes the fact that the quantity should be
evaluated at the time when the scale $k$ left the Hubble horizon.  Given
the inverse dependence on $\dot\phi$, a natural way in which we might
seek to suppress the spectrum would be to require that the field be
rolling quickly during the period that large scales left the horizon,
i.e. during the first few observable e-foldings of
inflation.\footnote{Note that there are other possible mechanisms, see
e.g. \cite{Contaldi:2003zv,Cline:2003ve,Powell:2006yg,Kawasaki:2014fwa}.}
This mechanism for suppression has been considered in the literature,
see
e.g. \cite{Contaldi:2003zv,Jain:2008dw,*Jain:2009pm,Pedro:2013pba,Hazra:2014jka},
and in canonical single-field inflation models an enhancement of
$\dot\phi$ is associated with a steepening of the potential towards the
onset of inflation.  Whilst such a tuned steepening might seem
unnatural, this is not necessarily the case in the context of open
inflation \cite{Bousso:2013uia, Bousso:2014jca, Freivogel:2005vv,
Freivogel:2014hca, Linde:1998iw, Linde:1999wv}.

Models of open
inflation involve potentials of the form shown in figure \ref{openpot},
and the last stage of observable inflation is preceded by tunnelling from
a false vacuum described by a Coleman-De Luccia (CDL) instanton
\cite{Coleman:1980aw}.  The universe nucleated in the tunnelling process
has negative curvature, i.e. it is open.  In general, CDL instantons require
$|V_{\phi\phi}|> H^2$ during the tunnelling process \cite{Jensen:1983ac}, which
is in contradiction with the requirement $|V_{\phi\phi}|\ll
H^2$ for slow-roll inflation.  As such, it is expected that between the end of
the tunnelling process and the onset of standard slow-roll inflation one has an
intervening stage of ``fast-roll'' inflation, during which the potential
becomes progressively less steep.  In order that observable signatures of
this fast-roll phase remain, we require that the ensuing stage of
slow-roll inflation does not last too long.  If the number of
e-foldings is too small, however, then the model becomes tightly
constrained by observational constraints on $\Omega_{{\rm K}}$ today.  The
compatibility of these two competing effects was discussed by Freivogel
{\it et al.} and Bousso {\it et al.}
\cite{Freivogel:2005vv,Freivogel:2014hca,Bousso:2013uia,Bousso:2014jca}.
It was found that models providing sufficient suppression of the
scalar spectrum whilst satisfying bounds on $\Omega_{{\rm K}}$ are possible if the
steepening of the potential towards the barrier is gradual enough.

In this paper we revisit the toy models of open inflation
presented in \cite{Linde:1998iw} and \cite{Linde:1999wv}, and
investigate their ability to produce a suppression of the large-scale
scalar spectrum whilst evading observational constraints on $\Omega_{{\rm K}}$.
In section \ref{open} we begin by giving a brief summary of open
inflation and present the relevant background and perturbation equations.  In
section \ref{models} we then analyse two of the toy models discussed in
\cite{Linde:1998iw}.  Based on the two toy models, in section
\ref{general} we then summarise more generally the nature of
suppression from open inflation models and how they can be constrained
using observations.  Finally we conclude in section \ref{conclusions}
\begin{figure}
\begin{center}
\includegraphics[width = 0.9\columnwidth]{openpot.pdf}
\caption{\label{openpot}The general form of the potential associated with
 open inflation models.  Inflation is preceded by tunnelling from the
 false vacuum and the steepening of the potential in the vicinity of the
 barrier leads to a suppression of the scalar power spectrum.}
\end{center}
\end{figure}

\section{\label{open}Open inflation}

To describe the tunnelling process preceding single-field open inflation, we consider a class of model whose effective potential $V(\phi)$
has a local minimal at $\phi_0$ and global minimum at $\phi=0$ with
$V(0)=0$, as depicted in figure \ref{openpot}. The action is given by
\begin{eqnarray}\label{act}
\mathcal{S} =\int \sqrt{-g}
d^4x\left[\frac{R}{2}-\frac{1}{2}g^{\mu\nu}\partial_\mu\phi\partial_\nu\phi-V(\phi)\right]\,.
\end{eqnarray}
Note that here and throughout the paper we set $8\pi G=1/M_{{\rm Pl}}^2 = 1$.

Conventionally, the instanton solution is explored under the
assumption of $\mathcal{O}(4)$ symmetry, since it has been proved
that the $\mathcal{O}(4)$-symmetric solution gives the lowest value
of the Euclidean action for a wide class of scalar-field
theories~\cite{Coleman:1977th}. Hence, this solution is
exponentially favoured when calculating the corresponding tunnelling
probability. It is also reasonable to use this assumption with
gravity included~\cite{Coleman:1980aw}.
To be specific, the metric of an
$\mathcal{O}(4)$-invariant Euclidean spacetime can be written as
\begin{eqnarray}\label{Emetric}
ds_{\mathrm{E}}^2=d\xi^2+a_{\mathrm{E}}^2(\xi)\left(d\chi_{\mathrm{E}}^2+\sin^2\chi_{\mathrm{E}}d\Omega_2^2\right)\,.
\end{eqnarray}
The corresponding Euclidean background equations are given as
\begin{eqnarray}
&~&H_{\mathrm{E}}^2=
   \frac{1}{3}\left(\frac{\dot\phi^2}{2}-V  \right)+\frac{1}{a_{\mathrm{E}}^2}\,,\label{EFried}\\
&~&\ddot\phi+3H_{\mathrm{E}}\dot\phi-V_{\phi}=0\,,\label{Escalar}
\end{eqnarray}
where a dot denotes differentiation with respect to $\xi$ and
$H_{\mathrm{E}}\equiv\dot a_{\mathrm{E}}/a_{\mathrm{E}}$.  For the CDL instanton solution, we choose
$\xi=0$ to coincide with the centre of the nucleated $\mathcal O(4)$-symmetric
bubble, at which $\phi = \phi_{{\rm exit}}$, $a_{{\rm E}}= 0$ and $\dot
a_{{\rm E}}= 1$.  We further set $\xi = \xi_{{\rm F}}$ ($>0$) in the
false vacuum, where $\phi=\phi_{{\rm F}}$, $a_{{\rm E}}=0$ and $\dot
a_{{\rm E}} = -1$.

On making an analytic continuation to the Lorentzian space corresponding
to our open universe, the metric takes the form \cite{Sasaki:1994yt,Sasaki:1994yt,Yamauchi:2011qq,Sugimura:2013cra}
\begin{align}\label{Rmetric}
ds^2&=-dt^2+a^2(t)\left(d\chi^2+\sinh^2\chi d\Omega^2\right)\nonumber\\
&=a^2(\eta)\left(-d\eta^2+d\chi^2+\sinh^2\chi d\Omega^2\right)\,,
\end{align}
where $t = -i\xi$, $a = -ia_{{\rm E}}$, $\chi = i\chi_{{\rm E}}$ and $a
d\eta = dt$, and
the background equations of motion become
\begin{gather}
H^2 = \frac{1}{3}(\frac{1}{2}\dot\phi^2+V) + \frac{1}{a^2},\label{bgFe} \\
\ddot\phi + 3H\dot\phi + V_\phi = 0,\label{bgphiem}
\end{gather}
where a dot now denotes taking the derivative with respect to $t$.  The
initial conditions coming from the instanton solution are $\phi =
\phi_{{\rm exit}}$, $\dot\phi = 0$, $a = 0$ and $\dot a = 1$.  Using these
equations of motion and initial conditions, we then need to solve for the
ensuing inflationary dynamics.

Locating the
observer at the centre of the spherical coordinates, the only tensor
modes that contribute to the CMB temperature power spectrum are the even-parity modes, which can be expressed as
\begin{equation}
 \delta g_{ij} = a^2t_{ij};\quad t_{ij} = \sum\hat b_{plm} U_p(\eta)Y_{ij}^{(+)plm}+\rm{h.c.}.
\end{equation}
Here $\hat b_{plm}$ are the annihilation operators, $Y_{ij}^{(+)plm}$
are the even-parity tensor harmonics on a unit 3-hyperboloid and
$\eta$ is the conformal time. The
$U_p(\eta)$ are found to satisfy \cite{Garriga:1998he}
\begin{equation}\label{Ueom}
 U_p^{\prime\prime} + 2\mathcal H U_p^\prime + (p^2+1)U_p = 0,
\end{equation}
where a prime denotes $d/d\eta$ and $\mathcal H = a^\prime/a$.

The important scalar quantity is the comoving curvature perturbation
$\mathcal R_c^p$, but this is more conveniently expressed in terms of a
new variable $\bm q^p$ as
\begin{equation}\label{Rqrel}
 \mathcal R_c^p = -\frac{\dot\phi}{2} \bm q^p -
		    \frac{H}{a\dot\phi^2}\frac{d}{dt}\left(a\dot\phi\bm q^p\right).
\end{equation}
The variable $\bm q^p$ then satisfies the simple equation
\begin{equation}\label{qeom}
 \bm q^{p\prime\prime} - \left[\frac{\phi^{\prime 2}}{2} + \phi^\prime\left(\frac{1}{\phi^\prime}\right)^{\prime\prime}-(p^2+4)\right]\bm q^p = 0.
\end{equation}

The scalar and tensor power spectra are given in terms of $\mathcal
R_c^p$ and $U_p$ as
\begin{equation}
 \mathcal P_{\mathcal R} = \frac{p^3}{2\pi^2}\left|\mathcal R_c^p\right|^2\quad\mbox{and}\quad\mathcal P_{T} = \frac{p^3}{2\pi^2}\left|U_p\right|^2
\end{equation}
respectively.  In this work
we fit the power spectra using the following functional forms:
\begin{align}\label{scalar_mag}
 \mathcal P_{\mathcal R} &=
  \left(\frac{H^2}{2\pi\dot\phi}\right)^2_{t=t_{\mathcal R,p}}\frac{\cosh(\pi
  p)+\cos(\delta_p)}{\sinh(\pi p)}\frac{p^2}{c_1^2+p^2},\\
\mathcal P_{T} &=4\left(\frac{H}{2\pi}\right)^2_{t=t_{T,p}}\frac{\cosh(\pi
 p)-1}{\sinh(\pi p)}\frac{p^2}{c_2^2+p^2},\label{tensor_mag}
\end{align}
where the subscripts $t=t_{\mathcal R,p}$ and $t=t_{T,p}$ indicate that
the quantity should be evaluated at the horizon-crossing time for scalar
and tensor modes, respectively.  The phase $\delta_p$ is irrelevant for
large $p$ and behaves as $\delta_p - \pi \propto p$ for $p\rightarrow
0$.  In fact, we take $\delta_p = \pi$ throughout, which corresponds to
the case where maximal suppression is achieved.  The above expressions
for $\mathcal P_{\mathcal R}$ and $\mathcal P_{T}$ are based on analytic
approximations derived in \cite{Yamamoto:1996qq,Garriga:1998he}.  We can
see that they differ from the standard expressions by the $
p$-dependent ``suppression factors,'' which tend to unity for large $
p$ and $\pi p^3/(2c_{1,2}^2)$ for $p \rightarrow 0$.  This $
p$-dependent suppression is distinct from that associated with the
fast-roll dynamics, and reflects the systems memory of the quantum
tunnelling that preceded inflation \cite{Linde:1999wv}.  The parameters $c_1$ and $c_2$
determine the scales at which this additional suppression becomes active
for the scalar and tensor modes, respectively.  Under the weak
back-reaction approximation, the spectra were found to take the above
form with $c_1 = 1$ and $c_2 =1$ \cite{Yamamoto:1996qq,Garriga:1998he}.
In Appendix C of \cite{Garriga:1998he}, similar expressions were also
found for an analytically soluble model, but with $c_1 = 2$ and $c_2 =
1$.  In this paper we take the two parameters $c_1$ and $c_2$ as free
parameters that we use to fit the numerical results of
\cite{Linde:1999wv}.

In order to determine the horizon-crossing conditions we return to the
equations of motion.  Firstly, for $U_p$, we see from \eqref{Ueom} that
horizon crossing takes place when $a^2H^2 = p^2 + 1$, as in the standard
case.  In order to determine $t_{\mathcal R,p}$, we first need to use
\eqref{Rqrel} and \eqref{qeom} to find an equation of motion for
$\mathcal R_c^p$.  We find\footnote{This equation can also be derived using the action for $\mathcal R_c$ given in Appendix B of \cite{Garriga:1997wz}.}
\begin{equation}
 \mathcal R_{c}^{p\prime\prime} + 2A(\eta,p)\mathcal R_c^{p\prime} +
  B(\eta,p)\mathcal R_c^p = 0,
\end{equation}
with
\begin{align}
 A(\eta,p) &= \frac{k^2\mathcal H^2\frac{\mathcal
 H}{a\phi^\prime}\left(\frac{a\phi^\prime}{\mathcal H}\right)^\prime -
 \frac{\phi^{\prime 2}}{2}\mathcal H}{k^2\mathcal
 H^2-\frac{\phi^{\prime 2}}{2}},\\
B(\eta,p) &= \frac{k^4\mathcal H^2 + \left(1+\frac{\mathcal H^2}{\phi^{\prime
 2}}\left(\frac{\phi^{\prime 2}}{\mathcal H}\right)^\prime\right)k^2 -
 \frac{\phi^{\prime 2}}{2}}{k^2\mathcal
 H^2-\frac{\phi^{\prime 2}}{2}},
\end{align}
where $k^2 = p^2 + 4$.  Note that in the limit of large $p$
($\leftrightarrow$ large $k$) the equation of motion for $\mathcal
R_c^p$ reduces to the standard form, where, defining $z =
a\phi^\prime/\mathcal H$, $A(\eta,p) = z^\prime/z$ and $B(\eta,p) =
p^2$.  Defining horizon crossing as when $A(\eta,p)^2 = B(\eta,p)$,
noting $k^2\ge 4$ and assuming $\dot\phi^2/H^2\ll 1$ and
$1/(a^2H^2)\ll 1$ (which is always the case after curvature domination
in our particular models), we find the condition
\begin{equation}\label{hccond}
 p^2+4 = a^2H^2\left(1+\frac{\ddot\phi}{\dot\phi H}\right)^2 - \left(1+2\frac{\ddot\phi}{\dot\phi H}\right).
\end{equation}
One can see that if the slow-roll condition $\ddot\phi/(\dot\phi H)\ll1$
is satisfied, then this reduces to the standard horizon-crossing
condition for large $p$.

\section{\label{models}Toy Models}

Having summarised the key features of open inflation models and the
relevant background and perturbation equations, in this section we revisit
two of the toy models that were introduced and analysed numerically in
\cite{Linde:1999wv}.

\subsection{Model 1}

The potential of Model 1 takes the form
\begin{equation}\label{m1pot}
 V(\phi) = \frac{1}{2}m^2\phi^2\left(1 + \frac{\alpha^2}{\beta^2 + (\phi-\nu)^2}\right).
\end{equation}
We consider the same parameter values as considered in
\cite{Linde:1999wv}, except we work in reduced Planckian units instead
of Planckian units.  As such, our parameters are given as
\begin{align}\nonumber
 \nu &= 3.5\times\sqrt{8\pi}\\\nonumber
\beta^2 &= 2\alpha^2\\\nonumber
\beta &= 0.1\times \sqrt{8\pi}\\\nonumber
m &= 1.5\times10^{-6}\times \sqrt{8\pi}.
\end{align}
\begin{figure}
\begin{center}
\includegraphics[width = 0.9\columnwidth]{phisol.pdf}
\includegraphics[width = 0.9\columnwidth]{DDV.pdf}
\caption{\label{phisol}Upper Panel: $\phi$ as a function of $\xi$ for
the CDL instanton solution of Model 1. Lower panel: A plot of
$|V_{\phi\phi}|/H^2$ as a function of $\xi$ for the CDL instanton
solution of Model 1.  We can see that for most of the trajectory the
condition $|V_{\phi\phi}|>H^2$ is satisfied.}
\end{center}
\end{figure}
By solving Eqs.~(\ref{EFried}) and (\ref{Escalar}) numerically, one
is able to find a CDL instanton solution, which is plotted in the upper
panel of figure
\ref{phisol}. The value of the field at the end of the tunnelling is
given as
\begin{equation}
 \phi_{{\rm exit}} = 17.14.
\end{equation}
In the lower panel of figure \ref{phisol} we plot $|V_{\phi\phi}|/H^2$ along
the CDL instanton trajectory, and we see that $|V_{\phi\phi}|>H^2$ is indeed
satisfied for most of the trajectory.

\begin{figure}[h]
 \centering \includegraphics[width =
\columnwidth]{m1_hubble_evolution.pdf} \caption{Evolution of the Hubble
scale as a function of the number of e-foldings before the end of
inflation (solid black curve).  Also plotted are the evolution of the
curvature scale (upper dashed line), the current Hubble scale as determined
assuming that the observational constraint on $\Omega_{{\rm K}}$ is
saturated (middle dot-dashed line) and the scale associated with $l=100$
(bottom dotted line).  The left-most vertical dashed line corresponds to
the time at which the curvature term becomes sub-dominant to the
potential term in the Friedmann equation.  The next two lines, from left
to right, indicate the times at which the current Hubble scale and the
scale associated with $l = 100$ left the Hubble horizon.}\label{m1_he}
\end{figure}

Using the value of $\phi_{{\rm exit}}$ found above, we then solve
the background equations of motion \eqref{bgFe} and \eqref{bgphiem} for
$\phi$ and $a$ after the tunnelling.  In figure \ref{m1_he} we plot the
resulting evolution of the Hubble parameter as a function of the number
of e-foldings before the end of inflation (solid black
curve).\footnote{The end of inflation is taken to be when $\epsilon_V
\equiv (1/2)(V_\phi/V)^2= 1$.}  We also
plot three scales of interest, which are
\begin{enumerate}
 \item the curvature scale, corresponding to $p = 1$ (blue dashed curve),
\item  the upper limit on the current Hubble scale, as determined by the
       constraint $\Omega_{{\rm K}}\lesssim 0.01$, i.e. $p_{{\rm
       H_0}}= 1/\sqrt{\Omega_{{\rm K}}} \simeq 10$ (red dot-dashed curve), and
\item  the scale that approximately corresponds to the CMB multipole $l =
       100$, as determined assuming that $H_0$ is given by the upper
       limit mentioned above (green dotted curve).
\end{enumerate}
Finally, the times at which the last two of the aforementioned
scales leave the horizon (as simply determined by $p = aH$) are marked with vertical dashed
lines approximately 64 and 60 e-foldings before the end of inflation,
respectively.  The third, left-most vertical dashed line, located
approximately 66 e-foldings before the end of inflation, corresponds to the
time of potential-curvature equality.  Unless otherwise mentioned, the
vertical dashed lines shown in all following plots will correspond to
these same three events.  As we will see later, it is also important to know
the time at which the curvature becomes sub-dominant to the kinetic component
of $H^2$.  Interestingly, in Model 1 this time almost exactly coincides
with the time at which the scale $p_{H_0}$ left
the horizon, i.e. $\sim 64$ e-foldings before the end of inflation. The evolution of the three
components of $H^2$ are plotted explicitly in figure \ref{m1_fc}.

\begin{figure}[h]
\begin{center}
\includegraphics[width = \columnwidth]{m1_fried_comp.pdf}
\caption{\label{m1_fc}Evolution of the kinetic (solid black curve),
 curvature (dashed red curve) and potential (dot-dashed blue curve)
 contributions to $H^2$ as
 a function of the number of e-foldings before the end of inflation.
 The three vertical dashed lines are as in figure \ref{m1_he}.}
\end{center}
\end{figure}

We emphasise that the $p_{H_0}$ discussed above corresponds to the current Hubble scale {\it as
determined assuming that the observational constraint on $\Omega_{{\rm K}}$ is
saturated}.  For our choice of model parameters, we see that this
scale would leave the horizon approximately $64$ e-foldings before the
end of inflation.  However, this would seem a little too early, as
standard arguments dictate that the current horizon scale left the horizon
$\sim 60$ e-foldings before the end of inflation \cite{Liddle:2003as}.

In order to help relate features in the background dynamics and power
spectrum to the underlying potential, in figure \ref{mod1pot} we
indicate the location of the scalar field on the potential at the three
times discussed above.  Note that the order of the vertical dashed lines
is the reverse of that in other plots.  Looking at the form of the
potential \eqref{m1pot}, we see that it naturally lends itself to being
decomposed into a fiducial $m^2\phi^2$ potential and a correction to
this around the tunnelling barrier.  As such, in figure \ref{mod1pot} we
also plot the fiducial $m^2\phi^2$ potential.  We see that the
steepening of the potential relative to $m^2\phi^2$ remains
non-negligible until after the scale $p_{100}$ leaves the horizon.  As
such, we expect that observable scales will indeed be subject to
suppression in this model.

\begin{figure}[h]
 \centering
\includegraphics[width = \columnwidth]{m1_potential.pdf}
\caption{Plot of the potential in Model 1 of \cite{Linde:1999wv}, with the parameter choice shown in the main text.  The
 three vertical lines are as in figure \ref{m1_he}, except their order from
 left to right is reversed.}\label{mod1pot}
\end{figure}

In the upper panel of figure \ref{m1_sr_param_ev} we compare the evolution of the three slow-roll
parameters $\epsilon\equiv -\dot H/H^2$, $\epsilon_V\equiv
(V_\phi/V)^2/2$ and $\epsilon_m = 2/\phi^2$ with the evolution of
$\dot\phi^2/2H^2$.  As we can see from \eqref{scalar_mag}, the quantity
that we are interested in evaluating is $\mathcal P_{\mathcal R}\propto
H^4/\dot\phi^2$.  In the case of a flat universe, this can be
re-expressed in terms of $\epsilon$ as $\mathcal P_{\mathcal R}\propto
H^2/\epsilon$, which in turn, under the slow-roll approximation can be
re-expressed as $\mathcal P_{\mathcal R}\propto H^2/\epsilon_V$.  In the
case of open inflation, however, we have $\dot H = -\dot\phi^2/2 -
1/a^2$, meaning that we can no longer directly replace $\dot\phi^2$ with
$\epsilon$.  Moreover, as the slow-roll condition may be violated in the
early stages of evolution after tunnelling, we are also not necessarily able to
make the second approximation $\epsilon \simeq \epsilon_V$.  Of course,
at late times we expect the curvature to become negligible and for
slow-roll inflation to take place, so it is interesting to explicitly
see at what stage the equivalence of the three quantities is recovered.
As we can see from figure \ref{m1_sr_param_ev}, the equivalence is
recovered approximately $63$ e-foldings before the end of
inflation.  This time is shortly after that at which the curvature term
becomes negligible in comparison to both the potential {\it and} kinetic
terms in the Friedmann equation.
This is exactly as we would expect in light of the contribution of
$1/a^2$ to $\dot H$.  The fact that $\epsilon$ and
$\epsilon_V$ also become approximately equal at this time tells us that
the potential must already be relatively flat and the slow-roll
approximation is a good one.
Another point to note from this figure is the difference between
$\epsilon$ and $\dot\phi^2$ at early times.  As a result of the initial
conditions imposed by the CDL instanton and the initial curvature domination,
$\epsilon$ asymptotes to unity whilst $\dot\phi$ vanishes.  This is
evidently very important in determining the asymptotic behaviour of
$\mathcal P_{\mathcal R}$, as we will see shortly.  Finally the
$\epsilon_m$ that we have plotted corresponds to the slow-roll parameter
associated with the fiducial $m^2\phi^2$ potential.  We can see that
even 60 e-foldings before the end of inflation there is a
non-negligible difference between $\epsilon_m$ and $\epsilon_V$.

\begin{figure}[h]
 \centering \includegraphics[width =
0.9\columnwidth]{m1_sr_param_ev.pdf} \includegraphics[width =
0.9\columnwidth]{m1_eta.pdf} \caption{Upper panel: Evolution of the four
slow-roll quantities $\epsilon$ (blue dot-dashed curve), $\epsilon_V$
(red dashed curve), $\epsilon_m$ (black dotted curve) and  $\dot\phi^2/2H^2$ (solid green curve) as a
function of the number of e-foldings before the end of inflation.  Lower panel:
Evolution of the three slow-roll quantities $\tilde\eta \equiv
-\eta/2+2\epsilon$ (blue dot-dashed curve), $\eta_V$ (red dashed curve)
and $\eta_m$ (black dotted curve) as a function of the number of
e-foldings before the end of inflation.  In both panels the
three vertical lines are as in figure \ref{m1_he}.}\label{m1_sr_param_ev}
\end{figure}

To leading order in the slow-roll approximation, the tilt of the power
spectrum is given as
\begin{equation}
 n_s-1 = -2\epsilon-\eta,
\end{equation}
where $\eta \equiv \dot\epsilon/(H\epsilon)$.  Using
the fact that $\eta_V\equiv V_{\phi\phi}/V \simeq -\eta/2 + 2\epsilon$, this can be written
in the perhaps more familiar form
\begin{equation}
 n_s -1 = -6\epsilon_V + 2\eta_V.
\end{equation}
We know that the initial conditions prescribed by the CDL instanton
dictate that $\eta_V$ be large, which will tend to give a blue-tilted
spectrum.  Given that the spectrum must be red-tilted on small scales,
we are therefore interested in following the evolution of $\eta$ and in
determining when the transition from a blue- to a red-tilted spectrum
takes place.  As such, in the lower panel of figure \ref{m1_sr_param_ev}
we plot the evolution of $\tilde\eta \equiv -\eta/2 + 2\epsilon$,
$\eta_V$ and $\eta_m\equiv 2/\phi^2$, where the last quantity
corresponds to the equivalent slow-roll parameter associated with the
fiducial $m^2\phi^2$ potential.  Similar to the case with $\epsilon$ and
$\epsilon_V$, we note the different behaviour of $\tilde\eta$ and
$\eta_V$ at early times.  This is again due to the initial conditions
imposed by the CDL instanton and the domination of curvature, which give
$\eta = 0$ and $\epsilon = 1$ as $t\rightarrow 0$.  Comparing with the upper panel of
figure \ref{m1_sr_param_ev}, we also note that the time at which the three
different definitions of $\eta$ become equivalent is somewhat later than
the corresponding time for $\epsilon$.

\begin{figure}
\begin{center}
\includegraphics[width = \columnwidth]{m1_spectra.pdf}
\caption{\label{m1_spectra}The scalar and tensor power spectra as
functions of the comoving scale $p$.  In the case of the scalar spectrum we
plot three variants of the analytic expression \eqref{scalar_mag},
namely 1) the full expression (solid blue curve), 2) \eqref{scalar_mag}
without the $p$-dependent suppression factor that distinguishes
\eqref{scalar_mag} from the standard flat-universe expression (red
dot-dashed curve) and 3) \eqref{scalar_mag} taking $t_{\mathcal R, p}$
as the time when $a^2H^2 = p^2+1$ instead of when \eqref{hccond} is
satisfied (purple dotted curve).  We also plot the numerical solution
for $\mathcal R^p_c$ from \cite{Linde:1999wv} (upper solid black curve).
The fitted curves use $c_1 = 4$.  Finally, for comparison, we also plot
the form of the spectrum associated with the cut-off model considered in
\cite{Ade:2013uln} (solid orange curve).  In the case of the tensor
spectrum we simply plot the full expression \eqref{tensor_mag} with $c_2
= 1$ (solid green curve) and the numerical solution from
\cite{Linde:1999wv} (lower solid black curve). The vertical dashed line
at $p=10$ corresponds to the current Hubble scale if we assume
that observational constraints on $\Omega_{{\rm K}}$ are saturated.}
\end{center}
\end{figure}

In figure \ref{m1_spectra} we plot the scalar and tensor power spectra
associated with Model 1 as functions of the comoving scale $p$.
In the case of the scalar spectrum we plot four different curves.  Three
of these correspond to different variants of the analytic expression
\eqref{scalar_mag}, namely 1) the full expression \eqref{scalar_mag}
(blue solid curve) 2) \eqref{scalar_mag} without the $p$-dependent
suppression factor that distinguishes \eqref{scalar_mag} from the
standard flat-universe expression (red dot-dashed curve) and 3)
\eqref{scalar_mag} taking $t_{\mathcal R, p}$ as the time when $a^2H^2 =
p^2+1$ instead of when the condition \eqref{hccond} is satisfied (purple
dotted curve).  The upper solid black curve then corresponds to the
numerical solution for $\mathcal R^p_c$ found in \cite{Linde:1999wv}.
We take $c_1 = 4$ in \eqref{scalar_mag}, which we find to give the best
fit to the numerical results for small $p$.  Qualitatively, we see that
the full analytic result (solid blue curve) matches the numerics well,
especially for small and large $p$.  At intermediate scales, around $p
\simeq 10$, however, it tends to underestimate the amount of
suppression, with the discrepancy being on the order of 10\%. If we
assume that the observational bound on $\Omega_{{\rm K}}$ is saturated,
this scale coincides exactly with the current Horizon scale.  As such,
if we are interested in quantitatively constraining the suppression from
open inflation, we see that numerical calculations are necessary.

From curves 2) and 3) we are able to appreciate the importance of
corrections to the standard flat-universe expression for $\mathcal
P_{\mathcal R}$ for small $p$.  The fact that curve 2) always
underestimates the suppression of the scalar spectrum highlights the fact that
the fast-rolling of the inflaton is not the only source of suppression,
especially on very large scales.  The additional suppression reflects
the system's memory of the tunnelling process preceding inflation
\cite{Linde:1999wv}.  We also see that curve 3) underestimates the
suppression for $p\lesssim 1$.  This reflects the fact that large-scale
modes freeze out at later times than predicted by the standard
horizon-crossing condition, leading to a suppression in their amplitude.
As expected, all curves converge for large $p$, where the effects of the
tunnelling become negligible.

In addition to the four curves discussed above, for comparison we also
plot the spectrum associated with the so-called cut-off model that has
been considered in the literature, see
e.g. \cite{Contaldi:2003zv,Cline:2003ve}.  The model -- corresponding to
the solid orange curve in figure \ref{m1_spectra} -- is a
phenomenological example of a spectrum with suppression on large scales,
and takes the form
\begin{equation}
\mathcal P_{{\rm co}} = A_s\left(\frac{p}{p_\ast}\right)^{n_s-1}\left[1-\exp\left\{-\left(\frac{p}{p_c}\right)^{\lambda_c}\right\}\right].
\end{equation}
The {\it Planck} team found that such a form for the spectrum was preferred
over power-law $\Lambda$CDM with $2\Delta\ln \mathcal{L}_{{\rm max}}=2.9$
\cite{Ade:2013uln}.  The corresponding best-fit parameters were $\ln
(p_c/(a_0\, {\rm Mpc}^{-1})) = -8.493$ and $\lambda_c = 0.474$, where
the pivot scale $p_\ast/a_0 = 0.05\,{\rm Mpc^{-1}}$ was used.  Whilst it
is not our intention that our models quantitatively fit the data well,
it is nevertheless interesting to qualitatively compare our spectra with
this cut-off model.  For the orange curve in figure \ref{m1_spectra} we have
used {\it Planck}'s best-fit values for $p_c/a_0$ and $\lambda_c$ and
further assumed that $\Omega_{{\rm K}} = 0.01$ and $n_s = 0.96$.  The amplitude $A_s$ is adjusted so that
our spectrum and the phenomenological model agree at $p = 10^5$, as both
spectra have relaxed to the standard power-law form on these scales.

In comparing Model 1 with the cut-off spectrum, the main feature we note
is that Model 1 gives substantially more suppression.  This
feature is favourable in light of the fact that {\it Planck}'s best-fit
spectrum was found assuming no tensor modes.  In order to be able to
accommodate a tensor contribution to large-scale CMB
temperature fluctuations (as suggested by BICEP2), we expect that a
larger suppression of the scalar spectrum will be required.  The
discrepancy between the two models is, however, rather substantial, even
at $p \simeq 100$.  Better agreement could perhaps be achieved by
correcting our assumption that $\Omega_{{\rm K}} = 0.01$.  As such, in Model 1 we
may find that the best-fit value for $\Omega_{{\rm K}}$ is unobservably
small.

For reference, we also plot the tensor spectrum, but this time only
include the full analytic result given by \eqref{tensor_mag} with $c_2 =
1$ (solid green curve) and the numerical results from
\cite{Linde:1999wv} (lower solid black curve).  It is clear to see that
the onset of suppression occurs at much smaller $p$ in the case of the
tensor spectrum, i.e. on scales larger than the current Hubble horizon.
In contrast with the scalar spectrum, the tensor spectrum is not
affected by the fast rolling of the inflaton.  As such, the suppression
on large scales is purely a result of the pre-inflationary tunnelling
and the presence of the bubble wall.  See \cite{Linde:1999wv} for a more
detailed discussion.

\begin{figure}
\begin{center}
\includegraphics[width = \columnwidth]{m1_relpow.pdf}
\caption{\label{m1_relpow}The magnitude of the scalar power relative to
 the magnitude at the scale of
 transition between a blue- and red-tilted spectrum.  In this plot the
 vertical dashed lines, from left to right, correspond, respectively, to 1) the current
 Horizon scale $p_{H_0}$ as determined assuming that $\Omega_{{\rm K}}
 =0.01$ 2) the scale associated with $l = 100$$, p_{100}$, and 3) the
 scale at which the spectrum transitions from being blue- to red-tilted,
 $p_{{\rm red}}$.  On the upper
 horizontal axis we also give the number of e-foldings before the end of
 inflation that each given scale $p$ leaves the
 horizon.}
\end{center}
\end{figure}

Finally, in figure \ref{m1_relpow} we plot the magnitude of the scalar
power relative to that at the scale at which the spectrum transitions
from being blue- to red-tilted.  This transition occurs approximately 56
e-foldings before the end of inflation, when the scale $p_{{\rm
red}}\simeq 10^4$ left the horizon.  Given that
potential-curvature equality occurs approximately 66 e-foldings before
the end of inflation, we find that we have roughly 10 e-foldings of the
``fast roll'' phase before the spectrum becomes red-tilted and standard
slow-roll is achieved.

Even in the best-case scenario, it is only possible to detect a
$\Omega_{{\rm K}}\gtrsim 10^{-4}$, which would correspond to $p_{H_0} =
10^2$.  As such, we see that in Model 1 we have the possibility that
suppression of the scalar spectrum is observed on large scales whilst
$\Omega_{{\rm K}}$ remains undetectable.  Indeed, if we were to make the usual
assumption that $p_{H_0}$ leaves the horizon $\sim 60$ e-foldings
before the end of inflation, this would correspond to $p_{H_0}\sim
3\times 10^2$, giving an unobservable $\Omega_{{\rm K}}\sim 1\times 10^{-5}$.
However, in such a scenario the scale $p_{{\rm red}}$ would coincide
with $l \sim 100$, and suppression on the order of 10\% would be
observed at scales corresponding to the current Hubble horizon.

\subsection{Model 2}

The second model considered in \cite{Linde:1999wv} has a potential of
the form
\begin{equation}\label{pot2}
 V(\phi) = \frac{m^2}{2}\left(\phi^2 - B^2\frac{\sinh\left[A(\phi-\nu)\right]}{\cosh^2\left[A(\phi-\nu)\right]} \right),
\end{equation}
with parameters chosen as
\begin{align}\nonumber
 m &= 10^{-6}\times\sqrt{8\pi}\\\nonumber
A &= 20/\sqrt{8\pi}\\\nonumber
B &= 4\times \sqrt{8\pi}\\\nonumber
\nu &= 3.5\times\sqrt{8\pi}.
\end{align}

The CDL instanton solution and the evolution of $V_{\phi\phi}/H^2$
through the tunnelling process are shown in figure \ref{CDL2}.  From
the upper plot we are able to obtain $\phi_{{\rm exit}}=16.55$,
and from the lower plot we see that the condition $|V_{\phi\phi}|>H^2$ is
indeed satisfied for most of the trajectory.
\begin{figure}
\begin{center}
\includegraphics[width = 0.9\columnwidth]{phisolm2.pdf}
\includegraphics[width = 0.9\columnwidth]{DDVm2.pdf}
\caption{\label{CDL2}Upper Panel: $\phi$ as a function of $\xi$ for
the CDL instanton solution of Model 2. Lower panel: A plot of
$|V_{\phi\phi}|/H^2$ as a function of $\xi$ for the CDL instanton
solution of Model 2.  We can see that for most of the trajectory the
condition $|V_{\phi\phi}|>H^2$ is satisfied.}
\end{center}
\end{figure}

Qualitatively, the features of Model 2 are very similar to Model 1,
except that the dependence of the potential on hyperbolic trigonometric
functions makes deviations from the fiducial $m^2\phi^2$ potential
around the tunnelling barrier sharper and more localised.  Given the
similarities, we refrain from reproducing the full set of plots given in
the case of Model 1, focusing only on those we feel highlight the
differences between the two models.

In order to highlight the sharpness of the barrier feature,
in figure \ref{m2_pot} we plot Model 2's equivalent of figure
\ref{mod1pot}.  Namely, we plot the potential, the corresponding fiducial
$m^2\phi^2$ potential and the position of the field at the time of
potential-curvature equality, the time that $p_{H_0}$ left the
horizon and the time that $p_{100}$ left the horizon.  We see that
the potential relaxes to the fiducial one at around the time that $
p_{H_0}$ left the horizon, which is much sooner than in the case of
Model 1.  As such, we expect that a much smaller range
of scales will be subject to suppression.

\begin{figure}[h]
 \centering
\includegraphics[width = \columnwidth]{m2_potential.pdf}
\caption{Plot of the potential in Model 2 of \cite{Linde:1999wv} with the parameter choice shown in the main text.  The
 three vertical lines are as in figure \ref{mod1pot}.}\label{m2_pot}
\end{figure}

In figure \ref{m2_spectra} we plot the scalar and tensor power spectra
as a function of $p$, with the same set of curves included here as in
figure \ref{m1_spectra}.  Regarding the scalar spectrum, in the case of
Model 2 we find that using $c_1 = 3.5$ in \eqref{scalar_mag} gives a
better fit to the numerical results of Linde {\it et al.} for small $\rm
p$.  Once again we find that at intermediate scales, i.e. $p \simeq 10$,
the analytic fit differs from the numerical results by an amount on the
order of 10\%.  We also see that neglecting the $p$-dependent
suppression factor in \eqref{scalar_mag} and using only the naive
horizon-crossing condition to determine $t_{\mathcal R,p}$ have very
similar effects as in the case of Model 1.  The cut-off spectrum
represented by the solid orange curve is constructed in exactly the same
way as in the case of Model 1.  As with Model 1, we find that the
spectrum of Model 2 is much more suppressed on large scales compared to
the cut-off model.  A tensor contribution to CMB temperature
fluctuations could therefore be accommodated in the case that the
results of BICEP2 are confirmed.  In comparison with Model 1, Model 2 is
in much better agreement with the cut-off spectrum on smaller scales.
Regarding the tensor spectrum, we once again use $c_2 = 1$ in our fit,
and again we can see that suppression of the tensor spectrum only
becomes active at much lower $p$ than in the case of the scalar
spectrum.

\begin{figure}
\begin{center}
\includegraphics[width = \columnwidth]{m2_spectra.pdf}
\caption{\label{m2_spectra}The scalar and tensor power spectra as a function of
the comoving scale $p$ for Model 2.  The set of curves plotted is the same as in figure \ref{m1_spectra} for Model 1, except that solid black curves are now replaced with black dot-dashed curves.}
\end{center}
\end{figure}

Finally, in figure \ref{m2_relpow} we plot the magnitude of the scalar
power spectrum relative to its value at the scale of the transition
between a blue- and a red-tilted spectrum.  It is in this plot that the
differences between Models 1 and 2 become most apparent.  Comparing with
figure \ref{m1_relpow} we see that the transition from a blue- to
a red-tilted spectrum occurs a few e-foldings earlier in Model 2,
approximately 60 e-foldings before the end of inflation.  Given that
potential-curvature equality occurs approximately 66 e-foldings before
the end of inflation, this means that we only have around 6 e-foldings
of the ``fast-roll'' phase.  Correspondingly, a suppression of the scalar
power spectrum is observable over a much smaller range of scales, with
the smallest affected scales being quite close to $p_{100}$ in the case
that $\Omega_{{\rm K}} = 0.01$.  Nevertheless, the model is still able
to give rise to a suppression on observationally relevant scales whilst
also satisfying constraints on $\Omega_{{\rm K}}$.  Unlike Model 1,
however, we see that if $\Omega_{{\rm K}}$ is of non-detectable magnitude,
i.e. $\Omega_{{\rm K}}\lesssim 10^{-4}\rightarrow p_{H_0}\gtrsim
10^2$, then this model likely gives no suppression on observable scales.
Similarly, if we were to make the standard assumption that the current
Hubble scale left the horizon $\sim 60$ e-foldings before the end of
inflation, then we see that there would be no suppression on observable
scales.

\begin{figure}
\begin{center}
\includegraphics[width = \columnwidth]{m2_relpow.pdf}
\caption{\label{m2_relpow}The magnitude of the scalar power relative to
 the magnitude at the scale of
 transition between a blue- and a red-tilted spectrum for Model 2.  In this plot the
 vertical dashed lines, from left to right, correspond respectively to 1) the current
 horizon scale $p_{H_0}$ as determined assuming that $\Omega_{{\rm K}}
 =0.01$ 2) the scale associated with $l = 100$$, p_{100}$, and 3) the
 scale at which the spectrum transitions from being blue- to red-tilted,
 $p_{{\rm red}}$.  On the upper
 horizontal axis we also give the number of e-foldings before the end of
 inflation that each given scale $p$ leaves the
 horizon.}
\end{center}
\end{figure}

\section{\label{general}General constraints}

As discussed in the introduction, our motivation for considering open
inflation models is that they naturally give rise to a fast-roll phase
between tunnelling from the false vacuum and the onset of slow-roll
inflation, which in turn leads to a suppression of the scalar power
spectrum for modes leaving the horizon during this period.  However, in
order for this suppression to be observable, we
require that the total number of e-foldings of inflation be relatively small.  This
in turn means that the curvature of space may not get diluted enough to
satisfy current observational constraints on $\Omega_{{\rm K}}$.  In
this section we summarise how observational
constraints on these two competing effects might be used to constrain
models of open inflation.

As confirmed in the toy models of the preceding section, in general we expect
that the dynamics can be considered in three stages.  The first is the
curvature-dominated stage, where $H = 1/a$, such that $a = t$ and
$\dot{\phi}\simeq -V_\phi t/4$.  During this stage the Hubble friction
term is large, meaning that $\dot\phi$ is very small.  After the time
$t_\ast$, defined by $V(\phi_\ast)/3=1/a^2_\ast$, the potential energy
comes to dominate over the curvature term.  In the initial
potential-dominated stage the slope of the potential may still be
somewhat steeper than usually dictated by the requirement for slow roll.
As such, in this stage we might expect $\dot\phi$ to be somewhat
enhanced, leading to a suppression of the scalar power spectrum for
modes leaving the horizon during this period.  Finally, as the field
evolves away from the barrier, the potential flattens and usual
slow-roll inflation occurs.  The above scenario is depicted
schematically in figure
\ref{three_stages}.

\begin{figure}[t]
 \centering \includegraphics[clip=false, trim= 0 20 0 20, width =
\columnwidth]{three_stages.pdf} \caption{Schematic diagram showing the
 evolution of $\ln(1/H)$ (solid black curve) through the
three stages of post-tunnelling evolution referred to in the main text.
Initially the curvature dominates,
giving $1/H\propto a$.  In the second stage the potential now dominates
but the slope of the potential is too steep to allow for slow-roll
inflation, so the Hubble rate continues to evolve, but less rapidly.
Once the potential flattens out, a period of slow-roll inflation
commences, where the Hubble rate is approximately constant.  Also
 plotted are the curvature scale (blue dashed curve), the current Hubble
 scale (red dotted curve) and the scale
 associated with the onset of suppression (green long-dashed curve). The
 points at which these curves intersect the solid black curve corresponds to the
 horizon-crossing times. The shaded region corresponds to the fast-roll period.
}\label{three_stages}
\end{figure}

In addition to the evolution of the Hubble parameter, figure
\ref{three_stages} shows the evolution of the physical length
scales associated with the curvature scale, $p_{\rm K} = 1$, the current
Hubble scale, $p_{H_0}$, and
the scale associated with the onset of power suppression, $p_{\rm sup}$.  These scales can all be constrained using observations, which
allows us to determine the relative position of the corresponding
curves on the plot.  For a given open
inflation model, we would then require that the time at which the scale
associated with the onset of suppression leaves the horizon coincides
with the onset of slow-roll inflation.

To determine the separation between the curvature scale and the current
Hubble scale, we note that the current observational constraint on
$\Omega_{{\rm K}} \equiv 1/(a_0H_0)^2$ is $\Omega_{\rm K}\lesssim 0.01$.  We therefore
find $\ln(p_{H_0}/p_{{\rm K}}) \gtrsim \ln(10)\simeq 2.3$,
which corresponds to the vertical spacing between the blue and red lines
in figure \ref{three_stages}.

Next, in order to determine an estimate for the separation between the
current Hubble scale and the scale at which the onset of scalar
suppression is observed, we use the results of a recent analysis by
Easther {\it et al.} \cite{Abazajian:2014tqa}.  In this work they
considered a ``broken'' spectrum that is red-tilted up to some given scale
$p_{{\rm sup}}$ and then blue-tilted for larger scales, thus
explaining the suppression on large scales.  The transition is assumed
to take place instantaneously at $p = p_{{\rm sup}}$, and
the best fit is found to be
 $p_{{\rm sup}}/a_0=4.6\times10^{-3}\,{\rm Mpc}^{-1}$, which can be compared
to the value of $p_{H_0}/a_0=(h/3)\times10^{-3}\,{\rm
Mpc^{-1}}\simeq(0.23)\times 10^{-3}\,{\rm Mpc}^{-1}$.  We thus find
$\ln(p_{{\rm sup}}/p_{H_0})\simeq 2.98$, which corresponds
to the vertical separation between the red and green lines in figure \ref{three_stages}.

Combining the above results, we find $\ln(p_{{\rm sup}}/
p_{{\rm K}}) \gtrsim 5.28$, which corresponds to the total vertical separation
between the blue and green lines in figure \ref{three_stages}.  Of
course, we stress that this is a very simple estimate, as defining
and constraining the onset of suppression is non-trivial and
model dependent.

By considering the evolution of the Hubble parameter during the
transition from curvature domination to slow-roll inflation, we are able
to convert this constraint on the separation in scales to constraints on
the potential.  Let us use the subscripts ``$\ast$'' and ``S'' to denote
quantities at the time of curvature-potential equality and the onset of slow-roll
inflation respectively.  Then, using the definition of $\epsilon \equiv
d\ln H/dN$, we have
\begin{equation}
 H_S = H_\ast \exp\left[\int^S_\ast \epsilon dN\right].
\end{equation}
By definition, we also have $a_S = a_\ast \exp\left[\Delta N\right]$,
where $\Delta N$ is the number of e-foldings between curvature-potential
equality and the onset of slow-roll inflation. As such, for scales associated
with the onset of power suppression, i.e. scales leaving the horizon at the
time of the transition between the fast- and slow-roll phases, we
have
\begin{align}
 p_{{\rm sup}} = a_S H_S &= a_\ast H_\ast \exp\left[\Delta N +
						    \int_\ast^S\epsilon
 dN\right]\label{anapp},
\end{align}
where we have made the simplifying assumption that the
horizon-crossing condition $p = aH$ is valid, which should be true
for these scales.  We then note that prior to the time $t_\ast$ we have
curvature domination, during which $aH = 1 = p_{{\rm K}}$.  As
such, we can re-write the above expression as
\begin{align}\nonumber
 \Delta N + \int^S_\ast \epsilon dN &= \ln\left(\frac{p_{{\rm sup}}}{
					  p_{{\rm K}} }\right) \\
&=
 \ln\left(\frac{p_{{\rm sup}}}{p_{H_0}}\right)
 +\ln\left(\frac{1}{\sqrt{\Omega_{{\rm K}}}}\right).
\end{align}
Inserting the numbers discussed above, this gives
\begin{equation}
 \Delta N + \int^S_\ast \epsilon dN \gtrsim 5.3
\end{equation}

In the case that $\epsilon$ is constant during the fast-roll phase (as
is the case in figure \ref{three_stages}), this
simplifies further to
\begin{equation}
 \Delta N(1-\epsilon) = \ln\left(\frac{H_\ast}{H_S}\right)\frac{1-\epsilon}{\epsilon} \gtrsim 5.3.
\end{equation}

Another way in which we can try to constrain the form of the potential after
the tunnelling is by looking at how much the large-scale power spectrum
is suppressed.  Taking the naive relation $\mathcal P_{\mathcal R}
\propto H^2/\epsilon$, and further neglecting the variation in $H$, the
ratio of the power spectrum on large scales to that on small scales will
tell us about the ratio of the slow-roll parameters, i.e.
\begin{equation}
 \frac{\mathcal P_{\mathcal R}(p_1)}{\mathcal P_{\mathcal
  R}(p_2)} \simeq
  \frac{\epsilon_{p_2}}{\epsilon_{p_1}} \le 1,
\end{equation}
where $p_1\le p_2$ and $\epsilon_{p_i}$ indicates the value of $\epsilon$ when the scale $p_i$ left the horizon.
In the schematic example depicted in figure \ref{three_stages}, this
approach is particularly simple as $\epsilon$ simply takes on two
different but constant values in the fast- and slow-roll stages.  As we
have seen in Models 1 and 2, however, in more realistic models the
slow-roll parameter is likely to vary with time during the fast-roll
phase, so that the suppression becomes scale dependent.  Furthermore, we
have seen that the fast rolling of the field is not the only source of
suppression for small $p$.  The additional suppression from quantum
tunnelling effects is not captured in this simplified analysis.

\section{\label{conclusions}Conclusions}

The latest CMB measurement from WMAP and $Planck$ hint at there being a
deficit in the temperature power spectrum on large scales, and the
possible detection of primordial gravitational waves by the BICEP2 team
only stands to compound this tension.  In light of this, the fact that open inflation
models give rise to a suppression of scalar power on large
scales has led to them receiving renewed interest.
Intuitively, the main source of suppression in these models is
the steepening of the potential towards the barrier that separates the
true and false vacuums.  This steepening leads to a fast-roll phase in which
$\dot\phi$ is enhanced, so that $\mathcal P_{\mathcal R} \propto
H^4/\dot\phi^2$ becomes suppressed.  The requirement that this
fast-rolling should affect observable scales puts a limit on the
duration of ensuing slow-roll inflation.  If inflation does not
last long enough, however, then one risks violating observable
constraints on $\Omega_{{\rm K}}$.

In this paper we have revisited two of the toy models of single-field open
inflation introduced in \cite{Linde:1999wv}, with potentials given by \eqref{m1pot} and \eqref{pot2}.  We
have seen that both models are indeed capable of giving suppression of
the scalar power on the order of ten percent, and that they are also
able to evade current constraints on $\Omega_{{\rm K}}$.  The steepening
of the potential towards the barrier in Model 1 was of power-law form, and this gradual
steepening meant that the onset of suppression was also gradual,
potentially affecting three orders of magnitude of observable scales.  In Model 2, however,
the steepening was of exponential form, leading to a much shorter
fast-roll phase.  Correspondingly, the range of affected observable
scales was much narrower.

Finally, in both models we found that in addition to the source of
suppression mentioned above, one also has additional suppression that results
from the system's memory of the tunnelling phase. (See also \cite{Linde:1999wv}.)  In the
case that current bounds on $\Omega_{{\rm K}}$ are almost saturated,
we saw that this additional suppression can be non-negligible on
observable scales.  As such, any quantitative analysis would need to
take into account this additional suppression.

\begin{acknowledgments}
We would like to thank R.\,Saito, K.\,Sugimura, T.\,Tanaka and D.\,Yeom for valuable
discussions. This work was supported by the JSPS Grant-in-Aid for
Scientific Research (A) No.~21244033.
\end{acknowledgments}