text:
\section{INTRODUCTION}
Video interpolation, which aims to generate intermediate frames between given prior (or left) and post (or right) frames, is widely applied in video coding \cite{wu2018video} and video frame rate conversion \cite{castagno1996method}. However, natural videos include complicated appearance and motion dynamics, e.g., various object scales, different viewpoints, varied motion patterns, object occlusions, and dis-occlusions, making interpolation of realistic frames a significant challenge.
\begin{figure*}[tb]
\centering
    \subfigure[Accuracy / size tradeoff]{
        \includegraphics[width=.48\textwidth]{img/model/size.pdf}}
    \subfigure[Accuracy / runtime tradeoff]{
        \includegraphics[width=.48\textwidth]{img/model/time.pdf}}
\caption{{\bf Accuracy / efficiency tradeoff for video interpolation on Vimeo-90K dataset:} PDWN in comparison to previous work. Left: PDWN outperforms state-of-the-art methods in both accuracy and model size. Right: PDWN reaches the best balance between accuracy and runtime. PDWN++ is the enhanced PDWN model with input normalization, network improvements, and self-ensembling. PDWN++ further improves the performance with a small cost of model size and nearly 8 times of the runtime. The runtime is the time needed to interpolate one frame on GeForce RTX 2080 Ti GPU card.}
\label{fig:size}
\end{figure*}

Flow-based methods have been proven to work well in video interpolation \cite{bao2019depth,bao2019memc,jiang2018super,liu2017video,xu2019quadratic}. Many state-of-the-art methods first use an optical flow estimator to obtain optical flow between given frames, and then infer the optical flow between the missing middle frame and the left and right known frames, respectively, by prefixed motion assumptions such as linear motion \cite{bao2019depth,jiang2018super,Niklaus_CVPR_2020} or quadratic motion \cite{xu2019quadratic}. The middle frame is then obtained by backward warping input frames using the estimated optical flows. Such approaches are prone to flow errors caused by adopted flow estimators and the errors in the motion assumption. Thus, additional flow correction networks\cite{xu2019quadratic} or additional information such as depth \cite{bao2019depth} are usually required to refine the initial interpolated optical flows, leading to sophisticated models. Moreover, training such models require ground truth optical flow or depth information, which is expensive to obtain in large quantities.

Though flow-based methods have achieved great success in video interpolation, they are prone to errors and face the challenge of complicated dynamic scenes including nonlinear motions, lighting changes, and occlusions. Recently, deformable convolution (DConv) have been investigated in video interpolation to warp features and frames \cite{gui2020featureflow,lee2020adacof}. DConv produces multiple offsets for each pixel to be interpolated with respect to each input frame, and uses a weighted average of these offset pixels in the previous (or future) frame to predict target pixel. When the filter size of DConv is 1x1 and the filter coefficient is 1, DConv offset is the same as optical flow. When the filter size is larger than 1, DConv performs many-to-one weighted warping and thus the offsets can be considered as many-to-one flows. Generally, Dconv offsets are more robust than single optical flow. Furthermore, DConv filter coefficients enable the model to produce more complex transformations. However, the increased degree of freedom of DConv makes the model hard to train.

To alleviate the above issues, we propose a pyramid deformable warping network (PDWN) to perform coarse-to-fine frame warping. The coarse-to-fine structure has been proved to be powerful in optical flow estimation \cite{ilg2017flownet,ranjan2017optical,sun2018pwc}. In video interpolation, however, relatively few approaches explored the coarse-to-fine strategy. Amersfoort and Shi \cite{van2017frame} proposed a multi-scale generative adversarial network to generate the predicted flow and the synthesized frame in a coarse-to-fine fashion. Zhang et. al. \cite{zhang2020flexible} designed a recurrent residual pyramid architecture to refine optical flow using a shared network across pyramid levels. Other methods, despite the usage of multi-scale features, only generate one-stage optical flow \cite{bao2019depth,gui2020featureflow,liu2017video}. In our work, we exploit the advantages of the warping strategy and cost volume in addition to the pyramid structure to estimate DConv offsets from coarse to fine.

The proposed network follows a pyramid structure that extracts features at various resolution scales from each input frame. At every pyramid level, DConv is adopted to warp features from the past and future frames towards the middle frame, and a matching cost volume under different additional offsets between two warped features is constructed and exploited to infer residual DConv offsets. By warping features with the obtained offsets and passing the cost volume to the next pyramid level, the network refines the estimated offsets from coarse to fine. We demonstrate that such a methodology for video interpolation generates more realistic frames without requiring additional information such as ground truth optical flow information or depth during training. Our proposed network greatly reduces the number of model parameters and the inference time, while achieving better or on-par performance compared to state-of-the-art models as shown in Figure \ref{fig:size}. Furthermore, our proposed approach can be extended to using multiple input frames easily, and using four instead of two frames as input leads to significantly improved interpolation results.

\section{Related Work}
\subsection{Video interpolation} Video interpolation has been extensively explored in the literature \cite{bao2019depth,bao2019memc,gui2020featureflow,jiang2018super,lee2020adacof,liu2017video,Niklaus_CVPR_2020,niklaus2017video,xu2019quadratic,xue2019video}. Prior methods can be grouped into two categories: kernel-based approach and flow-based approach. Kernel-based approaches \cite{lee2020adacof,niklaus2017avideo,niklaus2017video} estimate convolution kernel parameters to hallucinate intermediate frame. However, kernel-based approaches typically fail in cases with large motions unless very large filter kernels are used, and suffer from large computational loads. Flow-based approaches estimate the optical flow to warp pixels to synthesize the target frame. Super SloMo \cite{jiang2018super} adopted one UNet to estimate optical flow between two input frames, and another UNet to correct the linearly interpolated flow vector. Beyond linear motion assumptions, Quadratic flow (QuaFlow) \cite{xu2019quadratic} adopted PWC-Net\cite{sun2018pwc} to estimate optical flow between input frames. Then the quadratically interpolated flow was refined through a UNet. MEMC-Net\cite{bao2019memc} estimated both motion vectors and compensation filters through CNNs. Note that four input frames are required for QuaFlow to construct a quadratic model. Instead of bilinear interpolation, MEMC proposed an adaptive warping layer based on optical flow and compensation filters to reduce blur. Based on MEMC-Net, DAIN \cite{bao2019depth} used depth information estimated by a pre-trained hourglass architecture \cite{li2018megadepth} to detect occlusions. Different from above methods, Softmax Splatting \cite{Niklaus_CVPR_2020} estimated forward flow using an off-the-shelf optical flow estimator and designed a differentiable way to do forward warping. Though flow-based methods can generate sharp frames, inaccurate flow estimation often leads to severe artifacts. Unlike methods described above, our method directly estimates the "flow" between given input frames and the unknown middle frame without assuming the trajectory is linear or quadratic or has other parametric forms. And we estimate many-to-one "flow" which is more robust compared to single optical flow. Furthermore, we estimate the flows in a coarse-to-fine manner, to efficiently handle large motions.

\subsection{Pyramid structure and the cost volume} Pyramid structure has been proved to be powerful in optical flow estimation. Ilg et al. \cite{ilg2017flownet} achieved state-of-the-art performance by stacking several UNets into a large model, called FlowNet2. To reduce over-fitting problem caused by large models, SpyNet \cite{ranjan2017optical} incorporated two classical principles, pyramid structure, and warping, into deep learning. A spatial pyramid network was constructed for each of the two frames, and it estimated the flow in each scale and warped the second image to the first one at each scale repeatedly to reduce motion between two images. PWC-Net \cite{sun2018pwc} further explored the trade-off between accuracy and model size. Instead of image pyramids, PWC-Net constructed feature pyramids that are invariant to shadows and lighting change. Partial cost volume is used to represent matching cost associated with different disparities. Inspired by classical pyramid energy minimization in optical flow algorithms, RRPN \cite{zhang2020flexible} designed a recurrent residual pyramid architecture for video frame interpolation to refine optical flow using a shared network for every pyramid level. Following above methods, we also exploit the advantages of classical principles of optical flow -- the pyramid structure, multi-scale warping, and cost volume. Different from RRPN, we replace the flow estimation in each scale by the estimation of many-to-one offset maps through the use of deformable convolution filters, significantly reducing artifacts that are associated with occasional wrong flow estimates. Furthermore, cost volume is incorporated into our model non-trivially. We demonstrate that cost volume between the warped features of the two known frames can provide useful information for estimating the flow between the unknown middle frame and the known prior and post frames.

\subsection{Deformable convolution} DConv operation \cite{dai2017deformable} is originally proposed to overcome the limitation of CNNs due to fixed filter support configuration and to enhance the transformation modeling capacity of CNNs. It estimates a set of $K$ offsets at each pixel and a global filter (non-spatially varying) with $K$ coefficients to be applied for the $K$ offset pixels. Zhu et. al. \cite{zhu2018deformable} further improved DConv by adding spatially adaptive modulation weights to modulate the global filter coefficient associated with each offset. The improved DConv thus has the ability to vary the attention to different offset pixels. Recognizing that DConv can be viewed as many-to-one weighted backward warping, FeFlow \cite{gui2020featureflow} used DConv to align input features from two known frames and fused aligned features to synthesize the middle frames. AdaCoF \cite{lee2020adacof} constructed a UNet to estimate both local filter weights and offsets for each target pixel to synthesize output frames. We have found that learning a global filter plus spatially-varying modulation weights as in \cite{zhu2018deformable} is better than directly estimating locally adaptive filters. Different from FeFlow and AdaCoF \cite{lee2020adacof} that estimate the DConv offsets directly in the original image resolution, we perform offset estimation and feature alignment in a coarse-to-fine successive refinement manner. Specifically, we successively refine DConv offsets from the coarser scales to the finer scales. We further utilize the cost volume computed from two aligned features at each scale to improve the accuracy of the offset update.

\section{Methods}
\begin{figure*}[htp]
\begin{center}
\includegraphics[width=\linewidth]{img/model/pipeline.pdf}
\end{center}
   \caption{{\bf (a) The architecture of PDWN}. Given the past frame $I_0$ and the future frame $I_2$, PDWN first generates two feature pyramids. Then DConv offsets $f_{1\to0}^l, f_{1\to2}^l$ and associated modulation weights $m_{1\to0}^l, m_{1\to2}^l$ are generated from the coarsest scale to the finest scale. Finally, two warped frames are adaptively blended to synthesize the middle frame. {\bf (b) Offset estimator module}. Features of scale $l$ are warped producing $\tilde{F_0^l}$ and $\tilde{F_2^l}$ via DConv with generated offsets and associated modulation weights. The cost volume between $\tilde{F_0^l}$ and $\tilde{F_2^l}$, together with input features, are fed to 2 convolutional layers to refine the next-scale DConv offsets and the associated modulation weights. The above process is repeated until it gets to the finest level.}
\label{fig:network_structure}
\end{figure*}
The structure of PDWN is shown in Figure \ref{fig:network_structure}. Given two input frames $I_0$ and $I_2$, we aim to synthesize the intermediate frame $I_1$ by gradually warping features of input frames to the middle frame using DConv. First, we construct a feature pyramid for each input frame using a shared feature extractor. Second, we generate the offsets and the associated modulation weights between each input frame and the middle frame, and then warp features of both input images toward the middle frame. This operation is taken at every level of the feature pyramid to refine the motion. Thus, the estimated DConv offsets, which can be considered as many-to-one flow, are refined from the coarse level to the fine level. Third, at the finest resolution level (same as the input frame), interpolation weight maps between the warped left and right frames are generated to handle occlusions. Finally, following the post-processing scheme of DAIN, we adopt a context enhance network to further enhance the interpolated frame, shown in Figure \ref{fig:res}.

\subsection{Shared feature pyramid encoder}
A multi-layer CNN is used to construct $L$-scale pyramids of feature representations for both input frames $\{F_i^l \mid i\in\{0,2\}, l\in\{1,2,...,L\}$. The features at the first scale, $F_i^1$,  have the same spatial resolution as the input frames. The $l$th scale feature $F_i^l$ is downsampled by a factor of 2 both horizontally and vertically from the $(l-1)$-th scale feature $F_i^{l-1}$. Each scale consists of two convolution blocks, with specifics shown in Table \ref{table:configuration}.

\subsection{Offset estimator module}
The Offset estimator module is used in every scale of PDWN. It jointly predicts the DConv offsets from the unknown intermediate frame to the given input frames and the associated modulation weights for each offset in order to warp input frames and features to the intermediate frame.

\paragraph{Deformable warping with spatially-varying modulation coefficients} A deformable convolution filter is specified by a global filter $w(j)$, a set of spatially-varying offsets $f(j, x)$, and modulation coefficients $m(j, x)$, where $j$ denotes $j$-th location in a filter support $\mathcal{R}$ and $x$ indicates pixel location. The global filter $w(j)$ here is the same convolution filter as regular convolutions except that the sampling is irregular. The support $\mathcal{R}= \{(-1, -1), (-1, 0), ..., (0, 1), (1, 1)\}$ specifies a $3\times 3$ filter in our model. The offset is defined by horizontal and vertical displacements. And every sampling point is associated with a modulation weight. Thus the offset tensor and the modulation tensor have channel dimensions of 18 and 9, respectively. The global filter has the size of $3\times 3$. To use DConv for video interpolation at multiple scales, we generate two sets of offsets and modulation coefficients at scale $l$, ${f_{1\to i} ^l(j, x)}, {m_{1\to i}^l (j, x)} $ with $i=0,2$ indicating the known prior and post frame and $i=1$ the unknown middle frame. The global filter weights $w^l(j)$ are learnt and stay fixed after training for each scale and shared for known input features. Specifically, we generate the warped feature at scale $l$ at pixel $x$ from the original features for frame i as follows:
\begin{equation}
\tilde{F}_i^l(x) = \sum_{j=1}^{\left| R\right|} w^l(j)m_{1\to i}^l(j, x)F_{i}^{l}(x+R(j)+ f_{1\to i}^l(j, x))
\label{align}
\end{equation}

\paragraph{Cost volume between features warped towards the middle frame} The notion of cost volumes has been widely used in optical flow methods \cite{hosni2012fast,sun2018pwc,vzbontar2016stereo} to provide explicit representation of matching cost under different displacements between two given frames for each pixel. In the PWC method for optical flow estimation, the cost volume is constructed between a warped image and a fixed image. Typically, for each pixel  $x$ in one frame, the correlation between the feature at $x$ in this frame and the feature at a displaced location $x+d$ in the other frame is computed, for a finite set of displacements $d\in {\mathcal{D}_k(x)}$. $\mathcal{D}_k(x)$ is a square neighborhood of pixel $x$ with neighborhood size $k\times k$. In our case, however, a cost volume is calculated between two sets of warped features $\tilde F_0^l$ and $\tilde F_2^l$ based on the estimated offsets from each known frame to the middle frame, determined in a lower scale. The cost volume indicates the correlation between the features for corresponding pixels in the left and the right warped features under different displacements. Specifically, given $\tilde{F}_{0}^{l}$ and $\tilde{F}_{2}^{l}$, a cost volume $C^l$ is constructed based on
\begin{equation}
C^l(x_1, x_2) = \frac{1}{k^2}\tilde{F}_0^l(x_1)^T\tilde{F}_2^l(x_2), \quad x_2 \in \mathcal{D}_k(x_1)
\label{cost}
\end{equation}
where $x_1$ and $x_2$ are pixel indexes. We set $k=9$, including displacement from -4 to 4 in both horizontal and vertical directions. Thus the cost volume has a channel dimension of 81.

Instead of using a pre-determined way to calculate the matching cost, one can also train a small network (learnt as part of the entire network) $v(\cdot)$ that takes the two warped features and outputs the cost volume:
\begin{equation}
C^l = v(\tilde{F}_0^l, \tilde{F}_2^l)
\end{equation}
We experimented with both approaches, where we used a network with two conv layers for the network $v(\cdot)$.

\paragraph{Multi-scale offset estimation} As shown in Figure \ref{fig:network_structure}, we estimate the offsets between the middle frame and each of the two input frames from coarse to fine scales with a total of $L$ scales ($L=3$ in Figure \ref{fig:network_structure}). DConv offsets are generated within each scale to gradually reduce the distance between two sets of features warped towards the middle frame.

At $l$-th scale, the offset estimation block first upsamples the estimated offsets $f^{l+1}_{1\to i}$ and modulation weights $m^{l+1}_{1\to i}$ at the lower scale $l+1$ to the current resolution using bilinear interpolater $h(\cdot)$, yielding
\begin{equation}
\hat{f}^l_{1\to i} = 2*h(f^{l+1}_{1\to i})
\end{equation}
\begin{equation}
    \hat{m}^l_{1\to i} = h(m^{l+1}_{1\to i})
\end{equation}
Then it warps the original features $F_i^l$ towards the middle frame based on $\hat{f}_{1\to i}^{l}$, $\hat{m}_{1\to i}^{l}$, and the learnt global filter $w^l$, generating the warped features $\tilde F_i^l$ using Eq. (\ref{align}). Then, the offset estimator computes the cost volume $C^l$ between the two warped features using Eq. (\ref{cost}). Next, it generates two sets of DConv offsets residuals $\Delta f_{1\to i}^{l}$ and two sets of modulation weight $m_{1\to i}^{l}$ from $C^l$, $\hat{f}_{1\to i}^{l}$, $\hat{m}_{1\to i}^{l}$, the original features $F_{i}^l$, and the upsampled features
$h(F^{l+1})$ from the features $F^{l+1}$ generated by the offset estimator in the previous scale:
\begin{equation}
\begin{aligned}
   \Delta f_{1\to i}^{l}, m_{1\to i}^{l} = g(C^l, F_{i}^{l}, \hat{f}_{1\to i}^{l}, \hat{m}_{1\to i}^{l}, h(F^{l+1})), i=0,2
\end{aligned}
\label{warp1}
\end{equation}
where $g(\cdot)$ denotes a three-layer CNN. The final offsets and modulation weights are obtained by
\begin{equation}
f_{1\to i}^{l} = \hat{f}_{1\to i}^{l} + \Delta f_{1\to i}^{l}
\label{warp2}
\end{equation}
\begin{equation}
m_{1\to i}^{l}= \sigma(m_{1\to i}^{l})
\label{warp3}
\end{equation}
\begin{equation}
\sigma(t) = \frac{1}{1+e^{-t}}
\label{warp4}
\end{equation}
where $\sigma(\cdot)$ denotes a sigmoid activation function. We can use a small subnetwork (consisting of three conv layers) to estimate the offset fields because the motion between two warped features is usually small. The same process repeats until we complete scale 1.

For the coarsest scale $L$, the offset estimator only takes the original features in that scale $F_0^{L}$ and $F_2^L$ as input and generates $f^L_{1\to i}$ and $m^L_{1\to i}$ directly.

To summarize, the offset estimator at each scale needs to generate two sets of offset tensors and two sets of modulation tensors, with a total channel dimension of 54. See Table \ref{table:configuration} for the specifics of the network structure.

\subsection{Adaptive frame blending}
Using the estimated offset $f_{1\to i}^1 $, modulation weights $m_{1\to i}^1$, and global filter $w^1$ at scale 1, we  warp frame $i$ towards the middle frame, generating two candidates estimates of the middle frames $\tilde{I}_i, i\in{0,2}$. Occlusions often happen due to the movement of objects. Therefore, in order to select valid pixels from two warped reference frames, we design a blending layer that generates a weight map $\alpha(x)$ to average the two transformed frames at position $x$. The layer is constructed by a three-layer CNN. See Table I, the network takes two warped frames, $\tilde{I}_0$ and $\tilde{I}_2$, and two warped features, $\tilde{F}_0^1$ and $\tilde{F}_2^1$, at first scale of the feature pyramids as input and generates the weight map with a softmax activation applied on the output layer. At position $x$, the blended frame is
\begin{equation}
\tilde{I}_1(x) = \alpha(x) * \tilde{I}_0(x) + (1-\alpha(x)) * \tilde{I}_2(x)
\end{equation}
The warped features provide contextual information to estimate the weight map.

\subsection{Context enhancement network}
To generate the final output, we construct a context enhancement network which takes warped images and features at scale 1 as input and outputs a residual image between the unknown ground truth intermediate frame and the blended frame. The network consists of five residual blocks, shown in Figure \ref{fig:res}.  See Table \ref{table:configuration} for the specific network configuration.
\begin{figure}[htb]
\begin{center}
\includegraphics[width=\linewidth]{img/model/res.pdf}
\end{center}
   \caption{{\bf Context enhancement network.} After getting the initial synthesized middle frame, warped input frames and warped 1th-level features are fed into five residual blocks to further enhance contextual details of the synthesized frame.}
\label{fig:res}
\end{figure}

\subsection{Extending to four input frames}
Quadratic flow \cite{xu2019quadratic} shows an improvement on moving trajectory estimation by estimating acceleration information from four input frames. We also extend our model to exploit the information in additional input frames and to estimate the motion more accurately. Our extended model takes four input frames (two previous and two following frames). A pyramid feature encoder is shared between four input frames to generate four feature pyramids. In the offset estimator, we input four feature maps of the four input frames instead of two in the first conv layer in Figure \ref{fig:network_structure}.(b). This allows the network to recognize the motion trajectory over a longer temporal scope and yield more accurate offset estimation. In higher scales, we still generate the warped feature maps for two closest past and future frames using the estimated offsets and modulation weights from the lower scale and determine the cost volume from these two warped features. Then the cost volume is concatenated with four original features of input frames as well as offsets and modulation weights and fed into the next scale to refine offsets and modulation weights in the next scale. Note that even though the input consists of four frames, the network only generates two sets of offsets, between the middle frame and its left and right neighboring frames, respectively. The final interpolated frame is the adaptively weighted average of these two closest frames warped by deformable convolution.

\subsection{Implementation detail}
\begin{table}
\caption{Architecture of PDWN}
\begin{center}
\renewcommand\tabcolsep{1pt}
\begin{threeparttable}
\begin{tabular}{cccc}
\toprule

Module & Scale & Output size & Configuration\\
\midrule
\multirow{13}{*}{Feature extractor} & \multirow{2}{*}{1} & \multirow{2}{*}{$H \times W$} & Conv 7 - 3 - 16 \\
\specialrule{0em}{1pt}{1pt}
  & & & Conv 5 - 16 - 16 \\
\cmidrule(r){2-4}
& \multirow{2}{*}{2} & \multirow{2}{*}{$H/2 \times W/2$} & Conv 3 - 16 - 32\\
\specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 32 - 32\\
\cmidrule(r){2-4}
& \multirow{2}{*}{3} & \multirow{2}{*}{$H/4 \times W/4$} & Conv 3 - 32 - 64\\
\specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 64 - 64\\
\cmidrule(r){2-4}
& \multirow{2}{*}{4} & \multirow{2}{*}{$H/8 \times W/8$} & Conv 3 - 64 - 96\\
\specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 96 -96\\
\cmidrule(r){2-4}
& \multirow{2}{*}{5} & \multirow{2}{*}{$H/16 \times W/16$}& Conv 3 - 96 - 128\\
\specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 128 - 128\\
\cmidrule(r){2-4}
& \multirow{2}{*}{6} & \multirow{2}{*}{$H/32 \times W/32$} & Conv 3 - 128 - 196\\
 \specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 196 - 196\\
\cmidrule(r){1-4}
  \multirow{24}{*}{Offset estimator} & \multirow{3}{*}{6} & \multirow{3}{*}{$H/32 \times W/32$} & Conv 3 - 473 - 256 \\
  \specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 256 - 256 \\
 \specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 256 - 54 \\
\cmidrule(r){2-4}
  & \multirow{4}{*}{5} & \multirow{4}{*}{$H/16 \times W/16$} & DConv 3 - 128 - 128 \\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 647 - 196 \\
  \specialrule{0em}{1pt}{1pt}
   & & & Conv 3 - 196 - 196 \\
   \specialrule{0em}{1pt}{1pt}
    & & & Conv 3 - 196 - 54 \\
\cmidrule(r){2-4}
  & \multirow{4}{*}{4} & \multirow{4}{*}{$H/8 \times W/8$} & DConv 3 - 96 - 96\\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 523 - 128 \\
  \specialrule{0em}{1pt}{1pt}
   & & & Conv 3 - 128 - 128 \\
   \specialrule{0em}{1pt}{1pt}
   & & & Conv 3 - 128 - 54 \\
\cmidrule(r){2-4}
  & \multirow{4}{*}{3} & \multirow{4}{*}{$H/4 \times W/4$} & DConv 3 - 64 - 64\\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 391 - 64 \\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 64 - 64\\
  \specialrule{0em}{1pt}{1pt}
   & & & Conv 3 - 64 - 54 \\
 \cmidrule(r){2-4}
  & \multirow{4}{*}{2} & \multirow{4}{*}{$H/2 \times W/2$} & DConv 3 - 32 - 32\\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 295 - 64 \\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3- 64 - 64\\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 64 - 54 \\
 \cmidrule(r){2-4}
  & \multirow{4}{*}{1} & \multirow{4}{*}{$H \times W$} & DConv 3 - 16 - 16 \\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 231 - 64 \\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 64 - 64\\
  \specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 64 - 54 \\
\cmidrule(r){1-4}
 \multirow{5}{*}{Adaptive frame blending} &  \multirow{5}{*}{1} & \multirow{5}{*}{$H \times W$} & DConv 3 - 3 - 3 \\
 \specialrule{0em}{1pt}{1pt}
 & & & DConv 3 - 16 - 16\\
 \specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 38 - 16 \\
 \specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 16 - 16 \\
 \specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 16 - 2 \\
\cmidrule(r){1-4}
 \multirow{3}{*}{Context enhancement} & \multirow{3}{*}{1} & \multirow{3}{*}{$H \times W$} & Conv 3 - 41 - 64 \\
 \specialrule{0em}{1pt}{1pt}
 & & & Conv 3 - 64 - 64 $\times$ 2 $\times$ 4\\
\specialrule{0em}{1pt}{1pt}
  & & & Conv 3 - 64 - 3\\
\bottomrule
\end{tabular}

\begin{tablenotes}
        \footnotesize
        \item[*]  The convolutional and deformable convolutional layer parameters are denoted as "Conv/DConv $<$filter size$>$ - $<$number of input channels$>$ - $<$number of output channels$>$". The leakyReLU activation function, max pool layer, bilinear upsample layer, and matching cost layer are not shown for brevity.

      \end{tablenotes}
\end{threeparttable}
\end{center}
\label{table:configuration}
\end{table}
\paragraph{Architecture configurations}
The configurations of PDWN with 6 scales and predefined matching cost calculation, evaluated in this paper, are outlined in Table \ref{table:configuration}.

\paragraph{Loss function} L1 norm has been proven to generate less blurry results in image synthesis tasks \cite{DBLP:journals/corr/GoroshinML15,mathieu2015deep}. Thus, L1 Reconstruction loss between the reconstructed frame and the ground truth frame is used to train the model:
\begin{equation}
    \mathcal{L} = || \tilde{I}_1 - I_1 ||_1
\end{equation}
We also explore a multi-scale L1 reconstruction loss for training. Specifically, we downsample the input frames and the ground truth middle frame. Then, we apply the estimated offsets and modulation weights to the downsampled input images to generate the interpolated frame at each scale. Finally, the L1 reconstruction losses between the reconstructed frame and the ground truth frame for all scales are combined. Through our experiment, we find that the multi-scale loss does not improve the final results compared to simple L1 reconstruction loss at the finest scale. But we do observe that the multi-scale loss could speed up the convergence during training. For simplicity, all results reported in this paper are obtained by using the simple L1 reconstruction loss at the finest scale.

\paragraph{Training dataset} We use Vimeo-90k training set \cite{xue2019video}, which has 51312 triplets, to train our model. Each triplet has 3 consecutive frames and each frame has a resolution of $448 \times 256$. Horizontal flipping and temporal reversing are adopted as data augmentation.

\paragraph{Training strategy} We train PDWN sequentially. In other words, we first train PDWN without context enhance network for 80 epochs, then finetune the whole system end-to-end for another 20 epochs. We use Adam \cite{kingma2014adam} with $\beta_1 = 0.9$ and $\beta2 = 0.999$ to optimize our model. The initial learning rate is set to 0.0002. Mini-batch size is set to 20. Following the techniques introduced in \cite{niklaus2021revisiting}, we also train a variant of PDWN, called PDWN++, with input normalization, network improvements, and self-ensembling. Specifically, each color channel of the input frames is normalized independently to have zero mean and unit variance. Then, we replace the two-layer convolution with residual blocks. Moreover, the global filter of the deformable convolution that warps frames at level 1 is shared not only between input frames but also across RGB color channels. Finally, 7 transforms, including reverse, flipping, mirroring, reverse and flipping, and rotation by 90, 180, and 270 respectively, are applied during the inference phase for self-ensembling.

\section{Results}
In this section, we first introduce evaluation datasets. Then, we conduct ablation studies to evaluate the contribution of each component and to compare our proposed model with state-of-the-arts on two input frames. Finally, we compare the performance of our models using two vs. four input frames and also compare with other models using four input frames.

\subsection{Evaluation Datasets and Metrics}

\subsubsection{Evaluation Datasets} Our model is trained on a single dataset (Vimeo-90K training set) but validated on multiple datasets including Vimeo-90K\cite{xue2019video} test dataset (448 $\times$ 256), UCF \cite{liu2017video,soomro2012ucf101} dataset (25 FPS, 256 $\times$ 256), and the Middlebury dataset \cite{baker2011database} (typically 640 $\times$ 480). The Middlebury dataset has two subsets. The OTHER set provides the ground-truth middle frames while the EVALUATION set hides the ground-truth and can only be evaluated by uploading the results to the benchmark website.

\subsubsection{Evaluation Metrics} We report PSNR, SSIM \cite{wang2004image}, and Interpolation Error (IE) for model comparison on multiple datasets with various resolutions and contents. IE is the average absolute color error. Higher PSNR or SSIM and lower IE indicate better performance.

\subsection{Ablation Studies}

\subsubsection{Optical flow V.S. DConv}
To analyze how well the proposed framework performs with different image warping techniques, we train two variants of our approach, one using optical flow and the other using DConv at each scale. To integrate optical flow into our model, PDWN-optical flow generates and refines two sets of optical flow in every pyramid level instead of deformable offsets and modulation weights. Features and frames are backward warped by optical flow in PDWN-optical flow to replace deformable convolution in PDWN. As shown in Table \ref{table:cv} (section 1), DConv outperforms optical flow in terms of all performance metrics, which demonstrates the effectiveness of DConv. In Figure \ref{fig:flow}.(i), we visualize the DConv sampling points in the past and future frame respectively of an occluded point. We observe that the proposed model is able to point to locations in the left frame where the color is similar to the occluded region. As discussed above, DConv offsets can be considered as many-to-one backward warping flow. The redundancy of many-to-one flow makes the model more robust. In Figure \ref{fig:flow}.(e) and \ref{fig:flow}.(g), we visualize the weighted averaged DConv offsets by:
\begin{equation}
    \bar{f}_{1\to i}(x) = \frac{\sum_{j=1}^{|R|} (R(j) + f_{1\to i}(j, x)) m_{1\to i}(j, x)}{\sum_{j=1}^{|R|}m_{1\to i}(j, x)}
    \label{flow}
\end{equation}.

\begin{table}
\caption{Ablation studies on different components of PDWN}
\begin{center}
\begin{threeparttable}
\begin{tabular}{cccccc}
\toprule
\multirow{2}{*}{Model} & \multicolumn{2}{c}{Vimeo-90k} & \multicolumn{3}{c}{Middlebury OTHER}\\
\cmidrule(r){2-3} \cmidrule(r){4-6}
 & PSNR & SSIM & PSNR & SSIM & IE\\
\midrule
PDWN-optical flow & 34.59 & 0.961 & 35.35 & 0.957 & 2.47\\
\specialrule{0em}{1pt}{1pt}
PDWN w/o modulation  & {35.23} & {0.965} & {\bf37.00} & {\bf0.966} & {2.02} \\
\specialrule{0em}{1pt}{1pt}
PDWN w/ modulation & {\bf 35.38} & {\bf 0.966} & {\bf 37.00} & {\bf 0.966} & {\bf 2.00} \\
\specialrule{0.02em}{1pt}{1pt}
PDWN w/o CV & 35.13 & 0.964 & 37.09 & 0.966 & 1.99 \\
\specialrule{0em}{1pt}{1pt}
PDWN w/ CV & 35.38 & 0.966 & 37.00 & 0.966 & 2.00 \\
\specialrule{0em}{1pt}{1pt}
PDWN w/ learnt CV & {\bf 35.42} & {\bf 0.966} & {\bf 37.17} & {\bf 0.967} & {\bf 1.98}\\
\specialrule{0.02em}{1pt}{1pt}
PDWN w/o coarse-to-fine & 34.54 & 0.959  & 35.95 & 0.961 & 2.19 \\
\specialrule{0em}{1pt}{1pt}
PDWN w/ coarse-to-fine & {\bf 35.42} & {\bf 0.966} & {\bf 37.17} & {\bf 0.967} & {\bf 1.98}\\
\specialrule{0.02em}{1pt}{1pt}
PDWN w/o c. e. & 35.42 & {\bf0.966} & 37.17 & {\bf0.967} & {\bf1.98}\\
\specialrule{0em}{1pt}{1pt}
PDWN w/ c. e. & {\bf 35.44} & {\bf0.966} & {\bf 37.20} & {\bf0.967} & {\bf1.98}\\
\specialrule{0em}{1pt}{1pt}
\bottomrule
\end{tabular}
\begin{tablenotes}
        \footnotesize
        \item[*] CV denotes cost volume and c.e. denotes context enhancement. All models presented here use 6 scales. Models in section 1, 2, and 3 are trained without context enhancement.
      \end{tablenotes}
\end{threeparttable}
\end{center}
\label{table:cv}
\end{table}
\begin{figure}[ht]
\includegraphics[width=\linewidth]{img/results/coarse-to-fine.pdf}
\caption{{\bf Effect of the coarse-to-fine structure.} By introducing the coarse-to-fine structure, PDWN generates more realistic interpolation results.}
\label{fig:coarse}
\end{figure}
\begin{table}
\caption{Effect of number of scales.}
\label{table:scale}
\begin{threeparttable}
\begin{tabular}{cccccccc}
\toprule
\multirow{2}{*}{Scale}& \multicolumn{1}{c}{Runtime}& \multicolumn{1}{c}{Param.}& \multicolumn{2}{c}{Vimeo-90k} & \multicolumn{3}{c}{Middlebury OTHER} \\
      \cmidrule(r){4-5} \cmidrule(r){6-8}
 & (second) & (million) & PSNR & SSIM & PSNR & SSIM & IE\\
\midrule
L=4 & {\bf 0.0056} & {\bf 1.7} & 35.02 & 0.963 & 36.63 & 0.964 & 2.07\\
\specialrule{0em}{1pt}{1pt}
L=5 & 0.0068 & 3.4 & 35.19 & {\bf0.965} & 36.85 & 0.965 & 2.04\\
\specialrule{0em}{1pt}{1pt}
L=6 & 0.0086 & 6.6 & {\bf 35.23} & {\bf 0.965} & {\bf 37.00} & {\bf 0.966} & {\bf 2.02}\\
\bottomrule
\end{tabular}
\begin{tablenotes}
        \footnotesize
        \item[*] L denotes the number of scales. Note that in this experiment we use a simpler version of DConv where the modulation weights are all set to 1 and the cost volume is predefined. The models trained here are all without context enhancement. The feature size is downsampled 8, 16, 32 times for L = 4, 5, 6, respectively. The runtime is evaluated for interpolating one middle frame of "DogDance" from Middlebury OTHER dataset, with a size of $640\times 480$, on GeForce RTX 2080 Ti.
      \end{tablenotes}
\end{threeparttable}
\end{table}

\begin{figure*}[tb]
\begin{center}
\includegraphics[width=\linewidth]{img/results/flow_scales2.pdf}
\end{center}
   \caption{{\bf Analysis on warping operations and network scales \& visualization of DConv offsets and adaptive blending weights.} (c)-(d) Optical flow V.S. DConv. (d), (f), and (h) compares models with different number of scales. The model with larger scales is able to generate more accurate and sharper contents. (i) visualizes the sampling points of DConv in the past and future frames respectively.

    (e) and (g) show weighted averaged offsets to the past and future frame respectively at each scale, calculated based on Eq. (\ref{flow}). (j) is the adaptive weight map $\alpha$ for warped past frame, i.e., the weight for the future frame is $1-\alpha$. Thus, the black regions around the hand and ball show PDWN's capacity to handle occlusion.}
\label{fig:flow}
\end{figure*}

\begin{figure*}[tb]
\includegraphics[width=\linewidth]{img/results/vimeo.pdf}
\caption{{\bf Visualized examples on Vimeo-90k test dataset.}}
\label{fig:vimeo}
\end{figure*}

\begin{table*}[htb]
\caption{Comparison with state-of-the-arts}
\begin{center}
\begin{threeparttable}
\begin{tabular}{lccccccccc}
\toprule
\multirow{2}{*}{Method}&
Runtime& Param.& \multicolumn{2}{c}{Vimeo-90k} & \multicolumn{3}{c}{Middlebury OTHER} & \multicolumn{2}{c}{UCF}\\
        \cmidrule(r){4-5} \cmidrule(r){6-8} \cmidrule(r){9-10}
 & (second) & (million) & PSNR & SSIM & PSNR & SSIM & IE & PSNR & SSIM \\
\midrule
DVF \cite{liu2017video} &-& {\bf\color{red}3.8} & - & - & - & - & - & 34.12 & 0.942\\
\specialrule{0em}{1pt}{1pt}
SepConv-L1 \cite{niklaus2017video} & {\bf\color{red}0.0032} & 21.6 & 33.80 & 0.956 & 35.89 & 0.959 & 2.24 & 34.69 & 0.945\\
\specialrule{0em}{1pt}{1pt}
SepConv++ \cite{niklaus2021revisiting} & - & 13.6 & 34.98 & - & \bf\color{blue}{37.47} & - & - & \bf\color{red}{35.29} & - \\
\specialrule{0em}{1pt}{1pt}
SuperSlowMo \cite{jiang2018super} & - & 39.6 & - & - & - & - & - & 34.75 & 0.947 \\
\specialrule{0em}{1pt}{1pt}
MEMC-Net* \cite{bao2019memc} & 0.122 & 70.3  & 34.40 & 0.962 & 36.48 & 0.964 & 2.12 & 35.01 & {\bf\color{blue}0.949}\\
\specialrule{0em}{1pt}{1pt}
DAIN \cite{bao2019depth} & 0.125 & 24.0 & 34.71 & 0.964 & 36.70 & 0.965 & 2.04 & 34.99 & {\bf\color{blue}0.949}\\
\specialrule{0em}{1pt}{1pt}
RRPN \cite{zhang2020flexible} & - & - & - & - & - & - & - & 34.76 & - \\
\specialrule{0em}{1pt}{1pt}
AdaCof \cite{lee2020adacof} & {\bf\color{blue}0.0043} & 21.8 & 34.35 & 0.956 &
35.69 & 0.958 & 2.26 & 34.90 & {\bf\color{blue}0.949} \\
\specialrule{0em}{1pt}{1pt}
FeFlow \cite{gui2020featureflow}& 0.7188 & 133.6 & 35.16 & 0.963 & 36.61 & 0.965 & 2.14 & 34.89 & {\bf\color{blue}0.949}\\
\specialrule{0em}{1pt}{1pt}
PDWN (L=6) & 0.0089 & {\bf\color{blue}7.8} & {\bf\color{blue}35.44} & {\bf\color{blue}0.966} & 37.20 & {\bf\color{blue}0.967} & {\bf\color{blue} 1.98} & 35.00 & {\bf\color{red}0.950} \\%34.92 & {\bf \color{blue}0.948}\\
PDWN++ (L=6) & 0.0669 & 8.6 & {\bf\color{red}35.69} & {\bf\color{red}0.968} & {\bf\color{red}38.35} & {\bf\color{red}0.971} & {\bf\color{red} 1.81} & {\bf\color{blue}35.10} & {\bf\color{red}0.950}\\%34.92 & {\bf \color{blue}0.948}\\
\bottomrule

\end{tabular}
    \begin{tablenotes}
        \footnotesize
        \item[*]  PDWN achieves on par performance with much fewer parameters compared to previous methods.
        \item[*]  The runtime of DAIN and MEMC-Net* is reported in their paper on a 640x480 image using an NVIDIA Titan X (Pascal) GPU card. Other runtime numbers reported are estimated for "DogDance" image on an Nvidia RTX 2080 Ti GPU card.
        \item[*] The number in {\color{red}red} and {\color{blue}blue} represents the best and second best performance.
     \end{tablenotes}
 \end{threeparttable}
\end{center}
\label{table:vimeo}
\end{table*}

\begin{figure*}[htb]
\includegraphics[width=\linewidth]{img/results/middle.pdf}
\caption{{\bf Visualized examples on Middlebury EVALUATION dataset.} PDWN generates high-quality details on the foot and girl's toe while other methods produce blurry output. Moreover, PDWN shows its capacity in dealing with occlusion and semantic shape distortion on the ball and white flowers.}
\label{fig:middle}
\end{figure*}

\begin{table*}[htb]
\caption{Results on Middlebury EVALUATION datase}

\begin{center}
\renewcommand\tabcolsep{2.5pt}
\begin{threeparttable}
\begin{tabular}{lcc|cccccccccccccccc}
\toprule
\multirow{2}{*}{Method} & \multicolumn{2}{c}{Average} & \multicolumn{2}{c}{Mequon} & \multicolumn{2}{c}{Schefflera} & \multicolumn{2}{c}{Urban} &  \multicolumn{2}{c}{Teddy} & \multicolumn{2}{c}{Backyard} & \multicolumn{2}{c}{Basketball} & \multicolumn{2}{c}{Dumptruck} & \multicolumn{2}{c}{Evergreen}\\
     \cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} \cmidrule(r){8-9} \cmidrule(r){10-11} \cmidrule(r){12-13} \cmidrule(r){14-15}\cmidrule(r){16-17}\cmidrule(r){18-19}
 & IE & NIE & IE & NIE & IE & NIE & IE & NIE & IE & NIE & IE & NIE & IE & NIE & IE & NIE & IE & NIE\\
\midrule
SuperSlowMo\cite{jiang2018super} & 5.31 &  0.78 &2.51 & 0.59 & 3.66 &  0.72 & 2.91 & 0.74 & 5.05 & 0.98 & 9.56 & 0.94 & 5.37 & 0.96 & 6.69 &  0.60 & 6.73 & 0.69\\
\specialrule{0em}{1pt}{1pt}
MEMC-Net*\cite{bao2019memc} & 4.99 & 0.74 & 2.39 & 0.59 & 3.36 & 0.64 & 3.37 &  0.80 & 4.84 &  0.88 & 8.55 & 0.88 & 4.70 & {\bf \color{blue}0.85} & 6.40 &  0.64 & 6.37 & 0.63\\
\specialrule{0em}{1pt}{1pt}
DAIN\cite{bao2019depth}& 4.85 &  {\bf \color{blue}0.71} & 2.38 &  0.58 & 3.28 &  0.60 & 3.32 &  {\bf\color{blue}0.69} & 4.65 &  {\bf\color{blue}0.86} & 7.88 &  0.87 & 4.73 &  {\bf \color{blue}0.85} & 6.36 & {\bf\color{blue}0.59} & 6.25 & 0.66\\
\specialrule{0em}{1pt}{1pt}
AdaCof\cite{lee2020adacof} & 4.75 & 0.73 & 2.41 & 0.60 & {\bf\color{blue}3.10} & 0.59 & 3.48 & 0.84 & 4.84 & 0.92 & 8.68 & 0.90 & {\bf \color{blue}4.13} & {\bf\color{red} 0.84} & {\bf \color{blue}5.77} & {\bf\color{red}0.58} &	{\bf \color{red}5.60} & {\bf\color{red}0.57} \\
\specialrule{0em}{1pt}{1pt}
FeFlow\cite{gui2020featureflow} & 4.82 &  {\bf \color{blue}0.71} &  {\bf \color{blue}2.28} & {\bf\color{blue}0.51} & 3.50 &  0.66 & {\bf \color{blue}2.82}  & 0.70 & 4.75 &  0.87 & {\bf \color{red}7.62} &  {\bf\color{red}0.84} & 4.74 &  0.86 & 6.07 &  0.64 & 6.78 & 0.67 \\
\specialrule{0em}{1pt}{1pt}
RRPN \cite{zhang2020flexible} & 4.93 & 0.75 & 2.38 & 0.53 & 3.70 & 0.69 & 3.29 & 0.87 & 5.05 & 0.94 & 8.20 & 0.88 & 4.38 & 0.88 & 6.50 & 0.65 & 6.00 & 0.62\\
\specialrule{0em}{1pt}{1pt}
SepConv++ \cite{niklaus2021revisiting} & {\bf \color{red}3.88} & 0.73 & 2.39 & 0.58 & {\bf\color{red}2.98} & {\bf\color{red}0.56} & 3.34 & 0.95 & {\bf\color{blue}4.49} &0.87& {\bf\color{blue}7.64} & {\bf\color{blue}0.85}& {\bf\color{red}3.77} & {\bf\color{red}0.84} & {\bf\color{red}5.26} & {\bf\color{blue}0.59} & {\bf\color{blue}5.71} & {\bf\color{blue}0.59}\\
\specialrule{0em}{1pt}{1pt}
PDWN (L=6) & {\bf \color{blue}4.71} & {\bf \color{red} 0.69} &{\bf \color{red}2.09} & {\bf\color{red}0.46} & 3.12 & {\bf\color{blue} 0.58} & {\bf \color{red} 2.38} & {\bf\color{red}0.64} & {\bf \color{red} 4.29} & {\bf\color{red}0.85} & 8.61 & 0.87 & 4.80 & 0.88 & 6.24 & 0.60 &	6.18 & 0.62\\
\bottomrule
\end{tabular}

\begin{tablenotes}
\item[*] NIE: normalized interpolation error.
\end{tablenotes}

\end{threeparttable}
\end{center}
\label{table:middle}
\end{table*}

\begin{figure*}[htb]
\includegraphics[width=\linewidth]{img/results/vimeo177.pdf}
\caption{{\bf Visualized examples of the extended PDWN with 4 input frames.} PDWN-2 takes only 1 past frame and 1 future frame as input. QuaFlow and PDWN-4 take 2 past frames and 2 future frames as input.}
\label{fig:vimeo177}
\end{figure*}

\begin{table}[htb]
\caption{Results of the extended PDWN with four input frames on Vimeo-90K spetulet dataset}
\begin{center}
\begin{threeparttable}
\begin{tabular}{lcccc}
\toprule
\multirow{2}{*}{Method}&
Runtime& Param.& \multicolumn{2}{c}{Vimeo-septuplet}\\
        \cmidrule(r){4-5}
 & (second) & (million) & PSNR & SSIM\\
\midrule
FeFlow \cite{gui2020featureflow} & 0.221 & 133.6 & 33.88 & 0.946 \\
\specialrule{0em}{1pt}{1pt}
PDWN-2 & {\bf 0.010} & {\bf 7.4} & 35.53 & 0.958\\
\specialrule{0em}{1pt}{1pt}
QuaFlow \cite{xu2019quadratic}  & 0.090 & 19.6 & 34.28 & 0.950\\
\specialrule{0em}{1pt}{1pt}
PDWN-4 & 0.012 & 8.3  & {\bf 35.93} & {\bf 0.960}\\

\bottomrule
\end{tabular}
    \begin{tablenotes}
        \footnotesize
        \item[*]  FeFlow and PDWN-2 take only 1 past frame and 1 future frame as input. QuaFlow and PDWN-4 take 2 past frames and 2 future frames as input.
        \item[*] Both PDWN-2 and PDWN-4 have 6 pyramid levels and no contexual enhancement module.
        \item[*] The runtime reported is the average runtime for Vimeo-septuplet dataset with image size $448\times256$ on an Nvidia Tesla V100 GPU card.
     \end{tablenotes}
 \end{threeparttable}
\end{center}
\label{table:4input}
\end{table}

\subsubsection{Cost volume}
To analyze the effectiveness of using cost volumes, we consider three variants of our approach. The first model takes warped features only as input to the first conv layer in the offset estimator in Figure \ref{fig:network_structure}.(b). The second model first computes the cost volume between two warped features, then concatenates the cost volume and the original features to estimate DConv offset residuals. The third model replaces the cost volume layer with a two-layer CNN to learn the matching cost between two warped features. As shown in Table \ref{table:cv} (section 2), cost volumes bring additional improvements without adding more parameters on Vimeo-90K dataset. Replacing the predefined cost with the learnt cost further improves the results for both datasets.

\subsubsection{Coarse-to-fine successive refinement manner}
In the proposed model, we warp features and construct the matching cost between warped features to estimate DConv offset {\it residuals} $\Delta f_{1\to i}^l$ at every pyramid level in a coarse-to-fine manner. It reduces the distance between two input frames gradually and is particularly important when the ground truth motion is large. We investigate the contribution of this coarse-to-fine structure via training another variant of our model, without the coarse-to-fine structure. In other words, this model is simply a UNet structure with 6 spatial scales that takes two images $I_0$ and $I_2$ as input and directly outputs DConv offsets and modulation weights in the finest scale. We show the quantitative results in Table \ref{table:cv} (section 3) and qualitative results in Figure \ref{fig:coarse}. By introducing the coarse-to-fine structure, the performance is significantly improved, demonstrating the effectiveness of our successive coarse-to-fine successive refinement approach.

\subsubsection{Impact of the number of scales}
To analyze the impact of the number of scales on the performance, we investigate three different pyramid scales ($L$ = 4, 5, 6). Quantitative results are shown in Table \ref{table:scale}, and the visual comparison is provided in Figure \ref{fig:flow}. We find that with model size increasing from 1.7, 3.4, to 6.6 million, the PSNR steadily get better from 36.63, 36.85, to 37.00 dB on Middlebury OTHER dataset. The example in Figure \ref{fig:flow} also shows that the model using more scales generates sharper outcomes. The gain on Vimeo-90K, however, is not as significant as that on Middlebury OTHER dataset. That is probably because Middlebury OTHER dataset has a larger image size (and hence larger motion in terms of pixels) than Vimeo-90K dataset. Even though the model size almost doubles with each additional scale, the runtime only increases slightly, as the lower scale images and features have a smaller spatial dimension.

\subsubsection{Adaptive Blending Weight} Figure \ref{fig:flow}.(j) shows an example of adaptive blending weight map. As discussed in section 3.3, $\alpha(x) = 0$ means pixel  x from $I_0$ is occluded and pixels x from $I_1$ is fully trusted. The black region around the ball in the weight map indicates that our model can detect and solve occlusion by selecting pixels from the previous and following frames softly.

\subsubsection{Context enhancement network}
To analyze the contribution of the context enhancement module, we train a variant of PDWN without context enhancement and show the results in Table \ref{table:cv}. Though DAIN gains significantly from adding the context enhancement module (0.27 dB on Vimeo-90k in terms of PSNR) \cite{bao2019depth},
the context enhancement network has little contribution to PDWN. By adding the context enhancement network, the number of model parameters increases from 7.4 million to 7.8 million and the runtime increases from 0.0082 to 0.0086 for interpolating "DogDance" image (640$\times$480) in Middlebury-OTHER dataset, using an NVIDIA RTX 8000 GPU card.

\subsection{Comparison with state-of-the-arts}
We compare our model with state-of-the-art video interpolation models both quantitatively and qualitatively, including deep voxel flow (DVF) \cite{liu2017video},  SepConv\cite{niklaus2017video}, SepConv++ \cite{niklaus2021revisiting}, SuperSloMo \cite{jiang2018super}, MEMC-Net* \cite{bao2019memc}, DAIN \cite{bao2019depth}, AdaCof \cite{lee2020adacof}, FeFlow \cite{gui2020featureflow}, on three different datasets, Vimeo-90K, UCF, and Middlebury dataset. Note that we only compare with methods which use backward optical flow or DConv for backward image warping. For SepConv, AdaCof, and FeFlow, we download their published models and test on the testing datasets. For DVF, SuperSloMo, MEMC-Net*, and DAIN, we calculate the numbers from their published interpolated data. For RRPN and SepConv++, we directly report their published numbers.

As shown in Table \ref{table:vimeo}, our proposed model outperforms all methods on Vimeo-90k dataset and Middlebury OTHER dataset except SepConv++. Using similar techniques applied to SepConv++, PDWN++ surpasses SepConv++ for 0.88 dB on Middlebury OTHER dataset with respect to PSNR. Meanwhile, the number of model parameters increases from 7.8 million to 8.6 million and the runtime increases nearly 8 times. On UCF dataset, our model achieves on par performance with state-of-the-art methods. Note that DAIN uses additional depth information to detect occlusion in order to compensate errors in the linear interpolated optical flow. DAIN relies on the accuracy of depth information, i.e., their model cannot learn meaningful depth information without a good initialization of (pretrained) depth estimation network and thus yields lower quality results than MEMC-Net. Our model does not need depth information for training information but still achieve 0.73 dB higher PSNR than DAIN on Vimeo-90K. FeFlow uses multiple groups of DConv offsets in every layer to avoide occlusion and edge maps generated by BDCN \cite{He_2019} as structure guidance. Compared to FeFlow, our model performs better on Vimeo-90K without edge maps and with only a single group of DConv offsets, which demonstrated the supremacy of using DConv in a coarse-to-fine manner. Moreover, our model size is only 5.8\% of that of FeFlow. Figure \ref{fig:vimeo} presents two examples from Vimeo-90k dataset. Notably, our model generates the sharpest results among all compared methods.

Table \ref{table:middle} shows the comparison on Middlebury EVALUATION dataset. Our proposed method performs favorably against state-of-the-art methods. Our model performs well quantitatively on sequences with small motion or fine textures such as {\it Mequon, Teddy} and {\it Schefflera}. For videos with complicated motions, Figure \ref{fig:middle} shows a visualized example. Our model produces more details at the girl's toe in the {\it backyard} example while other methods output blurry results. And our model handles occlusion well around the boundary of the orange ball.

\subsection{Extending to four input frames}
Vimeo-90K septuplet dataset is used to train and test our extended model PDWN-4 which takes four input frames as input and has 6 pyramid levels. We use frame 1, 3, 5, and 7 to interpolate frame 4 and compare the interpolated frame 4 with the original frame 4 for every sequence in Vimeo-90K septuplet dataset. We compare the results with our two-input model PDWN-2 and state-of-the-art methods including FeFlow \cite{gui2020featureflow} and QuaFlow \cite{xu2019quadratic}. PDWN-2 is pretrained on Vimeo-90K triplet dataset and finetuned on Vimeo-90K septuplet dataset. Results are given in Table \ref{table:4input}. Figure \ref{fig:vimeo177} shows visualized results on the Vimeo-90K septuplet test dataset. Both the quantitative and visual evaluations demonstrate that the extended PDWN with four input frames can significantly improve the interpolation accuracy over using two input frames, with only modest increases in the model size and the runtime. Furthermore, both PDWN-2 and PDWN-4 yield better results than QuaFlow that uses four input frames.

\section{Conclusion}

In this work, we propose a pyramid video interpolation model that estimates the many-to-one flows with modulation maps of the middle frame to the left and right input frames. We show that the offset estimator can benefit from using the cost volumes computed from the aligned features, compared to using the aligned features directly. Our model is significantly smaller in model size and requires substantially less inference time compared to state-of-the-art models and yet achieves better or on-par interpolation accuracy. Besides, our model does not rely on additional information (e.g. ground truth depth information or optical flow) for training. Moreover, our model that uses two input frames can be extended to use four input frames easily, with only a small increase in the model size and the inference time, and yet the extended model significantly improves the interpolation accuracy.

A recent work \cite{Niklaus_CVPR_2020}, which proposes a differentiable forward warping operation using forward optical flow to handle occlusion and dis-occlusion regions directly, outperforms all backward-flow-based methods. It shows a promising direction for video interpolation. In future work, we will also explore how to combine forward warping with a coarse-to-fine structure. Furthermore, we will explore the integration of PDWN in video coding, where the encoder can encode every other frame; Skipped frames will be interpolated by the PDWN method and the interpolation error images can be additionally coded.

\clearpage
\clearpage
\printbibliography

\end{document}

redpajama_set_name: RedPajamaArXiv


text:
Cloudy on the other side
Behind the cloud
london based female producer ro returns with electro pop dazzler 'low in the evenings'
After a well-received 2019 debut ('do you think it's fine?), Ro released her second single ('forgot'), which gained support from tastemakers such as Harriet Rose at Fubar Radio. Following a recent feature with LA producer Damon Steele (via PUZL Records), she's back with a kick. Single no.3 is titled 'low in the evenings' and it makes you want to dance and cry at the same time.
Blending a signature 80's touch with a 00's pop influence, the track delivers glittery vocals and charged-up choruses, reminiscent of some of Ro's idolised female creatives, Grimes and Christine and the Queens. Growing up making demos with tape recorders, the now London-based artist produces all of her music, which has created a sonic ID with an all-in-one approach.
Last year saw Ro performing at Laylow in Notting Hill for Trash Like You Records' collaboration with The Circle (a charity founded by Annie Lennox) and 2020 has seen her also grace the stage at Ronnie Scott's.
Check out her Ro's new single here:
Ro · low in the evenings
Blog / Features
Claudia eRecords

redpajama_set_name: RedPajamaCommonCrawl


text:
WA teen soccer player snapped up by Bundesliga team
Jacob Italiano has shone in his rookie year with Perth Glory.
Alexis Sanchez to Manchester United transfer news: Debut details, first training, new house
Jake RosengartenPerthNow
NEW signing Alexis Sanchez could make his Manchester United debut when the Red Devils travel to Yeovil Town in the fourth round of the FA Cup on Saturday morning.
It's probably not the lavish setting the big money Chilean had in mind for his bow, but he'll be keen to make an impact during his first minutes as a United player.
Sanchez is eligible to play for United after being rested for Arsenal's third round defeat to Nottingham Forest.
His potential opponents are enjoying the prospect of a debut for the attacker, with their social media team posting a tongue in cheek jibe aimed at the ex-Gunner.
"You said you've joined 'the biggest club in the world.' So it's only right that you start your United career in the biggest away dressing room in League Two," they tweeted.
Meanwhile, Sanchez has linked up with his new teammates for the first time. The attacker arrived at his new club's Carrington training base in his flashy Bentley, before joining his first session.
MOURINHO'S NEW ROOMMATE
After working hard to sign the Chilean, it appears Jose Mourinho will have the honour of sharing a residence with him!
The Mirror report that Sanchez has moved into Manchester's Lowry Hotel, the very same building Mourinho calls home.
Sanchez's beloved dogs have moved with him to Manchester, and were snapped wearing United jerseys after the move was completed.
Mourinho is reportedly living the high life in Manchester, despite still often returning to his London home. The Special One's digs cost a reported £816 (AU$1,400) a night!
Although he may not have the pleasure of being neighbours with Sanchez for long…
ALEXIS TOURS MANSION
The United new boy toured a £2m (AU$3m) mansion in greater Manchester after signing a contract which made him the Premier League's highest earner.
The property includes five bedrooms, one of which fits two double beds, a piano room and garden large enough to fit a full size goal with room left over for entertaining!
Sanchez was snapped arriving at the property in his Bentley, and could live down the road from international teammate and Manchester City goalkeeper Claudio Bravo.
He's got to spend his pay cheque somehow!
SANCHEZ CLARIFIES HENRY COMMENTS AFTER OUTRAGE
In his final act as an Arsenal player, Alexis Sanchez threw club legend Thierry Henry under the bus.
Sanchez implied that Henry had a hand in his decision to leave the club, stating "I remember today, a conversation I had with Henry, a historic Arsenal player, who changed club for the same reason and today is my turn."
His comments prompted Arsenal faithful to label their former hero a "snake."
Alexis took to twitter to defend Henry and suggested his comments were misinterpreted.
"Henry never told me to leave the club, it was a personal decision," Sanchez tweeted.
"He LOVES the club and someday it would be nice to see him as the Arsenal coach because he loves the club.'"
Camera IconThierry Henry is an Arsenal legend Credit: AFP
MKHI'S OZIL PRAISE
The man who took Sanchez's place at Arsenal, Henrikh Mkhitaryan, has praised Gunners playmaker Mesut Ozil, labelling the German an 'assist king.'
Mkhitaryan trained with his new side for the first time ahead of their second-leg Carabao Cup semi-final victory over Chelsea, although the Armenian was unable to take part in the clash having already represented United in the competition earlier this season.
When asked about his new teammates by the Arsenal website, the midfielder cited Mesut Ozil as someone he wished to build a relationship with.
"Here at Arsenal I know that lots of players move a lot and it's incredible the way [the team] plays. I will do my best to assist them as well," he told the Gunners website.
"I think it will be really fun to play alongside Ozil. Everyone knows he's a fantastic player, he's an assist king! I look forward to it."
"[I admire] his intelligence, the way that he can understand the game, reads the game, plus the way he gives pace to the game and of course his final decision."
Camera IconHenrikh Mkhitaryan praised Mesut Ozil after joining Arsenal Credit: AFP


Surge protection devices type 1 Welcome to Hager Hong Kong!
This type of surge protective devices are recommended on electric installations where the buildings are fitted with lightning conductor. The minimum value of shock current (I imp) = 12.5 kA.
In order to ensure optimum protection, those surge protective devices are using the "gas discharge spark gap" technology, associated with an electronic triggering system, which ensures a high discharging capacity with very short response time.
With a discharge current wave 10/350 µs (I imp) which is similar to lightning current on direct strike, those SPD's must have the capacity to divert out this energizing wave.
Monobloc SPD's type 1 have an LED for function indication for each phase on the front.
Connection capacity: 35mm² flexible conductor, 50mm² rigid conductor.
These surge protective devices complies with IEC 61643-11 september 2002.


Looking for a winter wonderland eco adventure? The Whitepod eco resort located in the village of Les Cerniers in the Swiss Alps offers a private ski resort, mountain chalets and 15 geodesic-dome pods that offer low impact accommodations. Guests tour the resort by skis, snowboards, dog sleds or snowshoes in the winter and relax in the luxurious and efficient Whitepod domes. The Swiss adventure resort won the World Prize for Sustainable Tourism in 2005.
Whitepod is a collection of uber-cosy, igloo-shaped pods (tents), clustered around the Pod House (a central wooden chalet) and set in the pristine Swiss mountains. The camp philosophy is very much about saving the environment and getting away from it all. There are no noisy bars or busy streets nearby, nor are there crowds of speed-junky skiers; just the camp, its private slopes and nature – lots of it. It is utterly still and unbelievably beautiful.


text:
Jay Nordlinger
American journalist
Jay Nordlinger (born 1963) is an American journalist. He is a senior editor of National Review, and a book fellow of the National Review Institute. He is also a music critic for The New Criterion and The Conservative. In the 1990s, Nordlinger worked for The Weekly Standard magazine. In the 2000s, he was music critic for the New York Sun. Nordlinger assisted the speechwriting team of George W. Bush in the last six weeks of the 2000 presidential election.
One more word on talking, before I stop talking.
1.1.1 Change and Determination: After 9/11, a shaking up (2002)
1.2.1 Baseball and Its Worriers (2018)
1.2.2 The Disinvitation Game, or, Against Weenification (2018)
1.2.3 Some Talk about Talk (2018)
Talk can be cheap, very cheap. It can also be costly. "Speak out!" we say. "Why are you afraid to speak out?" we say. In dictatorships, it can be very, very hard to speak out. Many people have been imprisoned or worse for talk.
People are always criticizing Twitter. "Twitter is crazy!" they say. I think that's misguided. Twitter is simply an avenue -- there are many -- by which people reveal who they are.
In capitalism, your narrowest interests are advanced by cooperation. A genius system.
I grew up with the phrase "Believe the woman." It was almost a slogan. You didn't hear it much during the Clinton years.
2000sEdit
Change and Determination: After 9/11, a shaking up (2002)Edit
"Change and Determination: After 9/11, a shaking up" (11 September 2002), National Review
But understand them, people say. And one does. But sometimes understanding is not comforting or flattering to the understood.
Nemtsov sacrificed a lot for politics, for a freer Russia — and I'm not just talking about his life, his mortal life.
"A Russian Patriot and His Country, Part I" (10 April 2017), National Review
[T]he nutcase Left and the nutcase Right are alike in virtually every particular. They should get a room somewhere -- far from here.
Twitter post (4 September 2018)
"Globalism" is often the most self-interested and "nationalist" thing you can do. It has been a bonanza to the United States, for sure. In capitalism, your narrowest interests are advanced by cooperation. A genius system.
More and more, I tune out when I see the word "elites," regarding it as a lazy Marxist slur, from both Left and Right. I still use it, when I believe it is absolutely the right word. But more and more, it makes me say, "Bye."
Twitter post (10 September 2018)
There's a point at which left and right join.
"Jaywalking" (12 September 2018), National Review
Too often, claims of "fake news" are the real fake news. We are taught, rightly, to be skeptical of the media (and indeed of most things). But it pays to be skeptical of their critics, taunters, and defamers, too.Jay Nordlinger added,
More and more, I like Utah.
Twitter post (19 April 2019)
Individuals are responsible for their actions. And yet . . .
No man is an island. We are all subject to influences, good and bad.
"Thoughts on El Paso" (August 2019), National Review
Baseball and Its Worriers (2018)Edit
"Baseball and Its Worriers" (3 September 2018), National Review
Maybe baseball has to accept that it is now more like classical music than popular music, with football and basketball — and soccer? — being the Justin Biebers and Lady Gagas of sports. Baseball need not hang its head in shame. A lot of things that are good and worthy are not popular. And baseball is plenty popular, for heaven's sake.
The Disinvitation Game, or, Against Weenification (2018)Edit
"The Disinvitation Game, or, Against Weenification" (4 September 2018), National Review Online
[T]he entire country should man up. We are drowning in weenification and snowflakiness. Shall we grow a national pair?
Some Talk about Talk (2018)Edit
"Some Talk about Talk" (4 September 2018), National Review Online
Talk can be cheap, very cheap. It can also be costly. "Speak out!" we say. "Why are you afraid to speak out?" we say. In dictatorships, it can be very, very hard to speak out. Many people have been imprisoned or worse for talk. But even in democratic societies, talk can be hard. It can be hard not only in politics but also in high schools and families and churches and professional communities and other arenas. But even in democratic societies, talk can be hard. It can be hard not only in politics but also in high schools and families and churches and professional communities and other arenas.
One more word on talking, before I stop talking, at least about this subject: You or I may not like the talker, but talk matters, much of the time. It is especially significant when the talker is lonely — when most around him are keeping mum.
Nordlinger's profile at National Review Online
Archive of Nordlinger's writings at the New York Sun
Archive of Nordlinger's writings at The New Criterion
Retrieved from "https://en.wikiquote.org/w/index.php?title=Jay_Nordlinger&oldid=2854901"


RETRO KIMMER'S BLOG: NEW! GEORGE HARRISON VINYL COLLECTION!
George Harrison - The Vinyl Collection is due for release on 24 February to mark George's 74th birthday. All 12 of George's studio albums plus Live in Japan and two 12" Picture Disc singles in one specially designed box.
Limited badge set available exclusively when you order from the official store, while stocks last.
An expanded edition of George's much-loved autobiography, I, Me, Mine, originally released through Genesis Publications in 1980, will also be released on February 24.
The George Harrison estate have teamed up with one of the worlds most recognised and respected turntable manufacturers, Pro-Ject to bring you this limited edition George Harrison turntable!
The vinyl box set includes all twelve of George's studio albums with exact replicas of the original release track listing and artwork. Also included in the box set are George's classic live album Live In Japan (2LP), and two 12" single picture discs of 'When We Was Fab' and 'Got My Mind Set On You'. All the discs are 180-gram heavyweight vinyl and are housed in a high-quality two-piece rigid slipcase box. The original analogue master tapes were used for the new re-masters and were cut at the legendary Capitol studios to ensure exceptional audio quality throughout. The individual albums from the collection will also be available separately, with All Things Must Pass as a limited edition title.


Whether you're looking for the most up-to-date items Urparcel Pet Dog Lace Tutu Dress Striped Spider Skirt Princess Clothes with the cheapest price and also have a discount? Urparcel Pet Dog Lace Tutu Dress Striped Spider Skirt Princess Clothes using high quality products. You can see evaluation and features with this post.
This item runs small a lot!!Pls pay attention to the size chart before payment!!Thanks!!
Fits just perfect! ! And came in quicker than I expected! !
This outfit looks just like the picture I could not be happier with this dress I let her wear it now but I can't wait for Halloween. It fits amazing looks a asking and is not made cheaply.
Size runs small, but nice item. Seller accepts returns but with very limited return and suggest for you to sale the item yourself.
It is so sweet on my Bella Sophia! Fits her well. Thank you!
This entry was posted in Urparcel on August 22, 2014 by marjinal.


text:
Ranked No. 1
Best Infrastructure
Bank Loan Facility
100+ Scholarship
1000+ Alumni
Welcome to Heritage Institute of Management & Communication
Best Mass Communication College in Delhi
Heritage Institute of Management & Communication is the most highly developed Institute of its kind provides practical training to the students in the field of Journalism & Mass Communication from day one with its highly qualified and experienced core and guest faculty. There is a huge potential for creative talent in India. The sector is facing an acute shortage of trained personnel. Media platforms-channels and production houses-both desperately seek quality-trained manpower in various functions involved in the making of a successful media product. The Heritage Institute of Management & Communication. (HIMCOM), Delhi, has been set up to offer contemporarily credible and relevant media education. We offered courses like Bachelor Degree in Journalism and Mass Communication, Masters Degree in Journalism and Mass Communication, PG Diploma in Journalism and Mass Communication, Certificate program in Videography, Certificate program in Video Editing etc. HIMCOM is now awarded International excellence award for Best Mass Communication College in North India 2017.
Why HIMCOM
Get 100 % assured placement on the basis of our live TV & Radio studio, newsroom, PCR, MCR, training in 24X7 News Channels.
Get UGC & NAAC Accredited University Degrees + Industry Integrated Diplomas (Autonomous) based on rigorous practical training.
Get dynamically trained by top-notched broadcast professionals from various news channels.
Get specialization in Print journalism and get a chance to learn from esteemed print organizations.
Get individual attention in the classes because of low student-teacher ratio. The parents will be periodically informed about the progress of their wards.
Get prepared on Apple Editing Machines with FCP (Final Cut Pro), Sony 4K Camera & Studio DSLR Cameras.
Get geared up on High-end Graphic Machines and Live Broadcast Graphics softwares.
Get Scholarships if you are a highly talented and deserving candidate.
Get assistance for getting Education Loan from public sector banks.
Get adequate Hostel Facilities (for all out-station students)
Prakash Javadekar
I am extremely happy to know that a media institute, Heritage Institute of Management & Communication (HIMCOM), has been giving the quality education since 5 years. The institute has been imparting rigorous practical training along theoretical inputs to nurture professional as per the need of media industry.
My support and best wishes to the institute.
P.P. Chaudhary
I am glad to know that "Heritage Institute of Management & Communication" New Delhi is imparting quality education in the field of journalism and mass communication and also dedicating themselves in social development and growth through participative journalism.
I also extend my heartiest good wishes for the welfare activities.
I am happy to learn that the Heritage Charitable Trust is establishing "Heritage Institute of Management & Communication (HIMCOM)" in Delhi, with the objective of training the students in the field of journalism & mass communication.
I wish HIMCOM, the faculty and the students all the best.
Sheila Dikshit
I am glad to learn that the Heritage Institute of Management & Communication (HIMCOM) is being inaugurated under the aegis of Heritage Charitable Trust.
I do hope that the institutes would be able to impart quality education in journalism and Mass Communication.
It gives me immense pleasure to know that Heritage Institute of Management & Communication (HIMCOM) has been imparting education & training in Mass Communication and Journalism under the aegis of Heritage Charitable Trust.
I also believe that Himcom will play a significant role in enhancing the standards and quality of media education by preparing future journalists and media professionals as per its needs.
Prof. Kiran Walia
I am glad to learn that Heritage Institute of Management & Communication (HIMCOM) is being inaugurated under the aegis the Heritage Charitable Trust.
I do hope that the institute would be able to import quality education in Journalism and Mass Communication.
Rajeev ShuklaSpokesperson, Congress
I am extremely delighted to learn about the inception of HIMCOM under the revered aegis if Heritage Charitable Trust. The institute is a great initiatives as it shall adroitly train and enhance the students in the area of journalism and mass communication.
Syed Shahnawaz HussainSpokesperson, BJP
I am happy to learn that HIMCOM is being opened under the aegis of Heritage Charitable Trust to train students in journalism and mass comm. The efforts of the founder of HIMCOM have now borne fruit. I wish the institute all success and convey my best wishes to its promoters.
Himcom Courses
10+2 from a recognized board.
Graduate in any discipline.
10+2 and above
Certificate program in Videography (Camera & Lighting)
Certificate Program in Radio Jockeying
HIMCOM Video
HIMCOM Photo Gallery
Placement Partners


text:
I Was Crazy
Crazyness
for my wife
told me
Crazy, Forever, and Happy: This is to him, With you, everything was perfect No matter how worst the day was as long as we were together, the world was perfect You made me completly happy. I was crazy for you. So please come back I don't know how to stop loving you Every night I cry because of you, every night pray for you to come back to me I never loved someone like I loved you. It was you, it is you and it'l be you forever. https://iglovequotes.net/
Crazy, Net, and You: You should have opened your eyes. I was crazy for you https://iglovequotes.net/
Crazy, Net, and You: You should have opened your eyes. I was crazy for you. https://iglovequotes.net/
Crazy, Disney, and Family: 1 hr 4/10/19 Where do l even start!? Today has been a month + in the making and it was such an incredibly special and fun day! Cancer has taken and continues to take so much from Trinity and our family, so we try to think outside of the box to come up with super fun, once in a lifetime things to do, see, or go to. I orginally reached out to Ink Wolves with an idea, not sure if they would even reply, and if they did reply I wasn't sure if they would think I was crazy! To my delight they jumped right into this crazy idea with me! I wanted to let Trinity get some "tattoos" because she loves the ones her daddy has. Ink Wolves took it upon themselves to get Tattoo markers, practice with them, have 2 artists come in on their day off, have snacks, and even jam out to Disney music for over 4 hours with a 3 year old who is fighting cancer, all while making sure Trinity had the BEST day! Never in a million years did I ever think I would take my 3 year old toddler to a tattoo shop, and never in a million years did I think that her and I would get our first tattoos together! But let me tell you I am SO thankful that we took a big jump out of the box and just "went for it" because we all had the BEST DAY TODAY!!!!! Thank you again Ink Wolves team! Many more pictures to come! #Trinitytough #WeFightTogether uesseu he wrong Trinity is a 3 year old girl that got diagnosed with Neuroblastoma cancer in July 2018. This week a tattoo shop, gave her and her mom an unforgettable experience by giving them their own person 4 hour "tattoo session".
Trinity is a 3 year old girl that got diagnosed with Neuroblastoma cancer in July 2018. This week a tattoo shop, gave her and her mom an unf...
Crazy, Forever, and Happy: This is to him, With you, everything was perfect. No matter how worst the day was, as long as we were together, the world was perfect. You made me completly happy I was crazy for you. So please come back. I don't know how to stop loving you Every night I cry because of you, every night I pray for you to come back to me. I never loved someone like I loved you. It was you, it is you and itl be you forever. http://iglovequotes.net/
Crazy, Doctor, and Dogs: vaspider When I was ten years old, a dog bit the back of my head. The doctor said, within earshot but out of sight- he didnt think I could hear him-that had the dog's teeth been a little longer, they could have gone in under my skull. Hit my brain stem. Killed me, crippled me I don't know whether or not he was right. All I know is that for a decade and a half after that, I harbored a complete and unreasoning terror of dogs. It didn't matter how big they were, or how tame, or how kind. Someone else could assure me that they were the best dog in the world, that they knew the dog that he would never hurt anyone, and it didn't matter, because I was convinced that any dog could suddenly turn on me, bite me hard enough to kill me There were two dogs I slowly learned to trust during that time period. And eventually, I learned to understand dogs again, to understand their body anguage, to like them again, because my husband convinced me to get a puppy, to raise a dog from when it was smal, that this would help me get over my fea No one ever told me I was crazy or irrational for not wanting to be around dogs after l'd been attacked and nearly killed by one. No one. For fifteen years, it was understandable that I would be afraid -terribly, unreasoningly afraid - of dogs. A dog attacked me. I bore the traumatic scars. I found dogs terrifying unpredictable. I could not trust any of them no matter how kind they'd beern to my friends, no matter how well-recommended they came or how well they'd been raised not to turn on me and injure me When I was fifteen, I was raped. Do I even need to finish this? goddess3 THIS IS WHAT I SAY ALL THE FUCKING TIME AND NO ONE UNDEFSTANDS What some people really need to understand
What some people really need to understand
Crazy, Dank, and Spider: stan lee @TheRealStanLee Never give up on your dreams! When l first wrote Spider-Man my publisher said I was crazy because people hate spiders and insects and he was not going to publish it. But I never gave up, until it was published. 5/19/18, 12:44 PM 40K Retweets 133K Likes RIP Legend
RIP Legend
Crazy, Tumblr, and Blog: papabearrj: myrrhguy: sandersstudies: This is so Unnecessary how do you explain to someone that this is your sense of humour This killed me! I showed it to three people who just looked at me like I was crazy.
papabearrj: myrrhguy: sandersstudies: This is so Unnecessary how do you explain to someone that this is your sense of humour This kill...
Crazy, Http, and Net: You should have opened your eyes. I was crazy for you. http://iglovequotes.net/
Crazy, Memes, and Penguins: This is gonna sound crazy, but lmet someone up on this roof four years ago and they mixed that cocktail, and they loved penguins. By any chance, was that you? It's Everyone thought I was crazy, but I. You are such a loser. Who remembers this? 😂 #HIMYM https://t.co/JmJGp6svri
Who remembers this? 😂 #HIMYM https://t.co/JmJGp6svri
Crazy, Love, and All The: You should've opened your eyes, I was crazy about you. I was ready to love you all the way baby
Being Alone, Ass, and Beef: 2017 AP Vine Exam Multiple-Choice Questions 26) "Officer I've got one question for you. a. Day Care lookin ass muthaf***tr b. Flex on him cuh c. Honeyyy No!! d. What are those! 27) "I Smell like a, cheese b. beef c. a foot fetish d. the inside of a Mormon cult house 28) "It is , my dudes." a. My cheat day b. Wednesday c. Saturday d. Hump Day 29) " , Richard!" a. What the F**k b. What are you doing c. Your moms a hoe d. You were a failed abortion 30) "Honestly I don't remember I was crazy back then." a. I was probably Fx***d up b. I was not of legal age c. It was so hazy, I could taste colors. d. I was too busy respecting women 31) l'm washing b***h a. my sins b. me and my clothes c. my whole self d. the minority off me 2017 APTM VINE FREE-RESPONSE QUESTIONS RIP Vine SECTION I, Part B Time-50 minutes 4 Questions rections: Write your responses in the Section I, Part B: Short-Answer Response booklet. You must write your response to each question on the lined page designated for that response. Each response is expected to fit within its designated page. Answer all parts of every question. Use complete sentences; an outline or bulleted list alone is not acceptable. You may plan your answers in this exam booklet, but no credit will be given for notes written in this booklet. 1. Answer (a), (b), and (c). a) Briefly explain how ONE specific vine the millennial society during the year of 2017. b) Briefly explain how the vine, "This is how I kiss doggies", emulates popular culture in the United States. Please analyze its positive and negative effects. c) Briefly explain ONE specific way in which the Tide Pod meme brings on criticism finally an exam i can pass https://t.co/ynEK0vNdWO
finally an exam i can pass https://t.co/ynEK0vNdWO
Being Alone, Ass, and Beef: 2017 AP Vine Exam Multiple-Choice Questions 26) "Officer I've got one question for you. a. Day Care lookin ass muthaf***tr b. Flex on him cuh c. Honeyyy No!! d. What are those! 27) "I Smell like a, cheese b. beef c. a foot fetish d. the inside of a Mormon cult house 28) "It is , my dudes." a. My cheat day b. Wednesday c. Saturday d. Hump Day 29) " , Richard!" a. What the F**k b. What are you doing c. Your moms a hoe d. You were a failed abortion 30) "Honestly I don't remember I was crazy back then." a. I was probably Fx***d up b. I was not of legal age c. It was so hazy, I could taste colors. d. I was too busy respecting women 31) l'm washing b***h a. my sins b. me and my clothes c. my whole self d. the minority off me 2017 APTM VINE FREE-RESPONSE QUESTIONS RIP Vine SECTION I, Part B Time-50 minutes 4 Questions rections: Write your responses in the Section I, Part B: Short-Answer Response booklet. You must write your response to each question on the lined page designated for that response. Each response is expected to fit within its designated page. Answer all parts of every question. Use complete sentences; an outline or bulleted list alone is not acceptable. You may plan your answers in this exam booklet, but no credit will be given for notes written in this booklet. 1. Answer (a), (b), and (c). a) Briefly explain how ONE specific vine the millennial society during the year of 2017. b) Briefly explain how the vine, "This is how I kiss doggies", emulates popular culture in the United States. Please analyze its positive and negative effects. c) Briefly explain ONE specific way in which the Tide Pod meme brings on criticism finally an exam i can pass https://t.co/xfvwmVYCyB
finally an exam i can pass https://t.co/xfvwmVYCyB
Crazy, Disappointed, and Sorry: I'M SORRY-- YOU BUILT THIS IN YOUR GARAGE? I MEAN--YEAH I--THE GUYS AT THE LAB ALL SAID I WAS CRAZY, THEY-- YOU BUILT A JET PACK IN YOUR GARAGE AND YOUR FIRST THOUGHT ...WELL, WHEN YOU PUT IT THIS TO COMMIT A CRIMETHAT WAY.
mrperezsoso:

thefingerfuckingfemalefury:

theazureesper:

Look at his face, man.

He is so disappointed in you.

"WHY DID YOU NOT PATENT THIS"

mrperezsoso: thefingerfuckingfemalefury: theazureesper: Look at his face, man. He is so disappointed in you. "WHY DID YOU NOT PATENT THIS...
Crazy, Crying, and Memes: Shayb @Shaymarieee I KNEW I was CRAZY when l use to cry & look at myself in the mirror & smile while I was crying Imaooo00000o 11/20/17, 2:10 PM 😂😂
Crazy, Crush, and Driving: Four years ago, I wasjusta guy who had And I had to do the hardest thing that Ive a crush on a girl, who had a boyfriend. ever had to do, whichwas just to...wait. Don't getme wrong, Iflirted with her. Pam, I can now admit in front of friends and family, that I do know how to make And, uh, do you remember how long it a photocopy took you to teach me how to drive stick? For a really long time that's all I had. I've been driving sticke since high school, so... Litdle moments With*l who saw me as a friend And, a lot of people told me I was crazy to wait this long for a dátewith a girl I work with but I think, even then I knew that... I was waiting for my wife. "I was waiting for my wife." https://t.co/0iS2k55JOJ
"I was waiting for my wife." https://t.co/0iS2k55JOJ
Apple, Chance the Rapper, and Chicago: #ThatsBaller: Chance the Rapper, Kendrick Lamar, and DJ Khaled Among Top 10 on Forbes Highest Paid Hip-Hop Artists List @balleralert ˊ嵛 ThatsBaller: Chance the Rapper, Kendrick Lamar, and DJ Khaled Among Top 10 on Forbes Highest Paid Hip-Hop Artists List-blogged by @thereal__bee ⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀ Every year Forbes acknowledges the year's highest earners in the hip-hop industry and this year we have a few newcomers to the list. ⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀ Chicago native, ChancetheRapper, ranked No. 5 on the list at $33 million. Though the 24-year-old has never sold a physical album and instead releases free music via streaming services, Chance has managed to get his coins from major festival gigs as well as partnerships with brands like KitKat. ⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀ Rapper Kendrick Lamar was right behind Chance at $30 million. According to Forbes, Lamar grosses more than $1 million per tour stop. DJKhaled was also in the top 10 of the list, ranking in at No. 9 with $24 million. From his deals with a variety of brands such as Apple, Mentos, and Champ Sports, along with his DJ gigs, Khaled made a nice little penny this year. ⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀ "People probably thought I was crazy to just spend my last penny to invest in myself," Khaled tells Forbes. "Now it's paying off … one thing I learned is some investments might take years to kick in, and they are kicking in."
ThatsBaller: Chance the Rapper, Kendrick Lamar, and DJ Khaled Among Top 10 on Forbes Highest Paid Hip-Hop Artists List-blogged by @thereal__...
Crazy, Target, and Tumblr: Kick Prussia's as 2. chapteruntitledddd: I tried many times and failed to draw a saitisfying Toris, so I was crazy and began drawing random chibis. It's so hard to keep a stable style…T.T
chapteruntitledddd: I tried many times and failed to draw a saitisfying Toris, so I was crazy and began drawing random chibis. It's so hard...
Crazy, You, and For: You should have opened your eyes. I was crazy for you
Anaconda, Crazy, and Crush: "l was waiting for my wife." Four years ago, I wasjusta guy who hadAnd I had to do the hardest thing that 've a crush on o girl, whg hada boyfriend. ever had to do, whichwajust to. wait. a crush on a girl, Don't getime wrong, I tirted with her Pam, I cannow admit in front of friends and family, that'l do know how to makeAnd, uh, do you remember how long it a photocopy took you to teach me how to drive stick? Fora really long time that's all I had. I've been driving stick since highL i who saw me school, so.! Little moments with sgi as a frien And, alot of people toldme I was crazy to wait this long for a dáte with a girlI work with but I think, even then I knew that... Iwas warting for my wite. all orders from the store today and tomorrow are going to be entered into a $100 gift card giveaway! you have until tomorrow night 11:59pm to enter :)
all orders from the store today and tomorrow are going to be entered into a $100 gift card giveaway! you have until tomorrow night 11:59pm t...
Crazy, Instagram, and Memes: This is gonna sound crazy, but et someone up on this roof four years ago and they mixed that cocktail, and they loved penguins. By any chance, was that you? how imetyourmotherthe age instagram It's you. Everyone thought I was crazy put You are such a loser. {1x6} Thank you all for 181k😘💙 -- Scene requested by @dhruv14rana himym howimetyourmother sitcom tedmosby joshradnor barneystinson neilpatrickharris
{1x6} Thank you all for 181k😘💙 -- Scene requested by @dhruv14rana himym howimetyourmother sitcom tedmosby joshradnor barneystinson neilpatri...
Crazy, Crush, and Saw: Four years ago, I was just a guy who had a crush on a girl who had a boyfriend. And I had to do the hardest thing that I've ever had to do which was just to wait For a really long time that's a I had. I just had little moments with a girl who saw me as friend. And a lot of people told me I was crazy to wait this long for a date with a girl who I worked with, but I think even then I knew that I was waiting for my wife. hands down my favorite episode and lines of The Office https://t.co/WYYMuaWO1D
hands down my favorite episode and lines of The Office https://t.co/WYYMuaWO1D
Crazy, Crush, and Saw: Four years ago, I was just a guy who had a crush on a girl who had a boyfriend. And I had to do the hardest thing that I've ever had to do which was just to wait... For a really long time that's all I had.Ojust had little moments with a girl who saw me as friend. And a lot of people told me I was crazy to wait this long for a date with a girl who I worked with, but I think even then I knew that I was waiting for my wife. https://t.co/f4Sxjuhubb
https://t.co/f4Sxjuhubb
Crazy, Facebook, and Instagram: @GARYVEE APRIL 2012 FACEBOOK BUYS INSTAGRAM FOR $1 BILLI That time people said I was crazy that FB bought Instagram for a Billy .. what's your big prediction or current observation that most don't agree with ... entrepreneur
That time people said I was crazy that FB bought Instagram for a Billy .. what's your big prediction or current observation that most don't ...
Crazy, Crush, and Jim Halpert: Four years ago, I was just a guy who had a crush on a girl, who had a boyfriend. And I had to do the hardest thing that I've ever had to do, which was just to wait. And, a lot of people told me I was crazy to wait this long for a date with a girl I work with but I think, even then I knew that I was waiting for my wife.-Jim Halpert 😍 https://t.co/Zt997HCBdW
😍 https://t.co/Zt997HCBdW
Crazy, Crush, and Saw: Four years ago, I was just a guy who had a crush on a girl who had a boyfriend. And I had to do the hardest thing that l've ever had to do which was just to wait... For a really long time that's all I had. I just had little moments with a girl who saw me as friend. And a lot of people told me I was crazy to wait this long for a date with a girl who I worked with, but I think even then I knew I was waiting for mny wi that I was waiting for my wife https://t.co/pyK5IURBf6
https://t.co/pyK5IURBf6
Girl Memes, Hook, and Whore: they call us females crazy but have you ever turned down a dude then suddenly you're a whore and he's threatening your life lol k My ex once told everyone I was crazy because I yelled at him after I found out he hooked up with another girl 🙃
My ex once told everyone I was crazy because I yelled at him after I found out he hooked up with another girl 🙃
Jim Halpert, Memes, and 🤖: Four years ago, was just a guy who had a crush on a girl, who had a boyfriend. And I had to do the hardest thing that I've ever had to do, which was just to wait. And, a lot of people told me I was crazy to wait this long for a date with a girl l work with but l think, even then knew that I was waiting for my wife.-Jim Halpert https://t.co/Zt997HCBdW
https://t.co/Zt997HCBdW
Crazy, Crush, and Jim Halpert: Four years ago, I was just a guy who had a crush on a girl, who had a boyfriend. And I had to do the hardest thing that I've ever had to do, which was just to wait. And, a lot of people told me I was crazy to wait this long for a date with a girl I work with but I think, even then I knew that I was waiting for my wife.-Jim Halpert https://t.co/Zt997HCBdW
Girl Memes, On Fire, and Fuckin: You tell him ur crazy and he thinks u tryna be cute. months later he's like "u fuckin crazy" l told u bitch. l really did I told @barrysbanterbus that I was crazy and he just laughed. So I set his house on fire and he didn't laugh 💅🏼🔥 @barrysbanterbus
I told @barrysbanterbus that I was crazy and he just laughed. So I set his house on fire and he didn't laugh 💅🏼🔥 @barrysbanterbus
Crazy, Crush, and Saw: Four years ago, I was just a guy who had a crush on a girl who had a boyfriend. And I had to do the hardest thing that l've ever had to do which was just to wait... For a really long time that's all I had. I just had little moments with a girl who saw me as friend. And a lot of people told me I was crazy to wait this long for a date with a girl who I worked with, but I think even then I knew I was waiting for mny wi that I was waiting for my wife https://t.co/uKnYnz8KCa
https://t.co/uKnYnz8KCa
Crazy, Crush, and Saw: Four years ago, I was just a guy who had a crush on a girl who had a boyfriend. And I had to do the hardest thing that l've ever had to do which was just to wait. For a really Iong time that's all I had. I just had little moments with a gir who saw me as friend. And a lot of people told me I was crazy to wait this long for a date with a girl who I worked with, but I think even then I knew that I was waiting for my wife. https://t.co/wU7qthjSWa
https://t.co/wU7qthjSWa
Crazy, Harry Potter, and Love: Funnily enough, in the first Harry Potter fim. Because that's just what I wasilke. If you watch carefullyin Sonne scenes, you con see me mouthlng Harry and Ron's lines, as well as my own. Was I was crazy I love her 😍 -Kreacher
I love her 😍 -Kreacher
Jim Halpert, Memes, and 🤖: Four years ago, was just a guy who had a crush on a girl, who had a boyfriend. And I had to do the hardest thing that I've ever had to do, which was just to wait. And, a lot of people told me I was crazy to wait this long for a date with a girl l work with but l think, even then knew that I was waiting for my wife.-Jim Halpert 😍
Memes, 🤖, and A Girl: Four years ago, was just a guy who had a crush on a girl who had a boyfriend. And I had to do the hardest thing that l've ever had to do which was just to wait For a really long time that's a I had. just had little moments with a girl who saw me as friend. And a lot of people told me I was crazy to wait this long for a date with a girl who l worked with, but l think even then l knew that I was waiting for my wife. https://t.co/pyK5IUzZQw
https://t.co/pyK5IUzZQw
Crazy, Crush, and Saw: Four years ago, I was just a guy who had a crush on a girl who had a boyfriend. And I had to do the hardest thing that l've ever had to do which was just to wait... For a really long time that's all I had. I just had little moments with a girl who saw me as friend. And a lot of people told me I was crazy to wait this long for a date with a girl who I worked with, but I think even then I knew I was waiting for mny wi that I was waiting for my wife https://t.co/pyK5IUzZQw
Jim Halpert, Memes, and 🤖: "Cottaweckaierwe Gotlitia weeklafter we starte didating sar When you're a kid, you assume your parents are soul mates. My kids are gonna be right about that Four years ago, was just a guy who had a crush on a girl, who had a boyfriend. And I had to do the hardest thing that I've ever had to do, which was just to wait. And, a lot of people told me I was crazy to wait this long for a date with a girl l work with but l think, even then knew that I was waiting for my wife. Jim Halpert They make me so emotional
They make me so emotional
Crazy, Crush, and Dating: "Gotita week after we Started dating When you're a kid you assume you parents are soul mates My kids are gonna be right about that. Four years ago, I was just a guy who had a crush on a girl, who had a boyfriend. And I had to do the hardest thing that I've ever had to do, which was just to wait. And, a lot of people told me I was crazy to wait this long for a date with a girl I work with but I think, even then I knew that I was waiting for my wife.-Jim Halpert They make me so emotional https://t.co/7jwWtmKukh
They make me so emotional https://t.co/7jwWtmKukh
Memes, 🤖, and for My Wife: Four years ago, I was just a guy who had a crush on a girl who had a boyfriend. And I had to do the hardest thing that I've ever had to do which was just to wait... For a really long time that's all I had.Ojust had little moments with a girl who saw me as friend. And a lot of people told me I was crazy to wait this long for a date with a girl who I worked with, but I think even then I knew that I was waiting for my wife. https://t.co/6Phgp9PVK3
https://t.co/6Phgp9PVK3
Crazy, Crush, and Saw: Four years ago, I was just a guy who had a crush on a girl who had a boyfriend. And I had to do the hardest thing that l've ever had to do which was just to wait. For a really Iong time that's all I had. I just had little moments with a gir who saw me as friend. And a lot of people told me I was crazy to wait this long for a date with a girl who I worked with, but I think even then I knew that I was waiting for my wife. https://t.co/6Phgp9PVK3
Jim Halpert, Memes, and 🤖: LOVE IS WORTH THEWAIT VIA 9GAGACOM "Four years ago, Iwas just a guy who had a crush on a girl, who had a boyfriend. And I had to do the hardest thing that I've ever had to do, which was just to... wait...And, a lot of people told me I was crazy to wait this long for a date with a girl l work with but l think, even then I knew that... I was waiting for my wife Jim Halpert Don't give up.
Don't give up.