text:
\section{Introduction}
\label{sec:introduction}

Current state-of-the-art approaches for video action recognition are based on convolutional neural networks (CNNs).  These include the best performing 3D models, such as I3D~\cite{I3D:carreira2017quo} and ResNet3D~\cite{ResNet3D:hara2017learning}, and some effective 2D models, such as Temporal Relation Networks (TRN)~\cite{TRN:zhou2018temporal} and Temporal Shift Modules (TSM)~\cite{TSM:lin2018temporal}. A CNN-based model usually considers a sequence of frames as input, obtained through either uniform or dense sampling from a video~\cite{I3D:carreira2017quo, TSN:wang2016temporal}. In general, Longer input sequences yield better recognition results. However, one problem arising for a model requesting more input frames is that the GPU resources required for training and inference also significantly increase in both memory and time.
For example, the top-performing I3D models~\cite{I3D:carreira2017quo} on the Kinetics~\cite{Kinetics:kay2017kinetics} dataset were trained with 64 frames on a cluster of 32 GPUs, and the non-local network \cite{Wang2018NonLocal} even uses 128 frames as input.
Another problem for action recognition is the lack of effective methods for temporal modeling when moving away from 3D spatiotemporal convolutions.
While 2D convolutional models are more resource-friendly than their 3D counterparts, they lack expressiveness over time and thus cannot take much benefit from richer input data.

In this paper, we present an efficient and memory-friendly spatio-temporal representation for action recognition, which enables training of deeper models while allowing for more input frames. The first part of our approach is inspired by the Big-Little-Net architecture (\textit{bLNet}\xspace~\cite{chen2018biglittle}).
We propose a new video architecture that has two network branches with different complexities: one branch processing low-resolution frames in a very deep subnet, and another branch processing high-resolution frames in a compact subnet. The two branches complement each other through merging at the end of each network layer.
With such a design, our approach can process twice as many frames as the baseline model without compromising efficiency. We refer to this architecture as ``Big-Little-Video-Net'' (\textit{bLVNet}\xspace).

In light of the limited ability of capturing temporal dependencies in \textit{bLVNet}\xspace, we further develop an effective method to exploit temporal relations across frames by a so called ``Depthwise Temporal Aggregation Module'' (\textit{TAM}). The method enables the exchange of temporal information between frames by weighted channel-wise aggregation. This aggregation is made learnable with 1$\times$1 depthwise convolution, and implemented as an independent network module. The temporal aggregation module can be easily integrated into the proposed network architecture to progressively learn spatio-temporal patterns in a hierarchical way. Moreover, the module is extremely compact and adds only negligible computational costs and parameters to \textit{bLVNet}\xspace.

Our main contributions lie in the following two interconnected aspects:
(1) We propose a lightweight video architecture based on dual-path network to learn video features, and (2) we develop a temporal aggregation module to enable effective temporal modeling without the need for computationally expensive 3D convolutions.

We evaluate our approach on the Kinetics-400~\cite{Kinetics:kay2017kinetics}, Something-Something~\cite{Something:goyal2017something} and Moments-in-time~\cite{Moments:monfort2019moments} datasets.
The evaluation shows that \textit{bLVNet-TAM}\xspace successfully allows us to train action-classification models with deeper backbones (i.e., ResNet-101) as well as more (up to 64) input frames, using a single compute node with 8 Tesla V100 GPUs. Our comprehensive experiments demonstrate that our approach achieves highly competitive results on all datasets while maintaining efficiency.
Especially, it establishes a new state-of-the-art result on Something-Something and Moments-in-time by outperforming previous approaches in the literature by a large margin.

\section{Experiments}
\label{sec:experiment}

\subsection{Experimental Setup}
\textbf{Datasets}.
We evaluate our approach on three large-scale datasets for video recognition, including the widely used Something-Something (Version 1 and Version 2)~\cite{Something:goyal2017something}, Kinetics-400~\cite{Kinetics:kay2017kinetics} and the recent Moments-in-time dataset~\cite{Moments:monfort2019moments}. They are herein referred to as \textit{SS-V1}\xspace, \textit{SS-V2}\xspace, \textit{Kinetics-400}\xspace and \textit{Moments}\xspace, respectively.

Something-Something is a dataset containing videos of 174 types of predefined human-object interactions with everyday objects. The version 1 and 2 include 108k and 220k videos, respectively. This dataset focuses on human-object interactions in a rather simple setup with no scene contexts to be exploited for recognition. Instead temporal relationships are as important as appearance for reasoning about the interactions. Because of this, the dataset serves as a good benchmark for evaluating the efficacy of temporal modeling, such as proposed in our approach.
Kinetics-400~\cite{Kinetics:kay2017kinetics} has emerged as a standard benchmark for action recognition after UCF101~\cite{ucf101:Soomro2012} and HMDB~\cite{HMDB:Kuehne2011}, but on a significantly larger scale. The dataset consists of 240k training videos and 20k validation videos, with each video trimmed to around 10 seconds. It has a total of 400 human action categories.

Moments-in-time~\cite{Moments:monfort2019moments} is a recent collection of one million labeled videos, involving actions from people, animals, objects or natural phenomena. It has 339 classes and each video clip is trimmed to 3 seconds long.

\textbf{Data Augmentation}.
During training, we follow the data augmentation used in TSN~\cite{TSN:wang2016temporal} to augment the video with different sizes spatially and flip the video horizontally with 50\% probability. Furthermore, since our models are finetuned on pretrained ImageNet, we normalize the data with the mean and standard deviation of the ImageNet images. The model input is formed by \textit{uniform sampling}, which first divides a video into $n$ uniform segments and then selects one random frame from each segment as the input.

During inference, we resize the smaller side of an image to 256 and then crop a centered 224$\times$224 region.  The center frame of each segment in uniform sampling is picked as the input. On Something-Something and Moments, our results are based on the single-crop and single-clip setting. On \textit{Kinetics-400}\xspace, we use the common practice of multi-crop and multi-clip for evaluation.

\textbf{Training Details}.
Since all the three datasets are large-scale, we train the models in a progressive way.
For each type of backbone (for example, \textit{bLResNet}\xspace-50), we first finetune a base model on ImageNet with a minimum input length (i.e. 8$\times$2 in our case) using 50 epochs. We adopt the Nesterov momentum optimizer with an initial weight of 0.01, a weight decay of 0.0005 and a momentum of 0.9. We then finetune a new model with longer input (for example, 16$\times$2) on top of the corresponding base model, but with 25 epochs only. In this case, the initial learning rate is set to 0.01 on Something-Something and 0.005 on Kinetics and Moments. The learning rate is decreased by a factor of $10$ at the 10-th and 20-th epoch, respectively.

This strategy allows to significantly reduce the training time needed for all the models evaluated in our experiments. All our models were trained on a server with 8 GPU cards and a total of 128G GPU memory. We set the total batch size to 64 whenever possible. For models that require more memory to train, we adjust the batch size accordingly to the maximum number allowed.

\begin{table}[t]
    \caption{Recognition Accuracy of Various Models on Something-Something-V1 (SS-V1).}

    \label{table:ssv1}
    \centering
    \resizebox{\linewidth}{!}{
    \begin{tabular}{c|c|cccc ccc c}
    \toprule
        \multirow{2}{*}{Model}  &  \multirow{2}{*}{Backbone} & \multirow{2}{*}{Pretrain} &  \multirow{2}{*}{Frames}  & \multirow{2}{*}{Modality}  & \multirow{2}{*}{Param (10$^6$)} & \multirow{2}{*}{FLOPs (10$^9$)}  & \multicolumn{2}{c}{Val} & Test \\
         &  &  &  &  &  &  & Top-1 (\%) & Top-5 (\%) & Top-1 (\%) \\
    \midrule
        I3D~\cite{I3D:carreira2017quo} & Inception   & ImageNet & 64 & RGB & 12.7 & 111 & 45.8 & 76.5 & 27.2\\
        NL I3D + GCN~\cite{GCN:wang2018gcn} & ResNet-50   & ImageNet & 32+32 & RGB & 303 & 62.2 & 46.1 & 76.8 & $-$  \\
        S3D~\cite{S3D:xie2018rethinking} & Inception   & ImageNet & 64 & RGB & 8.77 & 66 & 47.3 & 78.1 & $-$  \\
        ECO-Lite$_{En}$~\cite{ECO:zolfaghari2018eco}     & BNInception+ResNet18 & ImageNet  & 92    & RGB & 150 & 267 & 46.4 & $-$ & 42.3 \\
    \midrule
        TSN~\cite{TSN:wang2016temporal} & BNInception   & ImageNet & 8  & RGB & 10.7 & 16 & 19.5 & $-$ & $-$ \\
        \multirow{2}{*}{TRN~\cite{TRN:zhou2018temporal}} & BNInception   & ImageNet & 8  & RGB & 18.3 & 16 & 34.4 & $-$ & 33.6 \\
         & BNInception   & ImageNet & 8+8  & RGB+Flow & $-$ & $-$ & 42.0 & $-$ & 40.7 \\
         \midrule
         \multirow{3}{*}{TSM~\cite{TSM:lin2018temporal}} & ResNet-50     & Kinetics & 8  & RGB & 24.3 & 33 & 45.6 & 74.2 & \\
         & ResNet-50     & Kinetics & 16 & RGB & 24.3 & 65 &  47.2 & 77.1 & 46.0\\
         & ResNet-50     & Kinetics & 16+16 & RGB+Flow & $-$ & $-$ & 52.6 & 81.9 &  50.7 \\

    \midrule
        \multirow{6}{*}{\textit{bLVNet-TAM}\xspace} & \textit{bLResNet}\xspace-50  & ImageNet & 8$\times$2  & RGB & 25.0 & 23.8 & 46.4 & 76.6 & $-$ \\
         & \textit{bLResNet}\xspace-50  & SS-V1    & 16$\times$2 & RGB & 25.0 & 47.7 & 48.4 & 78.8 & $-$ \\
         & \textit{bLResNet}\xspace-101 & ImageNet & 8$\times$2  & RGB & 40.2 & 32.1  & 47.8 & 78.0 & $-$ \\
         & \textit{bLResNet}\xspace-101 & SS-V1    & 16$\times$2 & RGB & 40.2 & 64.3  & 49.6 & 79.8 & $-$ \\
         & \textit{bLResNet}\xspace-101 & SS-V1    & 24$\times$2 & RGB & 40.2 & 96.4  & 52.2 & 81.8 & $-$ \\
         & \textit{bLResNet}\xspace-101 & SS-V1    & 32$\times$2 & RGB & 40.2 & 128.6 & \textbf{53.1} & \textbf{82.9} & \textbf{48.9}\\
    \bottomrule
    \end{tabular}

\end{table}

\begin{table}[t]
    \caption{Recognition Accuracy of Various Models on Something-Something-V2 (SS-V2).}

    \label{table:ssv2}
    \centering
    \resizebox{\linewidth}{!}{
        \begin{tabular}{c|c|ccc cc cc cc}
        \toprule
            \multirow{2}{*}{Model}  &  \multirow{2}{*}{Backbone} & \multirow{2}{*}{Pretrain} &  \multirow{2}{*}{Frames}  & \multirow{2}{*}{Modality}  & \multirow{2}{*}{Param (10$^6$)} & \multirow{2}{*}{FLOPs (10$^9$)}  & \multicolumn{2}{c}{Val} & \multicolumn{2}{c}{Test} \\
             &  &  &  &  &  & & Top-1 (\%) & Top-5 (\%) & Top-1 (\%) & Top-5 (\%) \\
        \midrule
            \multirow{2}{*}{TRN~\cite{TRN:zhou2018temporal}} & BNInception   & ImageNet & 8  & RGB & 18.3 & 16 & 48.8 & 77.6 & 50.9 & 79.3  \\
             & BNInception   & ImageNet & 8  & RGB+Flow & 36.6 & 32 & 55.5 & 83.1  & 56.2 & 83.2\\
             \midrule
                \multirow{3}{*}{TSM~\cite{TSM:lin2018temporal}} & ResNet-50$\dagger$     & Kinetics & 8  & RGB & 24.3 & 33 & 58.9 & 85.5 & $-$ & $-$\\
                & ResNet-50$\dagger$     & Kinetics & 16 & RGB & 24.3 & 65 & 61.4 & 87.0 & $-$ & $-$ \\
                & ResNet-50     & Kinetics & $-$ & RGB+Flow & $-$ & $-$ & 66.0 & 90.5 & 66.6 & 91.3 \\
        \midrule
        \multirow{6}{*}{\textit{bLVNet-TAM}\xspace} & \textit{bLResNet}\xspace-50  & ImageNet & 8$\times$2  & RGB & 25.0 & 23.8 & 59.1 & 86.0 & $-$ & $-$\\
                & \textit{bLResNet}\xspace-50  & SS-V2    & 16$\times$2 & RGB & 25.0 & 47.7  & 61.7 & 88.1 & $-$ & $-$\\
                & \textit{bLResNet}\xspace-101 & ImageNet & 8$\times$2  & RGB & 40.2 & 32.1  & 60.2 & 87.1 & $-$ & $-$\\
                & \textit{bLResNet}\xspace-101 & SS-V2    & 16$\times$2 & RGB & 40.2 & 64.3  & 61.9 & 88.4 & $-$ & $-$\\
                & \textit{bLResNet}\xspace-101 & SS-V2    & 24$\times$2 & RGB & 40.2 & 96.4  & 64.0 & 89.8 & $-$ & $-$\\
                & \textit{bLResNet}\xspace-101 & SS-V2    & 32$\times$2 & RGB & 40.2 & 128.6 & 65.2 & 90.3 & $-$ & $-$ \\
                & \textit{bLResNet}\xspace-101$^*$ & SS-V2    & 32$\times$2 & RGB+Flow & $-$ & $-$ & 68.5 & 91.4 & \textbf{67.1} & \textbf{91.4} \\

        \bottomrule
        \multicolumn{9}{l}{$^\dagger:$ using their pretrained models and code to evaluate under the 1-crop and 1-clip setting for fair comparison} \\
        \multicolumn{9}{l}{$^*:$ model ensemble of RGB and Flow model, each is evaluated with 3 crops and 10 clips and uses 256 as the shorter side.}
        \end{tabular}

\end{table}

\subsection{Main Results}
\textbf{Something-Something}. We first report our results on the validation set of the Something-Something datasets in Table~\ref{table:ssv1} and Table~\ref{table:ssv2}.
With a moderately deep backbone \textit{bLResNet-50}, our approach outperforms all 3D models on \textit{SS-V1}\xspace while using much fewer input frames (8$\times$2) and being substantially more efficient. TSM~\cite{TSM:lin2018temporal} was the previously best approach on Something-Something. Under the same backbone (i.e. ResNet-50), our approach is better than TSM on both~\textit{SS-V1}\xspace and~\textit{SS-V2}\xspace while being more efficient (i.e our 8x2 model has $1.4$ times fewer FLOPs than a 8-frame TSM model).

When empowered with a stronger backbone \textit{bLResNet-101}, our approach achieves even better results at 32$\times$2 frames (\textbf{53.1\%} top-1 accuracy on \textit{SS-V1}\xspace, and \textbf{65.2\%} on \textit{SS-V2}\xspace), establishing a new state-of-the-art on Something-Something. Notably, these results while based on RGB information only, are superior to those obtained from the best two-stream models at no more computational cost. This strongly demonstrates the effectiveness of our approach for temporal modeling.
We further evaluated our models on the test set of Something-Something. Our results are consistently better than the best results reported by the other approaches in comparison including 2-stream models.

\begin{table}
    \caption{Recognition Accuracy of Various Models on Kinetics-400 (RGB-only).}

    \label{table:k400}
    \centering
    \resizebox{.8\linewidth}{!}{
    \begin{tabular}{c|c|c ccc}
    \toprule
        Net & Backbone & Pretrain & FLOPs (10$^9$) & Top-1 (\%) & Top-5 (\%) \\
    \midrule
        STC~\cite{STC:Diba2018spatio}    &  ResNeXt-101 & None    &  $-$ &   68.7    & 88.5 \\
        ARTNet~\cite{ARTNet:Wang2018appearance} &   ResNet-18 &  None &    23.5$\times$250 &   69.2   &   88.3    \\
        C3D~\cite{ARTNet:Wang2018appearance}    & ResNet-18 &   None    &   19.6$\times$250 &   65.6 & 85.7 \\
        I3D~\cite{I3D:carreira2017quo} & Inception & ImageNet &  108$\times$N/A & 71.1 & 89.3\\
        S3D~\cite{S3D:xie2018rethinking} & Inception & ImageNet & $-$ & 72.2 & 90.6\\
        R(2+1)D~\cite{R(2+1)D:tran2018closer}& ResNet-34 & None & $-$ & 72.0 & 90.0 \\
        SlowFast-4$\times$16~\cite{SlowFast:feichtenhofer2018slowfast} & ResNet-50 & None & 36.1$\times$30 & \textbf{75.6} & \textbf{92.1} \\
        \midrule
        TSN~\cite{TSN:wang2016temporal} & InceptionV3   & ImageNet &  142.8$\times$10 & 72.5 & $-$ \\
        ECO-Lite$_{En}$~\cite{ECO:zolfaghari2018eco}     & BNInception+ResNet18 & ImageNet &  267 & 70.7 & - \\
        TSM-8~\cite{TSM:lin2018temporal} & ResNet-50     & ImageNet  & 42.7$\times$30 & 74.1 & 91.2 \\
        TSM-16~\cite{TSM:lin2018temporal} & ResNet-50     & ImageNet  & 85.4$\times$30 & 74.7 & $-$ \\
    \midrule
        \textit{bLVNet-TAM}\xspace-8$\times$2 & \textit{bLResNet}\xspace-50  & ImageNet  &  31.1$\times$9  & 71.0 & 89.8 \\
        \textit{bLVNet-TAM}\xspace-16$\times$2 & \textit{bLResNet}\xspace-50  & Kinetics  & 62.3$\times$9  & 72.0 & 90.6 \\
        \textit{bLVNet-TAM}\xspace-24$\times$2  & \textit{bLResNet}\xspace-50  & Kinetics & 93.4$\times$9  & 73.5 & 91.2 \\
    \bottomrule
    \end{tabular}

\end{table}

\begin{table}
    \caption{Recognition Accuracy of Various Models on Moments-in-time.}

    \label{table:moments}
    \centering
    \resizebox{0.7\linewidth}{!}{
    \begin{tabular}{c|c|ccc cc}
    \toprule
        Net & Backbone & Pretrain & Frames & Modality & Top-1 (\%) & Top-5 (\%) \\
    \midrule
        SoundNet~\cite{Moments:monfort2019moments} & $-$ & $-$  & $-$  & Audio  & 7.60 & 18.0 \\
        TSN~\cite{Moments:monfort2019moments} & BNInception & ImageNet  & 16  & RGB  & 24.1 & 49.1 \\
        TSN~\cite{Moments:monfort2019moments} & BNInception & $-$ &16+16 & RGB+Flow  & 25.3 & 50.1\\
        TRN~\cite{Moments:monfort2019moments} & Inception & ImageNet & 16 & RGB  & 28.3 & 53.9\\
        I3D~\cite{Moments:monfort2019moments} & ResNet-50 & $-$ & 16 & RGB  & 29.5 & 56.1\\
        Ensemble~\cite{Moments:monfort2019moments} & $-$ & $-$ & $-$ & $-$  &31.2 & 57.7\\
    \midrule
        \multirow{2}{*}{\textit{bLVNet-TAM}\xspace} & \textit{bLResNet}\xspace-50  & ImageNet & 8$\times$2  & RGB  & 31.2 & 58.3\\
         & \textit{bLResNet}\xspace-50  & Moments & 16$\times$2  & RGB  & \textbf{31.4} & \textbf{59.3} \\
    \bottomrule
    \end{tabular}

\end{table}

\textbf{Kinetics-400}. Kinetics-400 is one of the most popular benchmarks for action recognition. Currently the best-performed models on this dataset are all based on 3D Convolutions. However, it has been shown in the literature that temporal ordering in this dataset does not seem to be as crucial as RGB information for recognition. For example, as experimented in S3D~\cite{S3D:xie2018rethinking}, the model trained on normal time-order data performs well on the time-reversed data on Kinetics.
In accordance to this, our approach (3 crops and 3 clips) mainly performs on par with or better than the current large-scale architectures, but without outperforming them as clearly as on the Something-Something datasets, where the temporal relations are more essential for an overall understanding of the video content.

\textbf{Moments}. We finally evaluate the proposed architecture on the \textit{Moments}\xspace dataset~\cite{Moments:monfort2019moments}, a large-scale action dataset with about three times more training samples than \textit{Kinetics-400}\xspace.
Since \textit{Moments}\xspace is relatively new and results reported on it are limited, we only compare our results with those reported in the Moments paper~\cite{Moments:monfort2019moments}.
As can been seen from Table~\ref{table:moments}, our approach outperforms all the single-stream models as well as the ensemble one. We hope our models provide stronger baseline results for future reference on this challenging dataset.

It is also noted that our model trained with $16\times2$ frames only produces slightly better top-1 accuracy than the model trained with $8\times2$ frames. We speculate that this has to do with the fact that the \textit{Moments}\xspace clips are only as short as 3 seconds and that there is only a limited impact in choosing a finer temporal granularity on this dataset.

\subsection{Ablation Studies}
\label{sec:experiment:ablation}
In this section, we conduct ablation studies to provide more insights about our main ideas.

\textbf{Is temporal aggregation effective?}.
We validate the efficacy of the proposed temporal aggregation module (\textit{TAM}), which is considered as a global fusion method (Section~\ref{sec:approach:deta}). \textit{Local fusion} here is referred to the branch merging in the dual path network (Section~\ref{sec:approach:dpn}). We compare \textit{TAM} with the temporal shift module used in TSM~\cite{TSM:lin2018temporal} in Table~\ref{table:temporal-modeling} under two different video architectures: TSN and \textit{bLVNet}\xspace proposed in this work. \textit{TAM} demonstrates clear advantages over TSM, outperforming TSM by over 2\% under both architectures.
Interestingly, with the here proposed \textit{bLVNet}\xspace baseline with local temporal fusion almost doubles the performance of a TSN baseline, improving the accuracy from 17.4\% to 33.6\%.
On top of that, \textit{TAM} boosts the performance by another 13\% in both cases, suggesting that \textit{TAM} is complementary to local fusion. This further confirms the significance of temporal reasoning on the Something-Something dataset.

\begin{wraptable}{R}{0.5\linewidth}
    \centering
    \caption{Temporal Modeling on SS-V1.}
    \label{table:temporal-modeling}
    \resizebox{\linewidth}{!}{
        \begin{tabular}{c|c|c|c|c}
            \toprule
            Net & Backbone & Local Fusion & Global Fusion & Top-1 (\%)   \\
            \midrule
                & ResNet-50  & None & None & 17.4 \\
            TSN & ResNet-50  & None & TSM  & 43.4 \\
                & ResNet-50  & None & TAM  & 46.1 \\
            \midrule
                       & \textit{bLResNet}\xspace-50  & $\checkmark$ & None  & 33.6 \\
            \textit{bLVNet}\xspace & \textit{bLResNet}\xspace-50  & $\checkmark$ & TSM  & 44.2 \\
                       & \textit{bLResNet}\xspace-50  & $\checkmark$ & TAM  & \textbf{46.4} \\
            \bottomrule
        \end{tabular}
    }
\vspace{-10pt}
\end{wraptable}

\begin{figure}[t!]
    \vspace{-10pt}
    \centering
    \includegraphics[width=.75\linewidth]{images/frame-complexity}
    \vspace{-10pt}
    \caption{Number of input frames v.s. model accuracy and memory usage. (a) A longer input sequence yields better recognition in our proposed \textit{bLVNet-TAM}\xspace on the Something-Something dataset~\cite{Something:goyal2017something}, but not in TSN~\cite{TSN:wang2016temporal} due to limited temporal modeling ability. (b) Compared to TSN, \textit{bLVNet-TAM}\xspace reduces memory usage by $\sim$2 times under the same number of input frames.}
    \label{fig:frame-complexity}
    \vspace{-10pt}
\end{figure}

\textbf{Does seeing more frames help?}. One of the main contribution of this work is an efficient video architecture that makes it possible to train deeper models with more input frames using moderate GPU resources. Fig.~\ref{fig:frame-complexity}a) shows consistent improvement of our approach on \textit{SS-V1}\xspace as the number of input frames increases. A similar trend in our results can be observed on \textit{Kinetics-400}\xspace in Table~\ref{table:k400}. On the other hand, the almost flattened line from TSN suggests that a model without effective temporal modeling cannot take much of the benefit from longer input frames.

\textbf{Memory Usage}. We compare the memory usage between our approach based on \textit{bLResNet}\xspace-50 and TSN based on ResNet-50. As shown in Fig.~\ref{fig:frame-complexity}b), our approach is more memory friendly than TSN, achieving a saving of $\sim$2 times at the same number of input frames.
The larger batch size allowed for training under the same computational budget is critical for our approach to obtain better models and reduce training time.

\section{Related Work}

Activity classification has always been a challenging research topic, with first attempts reaching back by almost two decades \cite{Aggarwal2011Review}; deep-learning architectures nowadays achieve tremendous recognition rates on various challenging tasks, such as Kinetics~\cite{I3D:carreira2017quo}, ActivityNet \cite{caba2015activitynet}, or Thumos \cite{THUMOS14}.

Most successful architectures in the field are usually based on the so-called two-stream model~\cite{Simonyan14TwoStream}, processing a single RGB frame and optical-flow input in two separate CNNs with a late fusion in the upper layers. Over the last years, many approaches extend this idea by processing a stack of input frames in both streams, thus extending the temporal window of the architecture form 1 to up to 128 input frames per stream. To further capture the temporal correlation in the input over time, those architectures usually make use of 3D convolutions as, e.g., in  I3D~\cite{I3D:carreira2017quo}, S3D~\cite{S3D:xie2018rethinking}, and ResNet3D~\cite{ResNet3D:hara2017learning}, usually leading to a large-scale parameter space to train.

Another way to capture temporal relations has been proposed by~\cite{TSN:wang2016temporal},~\cite{TRN:zhou2018temporal}, and~\cite{TSM:lin2018temporal}. Those architectures mainly build on the idea of processing videos in the form of multiple segments, and then fusing them at the higher layers of the networks.
The first approach with this pattern was the so-called Temporal Segment Networks (TSN) proposed by Wang \textit{et~al.}\xspace~\cite{TSN:wang2016temporal}. The idea of TSN has been extended by Temporal Relation Networks (TRN)~\cite{TRN:zhou2018temporal}, which apply the idea of relational networks to the modeling of temporal relations between observations in videos.
Another approach for capturing temporal contexts has been proposed by Temporal Shift Modules (TSM)~\cite{TSM:lin2018temporal}.
This approach shifts part of the channels along the temporal dimension, thereby allowing for information to be exchanged among neighboring frames.
More complex approaches have been tried as well, e.g. in the context of non-local neural networks~\cite{Wang2018NonLocal}.
Our temporal aggregation module is based on depthwise 1$\times$1 convolutions to capture temporal dependencies across frames effectively.

Separate convolutions are considered in approaches such as~\cite{S3D:xie2018rethinking, R(2+1)D:tran2018closer} to reduce costly computation in 3D convolutional models. More recently, SlowFast Network~\cite{SlowFast:feichtenhofer2018slowfast} uses a dual-pathway network to process a video at both slow and fast frame rates. The fast pathway is made lightweight, similar to Little Net in our proposed architecture. However, our approach reduces computation based on both a lightweight architecture and low image resolution.
Furthermore, the recent work Timeception~\cite{Timeception} applies the concept of ``Inception" to temporal domain for capturing long-range temporal dependencies in a video. The Timeception layers involve group convolutions at different time scales while our TAM layers only use depthwise convolution. As a result, the Timeception has significantly more parameters than the TAM (10\% vs. 0.1\% of the total model parameters).

\section{Network Architecture}
\label{sec:sup:net_arch}
Here we details our network architecture for \textit{bLVNet-TAM}\xspace-50 in Table~\ref{table:dta-50-conf}. We follow the notation used in \textit{bLNet}\xspace~\cite{chen2018biglittle} ($\alpha=2$ and $\beta=4$) but adding the proposed TAM module before branching out to \textit{Big-Net} and \textit{Little-Net} and the last shared residual block.
As noted before, the two branches work on different frames and then merged every stage; on the other hand, in ResBlock$_{TAM}$, the TAM module goes through the non-shortcut path.

\begin{table}[tbh]
\centering
\caption{Network configurations of \textit{bLVNet-TAM}\xspace-50 with temporal fusion.}
\label{table:dta-50-conf}
\begin{tabular}{c|c|cc}
\toprule
Layers & Spatial output size & \multicolumn{2}{|c}{\textit{bLVNet-TAM}\xspace-50}  \\
\midrule
Convolution & $112\times112$ & \multicolumn{2}{c}{$7\times7, 64, s2$} \\
\midrule
TAM-module & $112\times112$ & \multicolumn{2}{c}{Temporal Aggregation Module ($r=3$)} \\
\midrule
bL-module & $56\times56$ &  $3\times3, 64$, s2 &
            $\left ( \begin{tabular}{@{\ }l@{}} 3$\times$3, 32 \\ 3$\times$3, 32, s2 \\ 1$\times$1, 64 \end{tabular} \right )$  \\
\midrule
TAM-module & $56\times56$ & \multicolumn{2}{c}{Temporal Aggregation Module ($r=3$)} \\
\midrule
bL-module & $56\times56$ &  ResBlock$_B$, $256$ $\times2$ & ResBlock$_L$, $128$  $\times1$ \\
  & $28\times28$ & \multicolumn{2}{c}{ResBlock, $256$, s2} \\
\midrule
TAM-module & $28\times28$ & \multicolumn{2}{c}{Temporal Aggregation Module ($r=3$)} \\
\midrule
bL-module      & $28\times28$ & ResBlock$_B$, $512$ $\times3$ & ResBlock$_L$, $256$  $\times1$ \\
 & $14\times14$ &  \multicolumn{2}{c}{ResBlock, $512$, s2} \\
\midrule
TAM-module & $14\times14$ &  \multicolumn{2}{c}{Temporal Aggregation Module ($r=3$)} \\
\midrule
bL-module & $14\times14$ &   ResBlock$_B$, $1024$ $\times5$ & ResBlock$_L$, $512$  $\times1$ \\
 & $14\times14$ & \multicolumn{2}{c}{ResBlock, $1024$} \\
\midrule
ResBlock$_{TAM}$  & $7\times7$ &   \multicolumn{2}{c}{ResBlock$_{TAM}$, $2048$ $\times3$, s2}\\
\midrule
Average pool & $1\times1$   &  \multicolumn{2}{c}{$7\times7$ average pooling} \\
\midrule
FC, softmax &   \multicolumn{3}{c}{\# of classes} \\
\bottomrule
\multicolumn{4}{l}{\footnotesize{ResBlock$_B$: the first $3\times3$ convolution is with stride 2, and then restoring the size via the bi-linear upsampling.}}\\
\multicolumn{4}{l}{\footnotesize{ResBlock$_L$: a $1\times1$ convolution is applied at the end to align the channel size.}}\\
\multicolumn{4}{l}{\footnotesize{ResBlock$_{TAM}$: a residual block embedded with temporal aggregation module with $r=3$.}}\\
\multicolumn{4}{l}{\footnotesize{s2: the stride is set to 2 for the $3\times3$ convolution in the ResBlock.}}\\
\end{tabular}
\end{table}

\section{Data Preprocessing}
\label{sec:sup:datapre}
Here we describe how we convert the video data into images for our training and inference.
For the Something-Something dataset, we resize the smaller side of an image to 256 while keeping aspect ratio. For the Kinetics dateset, we resize the smaller side of an image to 331 since its original resolution is higher. For the Moments dataset, we we resize an image to 256$\times$256.

\section{Our Approach}
\label{sec:approach}
We aim at developing efficient and effective video representations for video understanding.
To address the computational challenge imposed by the desired long input to a model, we propose a new video architecture based on the Big-Little network (\textit{bLNet}\xspace)~\cite{chen2018biglittle} for learning video features.
We first give a brief recap of \textit{bLNet}\xspace~in Section~\ref{sec:approach:blnet}.
We then show, in Section~\ref{sec:approach:dpn}, how to extend \textit{bLNet}\xspace to an efficient video architecture that allows for seeing more frames with less computation and memory. An example of the proposed network architecture can be found in the supplementary material (Section A).

To make temporal modeling more effective in our approach, we further develop a temporal aggregation module (TAM) to capture short-term as well as long-term temporal dependencies across frames. Our method is implemented as a separate network module and integrated with the proposed architecture seamlessly to learn a hierarchical temporal representation for action recognition. We detail this method in Section~\ref{sec:approach:deta}.

\subsection{Recap of Big-Little Network}
\label{sec:approach:blnet}
The Big-Little Net, abbreviated as \textit{bLNet}\xspace~in~\cite{chen2018biglittle}, is a CNN architecture for learning strong feature representations by combining multi-scale image information.
The \textit{bLNet}\xspace~processes an image at different resolutions using a dual-path network, but with low computational loads based on a clever design. The key idea is to have a high-complexity subnet (\textit{Big-Net}) along with a low-cost one (\textit{Little-Net}) operate on the low-scale and high-scale parts of an image in parallel.
By such a design, the two subnets learn features complementary to each other while using less computation. The two branches are merged at the end of each network layer to fuse the low-scale and high-scale information so as to form a stronger image representation.
The \textit{bLNet}\xspace~approach demonstrates improvement of model efficiency and performance on both object and speech recognition, using popular architectures such as ResNet, ResNeXt and SEResNeXt. More details on \textit{bLNet}\xspace can be found in the original paper. In this work, we mainly adopt \textit{bLResNet}\xspace-50 and \textit{bLResNet}\xspace-101 as backbone for our proposed architecture.

\begin{figure}[b!]
\centering
\includegraphics[width=.8\linewidth]{images/video_arch}
\caption{\label{fig:video_arch} Different architectures for action recognition. a) \textbf{TSN}~\cite{TSN:wang2016temporal} uses a shared CNN to process each frame independently, so there is no temporal interaction between frames. b) \textbf{TSN-bLNet}~is a variant of TSN that uses \textit{bLNet}\xspace~\cite{chen2018biglittle} as backbone. It is efficient, but still lacks temporal modeling. c) \textbf{bLVNet} feeds odd and even frames separately into different branches in \textit{bLNet}\xspace. The branch merging at each layer (\textit{local fusion}) captures short-term temporal dependencies between adjacent frames. d) \textbf{bLVNet-TAM} includes the proposed aggregation module, represented as a red box, which further empowers \textit{bLVNet}\xspace to model long-term temporal dependencies across frames (\textit{global fusion}).}
\end{figure}

\subsection{Big-Little Video Network as Video Representation}
\label{sec:approach:dpn}
We describe our architecture in the context of 2D convolutions. However our approach is not specific to 2D convolutions and potentially extendable to any architecture based on 3D convolutions.

The approach of Temporal Segment Networks (TSN)~\cite{TSN:wang2016temporal} provides a generic framework for learning video representations. With a shared 2D ConvNet as backbone, TSN performs frame-level predictions and then aggregates the results into a final video-level prediction (Fig.~\ref{fig:video_arch}a)).
The framework of TSN is efficient and has been successfully adopted by some recent approaches for action recognition such as TRN~\cite{TRN:zhou2018temporal} and TSM~\cite{TSM:lin2018temporal}.
Given its efficiency, we also choose TSN as the underlying video framework for our work.

Let $\mathbf{F}=\{f_t|t=1\cdots n\}$ be a set of sampled input frames from a video. We divide $\mathbf{F}$ into two groups, namely odd frames $\mathbf{F_{odd}}=\{f_k \in \mathbf{F} |\bmod (k,2) \neq 0\}$ at half of the input image resolution, and even frames $\mathbf{F_{even}}=\{f_k \in \mathbf{F} |\bmod(k,2)= 0\}$ at the input image resolution. For convenience, from now on, $\mathbf{F_{odd}}$ is referred to as \textit{big} frames and $\mathbf{F_{even}}$ as \textit{little} frames. Note that big branch can take either of a pair of frames as input and the other frame goes to the little branch.

In TSN, all input frames are ordered as a batch of size $n$, where the $t^{th}$ element corresponds to the $t^{th}$ frame.
We denote the input and output feature maps of the $t^{th}$ frame at the $k^{th}$ layer of the model by $\mathbf{x}^k_t \in \mathbb{R}^{C\times W \times H}$ and $\mathbf{y}^k_t \in \mathbb{R}^{C\times W \times H}$, respectively. Whenever possible, we omit $k$ for clarity.

The \textit{bLNet}\xspace~can be directly plugged into TSN as the backbone network for learning video-level representation. We refer to this architecture as \textit{TSN-bLNet}\xspace to differentiate it from the vanilla TSN (Fig.~\ref{fig:video_arch}b)).
This network fully enjoys the efficiency of \textit{bLNet}\xspace, cutting the computational costs down by $1.6\sim2$ times according to~\cite{chen2018biglittle}. Mathematically, the output $\mathbf{y}_t$ can be written as
\begin{equation}
    \label{eq:blvnet}
    \mathbf{y}_{t} = \mathcal{F}(net_B([ \mathbf{x}_{t} ]_{1/2}) + net_L(\mathbf{x}_{t}), \theta_t).
\end{equation}
Here $[ {\cdot} ]_{s}$ is an operator scaling a tensor up or down by a factor of $s$ in the spatial domain; $net_B$ and $net_L$ are the Big-Net and Little-Net in the \textit{bLNet}\xspace~aforementioned; and $\theta_t$ are the model parameters. Following~\cite{chen2018biglittle}, $\mathcal{F}$ indicates an additional residual block applied after merging the big and little branches to stabilize and enhance the combined feature representation.

The architecture described above only learns features from a single frame, so there are no interactions between frames. Alternatively, we can feed the odd and even frames separately into the big and little branches so that each branch obtains complementary information from different frames. This idea is illustrated in Fig.~\ref{fig:video_arch}c) and the output $\mathbf{y}_t$ in this case can be expressed by
\begin{equation}
  \label{eq:fastblvnet}
  \mathbf{y}_t =\left\{
  \begin{array}{@{}ll@{}}
    \mathcal{F}(net_B(\lfloor {\mathbf{x}_{t}}\rfloor_{1/2}) + net_L(\mathbf{x}_{t+1}), \theta_t), & \text{if}\ \bmod(t,2) \neq 0 \\
    \mathbf{y}_{t-1}, & \text{otherwise}
  \end{array}\right.
\end{equation}

While the modification proposed above is simple, it leads to a new video architecture, which is called Big-Little-Video-Net, or \textit{bLVNet}\xspace for short.
The \textit{bLVNet}\xspace makes two distinct differences from \textit{TSN-bLNet}\xspace.
Firstly, without increasing any computation, it can take input frames two times as many as \textit{TSN-bLNet}\xspace. We shall demonstrate the benefit of leveraging more frames for temporal modeling in Section~\ref{sec:experiment}.
Furthermore, the \textit{bLVNet}\xspace has $1.5\sim 2.0 \times$ fewer FLOPs than TSN while seeing frames twice as many as TSN, thanks to the efficiency of the dual-path network.
Secondly, the merging of the two branches in \textit{bLVNet}\xspace now happens on two different frames carrying temporal information.
We call this type of temporal interaction by \textit{local fusion}, since it only captures temporal relations between two adjacent frames.
In spite of that, local fusion gives rise to a significant performance boost for recognition, as shown later in Section~\ref{sec:experiment:ablation}.

\subsection{Temporal Aggregation Module}
\label{sec:approach:deta}
Temporal modeling is a challenging problem for video understanding. Theoretically, adding a recurrent layer such as LSTM~\cite{lstm:donahue2015longterm} on top of a 2D ConvNet  seems like a promising means to capture temporal ordering and long-term dependencies in actions. Nonetheless, such approaches are not practically competent with 3D ConvNets~\cite{I3D:carreira2017quo}, which use spatio-temporal filters to learn hierarchical feature representations.
One issue with 3D models is that they are heavy in parameters and costly in computation, making them hard to train.
Even though some approaches like S3D~\cite{S3D:xie2018rethinking} and R(2+1)D~\cite{R(2+1)D:tran2018closer} alleviates this issue by separating a 3D convolution filter into a 2D spatial component followed by a 1D temporal component, they are in general still more expensive than 2D ConvNet models.

\begin{figure}[b!]
\centering
\includegraphics[width=.8\linewidth]{images/depthwise_temporal_aggregation}
\caption{\label{fig:fusion} Temporal aggregation module (TAM). The TAM takes as input a batch of tensors, each of which is the activation of a frame, and produces a batch of tensors with the same order and dimension. The module consists of three operations: 1) 1$\times$1 depthwise convolutions to learn a weight for each feature channel; 2) temporal shifts (left or right direction indicated by the smaller arrows; the white cubes are padded zero tensors.); and 3) aggregation by summing up the weighted activations from 1). }

\end{figure}

With the efficient \textit{bLVNet}\xspace architecture described above, our goal is to further improve its spatio-temporal representation by effective temporal modeling. The local fusion in \textit{bLVNet}\xspace only exploits temporal relations between neighbored frames. To address this limitation, we develop a method to capture short-term as well as long-term dependencies across frames. Our basic idea is to fuse temporal information at each time instance by weighted channel-wise aggregation. As detailed below, this idea can be efficiently implemented as a network module to progressively learn spatio-temporal patterns in a hierarchical way.

Let $\mathbf{y}_t$ be the output (i.e. neural activation) of the $t^{th}$ frame $f_t$ at a layer of the network (see Eq.~\ref{eq:fastblvnet}). To model the temporal dependencies between $f_t$ and its neighbors, we aggregate the activations of all the frames within a temporal range $r$ around $f_t$. A weight is learned for each channel of the activations to indicate its relevance. Specifically, the aggregation results can be written as
\begin{equation}
    \label{eq:dat}
  \mathbf{\hat{y}}_t = ReLU ( \sum_{j= - \left\lfloor r/2 \right\rfloor}^{j=\left\lfloor r/2 \right\rfloor } \mathbf{w}_j \otimes \mathbf{y}_{t+j} ),
\end{equation}
where $\otimes$ indicates the channel-wise multiplication and $\mathbf{w}_j \in \mathbb{R}^{C}$ is the weights. The $\otimes$ is defined as: for a vector $\mathbf{v}=[v_1~ v_2 ~ \cdots ~ v_C]$ and a tensor $\mathbf{M}=[\mathbf{m}_1~\mathbf{m}_2~ \cdots ~ \mathbf{m}_C]$ with $C$ feature channels, $\mathbf{v} \otimes \mathbf{M}=[v_1*\mathbf{m}_1 ~  v_2*\mathbf{m}_2 ~ \cdots v_C*\mathbf{m}_C]$.

We implement the temporal aggregation as a network module (Fig.~\ref{fig:fusion}). It involves three steps as follows,
\begin{enumerate}
\item apply 1$\times$1 depthwise convolution $r$ times to $n$ input tensors to form an output matrix of size $r\times n$;
\item shift the $i^{th}$ row left (or right) by $|i-\lfloor r/2 \rfloor|$ positions if $i > \lfloor r/2 \rfloor$ (or $i \leq \lfloor r/2 \rfloor$) and if needed, pad leading or trailing zero tensors in the front or at the end;
\item perform temporal aggregation along the column to generate the output.
\end{enumerate}

The aggregation module(\textit{TAM}\xspace), highlighted as a red box in Fig.~\ref{fig:video_arch}d), is inserted as a separate layer after the local temporal fusion in the \textit{bLVNet}\xspace, resulting in the final \textit{bLVNet-TAM}\xspace architecture.
Obviously none of the steps in the implementation above involve costly computation, so the module is fairly fast. A node in the network initially only sees $r-1$ neighbors. As the network goes deeper, the amount of context that the node involves in the input grows quickly, similar to how the receptive field of a neuron is enlarged in a CNN. In such a manner, long-range temporal dependencies are thus potentially captured. For this reason, the temporal aggregation is also called \textit{global temporal fusion} here, as opposed to the \textit{local temporal fusion} discussed above.

The work of TSM~\cite{TSM:lin2018temporal} has also applied temporal shifting to swap feature channels between neighboring frames. In such a case, TSM can be treated as a special case of our method where the weights are empirically set rather than learned from data. In Section~\ref{sec:experiment:ablation}, we demonstrate that the proposed TAM is more effective than TSM for temporal modeling under different video architectures. TAM is also related to S3D~\cite{S3D:xie2018rethinking} and R(2+1)D~\cite{R(2+1)D:tran2018closer} in that TAM is independent of spatial convolutions. However, TAM is based on depthwise convolution, thus has fewer parameters and less computation than S3D and R(2+1)D.

The \textit{TAM}\xspace~can also be integrated into 3D convolutions such as C3D~\cite{C3D:Tran2015learning} and I3D~\cite{I3D:carreira2017quo} to further enhance the temporal modeling capability that already exists in these models. Due to the difference in how temporal data is presented between 2D-based and 3D-based models, the temporal shifting now needs to operate on feature channels within a tensor instead of on tensors themselves.

\section{Conclusion}
\label{sec:conclusion}
We presented an efficient and memory-friendly video architecture for learning video representations. The proposed architecture allows for twice as many input frames as the baseline while using less computation and memory. This enables training of deeper models with richer input under the same GPU resources. We further developed a temporal aggregation method to capture temporal dependencies effectively across frames. Our models achieve strong performance on several action recognition benchmarks, and establish a state-of-the-art on the Something-Something dataset.

redpajama_set_name: RedPajamaArXiv


Top 10 remortgage mistakes and how to avoid them!
So here are the top 10 common mistakes that people make when remortgaging and how to avoid them.
1. Not starting the remortgage process early enough.
Completing remortgage can take a week or 6 months. If you don't start the remortgage process early enough, you could end up paying a standard variable mortgage rate. The lenders standard variable rate mortgage is generally a lot higher which could cost you hundreds of pounds extra each month.
The best way to avoid paying the lenders standard variable mortgage rate is to start the process early. We generally complete an initial remortgage assessment about 6 months before your current fixed or tracker mortgage rate ends, and set the time scales to start the process. Generally we can switch you over to a new lower fixed or tracker rate mortgage the day after your current fixed or tracker mortgage rate ends.
2. Using the lenders free legal incentive as part new remortgage package.
To switch your mortgage over to a new lender you will need a solicitor or conveyancer. Sometimes the new mortgage lender will offer a free legal incentive and cover the cost of this or offer a cashback of around £250 towards your own solicitor.
This is where to problems can start. The average cost of a solicitor to complete a remortgage is around £500. So, you can see there is a shortfall of £250 if you take the cashback and pay for a solicitor yourself.
Why would pay extra when the lender is offering it for free?
A remortgage loan that is exacting the same as what you have.
No one is being added or removed from the property deeds.
There are no second charges.
Anything that is slightly out of the ordinary or time is a factor you should instruct your own solicitor or conveyancer, take the cashback and pay the extra.
It will be worth it in the long run.
Our biggest cause of unhappy remortgage customers and errors are caused by the lender's free legal teams.
3. Not getting proper mortgage advice when choosing a rate type.
Your existing mortgage lender won't want you to leave and they may offer you a new rate on an unadvised basis.
e.g. Your current mortgage lender will show you a list of their products and ask you to choose.
The risk of doing this is the rate type, length, penalties or fees being charged may be completely wrong for your circumstances. This may cost you thousands in the long run.
Getting proper mortgage advice from a mortgage adviser will ensure the mortgage product that is recommended fits your current needs and your future needs are considered.
We will look at thousands of different mortgages for you, to ensure we find you the best mortgage rate.
mortgage, remortgage, mortgages, remortgages, mortgage plus, top 10 remortgage fails. How to remortgage.
4. Getting the value of their property wrong.
Most lenders will offer you a loan based on the current value of your home and the loan being applied for (Also known as equity). The lender will employ a surveyor to look at the property and give a best guess if the property was sold what price could be achieved within a 3 month period. They will ask the surveyor to provide the lender with some evidence of recent property completions in the last 3 months to support the final valuation figure.
When you apply for a remortgage you may be tempted to increase the value of your property or offer a finished value after proposed home improvements are finished and apply for a rate that isn't available based on the equity in the property.
Your property could also be worth more than you think and a better interest rate may be available to you, because you are a lower risk mortgage customer.
When considering your remortgage application we will check land registry for recent completions and contact local estate agents to ensure we get a figure that is as close as possible to your properties true value and base our mortgage advice on these figures.
5. Extending their mortgage term back to the start.
When changing your mortgage over to a new lender it is important to keep reducing the term in line with your current term. If you keep taking the term back to 25 years or 30 years you will be paying thousands in extra interest and risk still paying a mortgage into retirement.
We all want to see the back of our mortgage and getting your mortgage paid off early could be more viable than you think.
During our application process we will carefully consider your monthly budget and offer to the shortest term we can.
6. Not updating their life, lifestyle and family insurance.
Borrowing extra money on your mortgage and not increasing your insurance is easily done, you are focus on the mortgage and put off the insurance till the end.
This can then very easily be forgotten.
There are lots of reasons to remortgage, whether it be for home improvements, debt consolidation or just getting a better rate. When you change your mortgage terms it is important to review your protection and make sure that if the worst happens, your mortgage loan and family are taken care of. Over time your insurance needs change. We all sleep better at night knowing we have set everything up so if the worst happens, you have a plan B.
7. Not declaring everything to the mortgage lender.
Mortgage lenders need to demonstrate any new mortgage loan is affordable to you. To do this they will consider certain types of income and all or your expenditure. They will check your credit file and go through your bank statements.
It is important to declare all your regular expenditure at application stage as the lender may not see it at the start of the application process and find it at the end and change their mind on whether to approve the mortgage loan.
The most common things to be missed are hire purchase agreements on cars or 0% finance credit agreements.
During our application process we go over everything before we make our mortgage recommendation. Saving you time and worry.
8. Not checking how much they owe to their current mortgage lender.
There is nothing worst and getting to the end of your remortgage and then realising that you haven't borrowed enough to pay off your current loan.
Things that often get missed are the extra fees and charges that your existing mortgage lender levies to pay off the loan. Early redemption fees are also date sensitive so a completion date has be set after these fees don't apply.
The best thing to do is call your current lender and ask for a redemption statement before you apply for a remortgage. Also check with your current lender to find out the date the early redemption fees no longer apply to your mortgage loan.
This will prevent any nasty surprises at the end.
9. Cancelling the direct debit with your current lender to early.
Cancelling your direct debit with your current lender before the mortgage has been paid off is risky. Any delay in the remortgage application could mean your current mortgage isn't paid and goes into arrears. This could affect your credit score and the new mortgage lender may cancel your application as a result.
We suggest waiting for written confirmation from your current lender that your mortgage is paid off before you cancel it.
10. Not borrowing enough for home improvements.
Home improvements can cost more than you expect. Not borrowing enough to complete the works could mean having to apply for a further loan at a less favourable rate or worst still having to pay thousands in penalties to switch to a new lender.
Many lenders will allow over payments of up to 10% per year of the mortgage balance without penalties. We suggest you consider borrowing an extra contingency fund to cover any extra and if you don't use the money it can be paid back to the remortgage lender if penalties don't apply.
Why not sign up to our magical remortgage tracker and find out the perfect time to start the remortgage process.


Studying is different for everyone. Some people love to get up early, finish before the afternoon has begun and relax while others prefer to rest up and find their motivational groove later on. The only way to support both sides of the diverse coin when it comes to education is by offering a flexible set of courses such as ours at Oxford Learning College. What could be more flexible than studying at home online?
What if I need help with a part of the course?
Our courses are all designed to give you the best chance of succeeding. We know, therefore, that not everything in the course syllabus will instantly make sense. You can contact your specific tutor over email to get a better understanding.
Will employers look down on an online qualification?
On the contrary, you will be gaining more experience and learning more about yourself and how you work best. All of our fully accredited courses will provide you with exactly the same kudos as one studied physically in an institution.
If you are worried about taking the next step, whether that is starting a new course or progressing your career, speak with our team today.


text:
Some Android Rivals Simply Can't Afford to Compete With Apple's Face ID
Just because some of the components aren't all that expensive doesn't mean implementing 3D facial recognition comes cheap. For example, the vertical-cavity surface-emitting lasers (VCSELs) that go into Apple's (NASDAQ: AAPL) TrueDepth camera system are only estimated to represent $4 to $5 worth of content in iPhone X, which doesn't seem too expensive. But that's just a small piece of the puzzle. The modules include other parts like optical filters and infrared cameras, which initially suffered from low yield rates, driving up average costs per unit.
Then there are all the costs associated with developing the sophisticated software that does the heavy lifting behind the scenes, and integrating that software with the hardware. It's a lot to keep up with if you're a competing Android vendor looking to offer 3D sensing. Faced with such an expensive proposition, some original equipment manufacturers (OEMs) are simply giving up on the idea.
Looking for an alternative
DIGITIMES reports that rivals like Huawei are instead looking to introduce in-display fingerprint sensors instead of pursuing 3D sensing. Apple had been rumored to be exploring in-display Touch ID, but hardware engineering chief Dan Riccio shot that notion down last year, saying the company "spent no time looking at fingerprints on the back or through the glass." Apple knew early on that it wanted to bet big on Face ID, which is precisely what it did.
The fingerprint sensors that some Android OEMs are now considering utilize Qualcomm technology and are expected to dramatically improve performance while enabling thinner form factors. Even though these newer sensors are also about three times as expensive as traditional fingerprint sensors, the costs are nowhere near what it would take to implement 3D sensing, which DIGITIMES estimates to be upwards of $60 per unit after factoring in software development and hardware/software integration.
With most Android phones selling for far less than an iPhone (usually $200 to $300), there's really no room in the bill of materials (BOM) to afford $60 per unit. Even flagship Android phones that sell for much higher could potentially have trouble recouping those costs, seeing as all evidence suggests that iPhone X demand is suffering due to the high price point.
All of this is before even acknowledging that Android OEMs probably can't even procure VCSEL supply until at least 2019, since Apple has locked down much of the industry's capacity, according to suppliers.
Samsung will follow
However, it does appear that archrival Samsung is exploring 3D sensing for next year's Galaxy S10. The Galaxy S9 that launched this month includes facial recognition, but still uses a 2D implementation that is not secure enough to be comparable to Apple's Face ID. Samsung is one of the few Android OEMs that has the scale, vertical integration, and brand presence in the premium segment of the smartphone market to potentially pull it off.
As for the rest of the Android camp, they better hope that 3D facial recognition doesn't become a must-have feature in the years ahead.
10 stocks we like better than AppleWhen investing geniuses David and Tom Gardner have a stock tip, it can pay to listen. After all, the newsletter they have run for over a decade, Motley Fool Stock Advisor, has tripled the market.*
David and Tom just revealed what they believe are the 10 best stocks for investors to buy right now... and Apple wasn't one of them! That's right -- they think these 10 stocks are even better buys.
Click here to learn about these picks!
*Stock Advisor returns as of March 5, 2018
Evan Niu, CFA owns shares of Apple. The Motley Fool owns shares of and recommends Apple. The Motley Fool owns shares of QCOM and has the following options: long January 2020 $150 calls on Apple and short January 2020 $155 calls on Apple. The Motley Fool has a disclosure policy.

redpajama_set_name: RedPajamaCommonCrawl


Putting this out there now, we will need extra staff on Sunday, August 6. If you are interested in working, AND seeing a great show, please let me know! Hours approx 8:30am-3 or 4pm. The show should wrap up around 6.
If interested reach out to Island Dragway directly.
i'll be going there today, cant work or help, as I have to work at night. but I'll be there spectating for 4-5 hours.
welp, I went.. and had freaking BLAST! . I gotta say, I was very happy to see a whole TON of awesome cars . The nostalgia alcohol cars were never-ending. The nitro ones were bad-assss. Since I don't own a 9 second wheels-up car, I decided to just do the car show portion.
Last edited by IROCZman15; 08-13-2017 at 01:04 AM.


text:
RANDY PETERSON
Peterson: McKay is Iowa State's X-factor, the Cyclones' great unknown
rpeterson@dmreg.com
AMES, Ia. –
Jameel McKay sat back and watched as Iowa State overwhelmed Iowa last season in Iowa City. He wasn't yet eligible after transferring from Marquette, but that didn't stop him from attending the game.
He and Hallice Cooke, another sit-out Cyclone, drove to Iowa City. Together, they watched from behind the bench as their future teammates won in Carver-Hawkeye Arena.
And at 6:30 p.m. Thursday, the tallest of that pair suddenly becomes the most significant unknown when Cy and Hawk meet at Hilton Coliseum.
McKay is the shot blocker. He's the long and lanky rebounder. He's even often the fastest guy on the floor.
"He's really the X-factor in this game," teammate Georges Niang said before a practice this week.
If there's an Iowa State difference maker, it's the senior who's actually a rookie in this rivalry. McKay averages 13.9 points and 11.6 rebounds. He's blocked 12 shots, and he's deflected a bunch of others.
He's the intimidator and the sprinter, all rolled into one 6-foot-9, 225-pound frame.
"Jameel is like Usain Bolt when you're running the floor," Niang said of the Jamaican sprinter. "He's a guy that can protect the rim, rebound and adds toughness.
"It's a whole different dimension when you have a big guy out there that's going to outwork every other team's big guy."
McKay: The defensive difference-maker
He might not be the best perimeter defender, but that's not his thing. His deal is protecting the rim, owning the lane, and keeping as many opponents from entering it as possible.
He'll start out guarding 7-1 Adam Woodbury at the center position, while Niang takes on the Jarrod Uthoff challenge.
"Georges' matchup with Uthoff will be big," Iowa State coach Steve Prohm said. "They'll play him all over the floor."
McKay, meanwhile, will police the middle, where he has 71 blocked shots in 32 career games.
He'll do everything in his power to make sure the Hawkeyes don't get anything easy -- to see that they don't pick this game to suddenly score a rash of paint points.
"Jameel outworks every big man he plays against," Niang said. "If he can get us more possessions on offense, and take away their possessions and protect the rim — that's extremely effective."
McKay: The offensive difference-maker
He does his damage in the middle, where Iowa opponents have scored 46.6 of their points. He has 63 career dunks, including 25 this season. He's a 59 percent career field goal shooter.
He's the guy on the end of those often dynamic long lob passes from point guard Monte Morris. He's the instant highlight reel.
Roll tape.
"All you have to do is just throw it up there close to the basket," Morris said. "If Jameel can get his hands on the ball, it's going down."
And the crowd noise goes up.
"They love it," McKay said.
So does McKay. So does Morris – or whoever gets credit for the assist.
"When Jameel is flying all over the place dunking the ball, blocking shots, sprinting the floor, and defending – it pumps up the fans," Niang said. "The more energy he plays with, the better."
McKay: The great unknown
Iowa has faced Niang and Naz Mitrou-Long, Morris and Abdel Nader.
They know Niang is a mismatch, and that Mitrou-Long is a 3-point shooting machine . . .
That Morris has the eye-popping assist-to-turnover ratio of 5.1, and that Nader had a big game a year ago.
That talented Iowa State foursome starred during last season's 90-75 victory at Carver-Hawkeye Arena, so what haven't the Hawkeyes experienced?
The X-factor, aka . . .
Jameel McKay.
Cyclones columnist Randy Peterson has been reporting on ISU during the past five decades. Follow @RandyPete.


text:
Cooplands extends offer into wholesale business
Bakery chain Cooplands (Doncaster) is planning to expand into wholesale for the first time, after being awarded a BRC rating. The company was recently audited against BRC issue 5 and awarded a Grade B.
Previously, the firm had only manufactured products to sell in its 75-shop estate, but has worked with the Regional Food Group (RFG) for Yorkshire and Humber for the last three years on moving into wholesale. The RFG aims to develop the region's food and drink sector by delivering trade development, offered through targeted business assistance."Our objective over the past five years has been to develop a state-of-the-art pastry department that gives consistent quality and drives down production costs for the supply of fresh and frozen pastries for bake off in our retail outlets," explained Alan Jacques, purchasing and operations manager.This objective has seen the installation of a temperature-controlled production hall, a new Rondo Pastry Make-up line, a spiral freezer and a 250-pallet freezer. "From the outset we realised that this would give us large excess capacity, and enable us to develop a wholesale business. Our current focus is to make potential customers aware of our move into this market." Jacques said Cooplands' main focus will remain on its core retail business, "with no alterations to our existing plans for expansion." It will treat its wholesale business as a separate entity. "We have also recently purchased two Jiffy vans offering a delivery service to businesses in and around Doncaster," he added.
easyjet's first supermarket – easyfoodstore.com
Morrisons extends Free From range
Dawn Foods adds new flavours to Delifruit range
Pecan Deluxe invests in new premises
New sponsor for British Pie Awards


text:
Failure to File Marriage License Invalidates Marriage?
March 23, 2018 /in Law Review /Tags: Court of Appeal, Dissolution of Marriage, Domestic Partners, Family Code Section 306, Family Law, Joint Tenants, Real Estate Lawby Crysta Dwyer
In 2016, there were approximately 2,245,404 marriages in the United States and 827,261 divorces. That's discouraging.
A MARRIAGE
A marriage in California requires consent of the parties and issuance of a license and solemnization. The Family Code recites: "To solemnize the marriage, the parties shall declare, in the physical presence of the person solemnizing the marriage and necessary witnesses, that they take each other as spouses." Sounds easy enough.
In fact, so easy some of my friends have been married three or four times. Well not my friends, but some of yours.
Leanne Netterstrom and Michael Chaney began dating in 2008 and living together in 2011, finally getting married in a ceremony in Cambria on November 11, 2011. They obtained a marriage license, participated in the ceremony and exchanged vows before family and friends. And thereafter generally referred to themselves as husband and wife i.e. normal, normal.
On the other hand, they filed separate tax returns as "single" people for tax reasons and purchased a home together as "domestic partners," taking title as "an unmarried man and unmarried woman as joint tenants." Taking title as joint tenants was surely the wrong thing to do, but that's for another day.
In 2015, Chaney petitioned in court for dissolution of marriage. Remember when that was called a divorce? That law changed while I was in law school – 2 or 3 yeas ago as I recall.
Netterstrom never wanted to get married in the first place because she had been married twice before, and because Chaney supposedly gambled and was financially unstable. Nevertheless, I guess at a weak moment, Chaney proposed and Netterstrom accepted. I would not have put great odds on that marriage.
WE'RE NOT MARRIED!
Netterstrom, probably not happy because this would be her third divorce, claimed that the two of them were not really married because the wedding officiate failed to return and file the signed marriage license with the county clerk as required. Indeed the two agreed not to return the signed marriage license to the county and it remained in a desk drawer. Netterstrom claimed that she was off the hook. There never was a marriage.
COURT RULING
The Second District Court of Appeal looked at the case after the trial court ruled for Chaney: there was a marriage.
The Court of Appeal recited Family Code Section 306 which requires that a marriage be licensed, solemnized and authenticated and the license returned to the county recorder. So Netterstrom wins, right? Netterstrom loses because section 306 goes on to read "noncompliance with this part [return the signed license] by a nonparty to the marriage does not invalidate the marriage." And Section 423 of the Family Code requires the officiate (not the parties) to return the marriage license to the county recorder.
The officiant, who relied on the happily married couple to file the signed marriage license, didn't file the license as was required and because he was a nonparty to the marriage, his failure to do so did not invalidate the marriage.
These poor people not only could not get their marriage right, they couldn't get their divorce right. Wait till they learn they hold title as joint tenants – survivor gets the house.
Jim Porter is an attorney with Porter Simon licensed in California and Nevada, with offices in Truckee and Tahoe City, California, and Reno, Nevada. Jim's practice areas include: real estate, development, construction, business, HOA's, contracts, personal injury, accidents, mediation and other transactional matters. He may be reached at porter@portersimon.com or www.portersimon.com.
Like us on Facebook. ©2018
The content contained and opinions expressed in this blog are solely those of the author. This blog contains content and opinions concerning the law generally, and is not intended to constitute legal advice or to create any attorney‑client relationship with the reader. The reader should consult with an attorney about any specific legal issues prior to embarking on any course of action or inaction involving legal matters.
Tags: Court of Appeal, Dissolution of Marriage, Domestic Partners, Family Code Section 306, Family Law, Joint Tenants, Real Estate Law
Dangerous Conditions of Public Property Law vs. Trail Immunity Law
Snowboarder Runs Into Snowcat
When May Landlord Enter The House
Gang Members Threaten Cops Without Saying A Word
Shoplifter Banned From All Home Depot Stores
AEDs Required in Businesses?
Homesteading Your House
SUPREME COURT RULING ON EVICTION AFTER FORECLOSURE
CALIFORNIA'S DRAM SHOP LIQUOR LAWS July 12, 2019
SCHOOL IMMUNE FROM LIABILITY DURING FIELD TRIP INJURY? June 28, 2019
RIGHTS AND DUTIES OF EASEMENT OWNERS – FOR DUMMIES June 21, 2019
CYCLING SEASON SAFETY: SHARE THE ROAD, SHARE THE RESPONSIBILITY June 20, 2019
AIRBNB CHALLENGES SHORT-TERM VACATION RENTAL ORDINANCE June 14, 2019
Need legal help? Contact Porter Simon for a consultation today.
Trained Police Dog Bites Innocent Bystander Identity Theft Victims May Sue


Leadership Transition at Top Defense Contractor: Lockheed Martin's board chairman and CEO Robert Stevens has notified the company that he intends to retire after 25 years of service. Chris Kubasik, the company's president and chief operating officer, will become CEO next January, according to Lockheed Martin's April 26 release. Stevens intends to remain board chairman through January 2014, subject to election by shareholders and board approval. Marillyn Hewson will succeed Kubasik as president and COO next January. "Chris and Marillyn are superbly well prepared to guide the continuous evolution of Lockheed Martin's strategies and operating concepts in ways that will allow us to simultaneously address the evolving needs of customers and the expectations of shareholders," said Stevens, who became CEO in 2004 and board chairman the following year. "Bob Stevens has had an extraordinarily distinguished career, working tirelessly on behalf of Lockheed Martin's customers, shareholders, and employees," said Douglas McCorkindale, Lockheed Martin's lead director.


NEW BRAKES + ROTORS + TIRES JUST INSTALLED. ALL WHEEL DRIVE. FULLY LOADED ! ! !
BRAND NEW SILVERADO 2500HD. $10K UNDER BOOK ! ! !
ONLY 34K MILES. WHERE ELSE AT THAT PRICE ???
1 OWNER, NO ACCIDENTS, ONLY 18K MILES ! ! ! ! !
TRUE CLASSIC, ONLY 67K MILES, SHARP ! ! !
FUEL EFFICIENT + INSPECTED + NEW CLUTCH ! ! !
LIKE NEW ! ! !


text:
(-) Audio
Intro to Egalitarian Beliefs
Award-Winning Articles
Bible Translation
Biblical Word Studies
(-) Parenting / Youth
By Bellyn Whitteker | December 05, 2018
Educated: A Memoir is a story about surviving familial trauma as well as the transformation of a young woman as she becomes liberated from the oppressive beliefs and traditions of her childhood.
My Eyes Were Opened: The Treatment of Women in View of Biblical Texts About Children
By William Webb | July 31, 2008
Two Sides of the Same Coin: The Twin Epidemics of HIV and GBV
By Lyn Van Rooyen | October 09, 2016
A workshop from the 2016 international conference "Truth Be Told" in Johannesburg, South Africa.
Book Review: Steven R. Tracy's Mending the Soul: Understanding and Healing Abuse
By Scarlet Hai Yin Tsao | January 30, 2006
The increasing incidents of abuse in our society are a sad reality with which all church leaders must come to grips. Mending the Soul is, therefore, a valuable resource not only for abuse survivors and those ministering to them, but also for church leaders who have to explore the uncharted territory of abuse because of love for their congregations.
Shame and Abuse: Understanding and Healing a Deadly Legacy
By Steven R. Tracy | August 31, 2007
Seeking Justice and Loving Mercy: Gender and Equality in the Bible and our Culture
Sexual Trafficking, Prostitution, And The Global Sex Industry
By Lisa L. Thompson | July 31, 2005
This workshop explores the defnition of human traficking, the harms of sexual exploitation, and how the church can respond.
Is It My Fault?: Hope and Healing for Those Suffering Domestic Violence
There are many excellent books on the topic of domestic abuse, but this is one of the more practical ones that I have read. Included in its helpful resources is a detailed plan or "exit strategy" for the abuse "victim" who has decided she must leave the situation for the well-being of herself and any children involved. It also offers a section explaining how abuse in the home affects children, providing another point of reasoning as to why the best option might be to plan to leave. Throughout the book, the authors emphasize that abusive situations are not magically resolved, but offer biblical and wise practical counsel on how to proceed and why. Any person who reads this book will be greatly helped to sort through their emotions and be strengthened for path that lies ahead.
Confident Kids Created in His Image
By Bronwyn Stanford | October 09, 2016
A workshop from the 2016 "Truth Be Told" international conference in Johannesburg, South Africa.


Red, White and Bleu Slot - Try Playing Online for Free. Hoppa till Play Free Slots – Game List - You can see the full Rival slot games list below and immediately. Try a free demo of the Power Stars slots game or play for real and win for real! Red, White and Bleu Slot - Try Playing Online for Free wants just a little extra. Slot machine games free play book of ra, Play free slots app similar to real Ja, es passiert sehr oft, dass man die Kontrolle verliert und online casino reviews subsequent numbers, its advisable to try out the free credits before you go for 7 icons, along with Red Grapes, White Grapes, and a wedge of Bleu Cheese.
Flea Market MOBILE and ONLINE Slot for FREE PLAY Deine E-Mail-Adresse wird nicht veröffentlicht. Play Tycoon Towers slot machine for free. Das zweitbeste Symbol nach dem Sie Ausschau halten sollten, ist die goldene Glocke und der goldene Stern. Lost Secret of Atlantis Slot Game. Try Vintage Vegas slots online.
Try Vintage Vegas slots online. Most online casinos online casino games download require players to download their software in order to provide them with an improved gaming experience. This might seem like a strange name to give to a page with lots of slots to play, but these games can all be played to win real money, with our special offer that you will find on that page. View more diamond queen slots This free slot is a magical game filled with dazzling diamonds Rizk Casino is Fully Licensed by the Malta Gaming Authority (MGA) hidden gems. Mehr erfahren, oK, merkur spielautomat fehlermeldung mein Konto, suche, maps. We suggest you to try as many types of free slots as you can: Werfen Sie einen Blick auf die Liste unten. Abhängig von Ihrem Geschmack ist dieser Slot entweder das Kultigste was Sie je erlebt haben oder einfach nur. Octavian produziert ebenfalls andere Komponenten, wie Münzprüfer und Magazine. Slot machine kostenlos online spielen 5 dragons Spiele casino online kostenlos Roulette online spielen kostenlos ohne anmeldung pferde Casino merkur online redaktion Casino star games apk Kostenlos casino spielen ohne anmeldung vier gewinnt Online casino tricks 5 euro einzahlung Paypal online casino you win Merkur spielautomaten online kostenlos spielen Online casino bonus ohne einzahlung paypal Casino spiele kostenlos downloaden handy Merkur online casino kostenlos bonus sunmaker home Casino titan no deposit bonus codes july Deine E-Mail-Adresse wird nicht veröffentlicht. An awesome slot machine game has come to the rescue. Play Tycoon Towers slot machine for free. Try Lion's Roar slots online. We are delighted to welcome yet another market leader to Evolution. Swinging Sweethearts Video Slot Game.


text:
Front Matter Wonderland The Great God Pan The Six Pomegranate Seeds The Birth of Athene The Two Weavers The Purple Flowers Danae and Her Little Son The Quest of Perseus Andromeda and Sea-Monster Acrisius Killed by Perseus Achilles and Briseis Menelaus and Paris Do Battle Hector and Andromache The Horses of Achilles The Death of Hector Polyphemus the Giant Odysseus Escapes from Cave Odysseus Returns to Ithaca Argus the Hound Dies The Bow of Odysseus The Land of Hellas Lycurgus and His Nephew Lycurgus Returns to Sparta Training of the Spartans The Helots Aristomenes and the Fox The Olympian Games The Last King of Athens Cylon Fails to be Tyrant Solon Frees the Slaves Athenians Take Salamis Pisistratus Becomes Tyrant Harmodius and Aristogiton The Law of Ostracism The Bridge of Boats Darius Rewards Histiaeus Histiaeus Shaves His Slave Sardis Is Destroyed Sandal Sewn by Histiaeus Earth and Water Battle of Marathon Miltiades Sails to Paros Aristides is Ostracised The Dream of Xerxes Xerxes Scourges the Hellespont Bravest Men of All Hellas Battle of Thermopylae Battle of Artemisium Themistocles at Salamis Themistocles Tricks Admirals Battle of Salamis Battle of Plataea Delian League Themistocles Deceives Spartans Themistocles is Ostracised Eloquence of Pericles Pericles and Elpinice The City of Athens Great Men of Athens Thebans Attack Plataeans Attica Invaded by Spartans Last Words of Pericles Siege of Plataea The Sentence of Death Brasidas Loses His Shield The Spartans Surrender Brasidas the Spartan Amphipolus Surrenders Alcibiades the Favourite Socrates the Philosopher Alcibiades Praises Socrates Images of Hermes Destroyed Alcibiades Escapes to Sparta The Siege of Syracuse Athenian Army is Destroyed Alcibiades Returns to Athens Antiochus Disobeys Alcibiades Walls of Athens Destroyed March of the Ten Thousand Pelopidas and Epaminondas Seven Conspirators Battle of Leuctra Death of Epaminondas The Two Brothers Timoleon exiles Dionysius Icetes Attacks Timoleon Battle of Crimisus Demosthenes' Wish Greatest Orator of Athens The Sacred War Alexander and Bucephalus Alexander and Diogenes Battle of Granicus The Gordian Knot Darius Gallops from Battle Tyre Stormed by Alexander Battle of Gaugamela Alexander Burns Persepolis Alexander Slays Foster-Brother Porus and His Elephant Alexander Is Wounded The Death of Alexander Demosthenes in the Temple
Story of Greece - Mary Macgregor
The Battle of Salamis
On the morning of the battle, Xerxes ascended a golden throne which had been placed for him upon a rock that overlooked the sea. Around him sat scribes ready to record the events of the battle. That they would all be to the honour of his fleet Xerxes never doubted.
Themistocles saw with pleasure that the wind was rising, making it difficult for the Persians to manage their unwieldy vessels. As he watched their efforts he urged the Greeks to attack them at once.
The narrowness of the strait, as well as the force of the wind, added to the confusion of the enemy and made the number of its ships of little use. Yet the Persians fought bravely, remembering that the eyes of the great king were upon them.
One of the ships was commanded by a queen named Artemisia. She was fighting fiercely when her ship was attacked by an Athenian vessel at close quarters.
Artemisia tried to escape, but as her ship sailed away it was followed by the enemy. Straight in her path lay one of Xerxes' vessels. The queen did not try to avoid it, but pursuing her course struck the ship, so that her own countrymen who were on board were sent to the bottom.
When the Athenian captain saw what the queen had done, he thought, as perhaps she meant him to do, that she had deserted her own side and was now fighting for the Greeks, so he turned back and followed her no more.
From his golden throne, Xerxes too saw what Artemisia had done, and he supposed it was a Greek vessel that she had run down. In his delight he exclaimed, 'My men are become women, my women men.' This was a hard thing to say of his soldiers who were fighting gallantly for their king.
Meantime the Persian ships were driven into the narrow strait. Ship dashed against ship till the Persian dead strewed the deep 'like flowers.' When evening fell, two hundred Persian ships had been destroyed and the Greeks had won the great sea-battle of Salamis. The glory of the victory was due to Themistocles. There might indeed have been no battle at Salamis had he not tricked both the Persian king and the Greek admirals.
Ship dashed against ship, till the Persian dead strewed the deep "like flowers"
The Athenian was proud of his success, and he now determined by another crafty message to Xerxes to drive him out of Greece.
But first he sent for Aristides, and to test his wisdom he told him that he thought they should sail to the Hellespont to destroy the bridge by which Xerxes had crossed into Europe and by which he could return to Asia.
'Rather than break down the bridge,' answered Aristides, 'we should build another, if by so doing we may hasten his departure.'
Now this was what Themistocles himself really wished—to hasten the king's retreat. So although he did not mean to destroy the bridge, he sent once again to Xerxes, and this is what he said: 'O king, the Greeks are hastening to the Hellespont to destroy the bridge by which alone thou canst return to Asia. Hasten then to reach the bridge, while I delay the Greek fleet, lest evil overtake thee.'
Once more the king fell into the trap Themistocles had prepared for him. For he set out in haste with the main body of his army for the Hellespont, leaving Mardonius with a large force to carry on the war as well as he could.
The march to the Hellespont was a terrible one, for Xerxes had himself laid waste the land when he advanced upon Athens, and now there was neither food nor shelter for his army. The soldiers who were starving ate plants, grass, the bark of trees—anything to satisfy their hunger.
In their weakness they were attacked by plague, and hundreds perished long before the Hellespont was in sight. Even when at length the gleam of water gladdened the hearts of the soldiers, they were soon stricken again with fear, for where was the bridge?
The Greeks had not outstripped them, so this was not their doing. A storm had destroyed the bridge. Weak and hungry as they were, the soldiers had to rebuild it before they could cross over to Asia, where food and shelter awaited them.
When the Greeks saw that the Persians were marching to the Hellespont, they were eager to follow them. But Themistocles persuaded them to go back to Athens to rebuild the city.
Then he sent yet another message to Xerxes, saying, 'Themistocles, the leader of the Athenians and the best and wisest of the Greeks, has out of goodwill to thee held back the allies from chasing thy ships and breaking up the bridge at the Hellespont. So go thy way in peace.'
Although Themistocles sent these proud words to the great king, he really believed it was wiser for the Greeks not to pursue the retreating army. But he also wished to make Xerxes his friend, so that if at any time he was ostracized by the Athenians, he would find a welcome at the Persian court.
Greece was full of rejoicing when she heard of the victory of Salamis. The generals of the different states met at Corinth to propose a reward for the bravest and wisest among themselves.
Each general wrote on a tablet the names of two whom he believed to be worthy of a prize. They were not very modest, these brave soldiers of Greece, for each general wrote his own name first, though nearly all added beneath, the name of Themistocles.
The Spartans gave their meed of honour to the great Athenian, for a crown of olive was placed upon his head and he was presented with the most magnificent chariot that Sparta had ever produced.
Æschylus, one of the great Greek poets, wrote a tragedy on the fall of Xerxes, called The Persians, which was acted in 472 B.C., eight years after the battle of Salamis. Sculptors too wrought statues to commemorate the war, which were placed in the temple of Athene.


text:
The Lost Tomb 3: Ultimate Note
20 January 2021 DingooDramas Asian Dramas, Chinese Dramas 0
When Wu Xie (Zeng Shun Xi) receives a mysterious videotape, he, Xiao Ge (Xiao Yu Liang) and Pang Zi (Cheng Fang Xu) must embark on another quest, at the end of which may well lie the truth of Xiao Ge's missing memory.
If you disappeared from this world, I would notice.
Ultimate Note
I'll be honest, I want to scream a little. What a ride Ultimate Note was, what a ride. Before it came out, I really wasn't convinced by the cast and stills, but I'll freely admit that it hardly took a few episodes for me to get on board this adaptation, which I now rank as my absolute favorite amongst all the Lost Tomb series.
(If you're not sure what this series is about, check out my introductory guide right here!)
First and foremost, I still feel like this entire cast looks too young, but it's especially true for a few actors in particular. (Before I start, I'd like to make it clear that this section is nothing against the actors themselves, or their skills. I actually think they did a very good job, but they just look too young.)
Zeng Shun Xi
First, Zeng Shun Xi, who looks a smidge on the young side for 27-year-old Wu Xie. I think he lacks that maturity in his looks that Hou Ming Hao brought to the character in The Lost Tomb 2, but I ended up not minding too much simply because his acting was pretty good, especially in emotional scenes. And since Ultimate Note is the most emotional of the Lost Tomb installments, I'm convinced that he was a perfect fit for the role, so congratulations on (almost) making me change my mind.
He also had great chemistry with Xiao Yu Liang's Xiao Ge, who works perfectly as a follow-up to Cheng Yi's portrayal of the character. In The Lost Tomb 2, Xiao Ge was starting to be more human, to open himself up to his feelings, but he wasn't yet very good at expressing them, relying mostly on his eyes to convey his emotions. Cheng Yi, as we all know, excels at that sort of silent emoting, and didn't disappoint. In Ultimate Note, however, Xiao Ge grows to express himself a bit more, but also to participate in the Iron Triangle's shenanigans. Xiao Yu Liang, although a bit green at first, grew into the role quite well.
Liu Yu Han
Liu Yu Han as Xiao Hua didn't work for me either, because he looks far too boyish for a young man who's known to be ruthless and particularly brutal when it comes to protecting his family's interests. In The Lost Tomb 2, Liu Xue Yi was perfect for the role because he had this charismatic, imposing aura and arctic stare which did justify him having to wear soft colors so as not to look too intimidating.
If you tell me the guy on the left can and will murder me, I'll believe you. Say the same about the one on the right, and I'll laugh in your face.
Hankiz Omar
Another casting choice I had misgivings about is Hankiz Omar as A-Ning. It's nothing against her personally, really. She did a fine job acting, but she just looks far too small and delicate for a tough explorer. At one point, she said to Wu Xie "I'll pull you up", and I laughed to myself because there's just no way a slip of a girl like her can ever accomplish that, muscular or no.
Organic Relationships
I mentioned it above, Ultimate Note is the season which brings us the most emotion, and for good reason. For once, those relationships all felt real. Some characters, such as Wu San Xing, Wu Er Bai or Pan Zi, whom I used to hate with a passion for the very valid reason that they were absolute jerks to Wu Xie, whom they're supposed to care for, suddenly became much more than one-dimensional cryptic manipulators.
The human heart is more scary than god and ghost.
For once, I actually managed to feel their love for Wu Xie, instead of constant condescension, coercion and manipulation. It was quite a feat, because while I was indifferent to Wu San Xing in the Reboot, I positively hated him and his brother in all the others.
On a completely different note, the show also managed to establish HeiHua more firmly than all the other seasons combined. Where Xiao Hua and Hei Xiazi barely interacted at all before, we suddenly got a duo of competent men who banter and trust each other, equals in fighting prowess and strength of will. It was so much fun to see them interact throughout the show, to see their mutual respect build up and grow into a solid friendship.
All the Friendship
And speaking of, Ultimate Note also provided me with something I never knew I needed: all the friendship in the world. Between Wu Xie and Xiao Hua, Wu Xie and Pang Zi, the Mystic Nine heirs (Ultimate Note is actually the only installment which manages to make Huo Xiu Xiu somewhat interesting),… But the pair I loved especially was none other than Xiao Ge and Pang Zi. Most of the time, when the Iron Triangle are together, Xiao Ge stays focused on Wu Xie, who stares back in his usual lovestruck way. Which is fantastic, but ultimately leaves Pang Zi a bit on the sidelines.
But Ultimate Note corrected that, and it was glorious. For once, we were shown that their friendship and mutual understanding runs very deep as well, and that Xiao Ge does actually care a lot about Pang Zi (and vice versa).
Production-wise, Ultimate Note was nothing to write home about, either in good or bad terms. There were some really good instrumental tracks in the second part of the show, and the CGI was decent. I'm just not quite sure why they decided to have the story take place in 2004 when The Lost Tomb 2 clearly happens in 2019, but sure, why not. It wouldn't be The Lost Tomb if it made sense anyway.
From what I hear, the team was also very aware of what they could and could not do with the budget at their disposal, and elected to change creatures they knew they wouldn't be able to generate properly for more affordable ones. A strategy which worked perfectly: none of the monsters looked fake or horrible, and I thank them from the bottom of my heart for this excellent choice.
Another asset to the show were the beautiful illustrated vignettes which Ultimate Note used to portray their historical scenes instead of flashbacks.
What I get from Ultimate Note is love and respect for not only the story, but also the audience. This season is so carefully put together, faithful to the books (from what I understand), and does a fantastic job at laying the groundwork for the future of the Iron Triangle as seen in Tomb of the Sea and The Lost Tomb Reboot.
I didn't think anything could top The Lost Tomb 2 for me (mainly because I love Cheng Yi, the PingXie was out of this world, and the cast was my favorite), but Ultimate Note proved me wrong. It made up for a cast I didn't necessary like through good acting and stellar pacing, which the others dreadfully lacked. The screenwriters managed to pack three books in 36 episodes, refused to commit the sin of "the Xiao Ge waiting game", in one of my friends' words, and offered copious amounts of PingXie, HeiHua, and Iron Triangle feels on a silver platter.
Basically, they managed to give the fans what they wanted and what they needed, all in one fell swoop.
Some Xiao Ge screenshots, because I'm weak and we should promote Xiao Yu Liang.
Title: Ultimate Note
Starring: Zeng Shun Xi, Xiao Yu Liang, Cheng Fang Xu, Hankiz Omar
Number of episodes: 36 + 1 special
Genres: Action & Adventure, Mystery, Bromance, Supernatural, Friendship
Related Content: The Lost Tomb (prequel), The Lost Tomb 2 (prequel), Reunion: The Sound of the Providence (sequel), Voyage into the Lost Tomb series (Guide)
CDrama
Cheng Fang Xu
The Lost Tomb
The Lost Tomb 3
Xiao Yu Liang
All Aboard… Legend of Awakening
All Aboard… Word of Honor


Your racing suit source: Crow Enterprizes RJS Safequip and Pyrotect firesuits.
Auto Racing Gear by RJS Crow Racequip / Safequip Bell Pyrotect and more. Your source for helmet gear, racing shoes, gloves & racing suit accessories!
• Choice of 2" or 3"
See our NecksGen head and neck restraints for auto racing which is available in 3 sizes.
Racing gear needs vary, but regardless of your motorsport specialty, you love racing and we have the gear to keep you safe when things get dicey. You push hard and if they push back, you push back harder. That is why we are here, whether your thing is drag racing, sprint car, stock, dirt track, off-road, Jr. dragster, midget, tractor pull or any other form of motorsport we meet your needs for racing gear.
Everyone has different tastes, needs and applications for their racing gear. That's why we carry the best and safest available including, Bell Racing, Pyrotect, Crow Racing, RJS gear, Racequip, Safequip and more. Junior racers are constantly out growing their racing equipment and accessories and most importantly, you want to keep them safe. Kids will be kids. That's why we offer junior gear that is proudly made in the USA from manufacturers like RJS Racing, Crow Seat Belts and Safequip custom racing fire suits.
We carry all of the gear necessary to keep you competitive and safe. We have what it takes to get the job done like helmets, helmet accessories, racing gloves, racing shoes driving suits and more. You and your family deserve the best, so choose a custom racing suit, Nomex, Proban, Banox, SFI-1 thru SFI-20, helmet accessories, kids gear, seat belt harness 4 point, 5 point, 6 point, shoes, gloves, Nomex under gear, all of which is the highest quality gear.
It doesn't matter if you are a die-hard racer in one particular form of racing or even if you switch to a new type of motorsports, we have the stuff that you need. We even have Bell helmets, that are modular allowing you to have one helmet that with different kits, can morph to top-air, side-air, none or even both as well as a variety of aerodynamic kits, chin bar gurneys and top wicker gurneys or different sizes for different air speeds.
No matter where motorsports lead you, we are here for you and your family to keep you competitive, safe and of course, fast with outstanding customer service and the best racing gear!


Why are language skills so important when my child starts school?
How can I help my child develop his language skills?
How can I use books to encourage language skills?
How can singing and nursery rhymes help?
You want your child to be confident, happy and settled at school. You want him to make friends, get on with his teacher and have fun while he learns.
The secret weapon to making sure your child has the best possible opportunity to do all of this, is to help him develop his language and communication skills.
Being able to communicate confidently will help your child avoid becoming frustrated or upset. If he is able to ask for a turn on the sand table he's much less likely to throw a tantrum because someone else got there first! It's important for your child to be able to express his needs and feelings and ask for help when necessary.
Language lets your child join in. Reciting nursery rhymes, singing songs, playing games and listening to and discussing stories are all ways in which he'll be able to take part in his new life at school.
Good language skills will also help your little one to make new friends and build relationships. When he's away from you during the day, it helps to know that he has people he can trust and turn to. Being able to communicate confidently with his friends and his teacher makes the world less scary when mum's not around.
Get your child's attention. Sometimes parents think their children are ignoring them, or even have hearing problems, simply because they don't look up when their parents speak to them. The truth is that your child is probably just engrossed in what he's doing. It helps to start with his name at the beginning of what you say to him so that he tunes into you straight away.
Give your child time to speak. He'll need time to process what you've just said and find the words that he needs. You can help by showing that you're happy to wait and listen by looking at him and modding your head to show that you understand and that you're interested in what he has to say.
Always name the object. Try to name whatever your child plays with or shows an interest in. This helps him build his vocabulary and lets him express what he wants without getting frustrated.
Make talking fun. Turn language into games, like "I spy" or "Simon says". Imagine a different ending to a favourite story or make up characters of your own and give them lots of adventures.
Do it every day. Talk with your child as much as possible each day, whether you're eating dinner, driving in the car or walking to the park. When you're out and about, talk to him about what you can both see or what you're going to do. Your little one will hear a much wider range of words and have more opportunities to speak if he's interacting with you and others.
Stories are a fun way for your child to make sense of his world, learn new words and develop his concentration. Sharing books together encourages closeness so it's the ideal opportunity to share a cuddle at the same time.
Your bedtime story routine, whether it's one book or three, is the perfect time to snuggle up and read together. Whether your child prefers a fact-based book about dinosaurs or a vivid fairy tale, a wide range of books to choose from is a good idea, so neither of you gets bored!
For more active story-telling, use puppets or props to add another dimension. For example, you could make a fruit salad from some of the food in The Very Hungry Caterpillar or porridge to go with the story of Goldilocks and the Three Bears. Your child may love to dress up as a character from one of his stories. Role play is a great way to explore and develop language, especially if you get involved too!
Nursery rhymes are fun and easy to learn. They help your little one broaden his vocabulary and understand rhythm and rhyming. Number rhymes are especially useful in helping your child to learn to count. Children often know the sequence of numbers before they've developed a real understanding of them. If your child can say numbers in the right sequence, it will really help when he comes to count real objects.
Not all of us can remember rhymes from our own childhoods. If you're struggling to recall the words or the tune, pop along to your local children's centre or library. Most run "rhyme-time" sessions for parents and children on a weekly basis. Or check out some of our lullabies and nursery rhymes for inspiration. Whatever your level of music skill, never feel embarrassed to sing out loud! Your child will love to sing with you and will quickly become your biggest fan!
For more support and advice to give your child a happy start to Reception, visit readyforschool.co.uk - a new school-readiness app for you and your child, launching in autumn 2015.


The Sunice quarter-zip pullover is the ideal performance layer you will need on those cool and windy days. A stylish addition to any ladies wardrobe, whether on or off the course.
* SuperliteFX technology provides the ultimate stretch comfort, but allows the body to be kept at the right temperature.