text:
\section{Introduction}
	%
	Anticipating the near future is a natural task for humans and a fundamental one for intelligent systems when it is necessary to react before an action is completed (e.g., to anticipate a pedestrian crossing the street from an autonomous vehicle~\cite{de2016online}) or even before it starts (e.g., to notify a user who is performing the wrong action in a known workflow~\cite{soran2015generating}).
	%
	Additionally, tasks such as action anticipation~\cite{gao2017red,koppula2016anticipating,vondrick2016anticipating} and early action recognition~\cite{aliakbarian2017encouraging,de2016online,ma2016learning} pose a series of key challenges from a computational perspective. Indeed, methods addressing these tasks need to model the relationships between past, future events and incomplete observations. First Person (Egocentric) Vision~\cite{kanade2012first}, in particular, offers an interesting scenario to study anticipation problems. On one hand, whilst being a natural task for humans, anticipating the future from egocentric video is computationally challenging due to the ability of wearable cameras to acquire long videos of complex activities involving many objects and actions performed by a user from their unique point of view. On the other hand, investigating these tasks is fundamental for the construction of intelligent wearable systems able to anticipate the user's goal and assist them~\cite{kanade2012first}.

	%

	%

	In this paper, we address the problem of egocentric action anticipation.
	\begin{figure}
		\includegraphics[width=\linewidth]{figures/anticipation}
		\caption{Egocentric Action Anticipation. See text for notation.}
		\label{fig:anticipation}
		\vspace{-\topsep}
	\end{figure}
	As defined in~\cite{Damen2018EPICKITCHENS} and illustrated in \figurename~\ref{fig:anticipation}, the task consists in recognizing an action starting at time $\tau_s$ by observing a video segment \textit{preceding the action} starting at time $\tau_s-(\tau_o+\tau_a)$ and ending at time $\tau_s-\tau_a$, where the ``observation time'' $\tau_o$ indicates the length of the observed segment, whereas the ``anticipation time'' $\tau_a$ denotes how many seconds in advance actions are to be anticipated.
	%
	While action anticipation has been investigated in classic third person vision~\cite{abu2018will,gao2017red,huang2014action,koppula2016anticipating,vondrick2016anticipating,jain2016recurrent}, less attention has been devoted to the egocentric scenario~\cite{Damen2018EPICKITCHENS,furnari2018Leveraging,Ryoo2015a}. %

	%
	We observe that egocentric action anticipation methods need to address two sub-tasks: 1)~summarizing what has been observed in the past (e.g., ``a container has been washed'' in the observed segment in \figurename~\ref{fig:anticipation}), and 2)~making hypotheses about what will happen in the future (e.g., ``put-down container'', ``close tap'', ``take spoon'' in \figurename~\ref{fig:anticipation}). While previous approaches attempted to address these two sub-tasks jointly~\cite{aliakbarian2017encouraging,Damen2018EPICKITCHENS,ma2016learning,vondrick2016anticipating}, our method disentangles them using two separate LSTMs. The ``Rolling'' LSTM (R-LSTM) is responsible for continuously encoding streaming observations to summarize the past. When the method is required to anticipate actions, the ``Unrolling'' LSTM (U-LSTM) takes over the current hidden and cell states of the R-LSTM and makes predictions about the future.
	%
	Differently from previous works which considered a fixed anticipation time~\cite{Damen2018EPICKITCHENS,furnari2018Leveraging,vondrick2016anticipating}, the proposed method can anticipate an action at multiple anticipation times, e.g., $2s$, $1.5s$, $1s$ and $0.5s$ before it occurs. The network is pre-trained with a novel ``Sequence Completion Pre-training'' (SCP) technique, which encourages the disentanglement of the two sub-tasks. To take advantage of the complementary nature of different input modalities, the proposed Rolling-Unrolling LSTM (RU) processes spatial observations (RGB frames), motion (optical flow), as well as object-based features.
	%
	Multimodal predictions are fused with a novel ``Modality ATTention'' mechanism (MATT), which adaptively estimates optimal fusion weights for each modality by considering the outputs of the modality-specific R-LSTM components.
	%
	Experiments on two large-scale datasets of egocentric videos, EPIC-KTICHENS~\cite{Damen2018EPICKITCHENS} and EGTEA Gaze+~\cite{Li_2018_ECCV}, show that the proposed method outperforms several state-of-the-art approaches and baselines in the task of egocentric action anticipation and generalizes to the tasks of early action recognition and action recognition.
	%

	The contributions of this work are as follows: 1) we are the first to systematically investigate the problem of egocentric action anticipation within the framework of the challenge proposed in~\cite{Damen2018EPICKITCHENS}; 2) our investigation benchmarks popular ideas and approaches to action anticipation and leads to the definition of RU, an architecture able to anticipate egocentric actions at multiple temporal scales; 3) the proposed model is shown to benefit from two techniques specific to the investigated problem, i.e., i) ``Sequence Completion Pre-training'' and ii) adaptive fusion of multi-modal predictions through Modality ATTention; 4) extensive evaluations highlight the limits of previous approaches and show significant improvements of the proposed method over the state of the art. To support future research, the code implementing the proposed method will be released upon publication.

	%
	%
	%
	%
	%
	%
	%
	%

	%

	%
	%
	\section{Related Work}
	\noindent
	\textbf{Action Recognition\hspace{0.5em}} Our work is related to previous research on action recognition from third person vision~\cite{carreira2017quo,feichtenhofer2017spatiotemporal,feichtenhofer2016convolutional,karpathy2014large,laptev2005space,laptev2008learning,simonyan2014two,tran2015learning,wang2013dense,wang2013action,wang2016temporal,zhou2018temporal} and first person vision~\cite{fathi2011understanding,fathi2012learning,Li_2018_ECCV,li2015delving,ma2016going,pirsiavash2012detecting,ryoo2015pooled,singh2017trajectory,spriggs2009temporal,sudhakaran2018lsta,sudhakaran2018attention}. Specifically, we build on previous ideas investigated in the context of action recognition such as the use of multiple modalities for video analysis~\cite{simonyan2014two}, the use of Temporal Segment Networks~\cite{wang2016temporal} as a principled way to train CNNs for action recognition, as well as the explicit encoding of object-based features~\cite{fathi2011understanding,ma2016going,pirsiavash2012detecting,singh2016first,sudhakaran2018attention} to analyze egocentric video. However, in contrast with the aforementioned works, we address the problem of egocentric action \textit{anticipation} and show that approaches designed for action recognition, such as TSN~\cite{wang2016temporal} and early/late fusion to merge multi-modal predictions~\cite{simonyan2014two} are not directly applicable to the problem of egocentric action anticipation.

	\vspace{0.5\topsep}
	\noindent
	\textbf{Early Action Recognition in Third Person Vision\hspace{0.5em}}
	Early action recognition consists in recognizing an ongoing action as early as possible from partial observations~\cite{de2016online}. The problem has been widely investigated in the domain of third person vision~\cite{aliakbarian2017encouraging,becattini2017done,cao2013recognize,de2016online,de2018modeling,hoai2014max,huang2014sequential,ma2016learning,ryoo2011human}. Differently from these works, we address the task of \textit{anticipating} actions from egocentric video, i.e., the action should be recognized before it starts, hence it cannot even be partially observed at the time of prediction. Given the similarity between early recognition and anticipation, we consider and evaluate some ideas investigated in the context of early action recognition, such as the use of LSTMs to process streaming video~\cite{aliakbarian2017encouraging,de2018modeling,ma2016learning}, and the use of dedicated loss functions~\cite{ma2016learning}. Moreover, we show that the proposed architecture also generalizes to the problem of early egocentric action recognition, achieving state-of-the-art performances.

	\vspace{0.5\topsep}
	\noindent
	\textbf{Action Anticipation in Third Person Vision\hspace{0.5em}}
	Action anticipation is the task of predicting an action \emph{before} it occurs~\cite{gao2017red}. Previous works investigated different forms of action and activity anticipation from third person video~\cite{abu2018will,felsen2017will,gao2017red,huang2014action,jain2015car,jain2016recurrent,kitani2012activity,koppula2016anticipating,lan2014hierarchical,mahmud2017joint,vondrick2016anticipating,zeng2017visual}. While we consider the problem of action anticipation from egocentric visual data, our work builds on some ideas explored in past works such as the use of LSTMs to anticipate actions~\cite{abu2018will,gao2017red,jain2016recurrent}, the use of the encoder-decoder framework to encode past observations and produce hypotheses of future actions~\cite{gao2017red}, and the use of object specific features~\cite{mahmud2017joint} to determine which objects are present in the scene. Additionally, we show that other approaches, such as the direct regression of future representations~\cite{gao2017red,vondrick2016anticipating}, do not achieve satisfactory performance in the considered scenario.

	\vspace{0.5\topsep}
	\noindent
	\textbf{Anticipation in First Person Vision\hspace{0.5em}}
	%
	%
	%
	Past works on anticipation from egocentric visual data have investigated different problems and considered different evaluation frameworks~\cite{bokhari2016long,chenyou2017forecasting,Furnari2017,SooPark2016,rhinehart2017first,Ryoo2015a,singh2016krishnacam,soran2015generating,zhang2017deep,Zhou2015}. Instead, we tackle the egocentric action anticipation challenge recently proposed in~\cite{Damen2018EPICKITCHENS}, which has been little investigated so far~\cite{furnari2018Leveraging}. While a direct comparison of the proposed approach with the aforementioned works is unfeasible due to the lack of a common framework, our method incorporates some ideas from past approaches, such as the analysis of past actions~\cite{Ryoo2015a} and the detection of the objects present in the scene to infer future actions~\cite{Furnari2017}.

	%
	%
	\begin{figure}
		\includegraphics[width=\linewidth]{figures/processing}
		\caption{Video processing scheme of the proposed method with $S_{enc}=6$ and $S_{ant}=8$.}
		\label{fig:processing}
		\vspace{-\topsep}
	\end{figure}
	\section{Proposed Approach}
	\label{sec:method}

	\noindent
	\textbf{Processing Strategy \hspace{0.5em}}
	\figurename~\ref{fig:processing} illustrates the processing strategy adopted by the proposed method. The video is processed in an on-line fashion, with a short video snippet $V_t$ consumed every $\alpha$ seconds, where $t$ indexes the current time-step. Specifically, an action occurring at time $\tau_s$ is anticipated by processing a video segment of length $l$ starting at time $\tau_s-l-\alpha$ and ending at time $\tau_s-\alpha$. The input video ends at time $\tau_s-\alpha$ as our method aims at anticipating actions \textit{at least} $\alpha$ seconds before they occur. The processing is performed in two stages: an ``encoding'' stage, which is carried out for $S_{enc}$ time-steps, and an ``anticipation'' stage, which is carried out for $S_{ant}$ time-steps. In the encoding stage, the model summarizes the semantic content of the $S_{enc}$ input video snippets without producing any prediction, whereas in the anticipation stage the model continues to encode the semantics of the $S_{ant}$ input video snippets and outputs $S_{ant}$ action scores $s_t$ which can be used to perform action anticipation. This scheme effectively allows to formulate $S_{ant}$ predictions for a single action at multiple anticipation times.
	%
	In our experiments, we set $\alpha=0.25s$, $S_{enc}=6$ and $S_{ant}=8$. In these settings, the model analyzes video segments of length $l=\alpha(S_{enc}+S_{ant})=3.5s$ and outputs $8$ predictions at the following anticipation times: $\tau_a \in \{2s, 1.75s, 1.5s, 1.25s, 1s, 0.75s, 0.5s, 0.25s\}$. It should be noted that, since the predictions are produced \textit{while} processing the video, at time step $t$ the effective observation time will be equal to $\alpha \cdot t$. Hence, the $8$ predictions are performed at the following effective observation times: $\tau_o \in \{1.75s, 2s, 2.25s, 2.5s, 2.75s, 3s, 3.25s, 3.5s\}$. Our formulation generalizes the one proposed in~\cite{Damen2018EPICKITCHENS}, which is illustrated in~\figurename~\ref{fig:anticipation}. For instance, at time-step $t=11$, our model will anticipate actions with an effective observation time equal to $\tau_o=\alpha \cdot t=2.75s$ and an anticipation time equal to $\tau_a=\alpha(S_{ant}+S_{enc}+1-t)=1s$, as in~\cite{Damen2018EPICKITCHENS}.

	\vspace{\topsep}
	\noindent
	\textbf{Rolling-Unrolling LSTM\hspace{0.5em}}
	The proposed method uses two separate LSTMs to encode past observations and formulate predictions about the future. Following previous literature~\cite{simonyan2014two}, we include multiple identical branches which analyze the video according to different modalities. Specifically, at each time-step $t$, the input video snippet $V_t$ is represented using different modality-specific representation functions $\varphi_1,\ldots,\varphi_M$ depending on learnable parameters $\theta^{\varphi_1},\ldots,\theta^{\varphi_M}$. This process allows to obtain the modality-specific feature vectors $f_{1,t}=\varphi_1(V_t),\ldots,f_{M,t}=\varphi_M(V_t)$, where $M$ is the total number of modalities (i.e., the total number of branches in our architecture), and $f_{m,t}$ is the feature vector computed at time-step $t$ for the modality $m$. The feature vector $f_{m,t}$ is fed to the $m^{th}$ branch of the architecture. While our model can easily incorporate different modalities, in this work we consider $M=3$ modalities, i.e., RGB frames (spatial branch), optical flow (motion branch) and object-based features (object branch).
	%

	\begin{figure}
		\includegraphics[width=\linewidth]{figures/branch}
		\caption{Example of RU modality-specific branch with $S_{enc}=1$ and $S_{ant}=3$.}
		\label{fig:branch}
		\vspace{-\topsep}
	\end{figure}

	\figurename~\ref{fig:branch} illustrates the processing taking place in a single branch $m$ of the proposed RU model. For illustration purposes only, the figure shows an example in which $S_{enc}=1$ and $S_{ant}=3$. At time step $t$, the feature vector $f_{m,t}$ is fed to the Rolling LSTM (R-LSTM), which encodes its semantic content recursively, as follows:
	\begin{equation}\small
	(h_{m,t}^R,c_{m,t}^R)=LSTM_{\theta_m^R}(f_{m,t},h_{m,t-1}^R,c_{m,t-1}^R)
	\end{equation}
	where $LSTM_{\theta_m^R}$ denotes the R-LSTM of branch $m$, depending on the learnable parameters $\theta_m^R$, whereas $h^R_{m,t}$ and $c^R_{m,t}$ are the hidden and cell states computed at time $t$ in the modality $m$. The initial hidden and cell states of the R-LSTM are initialized with zeros: $h^R_{m,0}=\textbf{0},\ c^R_{m,0}=\textbf{0}$.

	In the anticipation stage, at time step $t$, the Unrolling LSTM (U-LSTM) is used to make predictions about the future. The U-LSTM takes over the hidden and cell vectors of the R-LSTM at the current time-step (i.e., $h_{m,t}^R$ and $c_{m,t}^R$) and iterates over the representation of the current video snippet $f_{m,t}$ for a number of times $n_t$ equal to the number of time-steps required to reach the beginning of the action, i.e., $n_t=S_{ant}+S_{enc}-t+1$. Hidden and cell states of the U-LSTM are computed as follows at the $j^{th}$ iteration:
	\begin{equation}\small
	(h_{m,j}^U,c_{m,j}^U)=LSTM_{\theta_m^U}(f_{m,t},h_{m,j-1}^U,c_{m,j-1}^U)
	\label{eq:ulstm}
	\end{equation}
	where $LSTM_{\theta_m^U}$ is the U-LSTM of branch $m$, depending on the learnable parameters $\theta_m^U$, and $h^U_{m,t}$, $c^U_{m,t}$ are the hidden and cell states computed at iteration $j$ for the modality $m$. The initial hidden and cell states of the U-LSTM are initialized from the current hidden and cell states computed by the R-LSTM: $h^U_{m,0}=h^R_{m,t},\ c^U_{m,0}=c^R_{m,t}$. Note that the input $f_{m,t}$ of the U-LSTM does not depend on $j$ (see eq.~\eqref{eq:ulstm}) because it is fixed during the ``unrolling'' procedure. The main rationale of ``unrolling'' the U-LSTM for a different number of times at each time-step is to encourage it to differentiate predictions at different anticipation times.

	Modality-specific action scores $s_{m,t}$ are computed at time-step $t$ by processing the last hidden vector of the U-LSTM with a linear transformation with learnable parameters $\theta^W_m$ and $\theta^b_m$: $s_{m,t}=\theta^W_m h^U_{m,n_t}+\theta^b_m.$
	%
	%
	%
	%
	%
	%
	%
	%
	%

	\begin{figure}
		\includegraphics[width=\linewidth]{figures/sequence_completion}
		\caption{Example of connection scheme used during SCP for time-step $t=2$.}
		\label{fig:sequence_completion}
		\vspace{-\topsep}
	\end{figure}

	\vspace{\topsep}
	\noindent
	\textbf{Sequence Completion Pre-Training (SCP)\hspace{0.5em}}
	The two LSTMs composing the RU architecture are designed to address two specific sub-tasks: the R-LSTM is responsible for encoding past observations and summarizing what has happened up to a given time-step, whereas the U-LSTM focuses on anticipating future actions conditioned on the hidden and cell vectors of the R-LSTM. To encourage the two LSTMs to specialize on the two different sub-tasks, we propose to train the architecture using a novel Sequence Completion Pre-training (SCP) procedure. During SCP, the connections of the U-LSTM are modified to allow it to process future representations, rather than iterating on the current one. In practice, the U-LSTM hidden and cell states are computed as follows during SCP:
	\begin{equation}\small
	(h_{m,j}^U,c_{m,j}^U)=LSTM_{\theta_m^U}(f_{m,t+j-1},h_{m,j-1}^U,c_{m,j-1}^U)
	\end{equation}
	where the input representations $f_{m,t+j-1}$ are sampled from future time-steps $t+j-1$. \figurename~\ref{fig:sequence_completion} illustrates an example of the connection scheme used during SCP for time-step $t=2$.
	The main goal of pre-training the RU with SCP is to allow the R-LSTM to focus on summarizing past representations without trying to anticipate the future.
	%

	\begin{figure}
		\includegraphics[width=\linewidth]{figures/matt}
		\caption{Example of the complete RU architecture with two modalities and the Modality ATTention mechanism (MATT).}
		\label{fig:matt}
		\vspace{-\topsep}
	\end{figure}

	\vspace{\topsep}
	\noindent
	\textbf{Modality ATTention (MATT)\hspace{0.5em}}
	Coherently with past work on egocentric action anticipation~\cite{Damen2018EPICKITCHENS}, we found it sub-optimal to fuse multi-modal predictions with classic approaches such as early and late fusion. This is probably due to the fact that, when anticipating egocentric actions, one modality might be more useful than another (e.g., appearance over motion), depending on the processed sample. Inspired by previous work on attention~\cite{bahdanau2014neural,xu2015show} and multi-modal fusion~\cite{mees16iros}, we introduce a Modality ATTention (MATT) module which computes a set of attention scores indicating the relative importance of each modality for the final prediction. At a given time-step $t$, such scores are obtained by processing the concatenation of the hidden and cell vectors of the R-LSTM networks belonging to all branches $m=1,\ldots,M$ with a deep neural network $D$ depending on the learnable parameters $\theta^{MATT}$:
	\begin{equation}\small
	\lambda_t=D_{\theta^{MATT}}(\oplus_{m=1}^M(h_{m,t}^R \oplus c_{m,t}^R))
	\end{equation}
	where $\oplus$ denotes the concatenation operator and $\oplus_{m=1}^M(h_{m,t}^R \oplus c_{m,t}^U)$ is the concatenation of the hidden and cell vectors produced by the R-LSTM at time-step $t$ across all modalities. Late fusion weights can be obtained normalizing the score vector $\lambda_t$ with the softmax function in order to make sure that fusion weights sum to one: $
	w_{m,t}=\frac{exp(\lambda_{t,m})}{\sum_k exp(\lambda_{t,k})}
	$,
	where $\lambda_{t,m}$ is the $m^{th}$ component of the score vector $\lambda_t$. The final set of fusion weights is obtained at time-step $t$ by merging the modality-specific predictions produced by the different branches with a linear combination as follows: $s_t=\sum_m w_{m,t} \cdot s_{m,t}$.
	%
	%
	%
	%
	%
	%
	%
	%
	%
	\figurename~\ref{fig:matt} illustrates an example of a complete RU with two modalities and the MATT fusion mechanism. For illustration purposes, the figure shows only three anticipation steps.

	\vspace{\topsep}
	\noindent
	\textbf{Branches and Representation Functions\hspace{0.5em}}
	We instantiate the proposed architecture with $3$ branches: a spatial branch which processes RGB frames, a motion branch which processes optical flow, and an object branch which processes object-based features. Our architecture analyzes video snippets of $5$ frames $V_t=\{I_{t,1},I_{t,2},\ldots,I_{t,5}\}$, where $I_{t,i}$ is the $i^{th}$ frame of the video snippet. The representation function $\varphi_1$ of the spatial branch computes the feature vector $f_{1,t}$ by extracting features from the last frame $I_{t,5}$ of the video snippet using a Batch Normalized Inception CNN~\cite{ioffe2015batch} trained for action recognition. The representation function $\varphi_2$ of the motion branch extracts optical flow from the $5$ frames of the current video snippet as proposed in~\cite{wang2016temporal}. The computed $x$ and $y$ optical flow form a tensor with $10$ channels, which is fed to a Batch Normalized Inception CNN trained for action recognition to obtain the feature vector $f_{2,t}$. Note that $\varphi_1$ and $\varphi_2$ allow to obtain ``action-centric'' representations of the input frame which can be used by the R-LSTM to summarize what has happened in the past. The representation function $\varphi_3$ of the object branch extracts objects from the last frame $I_{t,5}$ of the input snippet $V_t$ using an object detector. A fixed-length representation $f_{3,t}$ is obtained by accumulating the confidence scores of all bounding boxes predicted for each object class. Specifically, let $b_{t,i}$ be the $i^{th}$ bounding box detected in image $I_{t,5}$, let $b^c_{t,i}$ be its class and let $b^s_{t,i}$ be its detection confidence score. The $j^{th}$ component of the output representation vector $f_{3,t}$ is obtained by summing the confidence scores of all detected objects of class $j$, i.e., $f_{3,t,j}=\sum_i [b^c_{t,i}=j]b^s_{t,i}$, where $[\cdot]$ denotes the Iverson bracket. This representation only encodes the presence of an object in the scene, discarding its position in the frame, similarly to the representation proposed in~\cite{pirsiavash2012detecting} for egocentric activity recognition. We found this holistic representation to be sufficient in the case of egocentric action anticipation. Differently from $\varphi_1$ and $\varphi_2$, $\varphi_3$ produces object-centric features, which carry information on what objects are present in the scene and hence could be interacted next.\footnote{See supp. for details on implementation and training of our method.}
	%

	\vspace{-\topsep}
	\section{Experimental Settings}
	%

	%
	\noindent
	\textbf{Datasets\hspace{0.5em}}
	We perform experiments on two large-scale datasets of egocentric videos: EPIC-Kitchens~\cite{Damen2018EPICKITCHENS} and EGTEA Gaze+~\cite{Li_2018_ECCV}. EPIC-Kitchens contains $39,596$ action annotations, $125$ verbs, and $352$ nouns. We split the public training set of EPIC-Kitchens ($28,472$ action segments) into training ($23,493$ segments) and validation ($4,979$ segments) sets by randomly choosing $232$ videos for training and $40$ videos for validation. We considered all unique $(verb, noun)$ pairs in the public training set, thus obtaining $2,513$ unique actions. EGTEA Gaze+ contains $10,325$ action annotations, $19$ verbs, $51$ nouns and $106$ unique actions. Methods are evaluated on EGTEA Gaze+ reporting the average performance across the three splits provided by the authors of the dataset~\cite{Li_2018_ECCV}.

	\vspace{\topsep}
	\noindent
	\textbf{Evaluation Measures\hspace{0.5em}}
	Methods are evaluated using Top-k evaluation measures, i.e., we deem a prediction correct if the ground truth action falls in the top-k predictions. As observed in previous works~\cite{furnari2018Leveraging,koppula2016anticipating}, this evaluation scheme is appropriate given the uncertainty of future predictions (i.e., many plausible actions can follow an observation).
	Specifically, we use the Top-5 accuracy as a class-agnostic measure and Mean Top-5 Recall as a class aware metric. Top-5 recall~\cite{furnari2018Leveraging} for a given class $c$ is defined as the fraction of samples of ground truth class $c$ for which the class $c$ is in the list of the top-5 anticipated actions. Mean Top-5 Recall averages Top-5 recall values over classes. When evaluating on EPIC-Kitchens, Top-5 Recall is averaged over the provided list of many-shot verbs, nouns and actions. Results on the EPIC-Kitchens official test set are reported using the suggested evaluation measures, i.e., Top-1 accuracy, Top-5 accuracy, Precision and Recall.

	To assess the timeliness of anticipations, we design a new evaluation measure inspired by the AMOC curve~\cite{hoai2014max}. Let $s_t$ be the score anticipated at time-step $t$ for an action of ground truth class $c$, let $\tau_t$ be the anticipation time at time-step $t$, and let $tk(s_t)$ be the set of top-$k$ actions as ranked by the scores $s_t$. We define as ``time to action'' at rank $k$ the largest anticipation time (i.e., the time of earliest anticipation) in which a correct prediction has been made according to the top-$k$ criterion: $TtA(k)=max\{\tau(s_t) |c \in tk(s_t),\ \forall t\}$.
	%
	If an action is not correctly anticipated in any of the time-steps, we set $TtA(k)=0$.
	%
	The mean time to action over the whole test set $mTtA(k)$ indicates how early, in average, a method can anticipate actions.

	Performances are evaluated for verb, noun and action predictions on the EPIC-Kitchens dataset. We obtain verb and noun scores by marginalization over action scores for all methods except the one proposed in~\cite{Damen2018EPICKITCHENS}, which predicts verb and noun scores directly.

	\vspace{\topsep}
	\noindent
	\textbf{Compared Methods\hspace{0.5em}}
	We compare the proposed method with respect to $7$ state-of-the approaches and baselines. Specifically, we consider the Deep Multimodal Regressor (DMR) proposed in~\cite{vondrick2016anticipating}, the Anticipation Temporal Segment Network (ATSN) of~\cite{Damen2018EPICKITCHENS}, the anticipation Temporal Segment Network trained with verb-noun Marginal Cross Entropy Loss (MCE) described in~\cite{furnari2018Leveraging}, and the Encoder-Decoder LSTM (ED) introduced in~\cite{gao2017red}. We also consider baselines obtained adapting approaches proposed for early action recognition to the problem of egocentric action anticipation: the Feedback Network LSTM (FN) proposed in~\cite{de2018modeling}, a single LSTM architecture (we use the same parameters of our R-LSTM) trained using the Ranking Loss on Detection Score proposed in~\cite{ma2016learning} (RL), and an LSTM trained using the Exponential Anticipation Loss proposed in~\cite{jain2016recurrent} (EL). These baselines adopt the video processing scheme illustrated in \figurename~\ref{fig:processing} and are implemented as two stream networks with a spatial and a temporal branch whose predictions are fused by late fusion.\footnote{See supp. for implementation details of the considered methods.}

	%
	%
	\begin{table*}
		\begin{adjustbox}{width=\linewidth,center}
			\begin{tabular}{p{1.2cm}cccccccc||ccc|ccc|ccc}
				\hline
				\multicolumn{1}{c}{} & \multicolumn{8}{c||}{Top-5 ACTION Accuracy\% @ different $\tau_a$(s)} & \multicolumn{3}{c|}{Top-5 Acc.\% @1s} & \multicolumn{3}{c|}{M. Top-5 Rec.\% @1s} & \multicolumn{3}{c}{Mean $TtA(5)$}   \\  \hline %
				\multicolumn{1}{c}{} & $2$ & $1.75$ & $1.5$ & $1.25$ & $1.0$& $0.75$&$0.5$&$0.25$ & VERB & NOUN & ACT. & VERB & NOUN & ACT. & VERB & NOUN & ACT.\\ \hline
				DMR~\cite{vondrick2016anticipating} & /              & /              & /              & /              & 16.86          & /              & /              & /              & 73.66          & 29.99          & 16.86          & 24.50          & 20.89          & 03.23          & /              & /              & /              \\
				ATSN~\cite{Damen2018EPICKITCHENS}  & /              & /              & /              & /              & 16.29          & /              & /              & /              & \underline{77.30} & 39.93          & 16.29          & 33.08          & 32.77          & 07.60          & /              & /              & /              \\
				MCE~\cite{furnari2018Leveraging}  & /              & /              & /              & /              & 26.11          & /              & /              & /              & 73.35          & 38.86          & 26.11          & 34.62          & 32.59          & 06.50          & /              & /              & /              \\
				ED~\cite{gao2017red}           & 21.53          & 22.22          & 23.20          & 24.78          & 25.75          & 26.69          & 27.66          & 29.74          & 75.46          & 42.96          & 25.75          & \underline{41.77}          & \underline{42.59}          & \underline{10.97}          & \underline{01.60} & \underline{01.02} & \underline{00.63} \\
				FN~\cite{de2018modeling}       & 23.47          & 24.07          & 24.68          & 25.66          & 26.27          & 26.87          & 27.88          & 28.96          & 74.84          & 40.87          & 26.27          & 35.30          & 37.77          & 06.64          & 01.52          & 00.86          & 00.56          \\
				RL~\cite{ma2016learning}       & \underline{25.95} & \underline{26.49} & \underline{27.15} & \underline{28.48} & \underline{29.61} & \underline{30.81} & \underline{31.86} & 32.84          & 76.79          & \underline{44.53} & \underline{29.61} & 40.80 & 40.87 & 10.64 & 01.57          & 00.94          & 00.62          \\
				EL~\cite{jain2016recurrent}    & 24.68          & 25.68          & 26.41          & 27.35          & 28.56          & 30.27          & 31.50          & \underline{33.55} & 75.66          & 43.72          & 28.56          & 38.70          & 40.32          & 08.62          & 01.55          & 00.94          & 00.62          \\
				\textbf{RU}                             & \textbf{29.44} & \textbf{30.73} & \textbf{32.24} & \textbf{33.41} & \textbf{35.32} & \textbf{36.34} & \textbf{37.37} & \textbf{38.98} & \textbf{79.55} & \textbf{51.79} & \textbf{35.32} & \textbf{43.72} & \textbf{49.90} & \textbf{15.10} & \textbf{01.62} & \textbf{01.11} & \textbf{00.76} \\ \hline
				Improv.                                 & +3.49          & +4.24          & +5.09          & +4.93          & +5.71          & +5.53          & +5.51          & +5.43          & +2.25          & +7.26          & +5.71          & +1.95          & +7.31          & +4.13          & +0.02          & +0.09          & +0.13          \\ \hline
			\end{tabular}
		\end{adjustbox}
		\caption{Egocentric action anticipation results on the EPIC-KITCHENS dataset}
		\label{tab:anticipation_ek}
		\vspace{-\topsep}
	\end{table*}

	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%

	\section{Results}
	\vspace{-0.5\topsep}
	\noindent
	\textbf{Anticipation Results on EPIC-KITCHES\hspace{0.5em}} \tablename~\ref{tab:anticipation_ek} compares RU with respect to the state-of-the-art on our EPIC-Kitchens validation set. The left part of the table reports Top-5 action accuracy for the $8$ anticipation times. Note that some methods~\cite{Damen2018EPICKITCHENS,furnari2018Leveraging,vondrick2016anticipating} can anticipate actions only at a fixed anticipation time. The right part of the table breaks down Top-5 accuracy and Mean Top-5 Recall for verbs, nouns and actions, for anticipation time $\tau_a=1s$, and reports mean $TtA(5)$ scores. Best results are in bold, whereas second-best results are underlined. The last row reports the improvement of RU with respect to second best results.

	%
	The proposed approach outperforms all competitors by consistent margins according to all evaluation measures, reporting an average improvement over prior art of about $5\%$ on Top-5 action accuracy with respect to all anticipation times. Note that this margin is significant given the large number of $2,513$ action classes present in the dataset.
	Methods based on TSN (ATSN and MCE) generally achieve low performance, which suggests that simply adapting action recognition methods to the problem of anticipation is insufficient.
	Interestingly, DMR and ED, which are explicitly trained to anticipate future representations, achieve sub-optimal Top-5 action accuracy as compared to methods trained to predict future actions directly from input images (e.g., compare DMR with MCE, and ED with FN/RL/EL). This might be due to the fact that anticipating future representations is particularly challenging in the case of egocentric video, where the visual content changes continuously because of the mobility of the camera.
	RL consistently achieves second best results with respect to all anticipation times, except $\tau_a=0.25$, where it is outperformed by EL.
	RU is particularly strong on nouns, achieving a Top-5 noun accuracy of $51.79\%$ and a mean Top-5 noun recall of $49.90\%$, which improves over prior art by $+7.26\%$ and $+7.31\%$. The small drop in performance when passing from class-agnostic measures to class-aware measures (i.e., $51.79\%$ to $49.90\%$) suggests that our method does not over-fit to the distribution of nouns of the training set. It is worth noting that mean Top-5 Recall values are averaged over fairly large sets of $26$ many-shot verbs, $71$ many-shot nouns, and $819$ many-shot actions, as specified in~\cite{Damen2018EPICKITCHENS}. Differently, all compared methods exhibit large drops in verb and action performance when passing from class-agnostic to class-aware measures. Our insight into this different pattern is that anticipating the next object which will be used (i.e., anticipating nouns) is much less ambiguous than anticipating the way in which the object will be used (i.e., anticipating verbs and actions). It is worth noting that second best Top-5 verb and noun accuracy are achieved by different methods (i.e., ATSN in the case of verbs and RL in the case of nouns), while both are outperformed by the proposed RU.
	Despite its low performance with respect to class-agnostic measures, ED systematically achieves second best results with respect to mean Top-5 recall and mean $TtA(5)$. This highlights that there is no clear second-best performing method. Finally, mean $TtA(k)$ highlights that the proposed method can anticipate verbs, nouns and actions $1.62$, $1.11$ and $0.76$ seconds in advance respectively.

	%
	\begin{table}[t]
		\begin{adjustbox}{width=\linewidth,center}
			\begin{tabular}{llccc|ccc}
				%
				\hline
				& & \multicolumn{3}{c|}{Top-1 Accuracy\%} & \multicolumn{3}{c}{Top-5 Accuracy\%}  \\ \hline
				& & VERB & NOUN & ACT. & VERB & NOUN & ACT. \\ \hline
				\multirow{4}{*}{\rotatebox{90}{\textbf{S1}}} &
				2SCNN~\cite{Damen2018EPICKITCHENS} & \underline{29.76} & 15.15 & 04.32 & 76.03 & 38.56 & 15.21 \\
				&ATSN~\cite{Damen2018EPICKITCHENS} & 31.81 & \underline{16.22} & 06.00 & \underline{76.56} & \underline{42.15} & \underline{28.21} \\
				&MCE~\cite{furnari2018Leveraging} & 27.92 & 16.09 & \underline{10.76} & 73.59 & 39.32 & 25.28 \\
				&\textbf{RU} & \textbf{33.04} & \textbf{22.78} & \textbf{14.39} & \textbf{79.55} & \textbf{50.95} & \textbf{33.73} \\ \hline
				&Improvement & +1.23 & +6.56 & +3.63 & +2.99 & +8.80 & +5.52 \\

				\hline \hline
				\multirow{4}{*}{\rotatebox{90}{\textbf{S2}}} &
				2SCNN~\cite{Damen2018EPICKITCHENS} & 25.23 & 09.97 & 02.29 & \underline{68.66} & 27.38 & 09.35 \\
				&ATSN~\cite{Damen2018EPICKITCHENS} & \underline{25.30} & \underline{10.41} & 02.39 & 68.32 & \underline{29.50} & 06.63 \\
				&MCE~\cite{furnari2018Leveraging} & 21.27 & 09.90 & \underline{05.57} & 63.33 & 25.50 & \underline{15.71} \\
				&\textbf{RU} & \textbf{27.01} & \textbf{15.19} & \textbf{08.16} & \textbf{69.55} & \textbf{34.38} & \textbf{21.10} \\ \hline
				&Improvement & +1.71 & +4.78 & +2.59 & +0.89 & +4.88 & +5.39 \\

				\hline

			\end{tabular}
		\end{adjustbox}	\caption{Anticipation results on the EPIC-KITCHENS test sets.}
		\label{tab:anticipation_ek_test}
		\vspace{-\topsep}
	\end{table}

	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%

	\tablename~\ref{tab:anticipation_ek_test} assesses the performance of the proposed method on the official test sets of EPIC-Kitchens dataset.\footnote{See supp. for precision and recall results.} RU outperforms all competitors by consistent margins on both the ``seen'' test, which includes scenes appearing in the training set (\textbf{S1}) and the ``unseen'' test set, with scenes not appearing in the training set (\textbf{S2}). Also in this case, RU is strong on nouns, obtaining $+6.56\%$ and $+8.8\%$ in \textbf{S1}, as well as $+4.78\%$ and $+4.88$ in \textbf{S2}. Improvements in terms of actions are also significant: $+3.63\%$ and $+5.52\%$ in \textbf{S1}, as well as $+2.59\%$ and $+5.39\%$ on \textbf{S2}.
	%

	\vspace{\topsep}
	\noindent
	\textbf{Anticipation Results on EGTEA Gaze+\hspace{0.5em}}
	%
	%
	%
	\begin{table}
		%
		\begin{adjustbox}{width=\linewidth,center}
			\begin{tabular}{p{1.2cm}ccccccccc}
				\hline

				\multicolumn{1}{c}{} & \multicolumn{8}{c}{Top-5 ACTION Accuracy\% @ different $\tau_a$(s)}   & $TtA$ \\  \hline %
				\multicolumn{1}{c}{} & $2$ & $1.75$ & $1.5$ & $1.25$ & $1.0$& $0.75$&$0.5$&$0.25$& \\ \hline
				DMR~\cite{vondrick2016anticipating} & /              & /              & /              & /              & 55.70          & /              & /              & /              &  /\\
				ATSN~\cite{Damen2018EPICKITCHENS}   & /              & /              & /              & /              & 40.53          & /              & /              & /              &  / \\
				MCE~\cite{furnari2018Leveraging}    & /              & /              & /              & /              & 56.29          & /              & /              & /              & / \\
				ED~\cite{gao2017red}                & 45.03          & 46.22          & 46.86          & 48.36          & 50.22          & 51.86          & 49.99          & 49.17          & 1.24\\
				FN~\cite{de2018modeling}            & 54.06          & 54.94          & 56.75          & 58.34          & 60.12          & 62.03          & 63.96          & 66.45          & 1.26\\
				RL~\cite{ma2016learning}            & 55.18          & 56.31          & 58.22          & 60.35          & 62.56          & 64.65          & 67.35          & 70.42          & 1.29\\
				EL~\cite{jain2016recurrent}         & \underline{55.62} & \underline{57.56} & \underline{59.77} & \underline{61.58} & \underline{64.62} & \underline{66.89} & \underline{69.60} & \underline{72.38} & \underline{1.32}\\
				\textbf{RU}                                  & \textbf{56.82} & \textbf{59.13} & \textbf{61.42} & \textbf{63.53} & \textbf{66.40} & \textbf{68.41} & \textbf{71.84} & \textbf{74.28} & \textbf{1.41} \\ \hline
				Imp                                 & +1.20          & +1.57          & +1.65          & +1.95          & +1.78          & +1.52          & +2.24          & +1.89          & +0.09\\ \hline
			\end{tabular}
		\end{adjustbox}
		\caption{Anticipation results on EGTEA Gaze+.}
		\label{tab:anticipation_egtea}
		\vspace{-\topsep}
	\end{table} \tablename~\ref{tab:anticipation_egtea} reports Top-5 action accuracy scores achieved by the compared methods on EGTEA Gaze+ with respect to the $8$ considered anticipation times. The table also reports mean $TtA(5)$ action scores, denoted as $TtA$. The proposed method outperforms all competitors at all anticipation times. Note that the margins of improvement are smaller on EGTEA Gaze+ due to its smaller-scale ($106$ actions in vs $2,513$ actions in EPIC-KITCHENS). Differently from \tablename~{\ref{tab:anticipation_ek}}, EL systematically achieves second best performance on EGTEA Gaze+, which highlights again that there is no clear second best competitor to RU in our evaluations.

	\vspace{\topsep}
	\noindent
	\textbf{Ablation Study on EPIC-Kitchens\hspace{0.5em}}
	To assess the role of rolling-unrolling, we consider a strong baseline composed of a single LSTM (same configuration as R-LSTM) and three branches (RGB, Flow, OBJ) with late fusion (BL). To tear apart the influence of rolling-unrolling and MATT, \tablename~\ref{tab:ablation}(a) compares this baseline with the proposed RU architecture, where MATT has been replaced with late fusion. The proposed RU approach brings systematic improvements over the baseline for all anticipation times, with larger improvements in the case of the object branch.

	%

	\tablename~\ref{tab:ablation}(b) reports the performances of the single branches of RU and compares MATT with respect to late fusion (i.e., averaging predictions) and early fusion (i.e., feeding the model with the concatenation of the modality-specific representations). MATT always outperforms late fusion, which consistently achieves second best results. Early fusion always leads to sub-optimal results. All fusion schemes always improve over the single branches.
	\figurename~\ref{fig:correlations} shows regression plots of modality attention weights computed on all samples of the validation set. RGB and OBJ weights are characterized by a strong and steep correlation. A similar pattern applies to Flow and OBJ weights, whereas Flow and RGB weights are characterized by a small positive correlation. This suggests that MATT gives more credit to OBJ when RGB and Flow are less informative, whereas the it relies on RGB and Flow when objects are not necessary.

	\tablename~\ref{tab:ablation}(c) assesses the role of pre-training the proposed architecture with sequence completion. As can be noted, the proposed pre-training procedure brings small but consistent improvements for most anticipation times. \tablename~\ref{tab:ablation}(d) compares RU with the strong baseline of \tablename~\ref{tab:ablation}(a). The comparison highlights the cumulative effect of all the proposed procedures/component with respect to a strong baseline using three modalities. It is worth noting that the proposed architecture brings improvements for all anticipation times, ranging from $+1.53\%$ to $+4.08\%$.

	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%

	\begin{table}[t]
		\setlength{\tabcolsep}{5pt}
		\centering
		\begin{adjustbox}{width=\linewidth,center}
			\begin{tabular}{p{2cm}ccccccccc}
				\hline
				\multicolumn{1}{c}{} & \multicolumn{8}{c}{Top-5 ACTION Accuracy\% @ different $\tau_a$(s)} & $TtA$ \\  \hline %
				\multicolumn{1}{c}{} & $2$ & $1.75$ & $1.5$ & $1.25$ & $1.0$& $0.75$&$0.5$&$0.25$ & \\ \hline
				BL (Late)            & \underline{27.96} & \underline{28.76} & \underline{29.99} & \underline{31.09} & \underline{32.02} & \underline{33.09} & \underline{34.13} & \underline{34.92} & \underline{0.66}\\
				RU (Late)            & \textbf{29.10} & \textbf{29.77} & \textbf{31.72} & \textbf{33.09} & \textbf{34.23} & \textbf{35.28} & \textbf{36.10} & \textbf{37.61} & \textbf{0.73} \\ \hline
				Imp.                 & +1.14          & +1.01          & +1.73          & +2.00          & +2.21          & +2.19          & +1.97          & +2.69          & +0.07 \\ \hline
				&
			\end{tabular}
		\end{adjustbox}

		\vspace{1mm}
		\centerline{\footnotesize(a) Rolling-Unrolling Mechanism.}
		\vspace{1mm}

		\begin{adjustbox}{width=\linewidth,center}
			\begin{tabular}{p{2cm}ccccccccc}
				\hline
				RU (RGB)             & 25.44 & {26.89} & {28.32} & {29.42} & {30.83} & {32.00} & {33.31} & {34.47} & 0.69\\
				RU (Flow)            & {17.38} & {18.04} & {18.91} & {19.97} & {21.42} & {22.37} & {23.49} & {24.18} & 0.51 \\
				RU (OBJ)             & {24.56} & {25.60} & {26.61} & {28.32} & {29.89} & {30.85} & {31.82} & {33.39} & 0.67 \\ \hline

				Early Fusion         & 25.58          & 27.25          & 28.58          & 29.59          & 31.88          & 32.78          & 33.99          & 35.62          & 0.72 \\
				Late Fusion          & \underline{29.10} & \underline{29.77} & \underline{31.72} & \underline{33.09} & \underline{34.23} & \underline{35.28} & \underline{36.10} & \underline{37.61} & \underline{0.73} \\
				MATT                 & \textbf{29.44} & \textbf{30.73} & \textbf{32.24} & \textbf{33.41} & \textbf{35.32} & \textbf{36.34} & \textbf{37.37} & \textbf{38.98} & \textbf{0.76} \\ \hline
				Imp.                 & +0.34          & +0.96          & +0.52          & +0.32          & +1.09          & +1.06          & +1.27          & +1.37          & +0.03\\
				\hline&
			\end{tabular}
		\end{adjustbox}

		\vspace{1mm}
		\centerline{\footnotesize (b) Modality Attention Fusion Mechanism.}
		\vspace{1mm}

		\begin{adjustbox}{width=\linewidth,center}
			\begin{tabular}{p{2cm}ccccccccc}
				\hline
				%
				%
				w/o SCP              & \underline{29.22} & \underline{30.43} & \textbf{32.34} & \underline{33.37} & \underline{34.75} & \underline{35.84} & \underline{36.79} & \underline{37.93} & \underline{0.75} \\
				with SCP             & \textbf{29.44} & \textbf{30.73} & \underline{32.24} & \textbf{33.41} & \textbf{35.32} & \textbf{36.34} & \textbf{37.37} & \textbf{38.98} & \textbf{0.76} \\ \hline
				Imp. of SCP          & +0.22          & +0.30          & -0.10          & +0.04          & +0.57          & +0.50          & +0.58          & +1.05          & +0.01 \\ \hline
			\end{tabular}
		\end{adjustbox}

		\vspace{1mm}
		\centerline{\footnotesize (c) Sequence-Completion Pre-training.}

		\begin{adjustbox}{width=\linewidth,center}
			\begin{tabular}{p{2cm}ccccccccc}
				\hline
				%
				%
				%
				BL (Fusion)          & \underline{27.96} & \underline{28.76} & \underline{29.99} & \underline{31.09} & \underline{32.02} & \underline{33.09} & \underline{34.13} & \underline{34.92} & \underline{0.66} \\
				RU (Fusion)          & \textbf{29.44} & \textbf{30.73} & \textbf{32.24} & \textbf{33.41} & \textbf{35.32} & \textbf{36.34} & \textbf{37.37} & \textbf{38.98} & \textbf{0.76} \\ \hline
				Imp. (Fusion)        & +1.48          & +1.97          & +2.25          & +2.32          & +3.30          & +3.25          & +3.24          & +4.06          & +0.1 \\ \hline
			\end{tabular}
		\end{adjustbox}

		\vspace{1mm}
		\centerline{\footnotesize (d) Overall comparison wrt strong baseline.}

		\caption{Ablation study on EPIC-KITCHENS.}
		\label{tab:ablation}
		\vspace{-\topsep}
	\end{table} \begin{figure}
		\includegraphics[width=\linewidth]{figures/weights/correlations.png}
		\caption{Correlations between modality attention weights}
		\label{fig:correlations}
		\vspace{-1.5\topsep}
	\end{figure}

	\vspace{\topsep}
	\noindent
	\textbf{Qualitative Results\hspace{0.5em}}
	\figurename~\ref{fig:qualitative} reports two examples of anticipations made by the proposed method at four anticipation times. Under each frame we report top-4 predictions, whereas modality weights computed by MATT are reported in percentage on the right. Green bounding boxes are shown around the detected objects and the optical flow is illustrated in orange. In the first example (top), the model can predict ``close door'' based on context and past actions (e.g., taking objects out of the cupboard), hence it assigns large weights to RGB and Flow and low weights to OBJ. In the second example (bottom), the model initially predicts ``squeeze lime'' at $\tau_a=2s$. Later, the prediction is corrected to ``squeeze lemon'' as soon as the lemon can be reliably detected. Note that in this case the network assigns larger weights to OBJ as compared to the previous example.\footnote{See supp. and \textit{https://iplab.dmi.unict.it/rulstm/} for additional examples and videos.}

	\begin{figure*}
		\begin{tabular}{c}
			\includegraphics[width=\linewidth,clip=true,trim=0 0 0 90]{figures/qualitative/019_04502_flow.png}\\
			%
			\includegraphics[width=\linewidth,clip=true,trim=0 0 0 110]{figures/qualitative/084_38984_flow.png}
		\end{tabular}
		\caption{Qualitative examples (best seen on screen). Legend for attention weights: blue - RGB, orange - Flow, green - objects. }
		\label{fig:qualitative}
		\vspace{-\topsep}
	\end{figure*}

	\vspace{\topsep}
	\noindent
	\textbf{Early Action Recognition\hspace{0.5em}}
	%
	We also observe that the proposed method generalizes to the task of early action recognition. We adapt our processing scheme by sampling $8$ video snippets from each action segment uniformly in time and set $S_{enc}=0$, $S_{ant}=8$. The snippets are fed to the model, which produces predictions at each time-step, corresponding to the following observation rates:  $12.5\%$, $25\%$, $37.5\%$, $50\%$, $62.5\%$, $75\%$, $87.5\%$, $100\%$. Branches are fused by late fusion in this case. We compare the proposed method with respect to FN, EL and RL.
	%

	%
	%
	\begin{table}[t]
		\begin{adjustbox}{width=\linewidth,center}
			%
			\begin{tabular}{p{0.8cm}cccccccc}
				%
				\multicolumn{1}{c}{} & \multicolumn{8}{c}{Top-1 ACTION Accuracy\% @ different observation rates} \\ \hline
				& $12.5\%$ & $25.0\%$ & $37.5\%$ & $50.0\%$ & $62.5\%$& $75.0\%$&$87.5\%$&$100\%$ \\
				\hline

				FN~\cite{de2018modeling}& 19.61 & 23.85 & 25.66 & 26.85 & 27.47 & 28.34 & 28.26 & 28.38\\
				EL~\cite{jain2016recurrent}& 19.69 & 23.27 & 26.03 & 27.49 & 29.06 & 29.97 & \underline{30.91} & \underline{31.46}\\
				RL~\cite{ma2016learning} & \underline{22.53} & \underline{25.08} & \underline{27.19} & \underline{28.64} & \underline{29.57} & \underline{30.13} & 30.45 & 30.47\\
				\textbf{RU} & \textbf{24.48} & \textbf{27.63} & \textbf{29.44} & \textbf{30.93} & \textbf{32.16} & \textbf{33.09} & \textbf{33.63} & \textbf{34.07}\\

				\hline
				Imp.&+1.95 & +2.55 & +2.25 & +2.29 & +2.59 & +2.96 & +2.72 & +2.61\\

				\hline
			\end{tabular}
		\end{adjustbox}
		\caption{Early recognition results on EPIC-KITCHENS.}
		\label{tab:recognition_ek}
		\vspace{-1.5\topsep}
	\end{table} %
	\begin{table}[t]
		\begin{adjustbox}{width=\linewidth,center}
			%
			\begin{tabular}{p{0.8cm}cccccccc}
				%
				\multicolumn{1}{c}{} & \multicolumn{8}{c}{Top-1 ACTION Accuracy\% @ different observation rates} \\ \hline
				& $12.5\%$ & $25.0\%$ & $37.5\%$ & $50.0\%$ & $62.5\%$& $75.0\%$&$87.5\%$&$100\%$ \\ \hline
				FN~\cite{de2018modeling}    & 44.02          & 50.32          & 53.34          & 55.10          & 56.58          & 57.31          & 57.95          & 57.72          \\
				EL~\cite{jain2016recurrent} & 40.31          & 48.08          & 51.84          & 54.71          & 56.93          & 58.45          & \underline{59.55} & \underline{60.18} \\
				RL~\cite{ma2016learning}    & \underline{45.42} & \underline{51.00} & \underline{54.20} & \underline{56.54} & \underline{58.09} & \underline{58.93} & 59.29          & 59.50          \\
				\textbf{RU}                          & \textbf{45.94} & \textbf{51.84} & \textbf{54.39} & \textbf{57.05} & \textbf{58.15} & \textbf{59.31} & \textbf{60.10} & \textbf{60.20} \\
				\hline
				Imp.                        & +0.51          & +0.84          & +0.20          & +0.51          & +0.06          & +0.38          & +0.55          & +0.02         \\
				\hline
			\end{tabular}
		\end{adjustbox}
		\caption{Early recognition results on EGTEA Gaze+.}
		\label{tab:recognition_egtea}
		\vspace{-1.5\topsep}
	\end{table} \tablename~\ref{tab:recognition_ek} reports Top-1 early action recognition accuracy results obtained by the compared methods on our validation set of EPIC-Kitchens. The proposed method consistently outperforms the competitors at all observation rates. Interestingly, RU achieves an early action recognition accuracy of $33.09\%$ when observing only $75\%$ of the action, which is already comparable to the accuracy of $34.07\%$ achieved when the full action is observed. This indicates that RU can timely recognize actions before they are completed. RL achieves second best results up to observation rate $75\%$, whereas EL achieves second best results when more than $75\%$ of the action is observed, which confirms the lack of a clear second-best performer.
	%
	\tablename~\ref{tab:recognition_egtea} reports Top-1 accuracy results obtained on EGTEA Gaze+. The proposed RU outperforms the competitors for all observation rates by small but consistent margins. Coherently with \tablename~\ref{tab:recognition_ek}, second best results are obtained by RL and EL.

	%

	\begin{table}[t]
		\begin{adjustbox}{width=\linewidth,center}
			\begin{tabular}{llccc|ccc}
				%
				\hline
				& & \multicolumn{3}{c|}{Top-1 Accuracy\%} & \multicolumn{3}{c}{Top-5 Accuracy\%}  \\ \hline
				& & VERB & NOUN & ACTION & VERB & NOUN & ACTION \\ \hline
				\multirow{5}{*}{\rotatebox{90}{\textbf{S1}}} &
				2SCNN~\cite{Damen2018EPICKITCHENS} & 42.16 & 29.14 & 13.23 & 80.58 & 53.70 & 30.36\\
				&TSN~\cite{Damen2018EPICKITCHENS} & 48.23 & 36.71 & 20.54 & 84.09 & \underline{62.32} & 39.79\\
				&LSTA~\cite{sudhakaran2018lsta} & \textbf{59.55} & 38.35 & \underline{30.33} & \textbf{85.77} & 61.49 & \underline{49.97}\\
				&MCE~\cite{furnari2018Leveraging} & 54.22 & \underline{38.85} & 29.00 & 85.22 & 61.80 & 49.62\\
				&\textbf{RU} & \underline{56.93} & \textbf{43.05} & \textbf{33.06} & \underline{85.68} & \textbf{67.12} & \textbf{55.32} \\ \hline

				&Imp. & -2.62 & +4.20 & +2.73 & -0.09 & +4.80 & +5.35\\

				\hline
				\multirow{5}{*}{\rotatebox{90}{\textbf{S2}}} &
				2SCNN~\cite{Damen2018EPICKITCHENS} & 36.16 & 18.03 & 07.31 & 71.97 & 38.41 & 19.49\\
				&TSN~\cite{Damen2018EPICKITCHENS} & 39.40 & 22.70 & 10.89 & \underline{74.29} & \underline{45.72} & 25.26\\
				&LSTA~\cite{sudhakaran2018lsta} & \textbf{47.32} & 22.16 & \underline{16.63} & \textbf{77.02} & 43.15 & 30.93 \\
				&MCE~\cite{furnari2018Leveraging} & 40.90 & \underline{23.46} & 16.39 & 72.11 & 43.05 & \underline{31.34} \\
				&\textbf{RU} & \underline{43.67} & \textbf{26.77} & \textbf{19.49} & 73.30 & \textbf{48.28} & \textbf{37.15} \\ \hline
				& Imp. & -3.65 & +3.31 & +2.86 & -3.72 & +2.56 & +5.81 \\

				\hline

			\end{tabular}
		\end{adjustbox}	\caption{Recognition results on the EPIC-KITCHENS test sets.}
		\label{tab:recognition_ek_test}
		\vspace{-\topsep}
	\end{table}

	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	%
	\vspace{\topsep}
	\noindent
	\textbf{Egocentric Action Recognition Results\hspace{0.5em}}
	%
	The proposed method can be used to perform egocentric action recognition by considering the predictions obtained for the observation rate of $100\%$. \tablename~\ref{tab:recognition_ek_test} compares the performance of the proposed method with other egocentric action recognition methods on the two test sets of EPIC-Kitchens.\footnote{See supp. for precision and recall results.} Our RU outperforms all competitors in recognizing actions and nouns on both sets by significant margins, whereas it achieves second-best results in most cases for verb recognition. RU obtains an action recognition accuracy of $60.2\%$ on EGTEA Gaze+. Despite being designed for action anticipation, RU outperforms recent approaches, such as Li et al.~\cite{Li_2018_ECCV} ($+6.9\%$ wrt $53.3\%$) and Zhang et al.~\cite{zhang2018adding} ($+3.19\%$ wrt $57.01\%$ - reported from~\cite{sudhakaran2018lsta}), and obtaining performances comparable to state-of-the-art approaches such as Sudhakaran and Lanz~\cite{sudhakaran2018attention} ($-0.56\%$ wrt $60.76$) and Sudhakaran et al.~\cite{sudhakaran2018lsta} ($-1.66\%$ wrt $61.86\%$).\footnote{See supp. for the full table.}
	%
	%

	\vspace{-\topsep}
	\section{Conclusion}
	\vspace{-0.5\topsep}
	We presented RU-LSTM, a learning architecture which processes RGB, optical flow and object-based features using two LSTMs and a modality attention mechanism to anticipate actions from egocentric video. Experiments on two datasets show the superiority of the approach with respect to prior art and highlight generalization over datasets and tasks: anticipation, early recognition, and recognition. %

	\vspace{-\topsep}
	\section*{Acknowledgment}
	\vspace{-0.5\topsep}
	This research is supported by Piano della Ricerca 2016-2018, linea di
	Intervento 2 of DMI, University of Catania.

	%
	%
	%
	%
	%
	%
	%
	%


text:
Ime Mudkip kombinacija je engleskih riječi "mud" = blato, što je njegovo prirodno okruženje u staništu u kojem živi, i "skip" = skakutati, što se odnosi na englesku riječ "mudskipper" = riba skokunica, na čijem se liku temelji sam Mudkip. Ime Mudkip odnosi se na čitavu vrstu ovog Pokémona, kao i pojedinačne primjerke vrste u Pokémon igrama, animiranoj seriji i manga stripovima.

Biološke karakteristike

Peraja na Mudkipovoj glavi izrazito je osjetljiv radar. Koristeći ovu peraju kako bi osjetio kretanje zraka i vode, Mudkip može odrediti što se nalazi u njegovoj blizini bez korištenja očiju. Kada je u vodi, Mudkip diše kroz narančaste škrge smještene na njegovim obrazima. Izvan vode, može podizati nekoliko puta veće stijene od sebe, upirući svoje četiri noge u zemlju. Spava zakopan u mulju na rubu močvare ili jezera. U vodi, njegova ga velika repna peraja pokreće prema naprijed s visokim ubrzanjem. Ako se u borbi suoči s teškim izgledima, Mudkip će iznenada postati dovoljno snažan da raznese čitave kamene blokove nekoliko puta veće od njega samoga.

U videoigrama

Jedan od dosljednih aspekata svih Pokémon igara – od Pokémon Red i Blue za Nintendo Game Boy, do igara Pokémon Diamond i Pearl za Nintendo DS konzole – izbor je tri različita Pokémona na početku igračeve avanture; ta tri Pokémona etiketirana su kao Početni (ili starter) Pokémoni. Igrač može birati između Vodenog, Vatrenog ili Travnatog Pokémona, autohtonog za to područje; iznimka od ovog pravila je Pokémon Yellow (remake verzija prvotnih igara koja prati priču Pokémon animirane serije), gdje igrač dobiva Pikachua, Električnog miša, poznatog po tome što je maskota Pokémon medijske franšize.

Mudkip je izbor Vodenog početnog Pokémona u Pokémon Ruby, Pokémon Sapphire i Pokémon Emerald igrama, namijenjene igranju na Nintendo Game Boy Advance konzoli. Birajući Mudkipa za svog početnog Pokémona, težina igre postaje manja nego što bi bila da je igrač odabrao Vatrenog početnog Pokémona (Torchic), ali jednako teška kao kada izabere Travnatog početnog Pokémona (Treecko), prvenstveno zbog prednosti koju Mudkip i Treecko imaju nad prvim Vođom dvorane, Roxanne, koja koristi Kamene Pokémone – poput Geodudea – koji imaju prirodnu slabost na Vodene i Travnate napade.

Vodeni je Pokémon, što znači da se usavršio u ispaljivanju vode (u obliku mjehurića ili vodenih mlazova) kako bi napao protivnika. Mudkip prolazi kroz evoluciju, metamorfoznu promjenu unutar Pokémona uzrokovanu stjecanjem iskustva u borbi, dvaput. Evoluira u Marshtompa na 16 razini, te kasnije u Swamperta na 36. razini.

Mudkipa igrači često biraju za svog početnog Pokémona jer su njegovi razvijeni oblici dvostrukog Vodenog/ Zemljanog tipa, što znači da su se usavršili u Zemljanim napadima podjednako kao i u Vodenim napadima. Zemljani tip Marshtompu i Swampertu daje imunitet na Električne napade, na koje bi inače bili izrazito slabi. Istovremeno, Swampert je jedan od najčešće korištenih Pokémona u kompetitivnim borbama.

Još jedna zanimljivost vezana uz Mudkipa jest ta da njegovi evoluirani oblici ne mogu naučiti Vodenu crpku (Hydro Pump), moćni Vodeni napad koji Mudkip prirodno uči na 42. razini. Ostala dva početna Pokémona Hoenn regije također prirodno uče napade koje njihovi evoluirani oblici ne mogu naučiti, bar ne kroz iskustvo.

Mudkip je jedan od glavnih likova u Pokémon Mystery Dungeon igri za Nintendo DS konzolu. Na početku same igre, igrač je podvrgnut testu osobnosti. Ovisno o igračevim odgovorima, započet će igru kao jedan od 16 različitih Pokémona, uključujući i Bulbasaura, Torchica i Mudkipa. Ako je Bulbasaur ili neki drugi Pokémon odabran kao lik igrača, Mudkip može postati jedan od igračevih "partnera" na početku same igre.

U animiranoj seriji

Tijekom avantura u Hoenn regiji, Brock, Pokémon uzgajivač i bivši Vođa dvorane grada Pewtera koji putuje s Ashom, uhvati Mudkipa u gradu Dewfordu u epizodi 301 "A Mudkip Mission". Kako se Ash i društvo uspinju vodopadom, naiđu na mladog Mudkipa i Brock spasi malenog oslabljenog Mudkipa kojeg je gotovo odnijela struja. Brockov Lotad i Mudkip zajedno se bore protiv Tima Raketa, nakon čega Mudkip odluči pridružiti se Brockovu timu.

Mudkipova je prvotna uloga pomaganje Brocku tijekom situacija povezanih s vodom, poput traženja predmeta u oceanu, poput bisera koji je pripadao Spoinku. Mudkip isto tako čuva ostale Pokémone kada Ash, May i Brock nisu u blizini. Razvio se u Marshtompa u epizodi 425 "Chip Off The Old Brock!".

Još se jedan Mudkip pojavio u epizodi 281 "In the Knicker of Time!", gdje grupa susreće trenera Nicolaija, mladoga trenera koji trenira svog prvog Pokémona Mudkipa, koji kasnije pobijedi Mayinog Torchica u borbi. Nicolai podržava svoje Pokémone oblačeći se u kostime koji nalikuju na njegove Pokémone.

Vodeni Pokémoni
Početni Pokémoni
ca:Línia evolutiva de Mudkip#Mudkip
es:Anexo:Pokémon de la tercera generación#Mudkip
fi:Luettelo Pokémon-lajeista (252–280)#Mudkip
fr:Gobou et ses évolutions#Gobou
ko:포켓몬 목록 (252-280)#물짱이
pl:Lista Pokémonów (241-260)#Mudkip

redpajama_set_name: RedPajamaWikipedia


Water is compelling to photograph. When it's moving or situated by lights, the attraction grows. And if this is at night, then I cannot keep away. This combination can produce some interesting results – often art without the need to pick up a paintbrush.
This was captured at the Dubai Mall, just before a display of the Dubai Fountain. Water and the coloured lights of consumerism work well together for the lens.
In Mauritius, and an after dinner rain shower captures the night glow of Le Touessrok by the pool.
An illuminated waterfall plays close to Burj Khalifa, the world's tallest building. The long exposure gives the water a foggy sheen, while its colours shine through.
This is not a night shot, but taken in the relatively low light conditions of a shopping centre. It is a close-up of the Dubai Mall waterfall, showing water cascading. It's flipped to the side for effect and in sepia to bring out the textures. This is the most abstract of this foursome. Which one has the most impact on you?
Mike, The one I like best is the last one, the next is the third! Great shots!
Hi Tomaz, thanks so much for commenting. I guess the last one is the strangest of the lot!
Love the first image of the reflected colours at Dubai Fountains, lovely!
Thank you very much, Lucy. There's a certain symmetry to that one.
Cheers Stevie. I can't be kept away from moving water, lights while armed with a camera!
Many thanks Danita. The coloured shots are quite a contrast to the sepia.


text:
Q: Save MinMaxScaler model in sklearn I'm using the MinMaxScaler model in sklearn to normalize the features of a model.
training_set = np.random.rand(4,4)*10
training_set

       [[ 6.01144787,  0.59753007,  2.0014852 ,  3.45433657],
       [ 6.03041646,  5.15589559,  6.64992437,  2.63440202],
       [ 2.27733136,  9.29927394,  0.03718093,  7.7679183 ],
       [ 9.86934288,  7.59003904,  6.02363739,  2.78294206]]

scaler = MinMaxScaler()
scaler.fit(training_set)
scaler.transform(training_set)

   [[ 0.49184811,  0.        ,  0.29704831,  0.15972182],
   [ 0.4943466 ,  0.52384506,  1.        ,  0.        ],
   [ 0.        ,  1.        ,  0.        ,  1.        ],
   [ 1.        ,  0.80357559,  0.9052909 ,  0.02893534]]

Now I want to use the same scaler to normalize the test set:
   [[ 8.31263467,  7.99782295,  0.02031658,  9.43249727],
   [ 1.03761228,  9.53173021,  5.99539478,  4.81456067],
   [ 0.19715961,  5.97702519,  0.53347403,  5.58747666],
   [ 9.67505429,  2.76225253,  7.39944931,  8.46746594]]

But I don't want so use the scaler.fit() with the training data all the time. Is there a way to save the scaler and load it later from a different file?

A: So I'm actually not an expert with this but from a bit of research and a few helpful links, I think pickle and sklearn.externals.joblib are going to be your friends here.
The package pickle lets you save models or "dump" models to a file.
I think this link is also helpful. It talks about creating a persistence model. Something that you're going to want to try is:
# could use: import pickle... however let's do something else
from sklearn.externals import joblib

# this is more efficient than pickle for things like large numpy arrays
# ... which sklearn models often have.

# then just 'dump' your file
joblib.dump(clf, 'my_dope_model.pkl')

Here is where you can learn more about the sklearn  externals.
Let me know if that doesn't help or I'm not understanding something about your model.
Note: sklearn.externals.joblib is deprecated. Install and use the pure joblib instead

A: Just a note that sklearn.externals.joblib has been deprecated and is superseded by plain old joblib, which can be installed with pip install joblib:
import joblib
joblib.dump(my_scaler, 'scaler.gz')
my_scaler = joblib.load('scaler.gz')

Note that file extensions can be anything, but if it is one of ['.z', '.gz', '.bz2', '.xz', '.lzma'] then the corresponding compression protocol will be used. Docs for joblib.dump() and joblib.load() methods.

A: You can use pickle, to save the scaler:
import pickle
scalerfile = 'scaler.sav'
pickle.dump(scaler, open(scalerfile, 'wb'))

Load it back:
import pickle
scalerfile = 'scaler.sav'
scaler = pickle.load(open(scalerfile, 'rb'))
test_scaled_set = scaler.transform(test_set)

A: Update: sklearn.externals.joblib is deprecated. Install and use the pure joblib instead. Please see Engineero's answer below, which is otherwise identical to mine.
Original answer
Even better than pickle (which creates much larger files than this method), you can use sklearn's built-in tool:
from sklearn.externals import joblib
scaler_filename = "scaler.save"
joblib.dump(scaler, scaler_filename)

# And now to load...

scaler = joblib.load(scaler_filename)

A: The best way to do this is to create an ML pipeline like the following:
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import MinMaxScaler
from sklearn.externals import joblib

pipeline = make_pipeline(MinMaxScaler(),YOUR_ML_MODEL() )

model = pipeline.fit(X_train, y_train)

Now you can save it to a file:
joblib.dump(model, 'filename.mod')

Later you can load it like this:
model = joblib.load('filename.mod')

redpajama_set_name: RedPajamaStackExchange


text:
Le championnat de Côte d'Ivoire de football 2 a été créé en 1960. La  Ligue 2 autrefois appelé division 2 (1960-2004) et Ligue 2 Orange (2005-2007) est une compétition annuelle mettant aux prises 24 clubs de football en Côte d'Ivoire.

Les clubs sont répartis en 2 poules de 12 clubs. Les 4 derniers sont relégués en , tandis que les vainqueurs de la Poule Abidjan et la Poule intérieur sont promus en MTN Ligue 1. En 2007, l'ASC Ouragahio vainqueur de la poule intérieure et l'USC Bassam vainqueur de la poule Abidjan, sont montés en MTN Ligue 1 2008.

Histoire

Palmarès
Nous avons que ces informations à partir de cette date.

Bilan

Notes et références

Liens externes

Championnat de Côte d'Ivoire de football de deuxième division


Where is the Dejarnatt family from?
You can see how Dejarnatt families moved over time by selecting different census years. The Dejarnatt family name was found in the USA between 1840 and 1920. The most Dejarnatt families were found in the USA in 1880. In 1840 there was 1 Dejarnatt family living in Indiana. This was about 33% of all the recorded Dejarnatt's in the USA. Indiana and 2 other states had the highest population of Dejarnatt families in 1840.
Use census records and voter lists to see where families with the Dejarnatt surname lived. Within census records, you can often find information like name of household members, ages, birthplaces, residences, and occupations.
What did your Dejarnatt ancestors do for a living?
In 1880, the most common Dejarnatt occupation in the USA was Farmer. 83% of Dejarnatt's were Farmers. A less common occupation for the Dejarnatt family was House Joiner. Farmer and Grocer were the top 2 reported jobs worked by Dejarnatt.
Census records can tell you a lot of little known facts about your Dejarnatt ancestors, such as occupation. Occupation can tell you about your ancestor's social and economic status.
What Dejarnatt family records will you find?
There are 3,000 census records available for the last name Dejarnatt. Like a window into their day-to-day life, Dejarnatt census records can tell you where and how your ancestors worked, their level of education, veteran status, and more.
There are 642 immigration records available for the last name Dejarnatt. Passenger lists are your ticket to knowing when your ancestors arrived in the USA, and how they made the journey - from the ship name to ports of arrival and departure.
There are 1,000 military records available for the last name Dejarnatt. For the veterans among your Dejarnatt ancestors, military collections provide insights into where and when they served, and even physical descriptions.
You've only scratched the surface of Dejarnatt family history.
What is the average Dejarnatt lifespan?
Between 1957 and 2004, in the United States, Dejarnatt life expectancy was at its lowest point in 2000, and highest in 1989. The average life expectancy for Dejarnatt in 1957 was 53, and 81 in 2004.
An unusually short lifespan might indicate that your Dejarnatt ancestors lived in harsh conditions. A short lifespan might also indicate health problems that were once prevalent in your family. The SSDI is a searchable database of more than 70 million names. You can find birthdates, death dates, addresses and more.


TemplateToaster-> go to BODY->Background-> set background accordingly.
Go to CONTAINER->Background-> set background transparent.
Go to HEADER-> Background-> set background transparent (repeat these steps for all the sections header, menu, footer, etc ).


The star was admittedly nervous asking permission for his future wife's hand.
Even Galaxy-saver and Marvel superhero Chris Pratt was a little intimidated before facing his hopeful father-in-law.
The 39-year-old actor admitted to feeling some nerves before asking for Arnold Schwarzenegger's blessing to propose to his oldest child, 29-year-old daughter Katherine.
"Well, I think any person who appreciates the tradition and asks a father for his daughter's hand in marriage, it's inevitably something that's going to make you a little nervous," Pratt recently told Extra. "But that's sort of the beauty of it."
But it turns out Pratt didn't have much reason to be nervous. A few weeks after Pratt popped the question, the Terminator actor, 71, sat down for an interview on Extra with Mario Lopez where he revealed his thoughts on his future son-in-law for the very first time.
"That was really fantastic. She is so happy, they both are so happy," Arnold said of the couple's engagement. "As I said to him, he is such a likable guy. I love Chris—he's fantastic."
The former bodybuilder went on to reveal that Pratt asked for his blessing in the "traditional way" before proposing to Katherine.
"He's a wonderful guy," Arnold added. "They both look really great together, they're both happy together, so I wish them both the best of luck."
Pratt also told reporters at the Lego Movie 2 premiere that he and Katherine hadn't "really" figured out too many wedding details yet.
Sharing a hint about their possible wedding date, he mentioned they were "maybe" thinking of a "fall, winter kind of thing," before adding that they still have "a lot to get done."
This post originally appeared on People.com by Ale Russian.


Very large neutron star explosed, formed the Milky Way. 超巨大中性子星の爆発で銀河系が出来た。今から46億年前に昔太陽系(≒中性子星)が爆発して現在の太陽系が出来た! Old solar (≈ neutron star ) exploded, could the solar system!
The effect of planets for extreme weather events.異常気象に対する惑星の影響。木星・土星・小惑星の影響が大きい。Jupiter Saturn Asteroids impact Great.
光速一定の原理と惑星楕円軌道との関係。 Correctness proof "the existence of negative Sun","principle of the speed of light constant'"★dark-star★ in the Ⅱfocal point of elliptical orbit".
Why elliptical 楕円 orbits of satellites and planets. 衛星や惑星の軌道が楕円になる訳。PS. Found photos captured to PS, sun light, subatomic particles and particles go into the pseudo Sun.
Black hole ブラックホールの正体 Identity. World consists of binary. 2進数[2進法]で成り立つ世界に我々は住んでいます。We live in the world consisting of Binary number of [base-2].
d 水銀と白金は金の成り損ね。Mercury, Platinum : detracts from made of Gold.
easy Baseball の曲がり方の研究 bends the way research.
miyamo on 電磁力って何?⇒「銀河は全て左巻き。それは横着な大自然の姿」Galaxies are all left-handed. It is very lazy nature's appearance.
black hole 内に住む住人より on Black hole ブラックホールの正体 Identity. World consists of binary. 2進数[2進法]で成り立つ世界に我々は住んでいます。We live in the world consisting of Binary number of [base-2].


text:
A talizmán bármilyen olyan tárgy, amelyik varázslatos erővel rendelkezik, és arra tervezték őket, hogy megvédjék, vagy meggyógyítsák azokat a személyeket, akiknek készítették (esetenként árthatnak is). A talizmánok általában valaki által viselt hordozható tárgyak. Szoros kapcsolatban állnak az amulettekkel, szerepük nagyjából ugyanaz, de a formájuk és az anyaguk különböző. A talizmánok sokszor különféle tárgyak, pl. ruhák vagy fegyverek alakját veszik fel.

Etimológia

A talizmán szó a francia talisman szóból ered; ez viszont az arab tilasm (طِلَسْم) szóból származik, amelynek az eredete pedig az ókori görög "telesma" (τέλεσμα) szó, amely "beteljesülést, vallási rítust, fizetséget" jelent. Ez a szó pedig a görög "teleō" (τελέω) szóból származik.

Jegyzetek

Varázslat
Tárgyak


A total of 140 motorcycles rolled around Barcelona streets and around some urban circuits like Pedralbes, Vallvidrera and La Rabassada.
Last June 12th, the Barcelona Motorcycle Museum has organized the 5th edition of the mythical Motoclàssica Barcelona, a city rallie for historic motorcycles produced before 1986. This event actually appears on the official calendar of the Motorcycle Week 2016, an initiative created by the FIVA (International Federation of Ancient Vehicles), which includes different activities happening at the same time in different countries.
Starting and ending point was at the Shopping Centre L'Illa Diagonal, where the general public was able to admire unique items. There were in total 140 motorcycles, around 200 participants and the itinerary a total of 50KM.
Most of the brands were Spanish: Bultaco (Metralla, Streaker, Mercurio,...), Montesa (Impala, Brio,...) and OSSA (125, comercial,...) as well as some off-road models!
The most veteran pilot was Mr. Victor Miarons, a well known collector, aged 79 years old, driving a Montesa D-51.
A big Thank You to all participants for their presence with their classic motorcycles as well as to all authorities and companies that cooperated with this event: Barcelona City Council, Sant Cugat City Council, Illa Diagonal Shopping Centre, Pans&Company, Impala Motorcycle Club, Royal Enfield Barcelona, Climax, Repco, Moturisme and Helmetdress.
All pictures from the V Edition of the Motoclàssica Barcelona, are available in this link.


Who says wine and football don't mix? Watch Super Bowl 53 at Brooklyn Winery!
Watch the Los Angeles Rams take on the New England Patriots during the Super Bowl on our 9 foot Atrium screen with surround sound. Sit back and enjoy the game while savoring our Super Bowl menu!
Come early to grab a seat and watch the game (or, let's be honest, the commercials) in comfort! Seating will fill up fast. Doors open at 6pm.