text:
\section{Introduction}

Recent breakthroughs in machine learning (ML) and artificial intelligence (AI) have shown a remarkable ability to extract relationships and correlations in data and events. Indeed, there now exist highly scalable solutions for object detection and recognition, machine translation, text-to-speech conversion, recommender systems, and information retrieval.  Recent advances in machine learning and data analytics have yielded transformative results across diverse scientific disciplines \cite{ref:Alipanahi2015,ref:Krizhevsky2012,ref:Lake2015,lecun2015deep,ghahramani2015probabilistic}. Enabled by the decreasing price to performance ratio of sensing, data storage, and computational resources in the past decade, data-driven machine learning strategies are taking center stage across many scientific disciplines.

In the realm of complex spatiotemporal dynamical systems, data-driven machine learning strategies have been employed for reduced-order models (ROMs) \cite{Bhatnagar2019,Wiewel2019,white2019fast,mohan2018deep,lee2017prediction}, discovery of system dynamics \cite{Tibshirani1996,YAO2007,Bongard2007,Schmidt2009,Kim2016,Pan16,Brunton16,lusch2017deep,maulik2018data,maulik2019subgrid,ayed2019learning,almomani2020entropic}, computation of dynamical system solutions \cite{raissi2017PINN1,raissi2017PINN2,raissi2018deep,raissi2018hidden,raissi2018multistep,pathak2018hybrid}, and prediction of future dynamics \cite{Ling2015,miyanawala2017efficient,raissi2018hidden,pathak2018hybrid,viquerat2019,LAPEYRE2019}.  These recent developments spurred by the current enthusiasm surrounding ML and AI strategies can be broadly classified into two categories: works that investigate the feasibility of existing ML/AI algorithms and architectures, and those centered around the development of new algorithms and architectures.  Existing work whose main objective is the former have focused on the power of ML/AI techniques to significantly reduce the steep computation and data storage costs associated with high-fidelity computational fluid dynamics (CFD) efforts \cite{Bhatnagar2019,Wiewel2019,white2019fast,mohan2018deep,lee2017prediction,Ling2015,miyanawala2017efficient,raissi2018hidden,viquerat2019,LAPEYRE2019}.  These works often leverage existing CFD models to generate ground truth, training, and testing data sets to evaluate well-studied convolutional neural networks (CNN) \cite{Bhatnagar2019,LAPEYRE2019,miyanawala2017efficient}, long short-term memory (LSTM) networks \cite{mohan2018deep}, generative adversarial networks (GAN) \cite{lee2017prediction}, and existing ML/AI frameworks \cite{Ling2015,white2019fast,viquerat2019}.  Nevertheless,  existing ML/AI strategies are predicated on access to large amounts of {\it labeled} data where explicit knowledge derived from well-established first principles are difficult to encode.

Works in the second category that directly address these challenges include sparse regression techniques \cite{YAO2007,Kim2016,Pan16,Brunton16,lusch2017deep,almomani2020entropic} and physics-informed neural networks (PINNs) \cite{raissi2017PINN1,raissi2017PINN2,raissi2018deep,raissi2018hidden}.  Sparse identification is a data-driven system identification strategy that balances model complexity with descriptiveness \cite{Brunton16}. Since the dynamics of most physical systems are governed by only a few important terms \cite{Brunton16}, sparse identification selects from a finite set of candidate dictionary functions whose linear combination describes the system dynamics \cite{Pan16}.  On the other hand, PINNs are neural networks that are trained to solve supervised learning tasks whose dynamics can be described by general nonlinear PDEs. The key advantage of PINNs is their {\it data-efficiency} in the training phase. Sparse regression techniques such as those found in \cite{YAO2007,Kim2016,Pan16,Brunton16,lusch2017deep,almomani2020entropic} require large amounts of relatively clean data to accurately compute numerical gradients, whereas PINNs do not require any data on gradients of the flow field (nor their numerical approximations).  As such, PINNs perform more robustly when data is sparse and/or noisy relative to the complexity of the underlying system dynamics \cite{raissi2017PINN1,raissi2017PINN2}.  In contrast, \citeauthor{ayed2019learning} uses actual observations of a system whose dynamics are given by an ordinary differential equation to train the neural network weights.  Once trained, the network provides an equation-free model representation of the system dynamics.  Different from \cite{YAO2007,Kim2016,Pan16,Brunton16,lusch2017deep,raissi2017PINN1,raissi2017PINN2,almomani2020entropic}, the work does not directly address the issue of data-efficiency but assumes the network has access to a sufficiently large set of training data.

In this work, we take inspiration from \cite{YAO2007,Kim2016,Brunton16,raissi2017PINN1,raissi2017PINN2,maulik2018data,pathak2018hybrid,maulik2019subgrid,almomani2020entropic} and present a data-driven Deep Learning framework capable of resolving the differences between the actual dynamics of a complex nonlinear system and that of the same system which has been improperly or inaccurately modeled.  Given an inaccurate or inadequate model of a system,  our proposed ML strategy combines data from this inaccurate/inadequate model with observational data from the actual system to learn the dynamics of the actual system.  The result is a neural network model that can accurately estimate the system states in regions with no observations and/or provide predictions for future states.  Different from \cite{YAO2007,Kim2016,Brunton16,raissi2017PINN1,raissi2017PINN2,almomani2020entropic}, our approach provides an equation-free representation of the system dynamics  that successfully estimates the underlying physics that drives the process. We evaluate the proposed framework using three different dynamical systems each with increasing complexity.  Our results show how the proposed strategy is not only capable of resolving improperly or inaccurately modeled dynamics but also can learn the dynamics of the actual system and provide accurate future predictions.

While our approach is similar to \cite{pgnn_lake,pathak2018hybrid}, we make use of LSTMs in our deep learning network rather than a simple multi-layer perceptron \cite{pgnn_lake} or reservoir computer \cite{pathak2018hybrid}. Our approach is general and may be used for a wide range of dynamical systems of different dimension and complexity, including examples in which the known model is missing external forcing functions or other known dynamics. Even for these complicated scenarios, we demonstrate in this article the power of our method to successfully predict the dynamics wherein simpler approaches will fail. Since our output is a neural network representation of the system model, the output of our network  can be fed into existing data-driven model discovery techniques \cite{Tibshirani1996,Bongard2007,Schmidt2009,Brunton16,Pan16} to obtain closed-form equation representations of the dynamical system.

The paper is organized as follows: we list our assumptions and provide a concise formulation of our problem in Sec. \ref{sec_problem}.  The design of the network architecture and our methodology is described in Sec. \ref{sec_architecture}.  We discuss how we evaluate our methodology in Sec. \ref{sec_eval} and present our results with discussion in Sec. \ref{sec_results}.  Conclusions and directions for future work are contained in Sec. \ref{sec_conclusion}.

\section{Problem Formulation}\label{sec_problem}
We consider a spatio-temporal process $u(x,t) \in \mathbb{R}^m$, where $x\in \mathbb{W}$ represents a point in the environment $\mathbb{W} \subset \mathbb{R}^n$ and $t\in [t_s,\; t_f]$ represents the time within an observation interval of interest.  The actual model of the process that governs $u$ is denoted by $M_{act}$ and is given by a partial differential equation (PDE) of the form
\begin{equation}
    \label{eqn_sec_prob_act_model}
    u_t =\mathcal{N}[u,f_1,\cdots,f_p,g_1,\cdots,g_r],
\end{equation}
where $\mathcal{N}[\cdot]$ is a nonlinear differential operator, where $f_i = f_i(x,t) \in \mathbb{R}^{n_{f_i}},\; i=1,\ldots,p$ and $g_i = g_i(x,t) \in \mathbb{R}^{n_{g_i}},\; i=1,\ldots, r$ are external phenomena that impact $u$.  Let $M_{curr}$ denote the model that is obtained from the current understanding of the physics of $u$. Then $M_{curr}$ is given by the PDE with form
\begin{equation}
    \label{eqn_sec_prob_curr_model}
    u_t = \tilde{\mathcal{N}}[u,f_1,\cdots,f_p],
\end{equation}
where $\tilde{{\cal N}}[\cdot]$ is also a nonlinear differential operator.  Here, the $f_i$ denote the $p$ external phenomena whose impact on $u$ are currently {\it known} and the $g_i$ denote the $r$ external phenomena that affect $u$ but are not captured in $M_{curr}$.  Note that in general, $g_i$ could represent some error in $f_i$ so that $g_i = f_i + \epsilon$ where $\epsilon$ denotes the difference between $f_i$ and $g_i$.  Furthermore, $\tilde{{\cal N}}$ is used to denote any differences in system parameters between $M_{curr}$ and $M_{act}$.  Thus, while $M_{curr}$ represents the current understanding of the process, this understanding is incomplete or inadequate and thus $M_{curr}$ is not an accurate representation of the process model.

Given a set of coordinates $S = \{s_j\vert s_j = (x_j, t_j),\; x_j\in \mathbb{W},\; t_j \in [t_s,\;t_f],\;j=1,\ldots, n_{data}\}$, let $\hat{U}_{act} = \{\hat{u}_{act_j}\vert j=1,\ldots, n_{data}\}$ be the set of observations of $u$ obtained by measuring the actual process at coordinates $s_j \in S$.  Similarly, let $U_{curr} = \{u_{curr_j}\}$ and $U_{act} = \{u_{act_j}\}$ be the solution sets obtained from $M_{curr}$ and $M_{act}$ respectively, at the coordinates in $S$. In this work, $U_{act}$ is based on computer simulations, but could in fact be measured experimentally. For simplicity, we assume that there are no measurement errors, \ie $\hat{u}_{act_j} \equiv u_{act_j}$ for each $\hat{u}_{act_j} \in \hat{U}_{act}$ and  $u_{act_j} \in U_{act}$ obtained at the same coordinate $s_j \in S$.

Given $U_{act}$, $U_{curr}$ and observations of a subset of the $g_i$ at the coordinates in $S$, the objective of this work is to develop a neural network based model $M_{nn}$ that better estimates the process $u$ in and potentially beyond the space-time domain $\mathbb{W}\times [t_s,\;t_f]$. Let $e_* = \Vert M_{act} -M_* \Vert \geq 0$ represent some measure of the error of the output of a given model with respect to the output of $M_{act}$ in a given domain. We want $e_{nn} \leq e_{curr}$ in all domains (ideally $e_{nn} = e_{curr}$ only when $e_{curr}=0$), \ie the neural network should be much better at predicting/estimating $u$ than the existing model.

To illustrate, consider a mass-spring-damper system with mass $m$, damping coefficient $c$ and spring constant $k$ that is subjected to two external forcing functions given by $F_1(t) = A_1 \cos{(\omega_1 t)}$ and $F_2(t) = A_2 \cos{(\omega_2 t)}$. If the displacement of the mass is denoted by $y$, the actual model of the system $M_{act}$ is given by the ordinary differential equation (ODE)
\begin{equation}
    \label{eqn_prob_msd_approx}
    m\Ddot{y} + c\dot{y} + ky = F_1 + F_2.
\end{equation}
Let's assume that due to modeling and measurement errors, the model that we have access to, $M_{curr}$, is given by
\begin{equation}
    \label{eqn_prob_msd_curr}
    \tilde{m}\Ddot{y} + \tilde{c}\dot{y} + \tilde{k}y = F_1.
\end{equation}
Note that this model only captures part of the forcing function and has errors in the mass, spring, and damping coefficients. Given measurements of the displacement $y$, our work seeks to develop a neural network, whose output closely resembles that of the actual model $M_{act}$ for the same initial conditions. Denoting the output of the actual, current and neural network models by $y_{act}(t)$, $y_{curr}(t)$ and $y_{nn}(t)$ respectively, we would like $\Vert y_{nn}(t) - y_{act}(t)\Vert < \epsilon < \Vert y_{curr}(t) - y_{act}(t)\Vert$, where ideally $\epsilon$ is small. In other words, we would like the trained neural network output to always be a better approximation of the ground truth than the current model output or match the ground truth exactly.  Lastly, in our proposed framework, the neural network model $M_{nn}$ only provides outputs for the ODE, {\it e.g.}, $y$, $\dot{y}$, and $\ddot{y}$ rather than the equation of the actual ODE.

\section{Methodology}\label{sec_architecture}

The proposed method uses a neural network based framework to ``bridge the gap'' between $M_{curr}$ and $M_{act}$. Neural networks have recently been used in a plethora of prediction and estimation problems. However, in most of these solutions, large quantities of training data is required to obtain good prediction performance. This is especially true for prediction/estimation problems involving complex dynamical systems. In this work, we mitigate this data inefficiency problem by incorporating existing knowledge of the process into the neural network architecture.

The fundamental hypothesis of our work is that the current understanding of the physics of $u$ given by $M_{curr}$, has substantial information that the neural network can exploit in order to provide better predictions of the process. Thus, in addition to the space-time coordinates ($x,\;t$) and, where applicable, external forcing terms $g_i$, we also use the output from $M_{curr}$ as an input to the neural network. This input may be presented to the network in different formats, \eg data generated from a reduced-order model \cite{KirbyBook2000,Sirovich1987TurbulenceStructures}, coefficients and functions from a sparse identification of the process \cite{Brunton16,almomani2020entropic}, output data from a numerical model, \etc

Furthermore, the behaviour of any dynamical system depends heavily on the initial and boundary conditions. In the absence of explicit initial and boundary conditions, these spatio-temporal dependencies have to be captured by the network in a purely data-driven manner. We facilitate this by 1) using Long Short-Term Memory (LSTM) stages in our network to capture temporal dependencies, and 2) providing the network with data in a space-time hypercube around the point of interest.

\subsubsection*{Neural Networks and LSTM Networks}

Artificial neural networks (ANN) are powerful nonlinear statistical models which consist of multiple layers of interconnected nodes such that every connection represents a weight. Each node calculates a weighted sum of the outputs of neurons which are connected to it as well as a bias term. By representing the system in terms of layers, neural networks are able to learn features exhibited by highly nonlinear and complex data in a powerful hierarchical fashion. The nonlinearity of these networks comes from the use of nonlinear activation functions in the neural net nodes. The neural net is trained  by minimizing a loss function. The minimization is commonly done by a gradient-based optimization algorithm that makes use of backpropagation -- a computationally efficient algorithm that computes the gradient of the loss function with respect to the weights at each layer. Common optimization algorithms include stochastic gradient descent, Adam \cite{adamoptim}, and Adagrad \cite{adagradoptim}. The optimization algorithm commonly performs updates to the weights using batches of the dataset. A complete pass through all the dataset batches is usually referred to as an {\it epoch}.

The most basic structure of a neural network is a fully connected or dense ANN as displayed in Fig. \ref{FC-arch}. Each node in the neural network is governed by an activation function
    $a_{l+1}(W_l  a_{l} + b_l)$
where $W_l$ and $b_l$ denotes the weights matrix and bias vector for layer $l$ respectively.  Common choices for $a_{l+1}$ include the sigmoid function commonly denoted by $\sigma(\cdot)$, the hyperbolic tangent function $tanh(\cdot)$, and rectified linear unit function $ReLU(\cdot)$. We refer the reader to \cite{activationfunctions} for a detailed review of activation functions.

\begin{figure}
    \centering
    \includegraphics[width = 0.7\linewidth]{figs/full_dense.png}
    \caption{A general dense layer architecture.}
    \label{FC-arch}
\end{figure}

In choosing a neural network architecture, we make note that our problem is in nature time-dependant. More concretely, the problem imposes an order on the sequence of observations that must be preserved. In general, standard artificial neural networks are not well-suited to learn such orders since the weights in each ANN layer are fully connected to the previous layer. This forces the ANN to  consider the entire sequence at once.  Recurrent Neural Networks (RNNs), on the other hand, are a different type of neural network that is well suited for sequence learning problems. They are equipped with a memory unit which is updated for each new observation. Thus, parameters of the network are shared for each step in the sequence. As such, RNNs rather than ANNs are most commonly employed to learn time dependencies.

The Long Short-Term Memory (LSTM) network is a variant of RNNs.  LSTMs address the bottlenecks in traditional RNNs such as the vanishing gradient problem  \cite{vanishinggradient} which hampers learning of long data sequences.  The LSTM memory unit is usually called the cell, denoted by $C$, which is regulated by three gates: an input gate ${\cal I}$, a forget gate ${\cal F}$, and an output gate ${\cal O}$. The input gate controls the contribution of the input to the cell, the forget gate controls what parts of the cell to keep, and the output gate controls the contribution of the cell to the output of the LSTM. A schematic of the architecture can be found in Fig. \ref{Lstm-arch}, with $h$ representing the output of the network while the input of the network is represented with $s$. The equations to compute the gates and states are given by
\begin{align}
    \label{eq:gs}
    {\cal F}_t & = \sigma(W_{\cal F} \cdot [h_{t-1}, s_t] + b_{\cal F}]), \nonumber \\
    {\cal I}_t & = \sigma(W_{\cal I} \cdot [h_{t-1}, s_t] + b_{\cal I}), \nonumber \\
    \bar{C_t} & = tanh(W_C \cdot [h_{t-1} ,s_t] + b_C), \nonumber \\
    C_t & = {\cal F}_t * C_{t-1} + {\cal I}_t*\bar{C_t}, \\
    {\cal O}_t & = \sigma(W_{\cal O} \cdot [h_{t-1}, s_t] + b_{\cal O}), \nonumber \\
    h_t & = {\cal O}_t*tanh(C_t), \nonumber
\end{align}
where $\bar{C}$ is the updated state, $W$ is the weights matrix, $b$ is the bias vector for each gate, $s_t$ is the input to the network at time $t$, and * denotes the Hadamard product.  The forget gate reduces overfitting by controlling how an incoming input contributes to the hidden state. This structure is the key reason why LSTMs do not suffer from the vanishing gradient problem exhibited by RNNs. For more detailed discussions on ANNs, RNNs, and LSTMs, we refer the interested reader to \cite{Goodfellow-et-al-2016,lstmexplained,SHERSTINSKY2020132306,chang2018antisymmetricrnn}.

\begin{figure}
    \centering
    \includegraphics[width=0.95\linewidth]{figs/LSTM.jpg}
    \caption{A general architecture for an LSTM layer.}
    \label{Lstm-arch}
\end{figure}

\subsubsection*{Input data format to the network}
The network predicts/estimates the process on a point by point basis. In order to capture the spatio-temporal dependencies between the inputs and the output at each coordinate $j$, we consider a $n+1$ dimensional space-time hypercube of the inputs around this coordinate. We consider $k$ data points along each dimension, resulting in  $k^{n+1}$  number of data points for each input.  In general, the larger the choice of $k$, the larger the input data and thus the higher the computational load.  In this work, we choose $k=3$ to limit the computational burden.  Thus for scenarios where $n=2$, as shown in Fig. \ref{fig_method_data_format}, we would consider a hypercube with 27 vertices for each input.

\begin{figure*}
    \centering
    \subfloat[]{\includegraphics[width = 0.4\linewidth]{figs/data_collec.png}\label{fig_method_data_format}}
    \subfloat[]{\includegraphics[width=0.6\linewidth]{figs/Network-arch.jpg}\label{Network-arch}}
    \caption{(a) Format of the inputs to the network. For each input to the network, we consider data in a $n+1$ hypercube around the point of interest. Along each dimension, $k$ data points are included resulting in $k^{n+1}$ number of data points for each input. In this figure, $k=3$.  (b) The general architecture of our final model which is composed of three stages. The number of layers and nodes in each stage depends on the problem.}
\end{figure*}

\subsection{Architecture of the Neural Network}
Our proposed neural network architecture is composed of three stages as shown in Fig. \ref{Network-arch}. We modify the architecture for each problem by changing the number of layers/nodes at different stages of the architecture. The three stages of the network are: \begin{itemize}
    \item Stage 1: Time distributed dense stage with $D_1$ layers;
    \item Stage 2: Long Short-Term Memory (LSTM) stage with $D_2$ layers; and
    \item Stage 3: Dense output stage with $D_3$ layers.
\end{itemize}
The three stages are described below in detail.

\subsubsection*{Stage 1: Time Distributed Dense Layers}
This stage consists of a set of parallel dense layers that work on the inputs at each time slice independently. The purpose of this stage is to give the network the ability to pre-process the data and learn a representation that is most optimal for the LSTM stage. While most research in the literature employing LSTM networks do so without this pre-processing layer, our experiments have demonstrated that adding this stage improves the convergence of the network. The activation function for layer $l$ in this stage is denoted as $a_{l, t}$ where $t$ denotes the time step. In this case, $W$ and $b$ are shared for each time step. The output of this layer is then passed to Stage 2.

\subsubsection*{Stage 2: LSTM Stage}
The LSTM is a type of Deep Learning architecture that is designed to exploit long term dependencies in time series data. Given the nature of dynamical systems data where time-based dependencies are abundant, LSTMs are a powerful choice to model such data. Thus, after the data has been processed by a sequence of dense layers in Stage 1, we apply a sequence of LSTM layers in Stage 2. The equations for the LSTM layer are given by Eq. (\ref{eq:gs}) with $s_t$ replaced by $a_{L, t}$, where $L$ is the number of the last layer in Stage 1.  The output of the LSTM layer from the final time step is then used as the input to the Stage 3.

\subsubsection*{Stage 3: Dense Output Stage}
Stage 3 consists of a sequence of dense layers. This stage serves as a final stop for processing the data before producing the output. The output of the last dense layer is the final predicted output $u_{nn}$ from the neural network. The output of the network is used in the following loss function to train the network
\begin{equation}
    Loss(u_{act}, u_{nn}) = \frac{1}{M} \sum_{i=1}^{M}  (u_{{act}_{i}} - u_{{nn}_{i}})^2,
\end{equation}
where  $M$ is the dimension of the output $u$.

\section{Methodology Evaluation}\label{sec_eval}
To quantitatively and qualitatively evaluate our methodology, we consider different dynamical systems each with increasing complexity.  The proposed learning framework is evaluated with respect to its ability to reproduce the dynamics of the actual system and its ability to predict future observations on a point-by-point basis.

\subsection{Candidate Systems}
We consider three candidate systems to test our hypothesis on, with each system being progressively more complex. Each candidate system exhibits one of the three types of differences between $M_{act}$ and $M_{curr}$: 1) differences in system parameters, {\it e.g.}, $u_t = \tilde{\mathcal{N}}[u,f_1,\cdots,f_p]$ with $g_i=0$ for all $i=1, \ldots, r$; 2) differences in external forcing functions and/or boundary conditions, {\it e.g.}, $u_t = \tilde{\mathcal{N}}[u,f_1,\cdots,f_p]$ with $g_i = f_i + \epsilon$ for $i=1, \ldots, r$ with $r \leq p$; and 3) missing terms in the partial differential equation describing the dynamics of the system, {\it e.g.}, $u_t = \tilde{\mathcal{N}}[u,f_1,\cdots,f_p]$ with $g_1\neq 0$.  We briefly describe the candidate systems below.

\subsubsection*{System 1: 1D Heat Equation} In our first system, we assume both the actual model $M_{act}$ and current model $M_{curr}$ system dynamics are given by
\begin{equation}
    \label{eqn_sec_prob_1D_heat}
    u_t = D_{*}u_{xx},
\end{equation}
where $x \in \mathbb{R}$, $u \in \mathbb{R}$ is the temperature, and $D_{*}$ is the diffusion coefficient and is set to either $D_{act}$ or $D_{curr}$. In this scenario, the discrepancy in the models arise due to a mismatch in the actual and assumed diffusion coefficients.

\subsubsection*{System 2: Lid Cavity Problem}
For our second system, we consider a modified version of the lid cavity problem presented in \cite{2DunsteadyNavierStokes}. The actual model, $M_{act}$, is given by
\begin{equation}
    \label{eqn_sec_prob_lid_cav_M_act}
    u_t = -(u\cdot\nabla)u -\nabla p +\frac{1}{Re}\nabla^2u + F.
\end{equation}
In \cite{2DunsteadyNavierStokes}, $F$ is chosen to be an external body force with a whirlpool effect. In this work, we employ the same $F$ as in \cite{2DunsteadyNavierStokes} but include a periodic element to $F$ whose components are given by
\begin{align*}
F_x &= (12-24y)x^4+(-24+48y)x^3+\\
&(-48y+72y^2-48y^3+12)x^2+ \\
&(-2+24y-72y^2+48y^3)x+\\
&(1-4y+12y^2-8y^3)120\sin{(e^{1.3t} + 80t)},\\
F_y & = (8-48y+48y^2)x^3+ \\
        & (-12+72y-72y^2)x^2+\\
        &(4-24y+48y^2-48y^3+24y^4)x+\\
        &(-12y^2+24y^3-12y^4)120\cos{(e^{1.3t} + 80t)}.
\end{align*}

In this system, the assumed model, $M_{curr}$, is given by the Navier-Stokes equation for incompressible flows,
\begin{equation}
    \label{eqn_sec_prob_lid_cav_M_curr}
    u_t = -(u\cdot\nabla)u -\nabla p +\frac{1}{Re}\nabla^2u
\end{equation}
with $\nabla u = 0$, where $x\in\mathbb{W}\subset\mathbb{R}^2$ denotes the position, $u\in \mathbb{R}^2$ is the flow velocity, $Re$ is the Reynolds number, and $p$ is the pressure. In contrast to the classical lid cavity problem, where the domain $\mathbb{W}$ is a square in which the top boundary moves with a constant speed, we assume the dynamics are subject to periodic boundary conditions at the top and bottom boundaries of the square given by
\begin{align*}
    u_{top} & = [2\sin{((e^{1.2t} + 60)t)}],\\
    u_{bottom} & = [2\sin{((e^{1.2t} + 50)t)}].
\end{align*}

\subsubsection*{System 3: Flow Around a Cylinder} For our third system, we consider the 2D flow around a cylinder modeled using the Navier-Stokes equations. The cylinder has a $1$$m$ radius and is centered at $(20,20)$ in a $50 m \times 40 m$ rectangular workspace. For the actual system, $M_{act}$, the cylinder moves vertically along the $y = 20$ axis such that its center moves periodically between $(20,21)$ and $(20,19)$ at a frequency of $0.3927$ $rad/sec$.  The velocity profile at the left boundary is set to be a uniform stream while a zero pressure outflow condition is imposed at the right boundary. The Reynolds number is set to $200$.  In this scenario, the system model or dynamics, $M_{curr}$, is assumed to be that of the stationary cylinder placed in the same uniform free stream flow, at the same location, with the same radius, operating at the same Reynolds number.  We note that the oscillation frequency for the moving cylinder in $M_{act}$ is set to be approximately the vortex shedding frequency of $M_{curr}$.

\subsection{Implementation}
The details of each system's architecture are summarized in Tables \ref{exp1-model}, \ref{exp2-model}, and \ref{exp3-model}. We use Adam \cite{adamoptim}, a powerful and computationally efficient optimization algorithm with the recommended default  parameters to initialize the algorithm. We set the algorithm batch size to 64, and used the Python package Keras \cite{keras} to train the network for a total of 50 epochs. Note that for our dense layers, we chose ReLU as our activation function. The function demonstrated the best performance on our tasks.

\begin{table}[H]
\centering
\resizebox{\columnwidth}{!}{
\begin{tabular}{|l|l|l|}
\hline
\textbf{Layer Kind}                              & \textbf{Activation Function} & \textbf{Number of Nodes} \\ \hline
Input 0: $U_{curr}$, Coordinates                    & N/A                      & N/A                      \\ \hline
Layer 1: TDDL{[}Input 0{]} & ReLU                     & 32                       \\ \hline
Layer 2: LSTM Layer{[}Layer 1{]}                   & Tanh/Sigmoid             & 64                       \\ \hline
Layer 3: LSTM Layer{[}Layer 2{]}                   & Tanh/Sigmoid             & 32                       \\ \hline
Layer 4: LSTM Layer{[}Layer 3{]}                   & Tanh/Sigmoid             & 32                       \\ \hline
Layer 5: Dense Layer{[}Layer 4{]}                  & ReLU                     & 10                       \\ \hline
Layer 6: Dense Layer {[}Layer 5{]}                 & Linear                   & 1                        \\ \hline
\end{tabular}
}
\caption{Neural network parameters for System 1. Note that TDDL stands for Time Distributed Dense Layer, LSTM stands for Long Short-Term Memory, and ReLU stands for Rectified Linear Unit.}
\label{exp1-model}
\end{table}

\begin{table}[H]
\centering
\resizebox{\columnwidth}{!}{
\begin{tabular}{|l|l|l|}
\hline
\textbf{Layer Kind}                              & \textbf{Activation Function} & \textbf{Number of Nodes} \\ \hline
Input 0: $U_{curr}$,$F$, Coordinates                  & N/A                      & N/A                      \\ \hline

Layer 1: TDDL{[}Input 0{]} & ReLU                    & 32                       \\ \hline

Layer 2:TDDL{[}Layer 1{]} & ReLU                     & 64                       \\ \hline
Layer 3: LSTM Layer{[}Layer 2{]}                   & Tanh/Sigmoid             & 64                       \\ \hline
Layer 4: LSTM Layer{[}Layer 3{]}                   & Tanh/Sigmoid             & 32                       \\ \hline
Layer 5: LSTM Layer{[}Layer 4{]}                   & Tanh/Sigmoid             & 32                       \\ \hline
Layer 6: Dense Layer {[}Layer 5{]}                 & ReLU                     & 10                      \\ \hline
Layer 7:Dense Layer {[}Layer 6{]}                 & Linear                   & 2                       \\ \hline
\end{tabular}
}
\caption{Neural network parameters for System 2. Note that TDDL stands for Time Distributed Dense Layer, LSTM stands for Long Short-Term Memory, and ReLU stands for Rectified Linear Unit.}
\label{exp2-model}
\end{table}

\begin{table}[H]
\centering
\resizebox{\columnwidth}{!}{
\begin{tabular}{|l|l|l|}
\hline
\textbf{Layer Kind}                                & \textbf{Activation Function} & \textbf{Number of Nodes} \\ \hline
Input 0: $U_{curr}$, Coordinates, Cylinder Position & N/A                      & N/A                      \\ \hline
Layer 1: TDDL{[}Input 0{]}   & ReLU                     & 32                       \\ \hline
Layer 2: TDDL{[}Layer 1{]}   & ReLU                     & 64                       \\ \hline
Layer 3: LSTM Layer{[}Layer 2{]}                     & Tanh/Sigmoid             & 64                       \\ \hline
Layer 4: LSTM Layer{[}Layer 3{]}                     & Tanh/Sigmoid             & 32                       \\ \hline
Layer 5: LSTM Layer{[}Layer 4{]}                     & Tanh/Sigmoid             & 32                       \\ \hline
Layer 6: Dense Layer {[}Layer 5{]}                   & ReLU                     & 10                       \\ \hline
Layer 7: Dense Layer {[}Layer 6{]}                   & Linear                   & 2                        \\ \hline
\end{tabular}
}
\caption{Neural network parameters for System 3. Note that TDDL stands for Time Distributed Dense Layer, LSTM stands for Long Short-Term Memory, and ReLU stands for Rectified Linear Unit.}
\label{exp3-model}
\end{table}

Given the lightweight nature of our networks and the small size of input data, we trained the networks on a CPU Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz. Tensorflow, the backend of Keras, automatically distributes training on multiple cores.  The average time for completing one epoch for Systems 1, 2, and 3 is 1, 60, and 40 seconds respectively.  The differences in training time between each system is mostly due to the training set size. The marginal difference  between each system architecture does not significantly change the training time.

It is important to note that expanding the neural net input size will impact the computational time. Adding more points to the hypercube will result in $d$ more connections where $d$ is the number of nodes in the Stage 1 first layer. These $d$ new connections represent the new input contribution to each node in the first layer. We can also apply the network on longer data sequences. This would not result in any new connections, but it will result in applying Stages 1 and 2 of the network on the added time steps. Both of these changes, when studied independently, will result in a constant increase in the number of operations for both prediction and training.

There is also an impact on computational time through the addition of more data. In training neural networks, we apply the same vectorized operations, mostly matrix multiplications, on batches of data. The nature of this computational process means that for each new data point, the number of operations for both training and prediction increases by a constant factor.

Finally we note that in solving new problems, we might need to expand the network representational capacity by adding more nodes and layers. The change in the computational cost of the network will heavily depend on the size and complexity of the new network. However, recent advances in GPU development tailored specifically for Deep Learning offers a range of solutions for building optimized and scalable implementations of complicated and heavy architectures.

\subsection{Datasets}

In this work, we employ numerical solutions to the actual and assumed models, $M_{act}$ and $M_{curr}$, to generate the ground truth, test, and training data sets.  The ground truth and actual system observations, $U_{act}$, are obtained by numerically solving $M_{act}$.
Similarly, the values for $U_{curr}$ are obtained by numerically solving $M_{curr}$ for the assumed parameter values, e.g., $\tilde{{\cal N}}$.
For System 1, the 1D heat equation given by Eq. \eqref{eqn_sec_prob_1D_heat} was solved using the finite volume based PDE solver in Python (FiPy).  The equation was discretized on a spatial $50 \times 1$ grid over $0.006$ seconds with time step of $0.000012$. The number of time frames is $500$. For System 2, a finite difference scheme was used to solve the Navier Stokes equations given by \eqref{eqn_sec_prob_lid_cav_M_act}-\eqref{eqn_sec_prob_lid_cav_M_curr}.  The equation was discretized on a spatial $30 \times 30$ grid over $2$ seconds with time step of $0.001$. The number of time frames is $2000$. Lastly, numerical solutions for System 3, flow around a cylinder, were obtained using OpenFoam \cite{OpenFoamVortexTutorial}.  The datasets for $U_{act}$ and $U_{curr}$ were obtained on a largely uniform grid consisting of approximately $100 \times 70$ points over $2000$ seconds using OpenFoam's {\tt pimpleFoam} solver with a solution time step of $0.001$ seconds.  However, $U_{act}$ and $U_{curr}$ consist of data obtained at every second over the total $2000$ second simulation run and thus the data consist of $2000$ time frames.

Training and test datasets are comprised of both $U_{curr}$ and $U_{act}$.
Assuming $U_{act}$ and $U_{curr}$ consists of $N$ frames ($N_x \times N_y$ grid points in each frame), we partition the data into  four sets: training, validation, local test, and future test. For the training, validation, and local test sets, we consider $K$ consecutive frames. In order to split the data in the $K$ frames between the three sets, we split the set of $N_x \times N_y$ grid points at each frame randomly between training, validation and local test sets. We choose to include $60\%$ of the grid points in the training set, 10\% of the grid points in the validation set, and $30\%$ of the grid points in the local test set. Note that the split is the same for each frame. For the future test set, we consider the remaining $T = N - K$ frames with all the grid points. We use the training set to train the network, the validation set to test the effect of hyperparameter optimization and different network architectures on the network performance, the local test set to measure the chosen network ability to generalize over unseen grid points, and the future test set to measure the network ability to generalize over unseen dynamics, {\it i.e.}, predict future observations.

For System 1, we include the first 150 frames in the training, validation, and local test sets ($ K=150$) and the last 350 frames ($T=350$) in the future test set. This setup results in a total of 4,144 points for training, 740 points for validation, 2,220 for local test, and 16,800 for future test. For System 2, we use the first 1,000 frames for training ($ K=1,000$), validation and local test and use the last 1,000 ($T=1,000$) frames for the future test. This results in a total of 469,060 points for training, 77,844 points for validation, 235,528 points for local test, and 784,000 points for future test. For System 3, we use 100 frames of data between the frames $500$ and $600$, \ie between $500$ and $600$ seconds, for the training, validation and local test set. We used the 1,400 frames after 600 seconds for the future test. This results in a total of 399,800 points for training, 66,600 for validation, 200,000 points for local test, and 9,786,000 points for future test.

\subsection{Evaluation Metrics}\label{subsec_eval}
To assess model performance, we use two benchmarks to measure the difference between two sets of $F$ frames: $Set_{1}$ (tested set) and $Set_{2}$ (ground truth). In our evaluation, $Set_{1}$ will either be $U_{curr}$ or $U_{nn}$ and $Set_{2}$ will be $U_{act}$.

\paragraph{Mean Squared Errors (MSE)} The first benchmark uses the mean squared difference between $Set_{1}$ and $Set_{2}$. For 2D output, we take the average of the two outputs for every point before computing the MSE. We also include the mean magnitude square difference (MMSD) and the mean cosine similarity (MCS) between the two sets in the benchmark for the Lid Cavity and the Flow Around a Cylinder problems in Systems 2 and 3 since both are 2D systems.

\paragraph{Proper Orthogonal Decomposition} The second benchmark compares the proper orthogonal decomposition (POD) modes that accounts for $99\%$ of the system variation.  Complex nonlinear dynamical systems can exhibit significant spatiotemporal variations, often at differing scales.  To extract the dominant dynamics of these systems, techniques for modal analysis are often used to construct a reduced order representation of the dynamics. POD is a data-driven reduced order modeling strategy that is often used to identify the dominant dynamics of a system purely from observations \cite{Sirovich1987TurbulenceStructures, KirbyBook2000}.

Given $N$ snapshots of the system states which can be obtained either through measurements and/or numerical simulations, let $\bm{x(t)} = [x_1(t),...,x_k(t)]^\top$ denote the set of spatial coordinates in $\mathbb{W}$ at $t = 1,\ldots,N$.  We note that the points in $\bm{x(t)}$ correspond to the grid points in which $u_{act}$ and $u_{curr}$ values are provided at some given time $t$.  Using $\bm{x(t)}$, we can construct a covariance matrix as
\begin{equation}\label{cov_mat}
	\bm{K} =  \frac{1}{m} \sum_{t=1}^{m} \bm{x(t)x(t)}^\top = \frac{1}{m} \bm{XX}^\top,
\end{equation}
where $\bm{X} \in \mathbb{R}^{n \times m}$ with its columns as $\bm{x(t)}$.  To extract the dominant dynamic modes from the data given by $\bm{x(t)} = [x_1(t),...,x_k(t)]^\top$ for $t = 1,...,N$, we obtain the low dimensional basis for the data by solving the symmetric eigenvalue problem
\begin{equation*}
	\bm{K \phi_{i}} =  \lambda_{i} \bm{\phi_{i}},
\end{equation*}
where $\bm{K}$ has $N$ eigenvalues such that $\lambda_{1} \geq \lambda_{2} ... \geq \lambda_{N} \geq 0$ and the eigenvectors $\bm{\phi}$ are pairwise orthonormal.

The original basis is then truncated into a new basis $\bm{\Phi}$ by choosing $k$ eigenvectors that capture the desired fraction, $E$, of the total variance of the system, such that their eigenvalues satisfy
\begin{equation*}
	\frac{\sum_{i=1}^{k} \lambda_i}{\sum_{i=1}^{n} \lambda_i}  \geq E.
\end{equation*}
Each term $\bm{x(t)}$ can be written as
\begin{equation} \label{field}
	\bm{x(t)} = \bm{\Phi c(t)},
\end{equation}
where $\bm{c(t)} = [c_1(t), ..., c_k(t)]^\top$ holds time-dependent coefficients and $\bm{\Phi} \in \mathbb{R}^{n \times k}$ with its columns as $\bm{\phi_1}$,...,$\bm{\phi_k}$. The low-dimensional, orthogonal subspace associated with $\bm{\Phi}$ is an optimal approximation of the data with respect to minimizing least squares error.

To compare the POD modes, we compute the inner product, {\it i.e.} the cosine similarity, between the two sets of principal components obtained for $Set_{1}$ and $Set_{2}$. We call this metric CS-POD, for short. We calculate the statistics of both benchmarks on two cases: Case 1, $Set_1$ is $M_{curr}$ and Case 2, $Set_1$ is $M_{nn}$. Case 1 provides a relative baseline for measuring the performance of the neural net in Case 2. We report the first benchmark statistics over training, local test, and future test sets, and the second benchmark statistics over the entire simulation.

\section{Results and Discussion}\label{sec_results}
We present and discuss the results of our proposed learning framework for each of the candidate systems.

\subsubsection*{System 1: 1D Heat Equation}
 Figure \ref{exp1_fig} shows the temperature data generated by $M_{act}$, $M_{curr}$, and $M_{nn}$ for the entire spatiotemporal domain.  In these simulations, $D_{act}$ and $D_{curr}$ were set to $15$ and $1$ $mm^2/s$ respectively. Qualitatively we see that the network model does an excellent job in resolving the inaccurately modeled dynamics and accurately captures the true dynamics of the system. Figure  \ref{exp1_b1} quantitatively shows the network's ability to generalize over local unseen grid points as well as data in the future set. In fact, one can see that the error between $U_{act}$ and $U_{nn}$ is orders of magnitudes less than that of $U_{act}$ and $U_{curr}$. Moreover, the error bars between $U_{act}$ and $U_{nn}$ are so small that the orange bars are not visible in the graph (the exact values for comparison are denoted in the figure). The quantitative results are further confirmed in Fig. \ref{exp1_b2} which shows the CS-POD for the POD modes. In this problem, the POD decomposition of $U_{nn}$ over the entire simulation yielded two principal modes as did the POD decomposition of $U_{act}$. The agreement between these POD modes is excellent as demonstrated by a CS-POD values that are very close to unity, as seen in  Fig. \ref{exp1_b2}.

\begin{figure*}[ht]
    \centering
    \subfloat[]{\includegraphics[width = 0.45\linewidth]{figs/exp1_fig.png}\label{exp1_fig}}
    \subfloat[]{\includegraphics[width = 0.275\linewidth]{figs/exp1_b1_mse.png}\label{exp1_b1}}
    \subfloat[]{\includegraphics[width = 0.275\linewidth]{figs/exp1_b2.png}\label{exp1_b2}}
    \caption{(a) System 1: Temperature as a function of the spatial and temporal coordinates for (left) $U_{curr}$, (middle) $U_{nn}$, and (right) $U_{act}$. (b) MSE between $U_{act}$ and $U_{curr}$ (light blue) and $U_{act}$ and $U_{nn}$ (dark blue) for system 1. (c) Cosine similarity between the first principal POD mode of $U_{act}$ and $U_{curr}$ (light blue) and $U_{act}$ and $U_{nn}$ (dark blue) for system 1.}
    \label{fig:system1}
\end{figure*}

\subsubsection*{System 2: Lid Cavity Problem}
  Figure \ref{exp2_fig1} shows a snapshot of the vector field generated by $M_{act}$, $M_{curr}$, and $M_{nn}$ for the entire domain at $t=1.147$ seconds. As with System 1, we see qualitatively that the network model does an excellent job in resolving the inaccurately modeled dynamics and does accurately capture the actual dynamics of the system. Figures \ref{exp2_b1_mse}-\ref{exp2_b1_mcs} quantitatively show the network's ability to combine $U_{curr}$ and observations of $g_1 = F$ to correctly predict $U_{act}$ over local unseen grid points as well as data in the future set.  Figures \ref{exp2_b1_mse} and \ref{exp2_b1_MMSD} show respectively that the MSE and MMSD between $U_{act}$ and $U_{nn}$ is orders of magnitudes less than that of $U_{act}$ and $U_{curr}$. The quantitative results are further
confirmed in Figs. \ref{exp2_b1_mcs} which shows that the mean cosine similarity between $U_{act}$ and $U_{nn}$ are close to unity thus demonstrating that $U_{nn}$ is resolving the actual dynamics to a far greater degree than is $U_{curr}$. Similarly, Fig. \ref{exp2_b2} shows the CS-POD for the first six principal POD modes over the entire simulation. The CS-POD values again demonstrate the network's ability to resolve the actual system's dynamics. In short, the high degree of accuracy  shows that our network is capable of correctly predicting observations both in previously unseen regions in the workspace as well as in future time steps.

\begin{figure}[H]
    \centering
    \includegraphics[width = 0.9\linewidth]{figs/exp2_fig1.png}
    \caption{System 2:Vector field at $t=1.147$ seconds for (left) $U_{curr}$, (middle) $U_{nn}$, and (right) $U_{act}$.}
    \label{exp2_fig1}
\end{figure}

\begin{figure*}
    \centering
    \subfloat[]{\includegraphics[width=0.49\linewidth]{figs/exp2_b1_mse.png}\label{exp2_b1_mse}}
    \subfloat[]{\includegraphics[width=0.49\linewidth]{figs/exp2_b1_mmsd.png}\label{exp2_b1_MMSD}}\\
    \subfloat[]{\includegraphics[width =0.49\linewidth]{figs/exp2_b1_mcs.png}\label{exp2_b1_mcs}}
    \subfloat[]{\includegraphics[width =0.49\linewidth]{figs/exp2_b2.png}\label{exp2_b2}}
    \caption{(a) MSE between $U_{act}$ and $U_{curr}$ (light blue) and $U_{act}$ and $U_{nn}$ (dark blue) for system 2. (b) MMSD  between $U_{act}$ and $U_{curr}$ (light blue) and $U_{act}$ and $U_{nn}$ (dark blue) for system 2. (c) Mean cosine similarity between $U_{act}$ and $U_{curr}$ and $U_{act}$ and $U_{nn}$ for system 2. (d) Cosine similarity between the first five POD modes of $U_{act}$ and $U_{curr}$ (light blue) and $U_{act}$ and $U_{nn}$ (dark blue) for system 2.}
    \label{fig:system2_compare}
\end{figure*}

It is important to note that the periods of the body force and the moving upper and lower boundaries in System 2 are not constant.  In fact, they change exponentially as a function of time.  Ideally, the trained network should capture this exponential change in the periods and be able to accurately predict future values outside of the training frames. In reality though, the prediction accuracy would degrade the farther out the prediction times are from the training times. To quantify this behaviour, three training regimes were considered with different training set lengths. The training sets for the three regimes contained the first 500 frames, first 750 frames, and the first 1000 frames of the data set respectively. We evaluated the system's predictive power using intervals of 250 future output frames and the results are shown in Fig. \ref{pre_power}. The metric (MSE, MCS and MMSD) for each interval is computed across all 250 frames in that interval. As expected, the prediction accuracy degrades the further out the prediction time is from the training set. For this particular case, the network is able to predict approximately one training period into the future, with a fair degree of accuracy.

\begin{figure*}
    \centering
    \includegraphics[width = 0.9\linewidth]{figs/exp2_b3.png}
    \caption{Comparison between the predictive power of the $M_{nn}$ trained using the first $500$, $750$, and $1000$ frames. The $x$ axis denotes the time interval in increments of $250$.  The metric for each interval represents the metric value computed for the frames in that interval.}
    \label{pre_power}
\end{figure*}

\subsubsection*{System 3: Flow Around the Cylinder}
 Figure \ref{exp3_fig1} shows a snapshot of the magnitude of the velocity field at $t=1390$ seconds generated by $M_{act}$, $M_{curr}$, and $M_{nn}$. In these results, the Reynolds numbers for both $M_{act}$ and $M_{curr}$ were set to $200$.  As with the previous two systems, we see qualitatively that the network model does an excellent job in resolving the inaccurately modeled dynamics and does accurately capture the actual dynamics of the system. In particular, note that the network model $M_{nn}$ is accurately capturing the vortex shedding frequency while the $M_{curr}$ vortices are out of phase with the actual vortex shedding pattern.  As in the previous systems, Figs. \ref{exp3_b1_mse}-\ref{exp3_b1_mcs} quantitatively show the network's ability to combine values of $U_{curr}$ as well as $g_1 = O(t)$ (where $O$ indicates the position of the cylinder at time $t$) to correctly predict $U_{act}$ over local unseen grid points as well as data in the future test set. Figures \ref{exp3_b1_mse} and \ref{exp3_b1_mmsd} show respectively that the MSE and MMSD between $U_{act}$ and $U_{nn}$ is orders of magnitudes less than that of $U_{act}$ and $U_{curr}$. The quantitative results are further confirmed in Figs. \ref{exp3_b1_mcs} which shows that the mean cosine similarity between $U_{act}$ and $U_{nn}$ are close to unity thus demonstrating that $U_{nn}$ is resolving the actual dynamics to a far greater degree than is $U_{curr}$. Similarly, Fig. \ref{exp3_b2} shows the CS-POD for the first six principal POD modes over the  entire simulation. The CS-POD values again demonstrate the network's ability to resolve the actual system's dynamics. In short, the high degree of accuracy shows that our network is capable of correctly predicting observations both in previously unseen regions in the workspace as well as in future time steps for systems exhibiting more complex dynamics.

\begin{figure}
    \centering
    \includegraphics[width = 0.7\linewidth]{figs/exp3_fig1.png}
    \caption{System 3: Magnitude of the velocity profile given by (left) $U_{curr}$, (middle) $U_{nn}$, and (right) $U_{act}$ at time $t= 1390s$.}
    \label{exp3_fig1}
\end{figure}

\begin{figure*}
    \centering
    \subfloat[]{\includegraphics[width =0.49\linewidth]{figs/exp3_b1_mse.png}\label{exp3_b1_mse}}
    \subfloat[]{\includegraphics[width =0.49\linewidth]{figs/exp3_b1_mmsd.png}\label{exp3_b1_mmsd}}\\
    \subfloat[]{\includegraphics[width =0.49\linewidth]{figs/exp3_b1_mcs.png}\label{exp3_b1_mcs}}
    \subfloat[]{\includegraphics[width =0.49\linewidth]{figs/exp3_b2.png}\label{exp3_b2}}
    \caption{(a) MSE between $U_{act}$ and $U_{curr}$ (light blue) and $U_{act}$ and $U_{nn}$ (dark blue) for system 3. (b) MMSD  between $U_{act}$ and $U_{curr}$ (light blue) and $U_{act}$ and $U_{nn}$ (dark blue) for system 3. (c) Mean cosine similarity between $U_{act}$ and $U_{curr}$ and $U_{act}$ and $U_{nn}$ for system 3. (d)Cosine similarity between the first six POD modes of $U_{act}$ and $U_{curr}$ (light blue) and $U_{act}$ and $U_{nn}$ (dark blue) for system 3.}
    \label{fig:system3_compare}
\end{figure*}

To evaluate the predictive performance of $M_{nn}$, we focus on the network's ability to identify the periodicity of the oscillations. Since System 3 is periodic, once the network learns the true periodicity of the dynamics, it has effectively learned the true dynamics of the system for all future times. To quantify the difference in periodicity between the model output and the ground truth, the following analysis was performed. For each point in the local test set, $\tau_i$, we consider its time series from frame $600$, the last frame in the training set, to frame $2000$ in both $U_{nn}$ and $U_{act}$. We denote these as $U_{nn}(\tau_i, 600-2000)$ and $U_{act}(\tau_i, 600-2000)$ respectively. We start by computing the frequency spectrums of $U_{nn}( \tau_i,600-2000)$ and $U_{act}(\tau_i,600-200)$ using the Fast Fourier Transform (FFT) which we denote as $FFT_{nn}$ and $FFT_{act}$.  Consider the percentage mean absolute difference between the frequencies that corresponds to the energy peaks between $FFT_{nn}$ and $FFT_{act}$ which we denote by $\%\Delta(FFT_{U_{nn}}, FFT_{U_{act}})$. The mean of $\%\Delta(FFT_{U_{nn}}, FFT_{U_{act}})$ is then computed for every grid point in the local test set which resulted in a value of $0.0239$. This analysis indicates that the neural network output not only accurately captures the periodicity of the underlying phenomena but it is able to correctly identify the global features of the dynamics. In short, once the network captures the periodicity, it can then predict the system's behavior at any time in the future.

\section{Conclusion and Future Work}\label{sec_conclusion}

We have proposed a data-driven modeling strategy based on a neural network machine learning framework that enables one to overcome improperly or inadequately modeled dynamics for systems that exhibit complex spatiotemporal behavior.  Given a system model that does not accurately capture the true dynamics, our machine learning strategy uses data generated from the improper system model combined with observational data from the actual system to create a neural network model. As we have shown with three complex dynamical systems, the network model that is created is capable of accurately resolving the incomplete or inaccurate dynamics to generate solutions that compare very favorably with the actual dynamics, both in previously unobserved regions as well as for future states.

Our approach leverages state-of-the-art machine learning frameworks and existing, but limited, knowledge of the physical constraints that drives a process.  The result is an equation-free representation of the system dynamics that encodes a baseline understanding of the underlying physics that drives the process. Since our output is a neural network representation of the system model, the output of our network consists of a set of pointwise inferences and thus is equation-free.  Nevertheless, the output can be fed into existing data-driven model discovery techniques to obtain closed-form equation representations of the dynamical system \cite{Brunton16,lusch2017deep}.

In the future, we plan to perform a detailed analysis on our learning framework performance for different error bounds to better understand acceptable deviations from the true model. Associated with this is the effect of noise, and to this end we plan to investigate how measurement uncertainty in $\hat{U}_{act}$ impacts the performance of $M_{nn}$. Since real-world systems are inherently noisy, we must be able to incorporate noisy observational data while still accurately capturing the system's dynamics. As such, it is important to be able to deal with situations where every observation is subject to a noise that is non-negligible or with situations where one has very noisy outlier observations.  While the impact of noise on a network's performance is well documented and studied in the computer vision literature \cite{ref:Plotz2017}, its impacts on networks modeling more complex phenomena is less well understood. A complete analysis of the effect of noise includes consideration of both additive and multiplicative noise, and involves analyzing simulated systems where deterministic and stochastic elements can be tightly controlled to establish ground truth for comparisons.

By developing methods that can deal with negligible and non-negligible noise, we will enable the study of complex and high-dimensional systems including those found in fluid dynamics and in particular geophysical fluid dynamics.  Fluid flows are complex and exhibit multi-scale phenomena whose dynamics are not at all well-understood.  Even the underlying physical mechanisms for flows are not fully understood. In the future, we plan to use the framework developed in this article to make predictions and estimations. For example, in a geophysical flow, information such as wind forcing or data from depth, may not be included in the models. Even with noisy and sparse observations, we would like to investigate if our framework can be used to accurately resolve the inadequately modeled dynamics.

 \clearpage

\bibliographystyle{plainnat}

redpajama_set_name: RedPajamaArXiv


The Course and Practice Range are open for play! Carts Available!
Weddings, Banquets, Bar Mitzvahs and More!
Lessons & Clinics from our PGA Pro Doug White!
Become part of the family and receive all the benefits!
Play Golf at Lake Forest!
Whether it's a shotgun start or tee times, Lake Forest Golf Club's staff will be there for you to help you organize and execute a golf outing that is well planned, fun and carefree.
Become part of the Lake Forest Golf Club Family! Purchase a 2019 Season Membership and save on golf all year long!
Save Up to 40% on all posted rates All Season Long! Discounts are applicable on all rates : 9 and 18 hole rates, senior, junior rates, and twilight rates. Click for more details on LFGC Saving Card benefits!
Our banquet menu features delicious hors d'oeuvres, a variety of buffet menus for casual and more formal functions, and delectable desserts. At Lake Forest, we use the freshest ingredients possible, and work with our customers to address special dietary needs. Contact us to today and we'll help plan all the details of your next event.
Contact Jenny Foster, Event Planner at 734-994-8580 Ext. 4 or at Jenny@lkforest.com.
Now is the time to purchase a 2019 Season Membership!


Toronto. Another delightful newsletter by our editor Sonja Pushchak. Sonja's newsletters have themes – this issue mixes Valentine's Day with Black History month.
She begins with a backgrounder on the song Paper Moon – a riff on Valentine's Day. Next up is a wonderful introduction to our next speaker, Dr Julie Crooks whose talk will celebrate Black History month as does page three on the Scurlock Studio in Washington DC and page four on a Toronto Exhibition called Rich and Poor at the RIC featuring photographs by Jim Goldberg.
This is followed by an update to an earlier story on pre photographic use of lenses by oil paint artists. David does a comparison of a new Samsung smartphone camera and a "Y2K" vintage Olympus digital camera. Louise tosses in her always interesting Web Talks column (with live links). In Photo History, Sonja discusses Kara Walker and Shadows on the Wall, a modern take on the pre photographic tool, the silhouette, this time using obviously Black antebellum scenes and images.
All the above plus ads and posters make this another very readable issue. Click here or on the Diana F+ icon to read and print.
This entry was posted in newsletter and tagged AGO, auction, Black History, camera, Dr Julie Crooks, exhibit, moon, Olympus digital, painting, paper, Rich and Poor, Scurlock, smartphone. Bookmark the permalink.


A letter that Adrian Rogers wrote for use in the event of his death is being mailed to contributors to his Love Worth Finding radio and TV ministry, making it clear he wanted the ministry to continue after he was gone.
Children no longer beg for scraps of food here as commonly as they did when an American missionary first set foot in the town known as Tengchow.
The GBC Executive Committee, meeting in Atlanta on Dec. 13, heard a motion from Wayne Robertson, pastor of Morningside Baptist Church in Valdosta, to discontinue the Convention's relationship with Mercer University. Robertson, representing the Administration Committee of the Executive Committee, reminded those present of the action of the Convention regarding Mercer at the annual meeting in Columbus on Nov. 15.
Jackson, a historic town located midway between Atlanta and Macon, annually hosts a Christmas parade in December. A year ago First Baptist Church capitalized on the event by having a "Happy Birthday, Jesus Party," complete with a church float in the parade.
Bertie Mae Garrett has traveled the world for missions. Now she wants to share those experiences with others.
Terry P. Moncrief, 60, whose ministry was featured on the front page of the Dec. 8 issue of the Index, died on Dec. 7 from complications of brain cancer. He slipped into a comma earlier that day and passed away at 4 p.m.
Yalda Hajey, draped in traditional Assyrian scarves around his neck and waist, with red and green feathers protruding from his hat, dropped his vote into a ballot box, dipped his finger into a purple ink sponge and sprang into an Iraqi jig.
In the latter stage of his ministry, Paul planned to go to Spain. His goal was to be on mission where the Word of God was not proclaimed and Christ was not known. Spain was then what public schools are now – are they not?
An elementary school in Wisconsin allegedly has written new lyrics for "Silent Night" for its winter program, stripping away all references to Christ and replacing them with winter weather themes.
In this Index exclusive, Georgia Baptists are given an inside look at the believers who comprise the membership of the church where Lottie Moon served at the turn of the 20th century. Lottie died on Christmas Eve 1912 in the port of Kobe, Japan, as she was being evacuated to the United States. Her 50-pound body was decimated with starvation, but her love for the Chinese people is the legacy that continues to produce a spiritual harvest in the world's most populous nation.
When the superintendent in this Bible Belt town yanked baby Jesus from a fifth-grade school play – but left in symbols of Hanukkah and Kwanzaa, along with Santa Claus – a small army of parents erupted in protest.
At First Baptist Eastman, the Lottie Moon Christmas Offering giving has increased by $57,000 in the past four years.
The modern culture is seemingly becoming less interested in what historically has been termed "the work ethic." Conducting a skill or managing an office seems to have lost its appeal to many in contemporary culture. The American dream once was that one would secure a career, marry, and purchase a home. Seemingly, more people are interested in the "dream" of enjoyment through recreation rather than seeing their work as a source of satisfaction.
So significant is the need for a Christian understanding of money that the late Larry Burkett made a ministry of teaching about it. Burkett helped Christians recognize that accumulation of possessions and wealth is glorified in this society. He emphasized the proper management of God-provided resources. Still, status is desired because it indicates that one has arrived at significance. Many people, even Christians, fall into the trap of believing that monetary gain is equivalent to significance in life.
One Christmas season when I was pastor at First Baptist Church in Paducah, Kentucky. I decided to do a children's sermon. I had all the children to join me down front and I told the Christmas Story after reading a portion of the Luke 2 account. I didn't normally do this sort of thing because we were on live television on the NBC affiliate and with the children there was no way to predict what might happen.
Our children were born more than 30 years ago, but they had the advantage of being born in sanitized hospitals with competent physicians clothed in Martinized smocks doing their work with disinfected hands washed with soap containing Chlorhexidine Gluconate.
Are you going to miss the party?
I admit it. I am a traditionalist. I am a conservative. I am sentimental. I am old fashioned, maybe archaic, but I never thought of calling off worship services when Christmas was on Sunday. It just never occurred to me that doing that was an option, and it wasn't an option as long as my mother was living. I guess my parents just instilled into the fabric of my very being a loyalty and an allegiance to the church that just won't permit me to accept the notion that calling off church is an option.
I know the following comments will reflect my age as a retiree, but I also know that many seniors feel the same way about the direction of the musical programs currently in the church today. Isn't it amazing that everyone is concerned about taking Christ out of Christmas, and asking for money to fight the ACLU to prevent this from happening, while at the same time, the churches are taking Christ out of the music?
As I read the front page article in last week's Christian Index about Terry Moncrief and the Techwood Baptist Center being renamed for him, I was taken back to the late 1980's and Terry's impact on my life.
Respectfully, I would ask Terry Queen (Letter, December 8) if he has read Jimmy Carter's latest book in which he explains his personal opposition to abortion. Reading a news story in the AJC is not the same as reading a man's own words.


Seeking a web-designer for a blockchain e-sports project.
1. Design and develop a landing page (one-page; example: [login to view URL]).
3. Design the bitcointalk announcement (it's almost cutting the same images that were used for the website; example: [login to view URL]).
If you are interested, please send the most relevant examples from your practice, your price and deadlines.


text:
@class SearchViewController;
@interface AppDelegate : UIResponder <UIApplicationDelegate>

@property (strong, nonatomic) UIWindow *window;
@property (strong,nonatomic) SearchViewController * viewController;
@end

redpajama_set_name: RedPajamaGithub


Thrilling theme parks, outlet shopping, championship golf courses, and beautiful year-round weather make Orlando the ideal destination for thrill-seekers, shopaholics, avid golfers, and anyone looking for a family vacation.
Whether you're looking for a romantic weekend trip or a long sojourn of rest and relaxation, we invite you to stay with us at Maingate Lakeside Resort. Bring your family, your loved one, or yourself and enjoy an unforgettable fall getaway this year!
Every stay at Maingate Lakeside Resort is a special occasion. Make it even better with our exceptional "Weekday Deal" created just for you! Why wait? Book Now!
Save an extra 20% on Sun-Thurs.


text:
Audio Review: Reckless: Leather & Chrome series book 1 by Kiki Clark
Prison pen-pals become so much more in Reckless: Leather & Chrome book 1 and Laora experienced it in the audiobook form:
Ride hard. Love harder. Live recklessly.
Tank is done with his old life. All it's brought him is trouble and misery in the form of a four-year prison term. Though he did find one good thing while behind bars: Charles "CJ" Crane, pen pal extraordinaire.
After months of exchanging increasingly personal letters, Tank knows exactly where he's headed as soon as the prison gates open. CJ doesn't know he's coming, and he's not sure what he'll say when he gets there, but he knows one thing for certain.
The promises he and his boy made to each other in their letters may have been reckless, but he's willing to risk everything he's got on finding out if what they have could be real.
Reckless is the first book in the Leather & Chrome series and features a motorcycle club's annual charity ride, a possessive biker whose new favorite color is pink, a 24-year-old overly eager to experience his first time, and an excessive appreciation for perfectly lined tattoos.
Laora's Review:
It has been a while since I read (or listened to) a book by Kiki Clark. And it was good to get back to her writings. I am looking forward to the other books in the series, be it in writing or audio.
CJ and Tank met through a pen pals scheme for inmates when Tank was incarcerated and CJ was roped into the scheme by his friend.
After Tank his release he goes in search of CJ as they wrote about meeting. Their writing are personal and became more spicy.
When they meet they hit it of and due to circumstances move in together quickly. They fit in really well with the inclusive Motorcycle club with a difference to which Tank transferred after being released. They also explore and discover their kink site. The Motorcycle club turns out to have a a real social conscious and I love to see how that will cone to the for in future books.
Narration:
I have listened to a fair few of the books narrated by John Solo, but I find he seems to have a certain way for a type of character, so I am bit disappointed by his narration
Story 4.5 Stars Narration 4 Stars Overall 4 Stars
Reckless is available in kindle e-book, audio book and paperback! If you haven't tried Audible, you can try Audible Premium Plus and Get Up to Two Free Audiobooks!
4 Stars Age Gap Audio Kink Laora Motorcycle Pen Pals

redpajama_set_name: RedPajamaCommonCrawl


text:
How long would it take to sail from California to Australia?
How long does it take to sail from America to Australia?
How long does it take to go from California to Australia by boat?
Can you sail to Australia from USA?
How long would it take to sail to Australia?
What are 3 major events that brought immigrants to Australia?
Can you travel to Australia by ship?
How much is it to fly to Australia from America?
How long is a boat ride from New York to Australia?
Is it legal to sail to another country?
Can I come back to Australia?
Is it safe to sail around the Philippines?
Can you drive to Australia from England?
USA to Australia – 50 days Approx. Australia to Europe – 80 days Approx.
Those leaving from mainland ports on the east and west coasts range from three weeks to over 50 days.
There are cruises from the United States to Australia, but there aren't many because it's a long trip. A cruise to Australia is the comfortable and luxurious way to get down under. US to Australia cruises depart from Los Angeles, Hawaii, Seattle, New York, and San Francisco.
In calm weather a sailing ship might take as long as four months, while a well-run clipper ship with favourable winds could make the journey in a little over half this time.
THIS IS INTERESTING: Frequent question: Where are wilderness kayaks made?
Australia's Immigration History
Driven by the promise of a new life the Great Southern Land, waves of immigrants came to find fortune in the gold rush, to escape the social upheaval of the Industrial Revolution, two world wars and the aftermath of the Vietnam War.
But there are no regular passenger ships to Australia from Singapore or Indonesia or anywhere in the Far East. … You therefore need an infrequent passenger-carrying freighter for the last leg from Southeast Asia to Australia. There are one or two Asia-Australia freighters that will carry passengers.
The cheapest ticket to Australia from the United States found in the last 72 hours was $150 one-way, and $404 round-trip. The most popular route is from Los Angeles to Sydney and the cheapest round-trip airline ticket found on this route in the last 72 hours was $839.
The Sea Princess is a special ship that goes from New York to Sydney over a period of 38 days. On this particular route, the ship makes multiple stops along the Latin American coast.
Absolutely. Sailing does not need to be more difficult than flying or driving as long as you know what is required. The main things to pay attention to are: ports of entry, passport and visa requirements, boat ownership and cruising permits, and custom and quarantine regulations.
THIS IS INTERESTING: What happens to gas solubility when the scuba diver ascends?
Australia has strict border measures in place to protect the health of the Australian community. You cannot come to Australia unless you're an Australian citizen, permanent resident or in an exempt category. The Department of Home Affairs looks after this entry ban.
Generally the Philippines is safe from piracy except for Mindanao and the Sulu Sea which should be avoided due to the activities of Islamic extremist group Abu Sayyaf. The Philippines has been a very popular cruising region, for many reasons, for many years.
A BRITISH brother and sister spent 149 days driving more than 26,000km on an epic road trip from the UK to Australia in a Mini – only to be beaten by the WA outback. … "We've done so much, we've been on the road for 4½ months, we've been constantly breaking down and constantly fixing it.


text:
Manuals, Guidelines & Frameworks
COVID 19 Vaccine Perceptions: A 15 country study
1.1 Introduction and Background
1.1.1 Introduction and research aims
Since the emergence of COVID 19 in Wuhan, China, in December 2019 the disease has been declared a global pandemic and has been associated with the deaths of more than 2 3 million people around the world. It continues to be a dynamic and evolving pandemic.
The first case in Africa was reported in Egypt on 14 February 2020 and cases have since been reported in all 55 African Union Member States. At the start of 2021
Africa faced a second wave, with far more reported cases and deaths than in the early months of the global pandemic. In addition, new virus variants have emerged.
Initial public health responses to control the pandemic focused on promoting protective behaviors among the general population, including frequent hand washing, physical distancing and the use of face masks in public spaces However, many saw these only as interim measures to reduce the spread of the virus and hopes for a return to a sense of ' rested on the development of a safe and effective vaccine.
COVID-19 Perception Survey Final Report – ENG Download
Outbreak Brief 156: Coronavirus Disease 2019 (COVID-19) Pandemic
Outbreak Briefs
Guidelines for Enhanced Surveillance of Sudan Virus Disease
Communique on the Sudden Increase in COVID-19 Cases driven by the Omicron BF.7 Variants


Dale and Liz both continue to stay busy with conference work and short engagements. Therefore, you may want to check their ministry schedule on this website. Note: the duck icon indicates Liz's presence while the camel icon shows Dale's presence for each engagement on their ministry schedule.
It may be possible to engage one or both of them at your church or organization when they are already in your area. They reside in the Denver area, so it is quite easy for them to book flights from Denver to other major cities. Dale can provide availability dates and fees and will make all the arrangements for booking. Liz and Dale travel together, but also take separate dates as requested.
Note: When contacting Dale, please provide your name, phone number, city and state, and email address along with the requested date and type of event you are wanting Dale or Liz to schedule with you.


"Red Hamilton" A light weight fabric featuring two chest pockets and shoulder epaulettes with piping and matching collar contrasting fabrics and embroidery on the back yolk.
Da Vinci of California was established in 1952 and is as current today as it was then. Our "New Vintage" Collection Est. 2011, still represents the California Lifestyle and targets a young man who wants to be different and appreciates the classics. We do that by combining heritage with a modern twist from the use of classic designs & fabrics in a slimmer more modern fit.
© 2016 Global Rebels Inc.


Dunkin' Is Back With A New Donut Sneaker And You Can Buy It Right Now!
Dunkin' Donuts' motto says "America Runs On Dunkin," and now you can live by those words too with new sneakers from the chain.
Teaming up with apparel brand Saucony for its second year to celebrate the Boston Marathon, the new kicks have Dunkin's iconic logo on the sides of the shoes, a plastic donut on the laces, and comes in a box just like a dozen would.
Those looking to get their hands on a pair can hit Marathon Sports' website on Friday, while in-store roll-outs across Beantown are on the way.
But shoppers will have to act like they're on a sugar and caffeine high. Last year's sneakers sold out in hours.


text:
Davinder Singh SC named Dispute Resolution Lawyer of the Year, Drew sweeps litigation awards at ALB
Davinder Singh SC named Dispute Resolution Lawyer of the Year, Drew sweeps litigation awards at ALB SEA Law Awards 2018
Executive Chairman Davinder Singh, SC was named Dispute Resolution Lawyer of the Year at the annual Asian Legal Business (ALB) SE Asia Law Awards 2018 tonight. The award was presented in recognition of Mr Singh's outstanding performance and significant contribution to the evolving legal landscape of the Southeast Asia region. Mr Singh beat nine other contenders to win this title.
Drew & Napier also won the Regional Litigation Law Firm of the Year award and Singapore Litigation Law Firm of the Year award. We have been the sole recipient of the latter title for the past 14 years since the inception of the awards, cementing our reputation as undisputed leaders in this field.
The firm bagged two "Deal of the Year" awards for our involvement in high value, complex corporate work. They were: Debt Market Deal of the Year (Premium) award for Bumi Resources' Global Debt Restructuring and M&A Deal of the Year (Premium) for the Privatisation of Global Logistic Properties.
The deal involving Bumi Resources' Global Debt Restructuring was led by Sushil Nair, Deputy CEO and co-head of Drew & Napier's Corporate Restructuring & Workout practice group. This billion-dollar restructuring was among the largest debt restructuring transaction in Indonesia and in Southeast Asia in 2017.
The deal involving the Privatisation of Global Logistic Properties was led by Director Steven Lo, Head of the Corporate/M&A practice. This transaction is Asia's largest private equity buyout and one of the more significant public M&A transactions in Singapore in recent times. The deal was also named M&A Deal of the Year: Southeast Asia at the Asian Lawyer's Asia Legal Awards 2018.
Drew & Napier's CEO, Senior Counsel Cavinder Bull said: "We thank our clients for their continued support and trust in us. Our achievements would not be possible without our dedicated team of lawyers and staff."
The Awards dinner was held at Mandarin Oriental on Thursday, 10 May 2018.
About Asian Legal Business
The annual ALB SE Asia Law Awards pay tribute to the outstanding performance of private practitioners and in-house teams that have significantly contributed to the evolving legal landscape of the region. This year, the winners were decided after a rigorous two-tier assessment by a panel of more than 30 reputable judges from the legal academe, in-house teams, law firms, legal associations and non-profit organisations in the region.
Drew & Napier ranked top law firm in 5 practices by The Legal 500 Asia Pacific 2020
14 Drew & Napier lawyers recognised by peers as leading practitioners in their fields in Who's Who Legal 2019/2020
Jon Nair, Tan Teng Sen and Lin Shumin recognised in SBR's 'Singapore's most influential lawyers aged 40 and under 2019'
Head of Tax & Private Client Services Practice Ong Ken Loon singled out as the leading practitioner in Tax Dispute Resolution at the 2019 Asia Women in Business Law Awards
Drew & Napier maintains formidable standing in Chambers Asia-Pacific 2020 rankings
Drew & Napier named "Contributor of the Year 2019" by The Law Society of Singapore


This grand New Year Party eve celebration party that seems to emerge out as a winner in flying colours because it has all the ingredients for a fabulous New Year eve themed party celebrations.
First, it is by Celebrity group that has been organizing wonderful New Year parties every year for several years now. And those who have been there on weekends know what they have on offer. Here, everyone is treated like a celebrity – personalized attention, care.. you will never have to yell for a water refill on your table or to get someone to clean up the icing spill on the floor.
At Celebrity, you can expect things to be saner and better organized than most parties you've been to. No moving from one gate to the other just to gain access to the venue. No need of shouting your guts out to reclaims some parking space. That the venue is in the centre of the city ensures that you not late for the party.
Even your reclusive girlfriend who thinks twice before accompanying for a long drive will say yes in a jiffy to this. C'mon,the party is at Begumpet yaar. What more can you ask for.
Then there is music and masti. Not too loud – not too slow. Not so local that you are watching a Telugu or Hindi movie. Not so exotic that you feel lost and disconnected. Not so traditional that you start yawning and not so modern that you start getting a migraine. A mix of this and a mix of that – a Melange, so to speak.
There are three options for partygoers – to come party and leave after the party; to stay over and spend the night in a deluxe room; to stay over and spend the night in a premium room. Choose what works best for you.
Some parties tend to focus only on the MC, DJ and a handful of pretty people who also dance well. But Melange promises an interactive session where every partygoer feels like he is part of it. Unique contests and prizes in a host of interactive sessions is what will certainly keep you engaged throughout.
Life is a mix of this and that, a bit of happiness and sorrow. Perhaps that is the credo of Melange 2015 with a mix of this and that, but in this case, it's all fun and frolic, music and masti. So what are you waiting for? Get started on the bookings boss!


text:
Office of the Provost and Vice-President (Academic)
Deans Selections and Reviews
Program Changes and Approval
Provost Biography
Planning 2021-22
2020 COVID-19 Remote Teaching Award
The transition to remote teaching in response to the COVID-19 pandemic required extraordinary efforts by all of our professors and instructors and the many people who support them. However, some were truly exceptional in what they were able to achieve. In order to recognize these outstanding individuals, the University of Alberta will award 10 (ten) 2020 COVID-19 Remote Teaching Awards to full-time faculty members (staff category A1.1) and full-time Academic Teaching Staff (ATS) (staff category A2.1) who transformed traditional face-to-face courses into remote teaching environments for the Winter 2020 and Fall 2020 terms while exemplifying the University of Alberta's values - excellence in teaching, creativity and innovation, delivering an intellectually rewarding education environment for all learners, and, above all else, an unwavering commitment to serving the public good in the face of adversity and unprecedented challenges. Each award will come with $1,000 as a token of our appreciation.
Please consider nominating yourself, a professor or instructor, or a colleague using this brief nomination form. We encourage nominations of individuals who have been historically underrepresented at the University of Alberta and within teaching awards including women, visible minorities, Indigenous persons, persons with disabilities, and LGBTQ2S+ persons.
Nominations will be open until January 29, 2021.
Andrea Patrick
Portfolio Initiatives Manager
2-40B South Academic Building
Email: andrea.patrick@ualberta.ca


text:
Амос Булл (*1744 — †1825) — один з найперших американських композиторів.
 Андерс Генрік Булль (*1875 — † після 1909) — норвезький інженер-електрик.
 Люсьєн Булл (*1876 — †1972) — ірландський вчений, один з піонерів швидкісного фото.
 Ентоні Булл (*1908 — †2004) — британський інженер транспорту, президент Інституту транспорту.
 Енді Булл — австралійський співак.
 Елеонора Булл (англ. Eleanor Bull; 1550-1596) – далека родичка Вільяма Сесіла та кузина Бланш Перрі – вірної камеристки Єлизавети I Тюдор. Сумно відома, як власниця пансіону, в якому за загадкових обставин було вбито англійського поета та драматурга Крістофера Марлоу.

Див. також
 Джон Булль (John Bull — буквально Джон Бик) — прізвисько типового англійця.
 Ред Булл

redpajama_set_name: RedPajamaWikipedia


If you receive a Penalty Charge Notice please do not ignore it.
You will need to wait until the following day from the date the notice was issued before you can view the details, including photographs of the contravention, pay or appeal your Penalty Charge Notice. Notices issued on a Friday or at the weekend will not be available to view until the following Monday.
You can view details including photographs regarding the contravention.
You cannot pay and then subsequently appeal a Penalty Charge Notice.
If you pay within 14 days of the date your Penalty Charge Notice was issued, the charge will be reduced by 50% to £30.00.
By post to Customer Services, PO Box 7644, Perth PH2 1GP.
Cheques should be made payable to Perth & Kinross Council. Post-dated cheques will not be accepted. Please ensure that the Penalty Charge Notice number is written on the reverse of your cheque..
View the Penalty Charge Notice cancellation policy [18Kb] and Traffic Orders.
Once you have submitted an appeal for your Penalty Charge Notice, it will be placed on hold whilst we consider your appeal. You will receive a written response to your appeal within 20 working days.
Please note we will not discuss your appeal by telephone.
We will respond in writing to you once only, giving our decision in relation to your Penalty Charge Notice informal appeal, unless we need to contact you to request further evidence prior to any decision.
The vehicle's owner will automatically receive a Notice to Owner form if the Penalty Charge Notice has not been paid within 28 days of it being issued. This form offers you the opportunity to formally challenge your Penalty Charge Notice or pay the full £60.00.
If you decide not to pay your Penalty Charge Notice, but to formally challenge it, you must wait until the Notice to Owner form arrives. We cannot discuss your case verbally or in writing or consider any further information or evidence from you until this form is received by us, along with any supporting evidence and documents in relation to your formal challenge. Any correspondence relating to this matter will not be considered and will be filed.
When you receive a 'Notice to Owner', a formal representation can be submitted or full payment of £60 can be made.
Representation can only be made on certain grounds which are explained on the 'Notice to Owner' form.
Representations will be carefully considered and if the Council agrees with your representation you will receive a 'Notice of Acceptance' and the case will be closed and the Penalty Charge Notice will be cancelled. If not, you will receive a 'Notice of Rejection. If you receive a 'Notice of Rejection' we will not enter into further discussion with you or communicate with you in writing regarding the Penalty Charge Notice, any correspondence relating to this matter from you will be filed. You do, however, have the right to appeal to an independent adjudicator, the Parking and Bus Lane Tribunal Service, details will be sent to you with the 'Notice of Rejection'.
In certain circumstances vehicles issued with a Penalty Charge Notice can be towed away for example in the interest of public safety or repeated infringements.
Any vehicle removed will be taken to a secure area until such time as it is reclaimed by the owner.
On receipt of proof of ownership and payment of a towing charge of £150 the vehicle will be released. A storage charge of £20 per day, plus the payment of all outstanding Penalty Charge Notices will also be required.


text:
Bureau Home > Tsunami Warnings > Deep Ocean Tsunami Detection Buoys
Deep Ocean Tsunami Detection Buoys
Need Emergency Advice? Please listen to your local radio and TV announcements or call 1300 TSUNAMI (1300 878 6264) for latest warning information. For emergency assistance, call your local emergency authority on 132 500
The surface component of an operational deep-ocean tsunami detection buoy.
What is a deep-ocean tsunami detection buoy?
Deep-ocean tsunami detection buoys are one of two types of instrument used by the Bureau of Meteorology (Bureau) to confirm the existence of tsunami waves generated by undersea earthquakes. These buoys observe and record changes in sea level out in the deep ocean. This enhances the capability for early detection and real-time reporting of tsunamis before they reach land.
How does a deep-ocean tsunami detection buoy work?
A typical tsunami buoy system comprises two components; the pressure sensor anchored to the sea floor and the surface buoy. The sensor on the sea floor measures the change in height of the water column above by measuring associated changes in the water pressure. This water column height is communicated to the surface buoy by acoustic telemetry and then relayed via satellite to the tsunami warning centre.
The system has two modes - 'standard' and 'event'. The system generally operates in 'standard' mode, where it routinely collects sea level information and reports via satellite at relatively low frequency transmission intervals (i.e. every 15 minutes). This helps to conserve battery life and hence extend the deployment life. The tsunami buoy is triggered into 'event' mode when the pressure sensor first detects the faster moving seismic wave moving through the sea floor. It then commences reporting sea level information at one minute intervals to enable rapid verification of the possible existence of a tsunami. The system returns to standard mode after 4 hours if no further seismic events are detected.
Data from Australia's tsunami detection buoys are made freely available to the international community and the tsunami warning centres of other countries in real-time using the World Meteorological Organization's dedicated Global Telecommunication System (WMO GTS).
A pressure sensor ready for deployment on the sea floor.
How is the location for deployment of a tsunami buoy determined?
The best location for deployment of a tsunami buoy is determined by careful consideration of a number of factors. The tsunami buoy needs to be far enough away from any potential earthquake epicentre to ensure there is no interference between the earthquake signal at the buoy and the sea-level signal from the tsunami. On the other hand, the tsunami buoy needs to be close enough to the epicentre to enable timely detection of any tsunami and maximise the lead time of tsunami forecasts for coastal areas. In addition, tsunami buoys must ideally be placed in water deeper than 3000m to ensure the observed signal is not contaminated by other types of waves that have shallower effects (e.g. surface wind-waves). International maritime boundaries must also be considered when deploying tsunami buoy systems.
Where are Australia's tsunami buoys being deployed?
Australia is potentially vulnerable to tsunami generated by undersea earthquakes along subduction zones (where the earth's tectonic plates are moving under each other) to the northwest, northeast, east and southeast of Australia. Deep-ocean tsunami detection buoys (and coastal sea level stations) are used to monitor the oceans in each of these regions. There are also spare buoys on hand to ensure maintenance regimes and emergency replacements can be carried out if necessary.
Australia's first tsunami detection buoy was deployed on 15 April 2007 in the South East Tasman Sea, some 1200 km from Tasmania. The deployment was carried out in collaboration with NOAA. This buoy captures critical tsunami data from the oceans near the Puysegur fault line southwest of New Zealand. A few more tsunami buoys have since been deployed strategically to protect Australia from tsunami threat due to earthquake sources to the northwest, northeast and southeast of the Australian mainland.
How do these buoys contribute to tsunami warnings in Australia?
Due to the complexity and uncertainty as to whether an undersea earthquake has the potential to generate a tsunami, the observation of sea levels is a critical factor in verifying whether a tsunami has actually been generated. The use of actual sea level observations, as compared with reliance on seismic observations alone, therefore helps to significantly reduce the risk of false tsunami warnings being issued. All Australian-owned buoys, as well as deep-ocean buoys operated by other countries in the Australian region, provide critical data to Australia's tsunami warning system.
The buoys are just one part of Australia's sea-level observing system, which also includes a number of new and long standing coastal sea level stations that now have the ability to report sea level variations in real-time to monitor for tsunamis. Combined, these technologies provide a constant stream of sea level data for the Joint Australian Tsunami Warning Centre (JATWC) operated by the Bureau of Meteorology and Geoscience Australia, enhancing tsunami warnings for the Australian public.
What is the in-water life of a tsunami detection buoy?
The life cycle of a deployed tsunami buoy is approximately 2 to 4 years. The Bureau's maintenance regime involves the replacement of the surface buoy and the sea-floor pressure sensor every one to two years. The devices retrieved during regular maintenance are refurbished and made ready for the next redeployment.
The deployment of a DART™ buoy in the Tasman Sea.
Deep-ocean tsunami detection buoy technology was initially developed in the United States of America by the Pacific Marine Environmental Laboratory (PMEL) of the National Oceanic and Atmospheric Administration (NOAA) as "DART™" (Deep-ocean Assessment and Reporting of Tsunami) buoys.
The latest DART™ II systems contain two independent and redundant communications systems as back-up.
These systems are capable of measuring sea-level changes of less than a millimetre in the deep ocean.
Two-way communication between the tsunami buoy and the tsunami warning centre means that the buoy can be controlled remotely. This two-way communication allows for troubleshooting of the system and also allows people to put the systems into 'event' mode in case of a possible tsunami or for research purposes.
Tsunami Warnings
Warning Status - Australia
Warning Status - Indian Ocean
About Tsunami Warnings
Tsunami: The Ultimate Guide
The Warning Centre
Deep Ocean Detection Buoys
Past Tsunami Events
Pacific Tsunami Capacity Assessment Project
Tsunami Facts and Information


Here's a new reason why good work is sometimes not recognized at awards time.
I received an e-mail last night from Gina Tenorio, the president of the Inland Southern California Professional chapter of the Society of Professional Journalists. Layoffs at Southern California papers have reached the point where the chapter can't effectively run the contest and they didn't want to run a contest with an expensive entry fee in light of everyone's economic outlook this year.
A grim sign of the times.
I just got back from the Riverside Marriott, where the Inland Southern chapter of the Society of Professional Journalists just gave out the 2006 Southern California Excellence in Journalism Awards.
The Hesperia Star won three awards, with Peter bringing home the biggest kill: In an all-circulation category (meaning he was up against the Press-Enterprise, the Inland Valley Daily Bulletin and all the rest), he came in first for Best Editorial/Opinion Piece. The piece — "Censorship is wrong answer" — was never posted onto our old Web site, so on Monday, I'll dig it out of the archives and post it on the Web for all to see.
The Daily Press also cleaned up, with a ton of awards for photographer Michael Stenerson. Reporter Tatiana Prophet pulled in the most reporting awards for the High Desert, by my guesstimate. El Mojave and the Desert Dispatch also won awards.
More details can be found in the Daily Press story and on the Star site this week.
Update #1: Peter's editorial can be found here.
Update #2: Sadly, only one of the three award-winning pieces got commentary passed along by the SPJ judges.
I picked up second place in Best Feature Story for They Ain't Scared of No Ghost, second place in Best Law Enforcement/Legal Affairs Story for High-Flying in Hesperia and third place in Government/Political Story for Is Hesperia Casino Really Terminated?
This is my second award for a law enforcement story in two years, which I suspect will give a deceiving picture of what I was covering at the Hesperia Star in years to come. It's also my second casino-related award, which is more indicative.
Judges' comments should be up on the official SPJ site next week, and I'll link to them when they're posted.


Are you looking to achieve financial security for your retirement years? Our pension advisers are experienced in evaluating pension schemes and advising on pension plans for clients of all ages and incomes around Croydon and the nearby areas.
If you're ready to start planning your retirement or have a query about pensions and would like some advice, then please call our pension advisers today on 0203 859 3320 or visit our Croydon branch. Alternatively, you can complete our short enquiry form and one of our retirement planners will get in touch straight away. Our retirement planners can help you find a pension plan that meets your needs.