text:
\section{Introduction}
\label{submission}

Deep neural networks (DNNs) has been successfully applied in image recognition \cite{[a21]}, speech recognition \cite{[a22]}, natural language processing \cite{[a23]}, and other fields. However, recent studies revealed that DNNs are seriously vulnerable to adversarial examples. Many methods were proposed to generate adversarial examples \cite{[a1]}, \cite{[a2]}, \cite{[a3]}, \cite{[a4]}, \cite{[a5]}, \cite{[a6]}. At the same time, the corresponding defenses have also been extensively studied \cite{[a2]}, \cite{[a5]}, \cite{[a7]}, \cite{[a11]}, \cite{[a12]}, in which adversarial training is considered to be one of the most effective defensive methods at present \cite{[a12]}.

\begin{figure
\centering
\includegraphics[width=1in, height=1in]{change_trend2.pdf}
\includegraphics[width=1in, height=1in]{generate2.pdf}
\includegraphics[width=1in, height=1in]{fine-tune2.pdf}
\caption{Illustration of the decision boundary between Class1, Class2, and Class3. The circle with solid line represents the example $\boldsymbol{x}$, and the circle with dotted line represents the perturbation bound. Left: distance to the decision boundary. Middle: an adversarial example in the input space corresponding to an EAE. Right: new decision boundary after EAE adversarial training.}
 \label{fig14}
 \end{figure}

Existing research shows that the computational cost of adversarial training is very high \cite{[a12]}. In particular, the generation of adversarial examples accounts for the main part of the total time-consumption. For improving the efficiency of adversarial training, Shafahi et. al \cite{[a12]} proposed a ``Free" method to reduce the number of gradient steps used for generating adversarial examples and Wong et. al \cite{[a13]} proposed a ``Fast" method recently that outperforms the ``Free" method. However, these methods still rely on creating real adversarial examples in the input space.

In this paper, we propose a novel adversarial training method without generating real adversarial examples in the input space. The main idea is to add perturbations to the output (logits) of the penultimate layer. This output space is termed as a \emph{latent space} in this paper. In contrast to an adversarial example in the input space, we denote an example added perturbations in the latent space as an \emph{endogenous adversarial example} (EAE).  Suppose $\mathcal{F}(\boldsymbol{x})$ is a pretrained model (or a classifier), and its logits $\boldsymbol{z}=(z_1, z_2, \dots, z_{C})$ of $\boldsymbol{x}$, where $C$ is the number of classes. Assuming $z_1$ is the largest component and $z_2$ is the second largest component, then the class (\emph{Class1}) corresponding to $z_1$ is considered as the predicted class with the maximum posterior rule. Accordingly, we denote the class responding to $z_2$ as \emph{Class2}. Due to the similarity of the confidence of Class1 and Class2, we speculate the example $\boldsymbol{x}$ is closer to the decision boundary of Class2 than that of the remained classes in the input space as illustrated in Fig. \ref{fig14}. Thus we think Class2 is suitable for an adversarial targeted class in the latent space to ensure the smallest perturbation.

Unfortunately, even if we get the smallest perturbation in the latent space, we still do not guarantee its corresponding perturbation in the input space within the range of constraints, which is critical to avoid detection by human. We define the concepts of \emph{seed examples} that can be successfully crafted as an adversarial example by a specific attack method. We notice the logit difference between Class1 and Class2 of a seed example and a non-example has significant difference in statistical distributions. Thus we can determine the seed examples via their logit difference, which guarantees their perturbations in the latent space satisfying the constraint in the input space.

The advantage of this approach is that it can avoid calculating the gradient of a loss with respect to the example and remarkably speed up the adversarial training process. Compared to previous work, the contributions of our work are summarized as follows:
\begin{enumerate}
  \item [(1)] To the best of our knowledge, we are the first to propose EAE adversarial training without generating real adversarial examples. Since no need to calculate the gradient of the loss with respect to the example, it remarkably improves the adversarial training speed.
  \item [(2)] We clearly define the concepts of seed examples and EAEs for the first time. With these concepts, we observe the difference of seed example and non-seed example in statistics, which helps craft an EAE satisfying the perturbation constraints.

  \item [(3)] Extensive experiments conducted on CIFAR-10 and ImageNet show that the training time required for EAE adversarial training is about 1/3 of ``Fast" adversarial training. Meanwhile, for clean examples, the model after EAE adversarial training does not significantly reduce its classification accuracy.

\end{enumerate}

\section{Related Work}

Szegedy et al. \cite{[a1]} first discovered the existence of adversarial examples in DNNs. After then, Goodfellow et al. \cite{[a2]} proposed FGSM for quickly generating adversarial examples through one-step update along the gradient direction. Subsequently, Kurakin et al. \cite{[a14]} proposed BIM, which is a multi-step attack with smaller steps to achieve a stronger adversarial example. Madry et al. \cite{[a5]} proposed PGD, strengthening iterative adversarial attacks by adding multiple random restart steps.  All these methods craft adversarial examples in the input space, while our method manipulate logits in the latent space.

Goodfellow et al. \cite{[a2]} proposed FGSM adversarial training method using FGSM adversarial examples. Madry proposed PGD adversarial training, which is considered as the best method so far. However, existing research has shown \cite{[a12]} that the computational complexity of adversarial training is very high, and the generation of adversarial examples accounts for the main part of the total time. To reduce the computational overhead of PGD, Shafahi et al. \cite{[a12]} proposed ``Free" adversarial training, which updated both model weights and input perturbation by using a single back-propagation. Zhang et al. \cite{[a18]} believed that when performing a multi-step PGD, redundant calculations can be cut down. Therefore, Wong et al. \cite{[a13]} proposed the ``Fast" method, which used the previous minbatch's perturbation or initialized a perturbation from uniformly random perturbation to add to the clean example. However, these methods all rely on generating real adversarial examples, and calculate the gradient of loss with respect to the example through a back-propagation.

\section{Problem Statement}
\subsection{Basic Definition}
In this paper, we denote training data as ${D}=\left\{\left(\boldsymbol{x}_{i}, {y}_{i}\right)\right\}_{i=1}^{N}$, where $\boldsymbol{x}_{i} \in\mathbb{R}^{D}$, $y_{i} \in\{1,\dots,C\}$, and $y_{i}$ is the class index of $\boldsymbol{x}_{i}$. Suppose $\mathcal{F}(\boldsymbol{x};\boldsymbol{\theta})$ is a pretrained neural network model with a parameter vector $\boldsymbol{\theta}$, then $\mathcal{F}(\boldsymbol{x};\boldsymbol{\theta})=\mathcal{F}^{(N)}\left(\ldots\left(\mathcal{F}^{(2)}\left(\mathcal{F}^{(1)}(\boldsymbol{x})\right)\right)\right)$. The output of the penultimate layer of the classifier $\mathcal{F}$ is a vector of logits $\mathcal{F}^{(N-1)}=\boldsymbol{z}=(z_1, z_2, \dots, z_{C})$, and the output of the last layer is a probability vector $P=(p_1, p_2, \dots, p_{C})$ obtained by a Softmax function,
where $p_{j}$ is the confidence score to determine the example classified as the $j$-th class. $\hat{y}=\arg \max _{j}\left\{p_{j}\right\}$ is the predicted class index of $\boldsymbol{x}$.

Szegedy et al. \cite{[a1]} first discovered adversarial examples in DNNs. We formally defines adversarial examples as follows:

\begin{definition}(Perturbed Examples and Adversarial Examples)
For a clean example $\boldsymbol{x} \in\mathbb{R}^{D}$, $y$ is the ground-truth class index of $\boldsymbol{x}$. If $\boldsymbol{x} $ is added by a perturbation $\boldsymbol{\delta}$ with $\left \| \boldsymbol{\delta} \right\|_2<\epsilon$ , i.e., $\boldsymbol{x}^{\prime}=\boldsymbol{x}+\boldsymbol{\delta}$, then $\boldsymbol{x}^{\prime}$ is referred to as a perturbed example. For a perturbed example $\boldsymbol{x}^{\prime}$, if it satisfies $\mathcal{F}(\boldsymbol{x}^{\prime})\neq y$, then $\boldsymbol{x}^{\prime}$ is called an adversarial example.
\end{definition}

Note here the constraint $\left \| \boldsymbol{\delta} \right\|_2<\epsilon$ is critical to avoid detection by human. Since the added perturbation $\boldsymbol{\delta}$ is constrained by $\epsilon$, not all clean examples can be successfully crafted to the corresponding adversarial examples. Accordingly, we further divide the clean examples into \emph{candidate seed examples} and \emph{seed examples} in this paper.

\begin{definition}(Candidate Seed Examples and Seed Examples)
Assuming that $G$ is an algorithm to generate adversarial examples, i.e., $\boldsymbol{x}^{\prime}=G(\boldsymbol{x})$, any clean example $\boldsymbol{x} \in\mathbb{R}^{D}$ that can be accurately classified by $\mathcal{F}$ is called a candidate seed example of $G$. Supposing that $y$ is the ground-truth class of $\boldsymbol{x}$, given a constraint bound $\epsilon$, if $\mathcal{F}(G(\boldsymbol{x}))\neq y$, then $\boldsymbol{x}$ is called a seed example of $G$ on the model $\mathcal{F}$.
\end{definition}

Traditional adversarial training needs to generate adversarial examples in the input space to train the classifier. Instead, our method is to perturb the logits of the penultimate layer of a network, which is called as endogenous adversarial examples. The formal definition is presented as follows:

\begin{definition}(Endogenous Adversarial Examples)
For a clean example $\boldsymbol{x} \in\mathbb{R}^{D}$, $y$ is the ground-truth class of $\boldsymbol{x}$. $\boldsymbol{z}$ is the logits of $\boldsymbol{x}$, and a perturbation $\Delta$ in the latent space is added to the logits $\boldsymbol{z}$, that is, $\boldsymbol{z}^{\prime}=\boldsymbol{z}+\Delta$. If there is a corresponding $\boldsymbol{x}^{\prime}$ in the input space, which satisfies $\mathcal{F}(\boldsymbol{x}^{\prime})=\operatorname{Softmax}(\boldsymbol{z}^{\prime})\neq y$, then $\boldsymbol{z}^{\prime}$ is referenced to as an endogenous adversarial example (EAE).
\end{definition}

\subsection{Problem Setup}
Here we obtain adversarial examples in the latent space of DNNs. Thus we model the adversarial training as a two-level optimization problem as follows:
\begin{equation}
\begin{array}{c}
\min \limits_{\boldsymbol{\theta}} \mathcal{L}(\boldsymbol{z}+\Delta, \boldsymbol{\theta}) \\
\text {s.t. } \underset{\Delta}{\min} \|\Delta\|_{2} \\
z_{y}+\Delta_{y}<\sup\limits_{\Delta_{j}} \left\{z_{j}+\Delta_{j} \mid 1 \leq j \leq C \wedge j \neq y\right\}
\end{array}
\label{equ1}
\end{equation}
where $\mathcal{L}(\cdot)$ is a loss function, and $z_y$ is the component of logits corresponding to the ground-truth class $y$. The up-level problem is to minimize the loss function $\mathcal{L}$, while the low-level problem is to find the minimum perturbation $\Delta$, as described in problem (\ref{equ1}).

Instead of generating adversarial examples in the input space, we attempt to obtain a minimum perturbation $\Delta$ in the latent space instead of perturbation $\boldsymbol{\delta}$ in the input space. Assume that the ground-truth class $y$ is consistent with the predicted class $\hat{y}$ of the clean example $\boldsymbol{x}$, i.e., $y=\hat{y}$. After calculating by the softmax layer, the final output of the EAE $x^{\prime}$ is:
\begin{equation}
\begin{aligned}
p_{j}^{\prime}&=\operatorname{Softmax}(\boldsymbol{z}+\Delta)_{j}\\
&=e^{z_{j}+\Delta_{j}}/\sum\limits_{k=1}^{C}e^{z_{k}+\Delta_{k}}, \qquad j=1,2,\dots,C
\end{aligned}
\end{equation}
where $y^{\prime}=\arg \max _{j}\left\{p_{j}^{\prime}\right\}$ is the predicted class of the EAE. From the definition of adversarial examples, we know that $y^{\prime}\neq y$ and $y^{\prime}\neq\hat{y}$. That is, $y\neq \arg \max_{j}\left\{p_{j}^{\prime}\right\}$. Therefore, $\mathcal{F}(\boldsymbol{x}^{\prime};\boldsymbol{\theta})\neq y$ means $\operatorname{Softmax}(\boldsymbol{z}+\Delta)_{y}<\sup  \left\{\operatorname{Softmax}(\boldsymbol{z}+\Delta)_{j} \mid 1 \leq j \leq C \wedge j \neq y\right\}$.

Since Softmax is a monotonically increasing function, the above inequality is equivalent to $z_y+\Delta_y<\sup\limits_{\Delta_{j}}  \left\{z_j+\Delta_j \mid 1 \leq j \leq C \wedge j \neq y\right\}$. Therefore, we express the process of generating EAEs as the following optimization problem:
\begin{equation}
\begin{array}{c}
\min \limits_{\Delta}\|\Delta\|_{2} \\\text {s.t.}\quad z_{y}+\Delta_{y}<\sup\limits_{\Delta_{j}}  \left\{z_{j}+\Delta_{j} \mid 1 \leq j \leq C \wedge j \neq y\right\}
\end{array}
\label{equ2}
\end{equation}

\section{Implementation}
Although we can find a minimum perturbation $\Delta$ in the latent space through an optimization algorithm, it cannot guarantee the corresponding perturbation $\boldsymbol{\delta}$ in the input space to meet the constraint $\left \| \boldsymbol{\delta} \right\|_2<\epsilon$, i.e., the EAE does not satisfy the definition of adversarial examples (Definition 1). We address this problem by selecting seed examples. According to the definition of seed examples, they are the clean examples guaranteed to be crafted as adversarial examples. A threshold-based method using statistical distribution is adopted to select seed examples. Then with these selected seed examples, we use an optimization method to obtain EAEs. At the same time, the adversarial training are conducted with these EAEs.

\subsection{Selecting Seed Examples}
 In this paper, we propose a method based on statistical distribution to observe the seed examples. In fact, under the constraint $\left \| \boldsymbol{\delta} \right\|_2<\epsilon$, only these examples nearby the decision boundary can finally be crafted as adversarial examples \cite{[p]}, which are referred to as \textit{seed examples} in this paper. If we select all possible seed examples, then we can craft \textit{endogenous adversarial examples} that $\mathcal{F}(\boldsymbol{x}+\boldsymbol{\delta})=\operatorname{Softmax}(\boldsymbol{z}+\Delta)$, s.t. $\left \| \boldsymbol{\delta} \right\|_2<\epsilon$.

\begin{figure}
\centering
\subfigure[CIFAR-10]{
\includegraphics[width=3.8cm,height=3cm]{cifar_LD2.png}
}
\subfigure[ImageNet]{
\includegraphics[width=3.8cm,height=3cm]{imagenet_LD2.png}
}
 \caption{LD distribution of seed examples and non-seed examples: (a) on CIFAR-10, when FGSM is carried out on $F_1$ and $\epsilon=0.05$, LD distribution of seed examples and non-seed examples; (b) on ImageNet, when FGSM is carried out on $F_3$ and $\epsilon=0.002$, LD distribution of seed examples and non-seed examples.}
 \label{fig.1}
 \end{figure}

Let $d=z_{y}-z_{s}$ denote the difference between the largest component and the second largest component in the logits. For convenient description, we refer to this difference as LD (Logit Difference). Our experiment shows that for seed examples, LD is smaller, while for non-seed examples, LD is larger. Fig. \ref{fig.1} shows that the LDs of both seed examples and non-seed examples follows the Gaussian distribution. Thus, we introduce a threshold $\gamma$ to select the seed examples. If a clean example satisfies $d<\gamma$, we believe it may be a seed example with a large confidence. With this simple rule, we can find most of the seed examples

We determine the value of $\gamma$ by empirical rules, which we will discuss in the experiment. First, some clean examples that correctly classified by the model are remained as a set of candidate seed examples  ${ \widetilde D}=\left\{\left(\boldsymbol{x}_{i}, {y}_{i}\right)\right\}_{i=1}^{M}$, where $\widetilde D \subset D$ and $|\widetilde D| \ll |D|$. If the clean example $\boldsymbol{x} \in \widetilde D$ can be perturbed successfully by an adversarial generation algorithm $G$, such as FGSM, then $\boldsymbol{x}$ is remained to the set of seed examples $D^+$, otherwise it  is remained to the set of non-seed examples $D^-$. Then, we can obtain the difference between the largest component and the second largest component in the logits of examples in $D^+$ and $D^-$, respectively (as shown in Fig. \ref{fig.1}). We use the mean of LD denoted by $\bar d$, which is called as MLD in this paper, in $D^+$ as the threshold $\gamma$. Since our EAE adversarial training does not strictly require that all seed examples must generate EAEs, the threshold determined by this statistical method is more reliable.

\subsection{Generating EAEs}
Taking a deep insight into problem (\ref{equ2}), we observe that obtaining the minimum perturbation with the constraint only needs to find the second largest value in $z_{j}(1\leqslant j \leqslant y)$  corresponding to the second likelihood class. Let $z_s$ denote the second largest value: $z_{s}=\max \left\{z_j \mid 1 \leq j \leq C \wedge j \neq y\right\}$, such that $z_{y}+\Delta_{y}<z_{s}+\Delta_{s}$, where $\Delta_{y}$ is the perturbation added on $z_y$ and $\Delta_{s}$ is the perturbation added on $z_s$. Obviously, when $\Delta_{j}=0$ $(1\leqslant j \leqslant C \wedge j\neq y \wedge j \neq s)$, $\|\Delta\|_{2}$ reaches the minimum and $\|\Delta\|_{2}=\sqrt{\Delta_{y}^{2}+\Delta_{s}^{2}}$. Since $\min \Delta_{y}^{2}+\Delta_{s}^{2}$ is equivalent to $\min\sqrt{\Delta_{y}^{2}+\Delta_{s}^{2}}$, the optimization problem (\ref{equ2}) can be expressed as:
\begin{equation}
\begin{array}{c}
\underset{\Delta_{y},\Delta_{s}}{\min}\quad \Delta_{y}^{2}+\Delta_{s}^{2}
\\\text {s.t.}\quad z_{y}+\Delta_{y} \leq z_{s}+\Delta_{s}
\end{array}
\label{equ3}
\end{equation}

Since the constraint $z_{y}+\Delta_{y}<z_{s}+\Delta_{s}$ cannot determine the boundary of feasible regions, we further relax the constraints as $z_{y}+\Delta_{y} \leq z_{s}+\Delta_{s}$ in problem  (\ref{equ3}). Although this relax cannot guarantee that the example added with the perturbation $\Delta$ can become an EAE, we can ensure $z_{y}+\Delta_{y}=z_{s}+\Delta_{s}$, i.e., the confidence score of Class1 and Class2 is equal. Accordingly, the perturbed example has a 50\% probability to be classified as one of the top-2 classes. Our adversarial training does not strictly require each clean example becomes an EAE.

We use the Lagrangian multiplier method to solve this problem. The Lagrangian function can be written as follows:
\begin{equation}
    L(\Delta_{y},\Delta_{s},\lambda)=\Delta_{y}^{2}+\Delta_{s}^{2}+\lambda(\Delta_{y}-\Delta_{s}+z_{y}-z_{s})
\end{equation}
Its optimality conditions are:
\begin{equation}
\left\{\begin{array}{l}
\frac{\partial L}{\partial \Delta_{y}}=2 \Delta_{y}+\lambda=0 \\
\frac{\partial L}{\partial \Delta_{s}}=2 \Delta_{s}-\lambda=0 \\
\lambda\left(\Delta_{y}-\Delta_{s}+z_{y}-z_{s}\right)=0 \\
\lambda \geq 0
\end{array}\right.
\end{equation}
So when $\lambda=0$, then $\Delta_{y}=0$, $\Delta_{s}=0$; when $\Delta_{y}-\Delta_{s}+z_{y}-z_{s}=0$, then $\Delta_{y}=-(z_{y}-z_{s})/2$, $\Delta_{s}=(z_{y}-z_{s})/2$, $\lambda=z_{y}-z_{s}$. Obviously, the former does not meet the solution of problem (\ref{equ3}). In summary, the approximate optimal solution of problem (\ref{equ2}) is $\Delta^*=(\Delta_{1},\dots,\Delta_{j},\dots,\Delta_{C})$,
\begin{equation}
\Delta^{*}=\left\{\begin{array}{cl}
-\left(z_{y}-z_{s}\right) / 2, & j=y \\
\left(z_{y}-z_{s}\right) / 2, & j=s \\
0, & j \neq y \wedge j \neq s
\end{array}\right.
\end{equation}

\begin{algorithm}
	\caption{Adversarial training with EAEs

    {\textbf{Input:} $D=\left\{\left(\boldsymbol{x}_{i}, y_{i}\right)\right\}_{i=1}^{N}$; $T$ is the number of epochs; $K$ is the number of minibatch; $M$ is  the size of minibatch; $\gamma$ is the threshold}\\
    {\textbf{Output:} $\boldsymbol{\theta}$ }

\begin{algorithmic

	\STATE \textbf{for}  $t=1,...,T$ \textbf{do}\\
	\STATE  \quad \textbf{for}  $i=1,...,K$ \textbf{do}\\
	\STATE  \quad\quad \textbf{for}  $j=1,...,M$ \textbf{do}\\
	\STATE  \quad\quad \quad $\boldsymbol{z}_{j} \leftarrow F^{(N-1)}\left(\ldots\left(F^{(2)}\left(F^{(1)}\left(\boldsymbol{x}_{j}\right)\right)\right)\right)$
	\STATE  \quad\quad \quad $z_{y} \leftarrow \max(\boldsymbol{z}_{j})$, $z_{s} \leftarrow \sec(\boldsymbol{z}_{j})$
	\STATE \quad\quad \quad \textbf{if}  $z_{y}-z_{s}<\gamma$ \textbf{then}\quad$\boldsymbol{z}_{j} \leftarrow \boldsymbol{z}_{j}+\Delta_{j}$
	\STATE  \quad\quad \quad \textbf{end if}
	\STATE  \quad\quad \textbf{end for}
	\STATE  \quad\quad $\mathcal{L} \leftarrow \frac{1}{M} \sum \limits_{i=1}^{M} \mathcal{L}\left(\boldsymbol{z}_{j}, y_{j}\right)$, $\boldsymbol{\theta} \leftarrow \boldsymbol{\theta}-\eta \cdot \nabla_{\boldsymbol{\theta}} \mathcal{L}$
	\STATE    \quad \textbf{end for}\\
	\STATE  \textbf{end for}\\

\end{algorithmic}
\end{algorithm}

\subsection{Adversarial Training with EAEs}
We utilize SGD to train the DNN and divide the training dataset $D$ into $K$ minibatchs. Suppose the initial parameters of the DNN are $\boldsymbol{\theta}_{0}$, the algorithm will generate the EAEs in the first minibatch. With these EAEs, the gradient of the loss function with respect to $\boldsymbol{\theta}$ is obtained and then the model parameters are updated by back-propagation. This iterative update will continue until $K$ minibatchs are processed.

\section{Existence of EAEs}
In this section, we make an attempt to interpret the existence of EAEs through the theory of manifold. We first introduce the basic definitions of the manifold for better understanding of our further analysis.

\begin{definition}(Manifolds)
A $d$-dimensional manifold $M$ is a set that is locally homeomorphic with $\mathbb{R}^d$. That is, for each $\boldsymbol{x}\in M$, there is an open neighborhood $N_{\boldsymbol{x}}$ around $\boldsymbol{x}$, and a homeomorphism $f:N_{\boldsymbol{x}}\to \mathbb{R}^d$. These neighborhoods are referred to as coordinate patches, and the map is referred to as coordinate charts. The image of the coordinate charts is referred to as parameter spaces \cite{[a19]}.
\end{definition}

According to manifold hypothesis, real-world data presented in high-dimensional spaces are expected to concentrate in the vicinity of a manifold $M$ of much lower dimensionality $\mathbb{R}^d$ embedded in $\mathbb{R}^D (D>d)$  \cite{[a16]}. Then, we assume the latent space is a $d$-dimensional manifold $M$ passed through the $D$-dimensional input space.

\begin{theorem}
Assume that $\boldsymbol{x}$ is a seed example and $\boldsymbol{x}'$ is an adversarial example of $\boldsymbol{x}$ in the input space, which means that $\mathcal{F}(\boldsymbol{x'})\neq y$ and $\left \| \boldsymbol{x}-\boldsymbol{x'} \right\|_2<\epsilon$,  then there exist an endogenous adversarial example $\boldsymbol{z'}$ in the latent space and a positive constant $c$ satisfying $\|\boldsymbol{z}'-\boldsymbol{z}\|_2 \leq c\| \boldsymbol{x}' - \boldsymbol{x}\|_2$.
\end{theorem}

\begin{proof}
Let $M$ be a $d$ dimensional manifold, i.e., the latent space. This manifold $M$ is an embedded submanifold of $\mathbb{R}^{D}$. Let $P:\mathbb{R}^{D} \to M$ be the projection operator onto $M$, i.e.,
\begin{equation}
P(\boldsymbol{v}) = \arg\min_{\boldsymbol{w} \in M}\| \boldsymbol{w} -\boldsymbol{v}\|_{2}
\end{equation}

Given a point $\boldsymbol{x}\in \mathbb{R}^{D}$, suppose that there exists a unique projection point $\boldsymbol{x}_*$ of $\boldsymbol{x}$ onto $M$, i.e.,
\begin{equation}
\boldsymbol{x}_*  = P(\boldsymbol{x}).
\end{equation}
By the continuity of projection operator, there exists a positive constant $\epsilon>0$ such that
for any $\boldsymbol{x}'\in B(\boldsymbol{x},\epsilon)$, the projection $P(\boldsymbol{x}')$ is well-defined. Fig. \ref{fig3} illustrates the projection procedure.
Since $B(\boldsymbol{x},\epsilon)$ is compact, there exists a positive constant $c_0>0$ such that
\begin{equation}
\begin{aligned}
{\rm dist}(P(\boldsymbol{x}'),\boldsymbol{x}_*)=& {\rm dist}(P(\boldsymbol{x}'),P(\boldsymbol{x})) \leq c_0\| \boldsymbol{x}' -\boldsymbol{x}\|_2\leq c_0\epsilon,\\
& \boldsymbol{x}'\in B(\boldsymbol{x},\epsilon)
\end{aligned}
\end{equation}
By the above inequality, we have
\begin{equation}
P\big(B(\boldsymbol{x},\epsilon)\big)\subset B(\boldsymbol{x}_*, c_0 \epsilon)
\end{equation}

\begin{figure
	\centering
	\includegraphics[width=5.0cm,height=3.0cm]{manifold2.pdf}
	\caption{The examples of $\boldsymbol{x}$ and $\boldsymbol{x}^{\prime}$ in a $D$-dimensional space are projected onto a $d$-dimensional manifold via $P$.}
	\label{fig3}
\end{figure}

Let $U$ be an open bounded subset of $\mathbb{R}^{d}$, $\boldsymbol{z} \in U$, and $H:U(\subset \mathbb{R}^{d}) \to M$
be a local parametrization of $M$ such that
\[
H(\boldsymbol{z}) = \boldsymbol{x}_*, \quad  B(\boldsymbol{x}_*, c_0 \epsilon) \subset H(U)
\]
Since $H$ is diffeomorphism defined from $U(\subset \mathbb{R}^{d})$ to $H(U) (\subset M)$,
the set $H^{-1} \big( B(\boldsymbol{x}_*, c_0 \epsilon) \big)$ is a compact neighborhood of $\boldsymbol{z}$.

Since $H$ is an embedded submanifold, the differential ${\rm D}H(\widetilde{\boldsymbol{z}}):\mathbb{R}^{d} \to T_{H(\widetilde{\boldsymbol{z}})}M$
is injective for any point $\widetilde{\boldsymbol{z}}\in U$.
By the compactness of $H^{-1} \big( B(\boldsymbol{x}_*, c_0 \epsilon) \big)$, there are two positive constants
$c_1 < c_2$ such that
\begin{equation}
\begin{array}{c}
c_1\|\Delta \widetilde{\boldsymbol{z}}\|_2 \leq \| {\rm D}H(\widetilde{\boldsymbol{z}})[\Delta \widetilde{\boldsymbol{z}}]\|_2 \leq c_2\|\Delta \widetilde{\boldsymbol{z}}\|_2,\\
 \forall \widetilde{\boldsymbol{z}}\in H^{-1} \big( B(\boldsymbol{x}_*, c_0 \epsilon) \big), \Delta \widetilde{\boldsymbol{z}}\in \mathbb{R}^{d}
\end{array}
\end{equation}
Given a point $\boldsymbol{x}'\in B(\boldsymbol{x},\epsilon)$, denote $\boldsymbol{x}_*' = P(\boldsymbol{x}')$.
On the one hand, if $\boldsymbol {x}_* \neq \boldsymbol{x}_*'\in B(\boldsymbol{x}_*, c_0 \epsilon)$, there exists a unique point $\boldsymbol{z}'\in $ $ H^{-1} \big(B(\boldsymbol{x}_*, c_0 \epsilon)$
such that $\boldsymbol{x}_*' = H(\boldsymbol{z}')$ \cite{[a20]}. Thus we have
\begin{equation}
\begin{aligned}
\gamma_{1}\int_{0}^1 \|  {\rm D}H\big(\boldsymbol{z}+ t(\boldsymbol{z}'-\boldsymbol{z})\big)[\boldsymbol{z}'-\boldsymbol{z}]\|_2 {\rm d}t
\leq  {\rm dist}(\boldsymbol{x}_*',\boldsymbol{x}_*) \\ \leq \int_{0}^1 \|  {\rm D}H\big(\boldsymbol{z} + t(\boldsymbol{z}'-\boldsymbol{z})\big)[\boldsymbol{z}'-\boldsymbol{z}]\|_2 {\rm d}t
\end{aligned}
\end{equation}
where $0<\gamma_{1}\leq \frac{{\rm dist}(\boldsymbol{x}_*',\boldsymbol{x}_*)}{\int_{0}^1 \|  {\rm D}H\big(\boldsymbol{z} + t(\boldsymbol{z}'-\boldsymbol{z})\big)[\boldsymbol{z}'-\boldsymbol{z}]\|_2 {\rm d}t} $.

By (13) and (14), we have
\begin{equation}
c_1\gamma_{1}\|\boldsymbol{z}'-\boldsymbol{z}\|_2 \leq  {\rm dist}(\boldsymbol{x}_*',\boldsymbol{x}_*) \leq c_2\|\boldsymbol{z}'-\boldsymbol{z}\|_2
\end{equation}
According to (11) and (15), we can obtain
\begin{equation}
\begin{aligned}
\|\boldsymbol{z}'-\boldsymbol{z}\|_2 \leq \frac{1}{c_1\gamma_{1}} {\rm dist}(\boldsymbol{x}_*',\boldsymbol{x}_*)  &= \frac{1}{c_1\gamma_{1}}{\rm dist}(P(\boldsymbol{x}'),P(\boldsymbol{x}))\\
&\leq  \frac{c_0}{c_1\gamma_{1}}\| \boldsymbol{x}' - \boldsymbol{x}\|_2
\end{aligned}
\end{equation}
On the other hand, if $\boldsymbol{x_*} = \boldsymbol{x_*'}$, according to the diffeomorphism of $H$, we have $\boldsymbol{z}'= \boldsymbol{z}$.\end{proof}

In summary, the above result indicates that if we obtain a seed example, it can guarantee a valid EAE in the latent space.

\section{Experiments}
\subsection{Experimental Setup}
Our experiment is implemented with PyTorch1.4, and conducted on a server with four 2080ti GPUs running Ubuntu16.04.6 LTS.

\textbf{Datasets:} Two popular benchmark datasets, CIFAR-10 and ImageNet, are used to evaluate our EAE adversarial training algorithm. CIFAR-10 has $60,000$ color images from $10$ categories with a size of $32 \times 32$. For ImageNet, we only randomly extract examples with $100$ categories and each category includes $1,350$ images with a size of $224 \times 224$.

\textbf{Models:}  (1) To select seed examples, we first train a ResNet-18 model $F_1$ on CIFAR-10 with $15$ epochs and an AlexNet model $F_3$ on ImageNet with $20$ epochs, respectively.

(2) To generate perturbed examples for test, we train a VGG-16 model $F_2$ on CIFAR-10 with $15$ epochs and a ResNet-18 model $F_4$ on ImageNet with $20$ epochs, respectively.

(3) On CIFAR-10, we conduct adversarial training on ResNet-18 models and on ImageNet we conduct adversarial traing on AlexNet models with EAE, FGSM, BIM, PGD, and FAST etc. Since our experiments are conducted in full black-box settings, these models are different from $F_2$ and  $F_4$.

\begin{table}
\caption{The number and MLD of seed examples and non-seed examples with respect to different perturbation bounds on CIFAR-10.}
\vskip 0.15in
\begin{center}
\begin{small}
\begin{sc}
\scalebox{0.75}{
\begin{tabular}{cccccr}
\toprule
$\epsilon$ & \#(Seed) & MLD(Seed) & \#(Non-Seed) & MLD(Non-Seed) \\
\midrule
$0.01$ & $2,299$ & $3.63$ & $6,883$ & $11.85$ \\
$0.02$ & $3,949$ & $4.68$ & $5,233$ & $12.36$ \\
$0.03$ & $5,051$ & $5.26$ & $4,131$ & $12.82$ \\
$0.04$ & $5,839$ & $5.66$ & $3,343$ & $13.21$ \\
$0.05$ & $6,441$ & $5.98$ & $2,741$ & $13.50$ \\
\bottomrule
\end{tabular}
}
\end{sc}
\end{small}
\end{center}
\vskip -0.1in
\label{tab1}
\end{table}

\begin{table}
\centering
\caption{The number and MLD of seed examples and non-seed examples with respect to different perturbation bounds on ImageNet.}
\vskip 0.15in
\begin{center}
\begin{small}
\begin{sc}
\scalebox{0.75}{
\begin{tabular}{cccccr}
\toprule
$\epsilon$     & \# (Seed) & MLD(Seed) & \# (Non-Seed) & MLD(Non-Seed) \\ \hline
$0.002$ & $241$       & $1.86$      & $1,081$          & $9.77$          \\
$0.004$ & $447$       & $3.36$      & $875$           & $10.87$         \\
$0.006$ & $642$       & $4.47$      & $680$           & $11.97$         \\
$0.008$ & $779$       & $5.17$      & $543$           & $12.87$         \\
\bottomrule
\end{tabular}
}
\end{sc}
\end{small}
\end{center}
\vskip -0.1in
\label{tab2}
\end{table}

\subsection{Validation of Threshold Selection}
We select seed examples on CIFAR-10 and ImageNet. As described in Sec. 4.1, an appropriate threshold plays a critical role in determining seed examples. On CIFAR-10, we perform FGSM on the model $F_1$ with perturbation bound $\epsilon=0.01$ to construct the seed example set $D^+$ and non-seed example set $D^-$, respectively. We obtain the MLD $\bar d_1=3.63$ on $D^+$ and $\bar d_2=11.85$ on $D^-$. Obviously, $\bar d_1$ is quite different to $\bar d_2$. To further explore the impact of $\epsilon$ on MLD, we continue to set $\epsilon=0.02$, $0.03$, $0.04$, and $0.05$  and repeat the above process. MLDs with respect to different $\epsilon$ are presented in Table \ref{tab1}. It shows that although the MLD of both seed examples and non-seed examples raises with the increase of $\epsilon$, the difference between them still maintains large (about $7.54\sim8.22$), which indicates the feasibility of our threshold selection mechanism.

On ImageNet, we similarly perform FGSM on model $F_3$ with $\epsilon= 0.002$, $0.004$, $0.006$, and $0.008$, and obtain $D^+$ and $D^-$, respectively. The results are showed in Table \ref{tab2}. Similar to Table \ref{tab1}, it also demonstrates that the MLD of seed examples is smaller than that of non-seed examples by $6.76\sim7.91$.

\begin{figure}
\centering
\subfigure[CIFAR-10]{
\centering
\includegraphics[width=1.1in]{cifar_time2.pdf}
\includegraphics[width=1.1in]{cifar_clean_acc2.pdf}
\includegraphics[width=1.1in]{cifar_per_acc2.pdf}
}
\subfigure[ImageNet]{
\includegraphics[width=1.1in]{imagenet_time2.pdf}
\includegraphics[width=1.1in]{imagenet_clean_acc2.pdf}
\includegraphics[width=1.1in]{imagenet_per_acc2.pdf}
}
\caption{The impact of the threshold $\gamma$ on time, clean accuracy, and perturbed accuracy of the model after EAE adversarial training: (a) the perturbed examples used for test are from CIFAR-10. They are generated by FGSM on the model $F_1$ with the perturbation bound $\epsilon=0.05$; (b) the perturbed examples used for test are from ImageNet. They are also generated by FGSM on the model $F_3$ with the perturbation bound $\epsilon=0.007$.}
\label{fig4}
\end{figure}

\subsection{Impact of Threshold}
As mentioned in Sec. 6.2, the threshold $\gamma$ is a function of the perturbation bound $\epsilon$, thus a different $\epsilon$ will lead to a different $\gamma$. Therefore, we need to determine a reasonable $\gamma$ by analyzing its influence on adversarial training with EAEs.  In this section, we demonstrate the relationship of $\gamma$ with (1) the adversarial training time, (2) the accuracy of  clean examples (\emph{Clean Acc}), and (3) the accuracy of perturbed examples (\emph{Perturbd Acc}).

\textbf{Adversarial training time:} The first column of Fig. \ref{fig4} shows the adversarial training time with respect to the threshold, which reveals that with the raise of threshold, the adversarial training time increases accordingly. On CIFAR-10, the training time increases dramatically before $\gamma=5.98$, while after $\gamma=6$, it hardly changes. The reason is when $\gamma=6$, most of the examples have been crafted as EAEs, thus increasing the threshold will not increase the training time. Similarly, on ImageNet, when $\gamma>15$, the training time tends to be stable.

\textbf{Accuracy of clean examples:} The second column of Fig. \ref{fig4} shows the accuracy of the model after adversarial training with different thresholds on clean examples. On both CIFAR-10 and ImageNet,  the accuracy on clean examples decreases as the threshold $\gamma$ raises. Especially on CIFAR-10, the difference between the maximum and minimum prediction accuracy is about $5.9$\%, which indicates that a larger $\gamma$ will severely degrade the accuracy of clean examples than a smaller $\gamma$.

\textbf{Accuracy of perturbed examples:} The last column of Fig. \ref{fig4} shows the accuracy of the model after adversarial training with different thresholds on perturbed examples. On CIFAR-10, the accuracy improves to some extent with the increasing threshold on perturbed examples. However, this improvement is not salient, which does not exceed $1.6$\%. On ImageNet, the accuracy of perturbed examples is only improved by $2$\% even the threshold $\gamma$ increased from $1.86$ to $19.17$, which indicates that a larger $\gamma$ does not improve the adversarial training than a smaller $\gamma$.

Based on the above analysis, we need to make a trade-off between the training time and the accuracy. A smaller threshold can speed up the adversarial training with EAEs. At the same time, it can achieve a sound accuracy on clean examples compared to a little loss of robustness. As a thumb-rule, we suggest $\gamma \in [2, 4]$ is proper for most of the data sets.

\subsection{Effectiveness of Adversarial Training}

In this section, we compare our EAE adversarial training with the following three types of adversarial training method: FGSM adversarial training, Fast adversarial training, and PGD adversarial training.

On CIFAR-10, we choose ResNet-18 as a classifier architecture. Four ResNet-18 classifiers will be obtained by four types of adversarial training with $epoch=4$, the minimum cyclic learning rate $clr_{min}=0$, and the maximum cyclic learning rate $clr_{max}=0.2$. Especially, for our EAE adversarial training, the threshold $\gamma=3.63$; for FGSM adversarial training, the perturbation bound $\epsilon=16/255$; for Fast adversarial training, $\epsilon=16/255$ and the step size $\alpha=20/255$; and for PGD adversarial training, $\epsilon=16/255$, $\alpha=8/255$, and the number of iteration $K=7$

On ImageNet, we choose AlexNet as a classifier architecture. Four AlexNet classifiers will be obtained by four types of adversarial training with $epoch=15$, $clr_{min}=0$, and $clr_{max}=0.02$. Especially, for our EAE adversarial training, the threshold $\gamma=2.0$; for FGSM adversarial training, $\epsilon=0.01$; for Fast adversarial Training, $\epsilon=0.01$, $\alpha=3/255$; and for PGD adversarial training, $\epsilon=0.01$, $\alpha=1/255$, $K=4$. In addition, we also train two ResNet-18 classifiers with clean examples as benchmarks on CIFAR-10 and ImageNet, respectively.

 \begin{figure}
\centering
\subfigure[CIFAR-10]{
\includegraphics[width=3.8cm,height=3cm]{cifar_clean.pdf}
}
 \subfigure[ImageNet]{
\includegraphics[width=3.8cm,height=3cm]{imagenet_clean.pdf}
}

 \caption{The accuracy of a normal trained model compared with adversarial trained models test by clean examples: (a) clean examples from CIFAR-10, all of which can be classified correctly by $F_2$; (b) clean examples from ImageNet, all of which can be classified correctly by $F_4$.}
 \label{fig5}
 \end{figure}

\textbf{The accuracy of the classifier after adversarial training tested by clean examples:} It can be seen from Fig. \ref{fig5} that compared to normal training, adversarial training in general will reduce the classification accuracy on clean examples, which has been confirmed by previous work \cite{[q]}. However, our EAE adversarial training does not decrease the accuracy of the clean examples on ImageNet, while the other three types of adversarial training decrease significantly. On CIFAR-10 the accuracy decreases as well. Nevertheless, the accuracy of our EAE adversarial training is yet higher than others by $7.21\%\sim14.68\%$.  On ImageNet, the accuracy of our EAE adversarial training is higher than others by $8.31\%\sim11.47\%$.

\begin{figure*}[htbp]
\centering
\subfigure[FGSM+CIFAR-10]{
\includegraphics[width=1.25in]{cifar_FGSM2.pdf}
}
\subfigure[FGSM+ImageNet]{
\includegraphics[width=1.25in]{imagenet_FGSM2.pdf}
}
\subfigure[BIM+CIFAR-10]{
\includegraphics[width=1.25in]{cifar_bim2.pdf} }
\subfigure[BIM+ImageNet]{
\includegraphics[width=1.25in]{imagenet_bim2.pdf}
}
\subfigure[PGD+CIFAR-10]{
\includegraphics[width=1.25in]{cifar_PGD2.pdf}
}
\subfigure[PGD+ImageNet]{
\includegraphics[width=1.25in]{imagenet_PGD2.pdf}
}
\subfigure[RFGSM+CIFAR-10]{
\includegraphics[width=1.25in]{cifar_RFGSM2.pdf}
}
\subfigure[RFGSM+ImageNet]{
\includegraphics[width=1.25in]{imagenet_RFGSM2.pdf}
}
\subfigure[Fast+CIFAR-10]{
\includegraphics[width=1.25in]{cifar_FFGSM2.pdf}
}
\subfigure[Fast+ImageNet]{
\includegraphics[width=1.25in]{imagenet_FFGSM2.pdf}
}
\caption{The accuracy of the models with Normal training, EAE, FGSM, Fast, and PGD adversarial training tested by the perturbed examples. (a)$\sim$ (j) present ``attack+dataset" to generate perturbed examples for test with paremeters: (a) $\epsilon/255$; (b) $\epsilon/255$; (c) $\epsilon=12/255$, $\alpha=4/255$; (d) $\epsilon=2/255$, $\alpha/255$; (e) $\epsilon=12/255$, $\alpha=4/255$; (f) $\epsilon=4/255$, $\alpha=1/255$; (g) $\epsilon/255$, $\alpha=4/255$, $K=1$; (h) $\epsilon=1/255$, $\alpha/255$, and $K=1$; (i) $\epsilon/255$, $\alpha=20/255$; (j) $\epsilon/255$, $\alpha=6/255$. Note on CIFAR-10, the model to generate perturbed examples is $F_2$ (VGG-16), while on ImageNet, it is $F_4$ (ResNet-18).
\label{fig7}
}
\end{figure*}

Why does our EAE adversarial training not reduce the accuracy of clean examples? Since our method generating EAEs is to move the example to the decision boundary, and they will still be correctly classified with a probability of $50\%$. Thus, with our method the parameters are fine-tuned and the decision boundary merely changes less than other methods, as shown in Fig. \ref{fig14}. Hence, it has tiny influence on the accuracy of the clean examples. Remind other adversarial training methods, e.g., PGD,  the adversarial examples that they employ may cross and go far from the decision boundary, which leads the adversarial training to adjust the decision boundary with a larger magnitude. As a result, some clean examples are misclassified by their newly adjusted decision boundary.

\textbf{The accuracy of the classifier after adversarial training tested by perturbed examples:} Fig. \ref{fig7} presents the accuracy of the models with normal training or adversarial training with respect to the perturbation bound $\epsilon$. Remind that $F_2$ and $F_4$ are used to generate perturbed examples respectively on CIFAR-10 and ImageNet  to evaluate the robustness of the classifier, respectively. We observed that the proportion of adversarial examples increases with the perturbation bound $\epsilon$ or the number of iterations, which is consistent with our intuition. In essence, the accuracy of $F_2$ or $F_4$ represents the proportion of adversarial examples in the perturbed examples, and the lower accuracy of $F_2$ or $F_4$ indicates the more adversarial examples contained.

We observed that on CIFAR-10, increasing $\epsilon$ makes the accuracy decrease by $2\% \sim 15\%$ for our EAE adversarial training compared to the other three adversarial training methods that are not obviously affected by $\epsilon$. However, compared with the normal training, our EAE adversarial training can improve the accuracy by $11\%\sim39\%$, which indicates it can enhance the robustness of the model.

On ImageNet, the robustness of the model after EAE adversarial training is better than other adversarial training methods, and its accuracy is higher than other methods by about $7\%\sim10\%$. We also found that the robustness of the model after FGSM, Fast, and PGD adversarial training is even worse than that of normal training. Especially, PGD adversarial training is the worst among them.

Why can our EAE adversarial training effectively improve the robustness of the model? As shown in Fig. \ref{fig14}, given a perturbation bound, if a clean example intends to be crafted as an adversarial example, our method moves the clean example towards the closest decision boundary. In fact, we merely change the maximum and the second largest component in the logits during the adversarial training, and then update the parameters through back-propagation. The decision boundary adjusted by our method can defend against a large number of adversarial examples whose corresponding clean examples belong to Class1 can easily be crafted into adversarial examples misclassified as Class2.

\begin{table}[htbp]
\caption{The training time required for four types of adversarial trainings}
\begin{center}
\begin{small}
\begin{sc}
\scalebox{0.85}{
\begin{tabular}{lccccr}
\toprule
Dataset  & EAE   & FGSM  & Fast  & PGD    \\
\midrule
CIFAR-10   & $2.86$ min & $7.63$ min & $7.73$ min & $17.37$ min\\
ImageNet & $4.25$ hrs & $12.54$ hrs & $15.59$ hrs & $53.45$ hrs\\
\bottomrule
\end{tabular}
}
\end{sc}
\end{small}
\end{center}
\label{tab3}
\end{table}

\subsection{Time Cost of Adversarial Training}
In Table \ref{tab3}, we can clearly see that our EAE adversarial training is faster than other methods, since it does not need to generate real adversarial examples, and hence does not need to calculate the gradient of the loss with respect to the example. Compared to state-of-the-art ``Fast" adversarial training on CIFAR-10 and ImageNet, our method is faster by $4.87$ minutes and $11.34$ hrs, i.e., reducing the time by $63\%$ and $72.74\%$, respectively.

\section{Conclusion}
In this paper, we propose a novel EAE adversarial training method without generating real adversarial examples to participate in training. The experiment results show that our method can speed up the adversarial training process, which outperforms state-of-the-art ``Fast'' method. Especially,our EAE adversarial training has little impact on the accuracy of clean examples, which is a challenge in previous methods. In the future, we will continue to explore other techniques, e.g., network pruning, to speed up adversarial training for a robust network. Besides this, it is likely to further improve the EAE-based adversarial training itself.

\nocite{[a21]}

redpajama_set_name: RedPajamaArXiv


text:
Symbiosis through exploitation and the merger of lineages in evolution
Law, R. & Dieckmann, U. ORCID: https://orcid.org/0000-0001-7089-0393 (1998). Symbiosis through exploitation and the merger of lineages in evolution. Proceedings of the Royal Society B: Biological Sciences 265 (1402) 1245-1253. 10.1098/rspb.1998.0426.
A model for the coevolution of two species in facultative symbiosis is used to investigate conditions under which species merge to form a single reproductive unit. Two traits evolve in each species, the first affecting loss of resources from an individual to its partner, and the second affecting vertical transmission of the symbiosis from one generation to the next. Initial conditions are set so that the symbiosis involves exploitation of one partner by the other and vertical transmission is very rare. It is shown that, even in the face of continuing exploitation, a stable symbiotic unit can evolve with maximum vertical transmission of the partners. Such evolution requires that eventually deaths should exceed births for both species in the free-living state, a condition which can be met if the victim, in the course of developing its defences, builds up sufficiently large costs in the free-living state. This result expands the set of initial conditions from which separate lineages can be expected to merge into symbiotic units.
Adaptive dynamics; Coevolution; Evolutionary transitions; Mutualism; Symbiosis; Vertical transmission
Proceedings of the Royal Society B: Biological Sciences; 265(1402):1245-1253 (7 July 1998)

redpajama_set_name: RedPajamaCommonCrawl


The 8th Annual Correll Early Literacy Conference includes the Correll Book Award for excellence in early childhood informational text. This year's theme is "Joyful Literacy! Supporting Children's Interests to Bring Joy Back to Literacy Learning." Workshops will focus on ways educators can help young children find their voices. Space is limited, so register early!
Register for the Correll Early Literacy Conference here!
Join the Barbara Bush Foundation for Family Literacy for the 11th annual Literacy Connections Conference in Maine. This year's day long conference includes over 30 workshop sessions and keynotes from Dr. Jayne Singer (Developmental, Social-Emotional, and Trauma Frameworks for Strengthening Learning Readiness) and Marti Stevens Interactive Improvisational Theater (Adverse Childhood Experiences and Resilience).
Register for the Literacy Connections Conference here!
Register for Reading Round Up here!


Hahahaha did you see what I did there?
So Titan was basically just like Avatar. Great visuals and flashy stuff, but with no dialogue or plot to speak of. But it's the message of the movie that makes the least sense.
I don't want to spoil it for you (HINT THE GOOD GUY WINS) so maybe you better stop reading if you feel emotionally invested in going into this one unspoiled.
Okay. So the movie begins with a very clear and labored message: the gods are tyrannous jerks and all the humans are just waiting for one of their own to rebel and bring them down. This actually gets you pumped up - you're now picturing some sort of magic or even a physical fight where Perseus will destroy the gods and free men from their villainy. They even reinforce the point by having one of the gods personally and maliciously kill all of Perseus' adopted family. And Perseus is rescued by the soldiers of a city that has declared war on the gods, and it's up to him to save them and also avenge his family! And our hero even disdains the magical sword granted him by his real father, Zeus, and openly scorns prayer.
"If I'm going to do this, I am going to do this as a man," he declares.
So we're all on board at the midpoint of the movie. The message and theme are a little clumsy and obvious, of course; it's hard to imagine anyone saying Perseus' lines without a camera hovering in front of them and post-production dramatic music. But we know where it's going, and it makes internal sense.
But suddenly at the midpoint, things get rough for Perseus. And he needs Daddy's help.
Now, in Olympus the viewer has been treated to a few scenes of Hades and another character, in which Hades helpfully points out that recent events are all his trick to regain power. This is not a huge surprise, since they pretty much stated this was the case in the opening thirty seconds of the film. But suddenly this means that Zeus has gone from being just one of the evil gods to being the noble god being attacked by an evil one. This is the same Zeus who, not twenty minutes ago, dramatically shouted, "Release the Kraken!" And now we're supposed to feel sorry for him? What's the loss if he dies? No more cities will be eaten by monsters?
Worse is that somehow Perseus seems to act like he knows what's going on. Presumably no one invited Perseus to the secret underground Hell-conference, so why does he suddenly abandon his moral stance to accept no help from the gods? All at once he's using a magical sword his father gave him, and he's taking Pop's money to pay a toll.
The pinnacle of stupid is at the end. The climactic moment arrives and the monster is dead. Hades appears before Perseus to taunt him (or... something? who knows) and mocks the hero. The hero raises his hand to summon his father's lightning - exactly like it happened in a flashback to which he was not a witness! - and then throws his sword at Hades to carry the lightning to his enemy. Hades folds up and vanishes under the ocean, defeated by... Zeus. Then Perseus and Zeus have a fun chat and exchange some, "Atta boy!"s.
So what's the message? The only thing I can get is that the moral must be, "Don't see anything with Sam Worthington in it."


text:
package org.infinispan.stress;

import org.infinispan.container.InternalEntryFactory;
import org.infinispan.container.InternalEntryFactoryImpl;
import org.infinispan.container.entries.InternalCacheEntry;
import org.infinispan.container.versioning.EntryVersion;
import org.infinispan.loaders.CacheLoaderException;
import org.infinispan.loaders.decorators.AbstractDelegatingStore;
import org.infinispan.loaders.decorators.AsyncStore;
import org.infinispan.loaders.decorators.AsyncStoreConfig;
import org.infinispan.loaders.dummy.DummyInMemoryCacheStore;
import org.infinispan.marshall.TestObjectStreamMarshaller;
import org.infinispan.test.TestingUtil;
import org.infinispan.util.concurrent.locks.containers.LockContainer;
import org.infinispan.util.concurrent.locks.containers.ReentrantPerEntryLockContainer;
import org.infinispan.util.logging.Log;
import org.infinispan.util.logging.LogFactory;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.Set;
import java.util.TreeMap;
import java.util.concurrent.Callable;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Lock;

import static java.lang.Math.sqrt;

/**
 * Async store stress test.
 *
 * // TODO: Add a test to verify clear() too!
 *
 * @author Galder Zamarreño
 * @since 5.2
 */
@Test(testName = "stress.AsyncStoreStressTest", groups = "stress",
      enabled = true, description = "Disabled by default, designed to be run manually.")
public class AsyncStoreStressTest {

   static final Log log = LogFactory.getLog(AsyncStoreStressTest.class);
   static final boolean trace = log.isTraceEnabled();

   static final int CAPACITY = Integer.getInteger("size", 100000);
   static final int LOOP_FACTOR = 10;
   static final long RUNNING_TIME = Integer.getInteger("time", 1) * 60 * 1000;
   static final Random RANDOM = new Random(12345);

   private volatile CountDownLatch latch;
   private List<String> keys = new ArrayList<String>();
   private InternalEntryFactory entryFactory = new InternalEntryFactoryImpl();
   private Map<Object, InternalCacheEntry> expectedState = new ConcurrentHashMap<Object, InternalCacheEntry>();

   // Lock container that mimics per-key locking produced by the cache.
   // This per-key lock holder provides guarantees that the final expected
   // state has not been affected by ordering issues such as this:
   //
   // (Thread-200:) Enqueuing modification Store{storedEntry=
   // ImmortalCacheEntry{key=key165168, value=ImmortalCacheValue {value=60483}}}
   // (Thread-194:) Enqueuing modification Store{storedEntry=
   // ImmortalCacheEntry{key=key165168, value=ImmortalCacheValue {value=61456}}}
   // (Thread-194:) Expected state updated with key=key165168, value=61456
   // (Thread-200:) Expected state updated with key=key165168, value=60483
   private LockContainer locks = new ReentrantPerEntryLockContainer(32);

   private Map<String, AbstractDelegatingStore> createAsyncStores() throws CacheLoaderException {
      Map<String, AbstractDelegatingStore> stores = new TreeMap<String, AbstractDelegatingStore>();
      stores.put("ASYNC", createAsyncStore());
      return stores;
   }

   private AsyncStore createAsyncStore() throws CacheLoaderException {
      DummyInMemoryCacheStore backendStore = createBackendStore("async2");
      AsyncStoreConfig asyncCfg = new AsyncStoreConfig();
      asyncCfg.modificationQueueSize(0);
      AsyncStore store = new AsyncStore(backendStore, asyncCfg);
      store.init(backendStore.getCacheStoreConfig(), null, new TestObjectStreamMarshaller());
      store.start();
      return store;
   }

   private DummyInMemoryCacheStore createBackendStore(String storeName) throws CacheLoaderException {
      DummyInMemoryCacheStore store = new DummyInMemoryCacheStore();
      store.init(new DummyInMemoryCacheStore.Cfg(storeName), null, new TestObjectStreamMarshaller());
      store.start();
      return store;
   }

   @DataProvider(name = "readWriteRemove")
   public Object[][] independentReadWriteRemoveParams() {
      return new Object[][]{
            new Object[]{CAPACITY, 3 * CAPACITY, 90, 9, 1},
            new Object[]{CAPACITY, 3 * CAPACITY, 9, 1, 0},
      };
   }

   @Test(dataProvider = "readWriteRemove", enabled = true)
   public void testReadWriteRemove(int capacity, int numKeys,
         int readerThreads, int writerThreads, int removerThreads) throws Exception {
      System.out.printf("Testing independent read/write/remove performance " +
            "with capacity %d, keys %d, readers %d, writers %d, removers %d\n",
            capacity, numKeys, readerThreads, writerThreads, removerThreads);

      generateKeyList(numKeys);

      Map<String, AbstractDelegatingStore> stores = createAsyncStores();

      for (Map.Entry<String, AbstractDelegatingStore> e : stores.entrySet()) {
         mapTestReadWriteRemove(e.getKey(), e.getValue(), numKeys,
               readerThreads, writerThreads, removerThreads);
         e.setValue(null);
      }
   }

   private void mapTestReadWriteRemove(String name, AbstractDelegatingStore store,
         int numKeys, int readerThreads, int writerThreads, int removerThreads) throws Exception {
      DummyInMemoryCacheStore delegate = (DummyInMemoryCacheStore) store.getDelegate();
      try {
         // warm up for 1 second
         System.out.printf("[store=%s] Warming up\n", name);
         runMapTestReadWriteRemove(name, store, readerThreads, writerThreads, removerThreads, 1000);

         // real test
         System.out.printf("[store=%s] Testing...\n", name);
         TotalStats perf = runMapTestReadWriteRemove(name, store, readerThreads, writerThreads, removerThreads, RUNNING_TIME);

         // Wait until the cache store contains the expected state
         System.out.printf("[store=%s] Verify contents\n", name);
         delegate.blockUntilCacheStoreContains(expectedState.keySet(), 60000);

         System.out.printf("Container %-12s  ", name);
         System.out.printf("Ops/s %10.2f  ", perf.getTotalOpsPerSec());
         System.out.printf("Gets/s %10.2f  ", perf.getOpsPerSec("GET"));
         System.out.printf("Puts/s %10.2f  ", perf.getOpsPerSec("PUT"));
         System.out.printf("Removes/s %10.2f  ", perf.getOpsPerSec("REMOVE"));
         System.out.printf("HitRatio %10.2f  ", perf.getTotalHitRatio() * 100);
         System.out.printf("Size %10d  ", store.loadAllKeys(null).size());
         double stdDev = computeStdDev(store, numKeys);
         System.out.printf("StdDev %10.2f\n", stdDev);
      } finally {
         // Clean up state, expected state and keys
         expectedState.clear();
         delegate.clear();
      }
   }

   private TotalStats runMapTestReadWriteRemove(String name, final AbstractDelegatingStore store, int numReaders, int numWriters,
         int numRemovers, final long runningTimeout) throws Exception {
      latch = new CountDownLatch(1);
      final TotalStats perf = new TotalStats();
      List<Thread> threads = new LinkedList<Thread>();

      for (int i = 0; i < numReaders; i++) {
         Thread reader = new WorkerThread("worker-" + name + "-get-" + i, runningTimeout, perf, readOperation(store));
         threads.add(reader);
      }

      for (int i = 0; i < numWriters; i++) {
         Thread writer = new WorkerThread("worker-" + name + "-put-" + i, runningTimeout, perf, writeOperation(store));
         threads.add(writer);
      }

      for (int i = 0; i < numRemovers; i++) {
         Thread remover = new WorkerThread("worker-" + name + "-remove-" + i, runningTimeout, perf, removeOperation(store));
         threads.add(remover);
      }

      for (Thread t : threads)
         t.start();
      latch.countDown();

      for (Thread t : threads)
         t.join();

      return perf;
   }

   private void generateKeyList(int numKeys) {
      // without this we keep getting OutOfMemoryErrors
      keys = null;
      keys = new ArrayList<String>(numKeys * LOOP_FACTOR);
      for (int i = 0; i < numKeys * LOOP_FACTOR; i++) {
         keys.add("key" + nextIntGaussian(numKeys));
      }
   }

   private int nextIntGaussian(int numKeys) {
      double gaussian = RANDOM.nextGaussian();
      if (gaussian < -3 || gaussian > 3)
         return nextIntGaussian(numKeys);

      return (int) Math.abs((gaussian + 3) * numKeys / 6);
   }

   private void waitForStart() {
      try {
         latch.await();
      } catch (InterruptedException e) {
         throw new RuntimeException(e);
      }
   }

   private Operation<String, Integer> readOperation(AbstractDelegatingStore store) {
      return new Operation<String, Integer>(store, "GET") {
         @Override
         public boolean call(String key, long run) {
            try {
               InternalCacheEntry ice = store.load(key);
               if (trace)
                  log.tracef("Loaded key=%s, value=%s", key, ice != null ? ice.getValue() : "null");
               return ice != null;
            } catch (CacheLoaderException e) {
               e.printStackTrace();
               return false;
            }
         }
      };
   }

   private Operation<String, Integer> writeOperation(AbstractDelegatingStore store) {
      return new Operation<String, Integer>(store, "PUT") {
         @Override
         public boolean call(final String key, long run) {
            final int value = (int) run;
            final InternalCacheEntry entry =
                  entryFactory.create(key, value, (EntryVersion) null);
            // Store acquiring locks and catching exceptions
            boolean result = withStore(key, new Callable<Boolean>() {
               @Override
               public Boolean call() throws Exception {
                  store.store(entry);
                  expectedState.put(key, entry);
                  if (trace)
                     log.tracef("Expected state updated with key=%s, value=%s", key, value);
                  return true;
               }
            });
            return result;
         }
      };
   }

   private Operation<String, Integer> removeOperation(AbstractDelegatingStore store) {
      return new Operation<String, Integer>(store, "REMOVE") {
         @Override
         public boolean call(final String key, long run) {
            // Remove acquiring locks and catching exceptions
            boolean result = withStore(key, new Callable<Boolean>() {
               @Override
               public Boolean call() throws Exception {
                  boolean removed = store.remove(key);
                  if (removed) {
                     expectedState.remove(key);
                     if (trace)
                        log.tracef("Expected state removed key=%s", key);
                  }
                  return true;
               }
            });
            return result;
         }
      };
   }

   private boolean withStore(String key, Callable<Boolean> call) {
      Lock lock = null;
      boolean result = false;
      try {
         lock = locks.acquireExclusiveLock(Thread.currentThread(), key, 30, TimeUnit.SECONDS);
         if (lock != null) {
            result = call.call().booleanValue();
         }
      } catch (CacheLoaderException e) {
         e.printStackTrace();
         result = false;
      } catch (InterruptedException e) {
         e.printStackTrace();
         result = false;
      } finally {
         if (lock == null) return false;
         else {
            lock.unlock();
            return result;
         }
      }
   }

   private double computeStdDev(AbstractDelegatingStore store, int numKeys) throws CacheLoaderException {
      // The keys closest to the mean are suposed to be accessed more often
      // So we score each map by the standard deviation of the keys in the map
      // at the end of the test
      double variance = 0;
      Set<Object> keys = store.loadAllKeys(null);
      for (Object key : keys) {
         double value = Integer.parseInt(((String )key).substring(3));
         variance += (value - numKeys / 2) * (value - numKeys / 2);
      }
      return sqrt(variance / keys.size());
   }

   private class WorkerThread extends Thread {
      private final long runningTimeout;
      private final TotalStats perf;
      private Operation<String, Integer> op;

      public WorkerThread(String name, long runningTimeout, TotalStats perf, Operation<String, Integer> op) {
         super(name);
         this.runningTimeout = runningTimeout;
         this.perf = perf;
         this.op = op;
      }

      public void run() {
         waitForStart();
         long startMilis = System.currentTimeMillis();
         long endMillis = startMilis + runningTimeout;
         int keyIndex = RANDOM.nextInt(keys.size());
         long runs = 0;
         long missCount = 0;
         while ((runs & 0x3FFF) != 0 || System.currentTimeMillis() < endMillis) {
            boolean hit = op.call(keys.get(keyIndex), runs);
            if (!hit) missCount++;
            keyIndex++;
            runs++;
            if (keyIndex >= keys.size()) {
               keyIndex = 0;
            }
         }
         perf.addStats(op.getName(), runs, System.currentTimeMillis() - startMilis, missCount);
      }
   }

   private static abstract class Operation<K, V> {
      protected final AbstractDelegatingStore store;
      protected final String name;

      public Operation(AbstractDelegatingStore store, String name) {
         this.store = store;
         this.name = name;
      }

      /**
       * @return Return true for a hit, false for a miss.
       */
      public abstract boolean call(K key, long run);

      public String getName() {
         return name;
      }
   }

   private static class TotalStats {
      private ConcurrentHashMap<String, OpStats> statsMap = new ConcurrentHashMap<String, OpStats>();

      public void addStats(String opName, long opCount, long runningTime, long missCount) {
         OpStats s = new OpStats(opName, opCount, runningTime, missCount);
         OpStats old = statsMap.putIfAbsent(opName, s);
         boolean replaced = old == null;
         while (!replaced) {
            old = statsMap.get(opName);
            s = new OpStats(old, opCount, runningTime, missCount);
            replaced = statsMap.replace(opName, old, s);
         }
      }

      public double getOpsPerSec(String opName) {
         OpStats s = statsMap.get(opName);
         if (s == null) return 0;
         return s.opCount * 1000. / s.runningTime * s.threadCount;
      }

      public double getTotalOpsPerSec() {
         long totalOpCount = 0;
         long totalRunningTime = 0;
         long totalThreadCount = 0;
         for (Map.Entry<String, OpStats> e : statsMap.entrySet()) {
            OpStats s = e.getValue();
            totalOpCount += s.opCount;
            totalRunningTime += s.runningTime;
            totalThreadCount += s.threadCount;
         }
         return totalOpCount * 1000. / totalRunningTime * totalThreadCount;
      }

      public double getHitRatio(String opName) {
         OpStats s = statsMap.get(opName);
         if (s == null) return 0;
         return 1 - 1. * s.missCount / s.opCount;
      }

      public double getTotalHitRatio() {
         long totalOpCount = 0;
         long totalMissCount = 0;
         for (Map.Entry<String, OpStats> e : statsMap.entrySet()) {
            OpStats s = e.getValue();
            totalOpCount += s.opCount;
            totalMissCount += s.missCount;
         }
         return 1 - 1. * totalMissCount / totalOpCount;
      }
   }

   private static class OpStats {
      public final String opName;
      public final int threadCount;
      public final long opCount;
      public final long runningTime;
      public final long missCount;

      private OpStats(String opName, long opCount, long runningTime, long missCount) {
         this.opName = opName;
         this.threadCount = 1;
         this.opCount = opCount;
         this.runningTime = runningTime;
         this.missCount = missCount;
      }

      private OpStats(OpStats base, long opCount, long runningTime, long missCount) {
         this.opName = base.opName;
         this.threadCount = base.threadCount + 1;
         this.opCount = base.opCount + opCount;
         this.runningTime = base.runningTime + runningTime;
         this.missCount = base.missCount + missCount;
      }
   }

   @Test(enabled = false) // Disable explicitly to avoid TestNG thinking this is a test!!
   public static void main(String[] args) throws Exception {
      AsyncStoreStressTest test = new AsyncStoreStressTest();
      test.testReadWriteRemove(100000, 300000, 90, 9, 1);
      test.testReadWriteRemove(10000, 30000, 9, 1, 0);
      System.exit(0);
   }

}

redpajama_set_name: RedPajamaGithub


text:
La fluorite, chiamata anche fluorina o spatofluore, è un minerale molto comune composto da fluoruro di calcio. È il più importante dei minerali fluorurati.

Storia
Il nome fluorite ha due possibili derivazioni:
 dal latino: fluere che significa scorrere, per il basso punto di fusione e il suo utilizzo come fondente in metallurgia;
 dai minatori inglesi del Medioevo che la chiamavano fiore di minerale per la sua bellezza.
Da essa deriva il nome del fluoro e del fenomeno della fluorescenza.
Era già utilizzata , nell'antica Grecia come pietra preziosa e dai Romani per la costruzione di vasi multicolori: la fluorite è infatti nota per le stupende colorazioni dei cristalli e la perfetta sfaldatura.

Abito cristallino
La fluorite forma spesso cristalli euedrali perfetti e grossi, di habitus generalmente cubico; frequente la geminazione secondo {111} di due individui compenetranti; ha una perfetta sfaldatura ottaedrica. Forma talvolta piccole particelle distinte, disperse in una matrice. Forma talvolta masse granulari, tessitura comune osservata in graniti ed altre rocce ignee.

La struttura è descrivibile come un reticolo cubico a facce centrate di ioni Ca2+, con tutte le cavità tetraedriche occupate da ioni F−. Il fluoro viene così ad avere una coordinazione 4 tetraedrica, il calcio con una coordinazione 8 cubica.

Origine e giacitura
In natura, spesso come matrice di giacimenti metalliferi, la fluorite si trova solitamente nei filoni idrotermali di temperatura media ed alta, dove è associata a solfuri di piombo, argento e zinco, e barite più quarzo. Raramente è associata ad altri minerali nelle pegmatiti. Si trova inoltre in cavità di rocce metamorfiche e magmatiche ricche di silice. Può trovarsi anche in rocce sedimentarie, per deposizione da acque ricche di soluzioni termominerali di origine vulcanica. Infine può formarsi per sublimazione di vapori vulcanici.

Forma in cui si presenta in natura

Raramente si presenta limpida e incolore; esibisce di solito bellissime colorazioni violette, blu, gialle, verdi, rosa, rosse, talvolta bianche, champagne e marroni.

Alcuni tipi, se esposti ai raggi ultravioletti, presentano un vistoso fenomeno di fluorescenza, fenomeno che proprio dal minerale prende il nome.

Ha un basso indice di rifrazione (1,4338) e isotropia, fonde a , è insolubile in acqua ed è attaccabile dagli acidi minerali.

Estrazione
Come molti minerali la fluorite non viene estratta allo stato puro. La separazione dalle rocce inutilizzabili può avvenire con due sistemi:
 il metodo gravimetrico, che permette di ottenere un prodotto poco concentrato adatto agli impieghi nell'industria siderurgica. Gli scarti possono essere smaltiti senza problemi sotto forma di ghiaietto;
 il sistema della flottazione, che permette una maggior concentrazione e fornisce un prodotto adatto agli impieghi nell'industria chimica. Per questo sistema viene utilizzata una grande quantità di acqua con l'aggiunta di schiumogeni; gli scarti, sotto forma di fango molto liquido e inquinante, sono depositati in una discarica per decantare. L'argine della discarica viene innalzato con la sabbia, separata dal fango residuato della lavorazione mediante centrifugazione in un apparecchio detto «ciclone». I limi più fini vengono depositati nel bacino di decantazione.

Usi
La fluorite è utilizzata in metallurgia come fondente, ad esempio nel processo di produzione dell'alluminio. Deve possedere un basso contenuto in zolfo (inferiore allo 0,3%) e rame. Il suo valore commerciale è stimato in base al tenore di fluoruro di calcio diminuito dalla presenza di silice.
Nell'industria ceramica e del vetro è richiesto un titolo non inferiore al 95%, deve essere limitata la presenza di alcune impurezze e di metalli che formano composti colorati. L'arricchimento del minerale è fatto normalmente per flottazione.
È la materia prima per la preparazione dell'acido fluoridrico da cui si estrae poi il fluoro. In ottica si utilizza per la fabbricazione di particolari lenti e prismi per il suo basso indice di rifrazione. Il minerale di particolare pregio è utilizzato per la produzione di oggetti ornamentali.

Viene usato per produrre lenti di qualità

Località di ritrovamento
La fluorite è assai diffusa; importanti giacimenti si trovano in varie località in Italia, Svizzera, Brasile, Norvegia, Canada, Gran Bretagna, Germania, USA, Francia, Spagna, Russia, Messico, Marocco, Madagascar, Namibia, Bulgaria.

In Italia riserve di fluorite si trovano a Corvara (Bolzano), a Dossena (Valle Brembana, Bergamo), Valle Imagna (Bergamo), Val Trompia (Brescia e Trento), Alpi Bergamasche e Bresciane, Veneto, Sardegna, Toscana Meridionale e Lazio settentrionale.

Varietà
L'ittrocerite è una varietà di fluorite ricca di ittrio e cerio.

Galleria d'immagini

Note

Altri progetti

Collegamenti esterni

  Miniere di Dossena

Alogenuri (minerali)

redpajama_set_name: RedPajamaWikipedia


If You Are Battling to Hang on – Don't Quit!
There is a God Who Wants to Help!
We have an enemy in this world. We are not ignorant of that. But just remember we have help available.
I encourage you … whatever battle you are going through – Don't Give Up! Keep fighting!
Cry out to The Living God and He will help. Ask His forgiveness. There is nothing anyone has done that the blood Jesus paid on the cross can't wash away and make as white as snow. Put your trust in Jesus. In Him is the power to overcome and be victorious!
God loves you! 🙂 And Jesus too!
P.S. The enemy tried to take my life, but The Living God pulled me out of the pit and makes me stand victorious! That is why I know He can do the same for you . . . and I know He will do for my daughters who I see the enemy attacking . . . don't fight your battles alone, keep crying out to The Living God for help!


Fantastic Maron Storage Ottoman By Ophelia & Co.
Wonderful Finley Upholstered Storage Bench By Simpli Home is necessary have in virtually any home. You want to get the best items, and also you want to successfully never overpay on their behalf. Appears to be a little bit difficult, correct? Properly, this article is right here to assist. Continue reading and locate some good specialist methods for choosing the discounts on furniture pieces you are going to enjoy. Before purchasing any cabinets, wide open each of the compartments and appear within. You're not only ensuring each of the drawers are built to last and open without any hitches, you should also make certain that the interior of your storage have some form of completing as well. When searching for Finley Upholstered Storage Bench By Simpli Home, calculate your room before heading shopping. There is certainly nothing a whole lot worse than picking out the best set of Finley Upholstered Storage Bench By Simpli Home only to find that it will not easily fit in your master bedroom when you get it home. As an alternative, accurately measure your areas measurements before heading remembering where by door and microsoft windows are located.
One such furniture piece home based may be the club stool. They come in handy in all kinds of places, from the kitchen to the porch to the club. Stools are not only for the club anymore. More and more households ask them to as furnishings. The great benefit of a bar feces is that its peak enables you to sit right at the counter height. For those who have a large kitchen area with an island inside it, several barstools let you change that island into a breakfast table or perhaps a spot to unwind and browse the newspaper. There are numerous types of bar gear that you can get when you go searching for some on the market. That's the place a problem begins since you'll be confused which one that is best for you. But we've restricted the extension of the types of barstools that you can buy and utilize. Thankfully, we as of now do the research for you, and in this list we came up with a list of the very best ten best bar feces seats evaluations in 2018 you can purchase for the bar. Once you review their list, youll effortlessly have the capacity to buy the appropriate barstools for your requirements.
Superb prices and excellent support! The two gentleman who were there, we're very useful and and also have large amount of excellent prices on their own choice of Finley Upholstered Storage Bench. We arrived looking for a queen Finley Upholstered Storage Bench and also got a complete established with box spring and all for a good deal! Extremely pleasant staff! I'd come back for other home furniture pieces!!!
Visited many different shops looking for the best Finley Upholstered Storage Bench looking for us. Had a good choice, but didn't have exactly what we were looking for. They let us know that they'll purchase Finley Upholstered Storage Bench using their company stores. We discovered a sofa, adore seat, and recliner that people really loved at Ashley Furniture. I went to a couple various Ashley stores attempting for the greatest offer. Returned to Furnishings Outlet, offered them the model figures, and they were able to conserve me hundreds of dollars.
I proceeded to go in today to check out the household furniture, I saw them online but desired to to check the sensation of each. As soon as I strolled in Manny greeted me and allow discover the home Finley Upholstered Storage Bench with no stress. After a few minutes came by to ask basically had query which he answered after which suggested I take it easy on the one I was interested in with full confidence to make sure that was the main one I needed. General, enjoyable and simple encounter I can't wait for my new Finley Upholstered Storage Bench!


Just a quick heads up here. My Neighbor's Secret is airing tonight on Lifetime. So if you missed it the first time, here's your chance to see it again. Vincent Ventresca has a sizeable role in this movie. It airs at 7:00 p.m. Pacific time and earlier if you live in the Central or Mountain regions, so be sure to check the listings.
Vincent Ventresca Returns to "Dollhouse"?
Wait a minute. Is that true? Didn't we see his character killed off then drained of blood and chopped up into pieces? Well, yes, but somehow Nolan Kinnard is back in this week's episode of Dollhouse entitled "The Attic." That's a good thing for us because we get to see Vincent Ventresca again at his creepy best.
"The Attic" airs this Friday, December 18, at 8:00 p.m. (7:00 p.m. Central Time) on Fox. If for some reason you miss it, you'll be able to catch the episode on Hulu the next day.


A pregnant woman was treated with brutality at Jalota Primary Health Centre in Banaskantha district of Gujarat. Rami Ben reached the health centre along with her mother-in-law on Friday, March 22 for delivery.
Instead of taking her to the labour room, the nurse asked her to stand upright holding the iron mesh. Rami Ben gave birth to a child standing right there. As per the media reports, the baby fell right there on the floor, though, the baby is believed to be safe.
The family of Rami Ben alleged that the nurse used her sari to clean the blood on the floor after the delivery. Also, Rami did not get any medical assistance from the health centre post-delivery. A media report further added that a large number of pregnant women were treated the way Rami Ben was treated.


Today my husband's brother's all-too-short time came to an end and he passed into eternity. While we rejoice that his pain is gone and he is finally at peace, he left a huge hole in all our hearts. And while I grieve for those he was closest to – his amazing wife, son, brother and his parents, I am truly sorry for those who never had a chance to meet him, play cards with him or taste his awesome culinary creations. He was one-of-a-kind, a devoted husband, loving father, and great brother, son and friend. We mourn for our loss, but take comfort knowing that we will see him once again thanks to the amazing gift of grace and salvation through Jesus.


text:
Arcade Archives: Urban Champion
July 8, 2019 by Indie Gamer Chick 8 Comments
Reputations are a fickle thing. They happen for a reason, are usually rooted in some form of reality, but just as often as not, are completely inaccurate. I have a reputation for being a tough but fair game critic, which I take great pride maintaining. But among some, my reputation is also that I'm anti-retro and anti-Nintendo. Those are fiction. The anti-retro thing comes from the fact that I don't bend the knee to every legendary older game based on their legacy alone. I hated Super Mario Bros. I hated Centipede. I owe them nothing and have no problem telling people I think they suck. But anti retro? Are you kidding? If the rep were true, would I have needed to pause my session of Gunstar Heroes just to cry tears of joy at how awesome it was? Because I totally did. It was so good and I was so blown away. And I even wrote a two-thousand word essay on how amazing Super Mario Bros. 2 is. If that's hate, by all means, hate me that way.
The anti-Nintendo rep comes from the fact that I'll criticize Nintendo when I think the situation calls for it. That's really it. That alone makes you "anti-Nintendo" to the generation of latchkey kids raised by an NES or a Wii who live their lives under the delusion that Nintendo is their bestie. Or, even worse, an insecure deity, and if they stop kissing its ass for even a split second, they might end up never seeing more entries in their favorite franchises. Because, right, Nintendo is totally going to stop making Metroid or Zelda games if anyone expresses even the faintest hint of rejection. Uh huh. By the way, my rep there is, again, obviously untrue. I've named two Nintendo-developed titles my Game of the Year since starting IGC (Link Between Worlds in 2013, Mario Odyssey in 2017) and my site has been focused recently more on Switch indies and releases. Strange way of being anti-Nintendo: covering games that would serve to help their bottom line. So, I've learned to take reputations with a grain of salt and judge people and things by my experience with them.
"I told you to make my hair look like Elvis! This isn't Elvis! His hair wasn't green!"
I bring up reputation because Urban Champion is, by reputation, the worst Nintendo game ever made. There's of course outliers who argued that the title belongs to Ice Climber, Clu Clu Land, NES Baseball, Stack Up (the R.O.B. game), or even later stuff like Wii Music or Pokemon Channel. Hell, I'd throw Kid Icarus: Uprising into that mix. The fact that they had to include an accessory just to play it without causing damage to your hands should have been a warning to them that maybe the game needed serious re-thinking. But, Urban Champion is the game that comes up most, at least from what I can tell. It's not even close. The argument is never that it's unplayable or broken, but rather that it's so bland and uninspired that it almost defies belief. Even while I was playing it and uploading videos today, people pointed out that, after five seconds, you'd seen everything the game had to offer. A baffling argument in my opinion. I mean, how many seconds do you need to see everything Pac-Man has to offer?
I'd never played Urban Champion properly. I'd played it as part of NES Remix or a microgame in WarioWare, but I'd never played the real deal. I wasn't sure what to expect. And then I turned it on, and I started playing it, and now I'm sitting here wondering how in the world this of all games became the worst Nintendo game title holder. Because it's not. It's not even the most bland early Nintendo game I've played. It's fine, honestly. There's not a whole lot of depth here, but the concept of two guys throwing punches on the street, high and low, jabs and heavy punches, works. And, in fact, being a fan of games where attacks feel like they have real world weight and impact, I liked Urban Champion's violence a lot. The punches feel like they're connecting and hurt. I'd be nice if the characters looked like they had damage, like a black eye or swollen faces just to really sell it, but still, it's not bad. It's almost unreal that a rushed, half-assed 1984 arcade game genuinely feels like a real fist fight between two angry people, but Urban Champion pulls that off.
You've got to appreciate that citizens of Urban Champion City keep confetti by their windows so that if someone below their apartment happens to punch some other poor SOB into a sewer, you can help them to celebrate their almost-certain manslaughter.
And it's slightly more than a button-masher. It's basically rock-scissors-paper with a fourth option. There's two types of punches: a jab and a "knock-out punch" that, if you land it, always knocks a person to the ground. Both types of punches can be thrown to the face or to the gut, giving you four total attack methods. Every landed punch moves your opponent backwards. After a character is knocked back two screens, the third screen will always have an open manhole, where to win the fight you have to punch them into it. It's a novel version of the round format, and really, a sort of precursor to Smash Bros when you think about it. They're both fighting games where you're trying to knock your opponents off the screen. I'm not saying Urban Champion is the grandfather of Smash Bros, but there's some shared DNA for sure. In the same way humans are related to sea sponges, but it still counts.
Of course, the problem is that there's not a lot of meat on these bones. Each opponent is identical in model with only the coloration changed. The difficult does ramp up, but rounds of Urban Champion are still long and slow. I can't imagine arcade operators ever liked this. Then again, most of my fans weren't even aware this was an arcade game. In fact, this (along with other current Arcade Archives releases like Clu Clu Land, Ice Climber, and Excitebike) were part of the Nintendo Vs. System that were basically made of slightly upgraded NES hardware. The games were interchangeable, with operators simply being given different marquees and other decorations to change the look of the cabinet. This is why the arcade version of Super Mario is called Vs. Super Mario Bros. It was a hugely profitable set-up, but it was limited to one, maybe two games, per unit. Nintendo eventually discontinued it in favor of the PlayChoice-10, which many of my older fans describe as "playing an NES, five minutes for 25¢ at a time." Anyway, Urban Champion was part of the Vs. Series, though it's so rare that not a single person registered to the Killer List of Video Games owns one, or even the board for it. As an NES game, it's not hard to find, but it's not exactly wanted as anything but a curio either.
Imagine how hilarious it would be if Urban Champion was announced as the next Smash Bros DLC. There'd be an internet riot.
I think history was a bit unfair to Urban Champion. If this had come out for something like the Intellivsion or Colecovision, it might have been remembered as one of the all-time greats. It even feels like it belongs more on one of those consoles. Maybe if it had been, today it'd be considered the rightful patriarch to games like Street Fighter or Mortal Kombat. Instead, titles like Karate Champ or Yie Ar Kung-Fu hold that distinction, even though I think Urban Champion is a better game. Compared to other early NES titles like Super Mario Bros. or Legend of Zelda, Champion feels so damn primitive. Maybe it was the victim of the worst timing in gaming history. And being slightly bland or limited in play mechanics aren't the only flaw. The police periodically resetting a round, seemingly at random, is annoying as fuck. The people dropping pots out the window, sometimes in your favor and sometimes not, break up the game's flow terribly. There's no special moves and I hate having to move manually after every knock-down and to start every round. But, I liked the no-frills fisticuffs it offered. Urban Champion isn't the worst Nintendo game. It's not even a bad game. It's genuinely decent. I liked it. I'm sure people will think I'm being sarcastic or taking the piss. I'm not. Urban Champion is underrated. It has the most undue reputation in all of gaming, positive or negative. I'm dead serious. Check it out if you get the chance.
Now, having said all that, the package and value of the Hamster release is pretty terrible. There's only three modes. The first allows you to mess with options, though the game defaults to easy so really the only thing you can tinker with is your life count. Which, even if you do, you can post a high score to online leaderboards with it. The second mode is high-score, which you have to use the default settings. The third is caravan mode, which is the typical five-minute timer mode with online boards. Thankfully, for this release, the timer actually stops when the action does between rounds. Nice touch. The problem with this mode is that you're playing a game where you'll be hitting the buttons a lot, including the B button. If you're doing that and the time runs out, you reject your own score and it doesn't get posted. I had a top 50 score and threw it away, and I'm fucking pissed off about that. It's such careless, lazy, stupid design. Have a fucking warning screen or something before tossing a score out. It's just common sense. Just because your company is called Hamster doesn't need you need to display the brainpower of one.
Haters gonna hate I guess. People will think I'm taking the piss but I'm not. I'm dead serious: I'm really enjoying Urban Champion. No joke. pic.twitter.com/Ice5xLRGpn
— Indie Gamer Chick (@IndieGamerChick) July 8, 2019
And finally, there's the price tag: yet another $7.99 game. And that's what presents a problem for me. If the game had been $1, I'd considered it one of the best dollars I've spent on a game in recent memory. EIGHT FUCKING DOLLARS FOR THIS? I bought six discounted Switch indies for that last night. All these Arcade Archives releases (and their Johnny Turbo cousins) are terrible values. $5 is a good price for an old game. That's an impulse buy. $7.99 is something most people will want to think about. And that's where I'm struggling here. You see, I liked Urban Champion. I just detest the price. But, my rule here is that price and value are not what I'm deciding on. I'm reviewing a game as a game, not a product. So, for that reason, I have to give Arcade Archives: Urban Champion the Indie Gamer Chick Seal of Approval. And I have to tell people to not buy it for $7.99. If you get a chance at $5 or less, give it a chance with an open mind and you'll probably walk away agreeing that Nintendo not only has done a lot worse, but that Urban Champion shouldn't even make the list. But for $7.99? Urban Champion and this whole series (which I'm not done with yet, two more to go) can jump in a manhole and become part of a fatberg. A fatberg is a congealed mess of wetwipes and cooking grease that clogs up sewers. There's also dozens of overpriced arcade relics on home consoles holding it together. I'm almost certain of this.
Arcade Archives: Urban Champion was developed by Hamster
Point of Sale: Nintendo Switch
$7.99 (TOO MUCH, DON'T BUY IT, IGNORE THE SEAL OF APPROVAL) said "maybe if I make the seal of approval smaller nobody will notice" in the making of this review.
Arcade Archives: Urban Champion is Chick-Approved but as a non-indie isn't ranked on the IGC Leaderboard.
Filed under Classic Reviews, Nintendo Switch Tagged with Arcade Archives, Arcade Archives urban champion, Ice Climber is clearly the worst Nintendo game and I'm reviewing it next but I don't want to so please kill me before I start it no wait I'm already loading it aaaaaaahhhhhhh, Urban Champion, Worst Nintendo Games


RecargaPay provides Latin American customers with the fastest and most secure way to purchase top apps and pay utility bills on their pre-paid Android, Apple and Microsoft mobile phones. With a strong presence in Brazil, RecargaPay's highly-rated Android, iOS and Windows apps service around two million paying customers.
RecargaPay is always looking for new and high quality traffic sources to continue to fuel their growth. However, their previous mobile attribution and marketing analytics provider used event-based pricing. As their Digital Marketing Specialist Richard Fenning stated, "In marketing, information is power. When we were charged for in-app events, we were reluctant to track all the events we needed. This meant that we had less data to drive decision making." Additionally, their brand reputation and owned media drove a significant amount of their installs. However, they suspected that their previous attribution provider was over-attributing their organic installs to ad networks due to an over-reliance on fingerprint attribution.
RecargaPay transitioned their mobile attribution and marketing analytics to AppsFlyer, in order to gain a better understanding of their marketing funnel and user activity. Using AppsFlyer's deep integrations with every ad-network in their marketing mix, including Google AdWords and Facebook, they were able to better understand which campaigns drove each install, in a single unified dashboard.
AppsFlyer's cost tracking and unlimited in-app-event reporting made it easy for their marketing team to understand the ROI of each ad campaign, in real-time. Additionally, AppsFlyer's deep integrations helped RecargaPay experiment with, attribute and optimize advanced network capabilities such as Google's Universal App Campaigns, a capability offered only by AppsFlyer.
The team at RecargaPay are tech savvy and extra vigilant against fraudulent installs. The AppsFlyer anti-fraud solutions and full raw data reports provide the confidence they need to make smarter assessments of potential fraud, and take action based on this data.
AppsFlyer's pay-once-per-install pricing gave the RecargaPay team the freedom to measure every in-app event, providing deep insights into their user LTV on a per media source and campaign level. This had a profound impact on their performance, doubling their monthly install volume. And thanks to AppsFlyer's full raw data reports and total transparency, they gained the clarity and confidence to grow their mobile marketing budget by over 40%.
Since switching to AppsFlyer, our confidence and peace of mind in our mobile performance data has increased dramatically. With this firm foundation in place, we now invest more and invest smarter in mobile user acquisition.


Davy's Sporting Club Ltd is a not-for-profit, multi-sports and leisure complex right at the heart of the local community. Our venue is the perfect place to hold your function, meeting or conference, wedding reception and much more! We also have facilities for a variety of sports including football, cricket, squash, crown green bowls, snooker, pool, darts and gym.
The club dates back to the 1960's, originally built by the now, Davy Markham Engineering Complex for its workers, over 2500 of them. It was a thriving company and so was the club, hosting regular evenings of entertainment, stage pantomimes, fundraising events, and of course it held many sporting events and competitions. The club was sold on by the company back in 1991 and is now a not-for-profit Limited company.
We are now in our 25th year of business as Davy's Sporting Club Ltd, providing a service for the local community through sport and leisure activities. The club still has many of the original features from the 1960's, most striking is our beautiful wooden, spiral staircase at heart of the club. There are of course many modern additions to the facilities, including an easy access ramp and lift to all floors, a 3G Astroturf Pitch and most recently the latest LED lighting in the main areas.


text:
WHO: Global COVID cases up 11% last week, omicron risk high – The Denver Post
BERLIN — The World Health Organization says the number of COVID-19 cases recorded worldwide increased by 11% last week compared with the previous week, with the biggest increase in the Americas. The gain followed a gradual increase since October.
The U.N. health agency said in its weekly epidemiological report released late Tuesday that there were nearly 4.99 million newly reported cases around the world from Dec. 20-26.
Europe accounted for more than half the total, with 2.84 million, though that amounted to only a 3% increase over the previous week. It also had the highest infection rate of any region, with 304.6 new cases per 100,000 residents.
WHO said that new cases in the Americas were up 39% to nearly 1.48 million, and the region had the second-highest infection rate with 144.4 new cases per 100,000 residents. The U.S. alone saw more than 1.18 million cases, a 34% increase.
Reported new cases in Africa were up 7% to nearly 275,000.
The agency said that "the overall risk related to the new variant … omicron remains very high." It cited "consistent evidence" that it has a growth advantage over the delta variant, which remains dominant in parts of the world.
It noted that a decline in case incidence has been seen in South Africa, and that early data from that country, the U.K. and Denmark suggest a reduced risk of hospitalization with omicron. But it said that more data is needed "to understand the clinical markers of severity including the use of oxygen, mechanical ventilation and death, and how severity may be impacted by vaccination and/or prior … infection."
WHO said that the number of newly reported deaths worldwide last week was down 4% to 44,680.
Brits to bask in four days of glorious sun after being battered by severe storms
Tokyo Olympics: Team GB rower Helen Glover 'proud' despite finishing fourth but excited to return home to her kids
How Europe, after a fumbling start, overtook the U.S. in vaccinations.
UK aiming for under 20,000 deaths from virus: government chief scientist
Mexico Earthquake: Massive 7.1 tremor rocks Acapulco – 'Like a horror movie'
AMD Ryzen And 12V Li-Ion Battery Spotted In New Model 3/Y In US
Zero-emission emergency: The fire brigade that's switching to Tesla
Trump impeachment trial faces challenge from Republican Senator Paul
Camera della Moda Reacts to Italian Government's Recovery Plan
Putin critic Alexei Navalny discharged from hospital in Germany after poisoning
Moderna shares jump on $1.5 billion U.S. contract for COVID-19 vaccine
England to set tough new socialising rules after virus spike


text:
Rosgal Invest New York City, NY - 10017
Rosgal Invest is categorized under in New York City, NY and active since 2006.
Rosgal Invest was established in 2006, and today employs 1 to 4, earning $100.000 to $499.999 per year. This is a business, which does work in the BOTH market, and is classified as a , under code number 523910 by the NAICS.
If you are seeking more information, feel free to contact at the company's single location by writing to 12 East 44th St Front, New York City, New York NY 10017 or by phoning (281) 252-2221 Full Phone Report. You can also visit this business on Facebook by going to their profile page at Facebook or on find them on Twitter.
Business Name: Rosgal Invest
Address: 12 East 44th St Front, New York City, New York 10017
Annual Revenue (USD): $100.000 to $499.999
Employee Number: 1 to 4
Rosgal Invest was started in 2006 to provide professional under the SIC code 6799 and NAICS code 523910. Since its inception, the company has gone on to take a total of 1 to 4 personnel under its employment and has achieved earnings of $100.000 to $499.999 per annum.
Feel free to contact for inquiries that concern Rosgal Invest by calling the company number (281) 252-2221 Full Phone Report, as your correspondence is most welcome. Additionally, the physical location of the single location of Rosgal Invest can be found at the coordinates as well as the street address 12 East 44th St Front in New York City, New York 10017.
For its online presence, you may visit Rosgal Invest's website at and engage with its social media outlets through on Twitter and on Facebook.
in United States in New York in New York City, NY in 10017 New York City, NY All businesses in New York StateAll businesses in New York City, NYAll businesses in 10017 New York City, NY
Related Businesses In New York City
Riverside Capital Advisors LLCMorris AssociatesAmp Capital InvestorsGold Coast CapitalCtl CapitalTrinity Capital ResearchIag The Investment AdvisoGreensward CapitalCoral Reef CapitalMetrock Capital GroupStormbridge CapitalSunrise Partners IncCello Capital Management LLCYonge CapitalRatafia & CO CpasAic Capital LLCD C Showrooms LtdEza Properties LLCThs Investment Fund LLCGeneral American Investors CO