text:
\section{GBDA: Gradient-based Distributional Attack}
\label{sec:method}

In this section, we detail GBDA---our general-purpose framework for gradient-based text attacks against transformers. Our framework leverages two important insights: (1) we define a parameterized \emph{adversarial distribution} that enables gradient-based search using the Gumbel-softmax~\cite{jang2016categorical}; and (2) we promote fluency and semantic faithfulness of the perturbed text using soft constraints on both perplexity and semantic similarity.

\subsection{Adversarial Distribution}
\label{sec:adv_dist}

Let $\mathbf{z} = z_1 z_2 \cdots z_n$ be a sequence of tokens where each $z_i \in \mathcal{V}$ is a token from a fixed vocabulary $\mathcal{V} = \{1,\ldots,V\}$. Consider a distribution $P_\Theta$ parameterized by a matrix $\Theta \in \mathbb{R}^{n \times V}$, which draws samples $\mathbf{z} \sim P_\Theta$ by independently sampling each token
\begin{equation}
    \label{eq:categorical}
    z_i \sim \text{Categorical}(\pi_i),
\end{equation}
where $\pi_i = \text{Softmax}(\Theta_i)$ is a vector of token probabilities for the $i$-th token.

We aim to optimize the parameter matrix $\Theta$ so that samples $\mathbf{z} \sim P_\Theta$ are adversarial examples for the model $h$. To do so, we define the objective function for this goal as:
\begin{equation}
    \label{eq:obj_func}
    \min_{\Theta \in \mathbb{R}^{n \times V}} \mathbb{E}_{\mathbf{z} \sim P_\Theta} \ell(\mathbf{z}, y; h),
\end{equation}
where $\ell$ is a chosen adversarial loss.

\paragraph{Extension to probability vector inputs.} The objective function in \autoref{eq:obj_func} is non-differentiable due to the discrete nature of the categorical distribution. Instead, we propose a relaxation of \autoref{eq:obj_func} by first extending the model $h$ to take probability vectors as input, and then use the Gumbel-softmax approximation~\cite{jang2016categorical} of the categorical distribution to derive the gradient.

Transformer models take as input a sequence of tokens that are converted to embedding vectors using
a lookup table.
In particular, let $\mathbf{e}(\cdot)$ be the embedding function so that the input embedding for the token $z_i$ is $\mathbf{e}(z_i) \in \mathbb{R}^d$ for some embedding dimension $d$. Given a probability vector $\pi_i$ that specifies the sampling probability of the token $z_i$, we define
\begin{equation}
    \label{eq:vector_input}
    \mathbf{e}(\pi_i) = \sum_{j=1}^V (\pi_i)_j \mathbf{e}(j)
\end{equation}
as the embedding vector corresponding to the probability vector $\pi_i$. Note that if $\pi_i$ is a one-hot vector corresponding to the token $z_i$ then $\mathbf{e}(\pi_i) = \mathbf{e}(z_i)$. We extend the notation for an input sequence of probability vectors $\boldsymbol\pi = \pi_1 \cdots \pi_n$ as:
$$\mathbf{e}(\boldsymbol\pi) = \mathbf{e}(\pi_1) \cdots \mathbf{e}(\pi_n)$$
by concatenating the input embeddings.

\paragraph{Computing gradients using Gumbel-softmax.} Extending the model $h$ to take probability vectors as input allows us to leverage the Gumbel-softmax approximation to derive smooth estimates of the gradient of \autoref{eq:obj_func}. Samples $\tilde{\boldsymbol\pi} = \tilde{\pi}_1 \cdots \tilde{\pi}_n$ from the Gumbel-softmax distribution $\tilde{P}_\Theta$ are drawn according to the process:
\begin{equation}
    \label{eq:gumbel_softmax}
    (\tilde{\pi}_i)_j := \frac{\exp((\Theta_{i,j} + g_{i,j}) / T)}{\sum_{v=1}^{V} \exp((\Theta_{i,v} + g_{i,v}) / T)},
\end{equation}
where $g_{i,j} \sim \text{Gumbel}(0,1)$ and $T > 0$ is a temperature parameter that controls the smoothness of the Gumbel-softmax distribution.
As $T \rightarrow 0$, this distribution converges towards the categorical distribution in \autoref{eq:categorical}.

We can now optimize $\Theta$ using gradient descent by defining a smooth approximation of the objective function in \autoref{eq:obj_func}:
\begin{equation}
    \label{eq:smooth_obj}
    \min_{\Theta \in \mathbb{R}^{n \times V}} \mathbb{E}_{\tilde{\boldsymbol\pi} \sim \tilde{P}_\Theta} \ell(\mathbf{e}(\tilde{\boldsymbol\pi}), y; h),
\end{equation}
The expectation can be estimated using stochastic samples of $\tilde{\boldsymbol\pi} \sim \tilde{P}_\Theta$.

\subsection{Soft Constraints}
\label{sec:soft_constraints}

Black-box attacks based on heuristic replacements can only constrain the perturbation by proposing changes that fall within the constraint set, \emph{e.g.}, limiting edit distance, replacing words by ones with high word embedding similarity, etc. In contrast, our adversarial distribution formulation can readily incorporate any differentiable constraint function as a part of the objective. We leverage this advantage to include both fluency and semantic similarity constraints in order to produce more fluent and semantically-faithful adversarial texts.

\paragraph{Fluency constraint with a language model.}
Causal language models (CLMs) are trained with the objective of next token prediction by maximizing the likelihood given previous tokens. This allows the computation of likelihoods for any sequence of tokens. More specifically, given a CLM $g$ with log-probability outputs, the negative log-likelihood (NLL) of a sequence $\mathbf{x} = x_1 \cdots x_n$ is given autoregressively by:
\begin{equation*}
    \mathrm{NLL}_g(\mathbf{x}) = -\sum_{i=1}^n \log p_g(x_i \mid x_1 \cdots x_{i-1}),
\end{equation*}
where $\log p_g(x_i \mid x_1 \cdots x_{i-1}) = g(x_1 \cdots x_{i-1})_{x_i}$ is the cross-entropy between the delta distribution on token $x_i$ and the predicted token distribution $g(x_1 \cdots x_{i-1})$ for $i=1,\ldots,n$.

In our adversarial distribution formulation, since the inputs are vectors of token probabilities, we extend the definition of NLL to:
\begin{equation*}
    \mathrm{NLL}_g(\boldsymbol\pi) := -\sum_{i=1}^n \log p_g(\pi_i \mid \pi_1 \cdots \pi_{i-1}),
\end{equation*}
with
\begin{align*}
    -\log p_g(&\pi_i \mid \pi_1 \cdots \pi_{i-1}) = \\
    &-\sum_{j=1}^V (\pi_i)_j g(\mathbf{e}(\pi_1) \cdots \mathbf{e}(\pi_{i-1}))_j
\end{align*}
being the cross-entropy between the next token distribution $\pi_i$ and the predicted next token distribution $g(\mathbf{e}(\pi_1) \cdots \mathbf{e}(\pi_{i-1}))$. This extension coincides with the NLL for a token sequence $\mathbf{x}$ when each $\pi_i$ is a delta distribution for the token $x_i$.

\paragraph{Similarity constraint with BERTScore.} Prior work on word-level attacks often used context-free embeddings such as word2vec~\cite{mikolov2013distributed} and GloVe~\cite{pennington2014glove} or synonym substitution to constrain semantic similarity between the original and perturbed text~\cite{alzantot2018generating, ren-etal-2019-generating, jin2020bert}. These constraints tend to produce out-of-context and unnatural changes that alter the semantic meaning of the perturbed text~\cite{garg-ramakrishnan-2020-bae}. Instead, we propose to use BERTScore~\cite{zhang2019bertscore}, a similarity score for evaluating text generation that captures the semantic similarity between pairwise tokens in contextualized embeddings of a transformer model.

Let $\mathbf{x} = x_1 \cdots x_n$ and $\mathbf{x}' = x'_1 \cdots x'_m$ be two token sequences and let $g$ be a language model that produces contextualized embeddings $\phi(\mathbf{x}) = (\mathbf{v}_1, \ldots, \mathbf{v}_n)$ and $\phi(\mathbf{x}') = (\mathbf{v}'_1, \ldots, \mathbf{v}'_m)$. The (recall) BERTScore between $\mathbf{x}$ and $\mathbf{x}'$ is defined as:
\begin{equation}
    \label{eq:bertscore}
    R_\mathrm{BERT}(\mathbf{x}, \mathbf{x}') = \sum_{i=1}^n w_i \max_{j=1,\ldots,m} \mathbf{v}_i^\top \mathbf{v}'_j,
\end{equation}
where $w_i := \mathrm{idf}(x_i) / \sum_{i=1}^n \mathrm{idf}(x_i)$ is the normalized inverse document frequency of the token $x_i$ computed across a corpus of data. We can readily substitute $\mathbf{x}'$ with a sequence of probability vectors $\boldsymbol\pi = \pi_1 \cdots \pi_m$ as described in \autoref{eq:vector_input} and use $\rho_g(\mathbf{x}, \boldsymbol\pi) = 1 - R_\mathrm{BERT}(\mathbf{x}, \boldsymbol\pi)$ as a differentiable soft constraint.

\paragraph{Objective function.} We combine all the components in the previous sections into a final objective for gradient-based optimization. Our objective function uses the margin loss (\emph{cf.} \autoref{eq:margin_loss}) as the adversarial loss, and integrates the fluency constraint with a causal language model $g$ and the BERTScore similarity constraint using contextualized embeddings of $g$:
\begin{align}
    \mathcal{L}(\Theta) = &\mathbb{E}_{\tilde{\boldsymbol\pi} \sim \tilde{P}_\Theta} \ell(\mathbf{e}(\tilde{\boldsymbol\pi}), y; h) \nonumber \\
    &+ \lambda_{\mathrm{lm}} \: \mathrm{NLL}_g(\tilde{\boldsymbol\pi}) + \lambda_{\mathrm{sim}} \: \rho_g(\mathbf{x}, \tilde{\boldsymbol\pi}),
\end{align}
where $\lambda_{\mathrm{lm}}, \lambda_{\mathrm{sim}} > 0$ are hyperparameters that control the strength of the soft constraints. We minimize $\mathcal{L}(\Theta)$ stochastically using Adam~\cite{kingma2014adam} by sampling a batch of inputs from $\tilde{P}_\Theta$ at every iteration.

\begin{table*}[t!]
\resizebox{\textwidth}{!}{
\begin{tabular}{l|ccc|ccc|ccc}
\toprule
& \multicolumn{3}{c|}{\textbf{GPT-2}} & \multicolumn{3}{c|}{\textbf{XLM} (en-de)} & \multicolumn{3}{c}{\textbf{BERT}} \\
\textbf{Task} & \textbf{Clean Acc.} & \textbf{Adv. Acc.} & \textbf{Cosine Sim.} & \textbf{Clean Acc.} & \textbf{Adv. Acc.} & \textbf{Cosine Sim.} & \textbf{Clean Acc.} & \textbf{Adv. Acc.} & \textbf{Cosine Sim.} \\
\midrule
\textbf{DBPedia}                 & 99.2                & 5.2                & 0.91       & 99.1                & 7.6               & 0.80           & 99.2 & 7.1 & 0.80 \\
\textbf{AG News}                 & 94.8                & 6.6               & 0.90      & 94.4                & 5.4                & 0.87           & 95.1 & 2.5 & 0.82 \\
\textbf{Yelp}                 & 97.8                &   2.9             & 0.94         & 96.3                & 3.4                & 0.93                 & 97.3 & 4.7 & 0.92 \\
\textbf{IMDB}                 & 93.8                & 7.6                & 0.98        & 87.6                & 0.1                & 0.97                 & 93.0 & 3.0 & 0.92 \\
\textbf{MNLI} (m.)                 & 81.7                    & 2.8/11.0                   & 0.82/0.88           & 76.9                    & 1.3/8.4                   & 0.74/0.80                    & 84.6 & 7.1/10.2 & 0.87/0.92 \\
\textbf{MNLI} (mm.)                 & 82.5                    & 4.2/13.5                   & 0.85/0.88           & 76.3                    & 1.3/8.9                   & 0.75/0.80                    & 84.5 & 7.4/8.8 & 0.89/0.93 \\
\bottomrule
\end{tabular}
}
\caption{Result of white-box attack against three transformer models: GPT-2, XLM (en-de), and BERT. Our attack is able to reduce the target model's accuracy to below $10\%$ in almost all cases, while maintaining a high level of semantic similarity (cosine similarity of higher than $0.8$ using USE embeddings).}
\label{tab:eval_whitebox}
\end{table*}

\subsection{Sampling Adversarial Texts}

Once $\Theta$ has been optimized, we can sample from the adversarial distribution $P_\Theta$ to construct adversarial examples. Since the loss function $\mathcal{L}(\Theta)$ that we optimize is an approximation of the objective in \autoref{eq:obj_func}, it is possible that some samples are not adversarial even when $\mathcal{L}(\Theta)$ is successfully minimized. Hence, in practice, we draw multiple samples $\mathbf{z} \sim P_\Theta$ and stop sampling either when the model misclassifies the sample or when we reach a maximum number of samples.

Note that this stage could technically allow us to add hard constraints to the examples we generate, \emph{e.g.}, manually filter out adversarial examples that do not seem natural.
In our case, we do not add any extra hard constraint and only verify that the generated example is misclassified by the model.

\paragraph{Transfer to other models.} Since drawing from the distribution $P_\Theta$ could potentially generate an infinite stream of adversarial examples, we can leverage these generated samples to query a \emph{target model} that is different from $h$. This constitutes a black-box \emph{transfer attack} from the source model $h$. Moreover, our transfer attack does not require the target model to output continuous-valued scores, which most existing black-box attacks against transformers rely on~\cite{jin2020bert, garg-ramakrishnan-2020-bae, li2020contextualized, li-etal-2020-bert-attack}. We demonstrate in \autoref{sec:quantitative} that this transfer attack enabled by the adversarial distribution $P_\Theta$ is very effective at attacking a variety of target models.

\section{Introduction}
\label{sec:intro}

Deep neural networks are sensitive to small, often imperceptible changes in the input, as evidenced by the existence of so-called \emph{adversarial examples}~\cite{biggio2013evasion, szegedy2013intriguing}.

The dominant method for constructing adversarial examples defines an \emph{adversarial loss}, which encourages prediction error, and then minimizes the adversarial loss with established optimization techniques. To ensure that the perturbation is hard to detect by humans, existing methods often introduce a perceptibility constraint into the optimization problem. Variants of this general strategy have been successfully applied to image and speech data~\cite{carlini2017towards, madry2017towards, carlini2018audio}.

However, optimization-based search strategies for obtaining adversarial examples are much more challenging with text data. Attacks against continuous data types such as image and speech utilize gradient descent for superior efficiency, but the discrete nature of natural languages prohibits such first-order techniques. In addition, perceptibility for continuous data can be approximated with $L_2$- and $L_\infty$-norms, but such metrics are not readily applicable to text data. To circumvent this issue, some existing attack approaches have opted for heuristic word replacement strategies and optimizing by greedy or beam search using black-box queries~\cite{jin2020bert, li2020contextualized, li-etal-2020-bert-attack, garg-ramakrishnan-2020-bae}. Such heuristic strategies typically introduce unnatural changes that are grammatically or semantically incorrect~\cite{morris-etal-2020-reevaluating}.

\begin{figure*}
    \centering
    \includegraphics[width=\textwidth]{figures/adversarial_nlp_3.pdf}
    \caption{\label{fig:overview}
    Overview of our attack framework. The parameter matrix $\Theta$ is used to sample a sequence of probability vectors $\tilde{\pi}_1,\ldots,\tilde{\pi}_n$, which is forwarded through three (not necessarily distinct) models: (i) the target model for computing the adversarial loss, (ii) the language model for the fluency constraint, and (iii) the BERTScore model for the semantic similarity constraint. Due to the differentiable nature of each loss component and of the Gumbel-softmax distribution, our framework is fully differentiable, hence enabling gradient-based optimization.

    }
\end{figure*}

In this paper, we propose a general-purpose framework for gradient-based adversarial attacks, and apply it against transformer models on text data.
Our framework, \textbf{GBDA} (Gradient-based Distributional Attack), consists of two key components that circumvent the difficulties of gradient descent for discrete data under perceptibility constraints. First, instead of constructing a single adversarial example, we search for an \emph{adversarial distribution}. We instantiate examples with the Gumbel-softmax distribution~\cite{jang2016categorical},
parameterized by a continuous matrix of coefficients that we optimize with a vanilla gradient-based method. Second, we enforce perceptibility and fluency using BERTScore~\cite{zhang2019bertscore} and language model perplexity, respectively, both of which are differentiable and can be added to the objective function as soft constraints. The combination of these two components enables powerful, efficient, gradient-based text adversarial attacks.

Our approach overcomes many ad-hoc constraints and limitations from the existing adversarial NLP literature. For instance, some black-box attacks~\cite{li-etal-2020-bert-attack} generate token-level candidates for replacement and then rescore all the combinations of these subword tokens to find the best word-level candidate: this can lead to exponential blow-up of the search space for rare or composite words. Another problem encountered by mask predictions for substitutions is that tokens have to be modified sequentially in arbitrary order.
In contrast, GBDA operates at the token level and the fluency of token combinations is handled directly by the language model constraint.
This makes our method generic and potentially applicable to any model for text sequence prediction.

We empirically demonstrate the efficacy of GBDA against several transformer models. In addition, we also evaluate under the transfer-based black-box threat model by sampling from the optimized adversarial distribution and querying against a different, potentially unknown target model. On a variety of tasks including news/article categorization, sentiment analysis, and natural language inference, our method achieves state-of-the-art attack success rate, while preserving fluency, grammatical correctness, and a high level of semantic similarity to the original input.

In summary, the main contributions of our paper are as follows:
\begin{enumerate}
    \itemsep -1ex
    \item We define a parameterized distribution of adversarial examples and optimize it using gradient-based methods. In contrast, most prior work construct a single adversarial example using black-box search.
    \item By incorporating differentiable fluency and semantic similarity constraints into the adversarial loss, our white-box attack produces more natural adversarial texts while setting a new state-of-the-art success rate.
    \item The adversarial distribution can be sampled efficiently to query different target models in a black-box setting. This enables a powerful transfer attack that matches or exceeds the performance of existing black-box attack. Compared to prior work that operate a continuous-valued output from the target model, this transfer attack only requires hard-label outputs.
\end{enumerate}

\section{Background}
\label{sec:background}

Adversarial examples constitute a class of robustness attacks against neural networks. Let $h : \mathcal{X} \rightarrow \mathcal{Y}$ be a classifier where  $\mathcal{X}, \mathcal{Y}$ are the input and output domains, respectively. Suppose that $\mathbf{x} \in \mathcal{X}$ is a test input that the model correctly predicts as the label $y = h(\mathbf{x}) \in \mathcal{Y}$. An (untargeted) adversarial example is a sample $\mathbf{x}' \in \mathcal{X}$ such that $h(\mathbf{x}') \neq y$ but $\mathbf{x}'$ and $\mathbf{x}$ are imperceptibly close.

The notion of perceptibility is introduced so that $\mathbf{x}'$ preserves the semantic meaning of $\mathbf{x}$ for a human observer. At a high level, $\mathbf{x}'$ constitutes an attack on the model's robustness if a typical human would not misclassify $\mathbf{x}'$ but the model $h$ does. For image data, since the input domain $\mathcal{X}$ is a subset of the Euclidean space $\mathbb{R}^d$, a common surrogate for perceptibility is a distance metric such as the Euclidean distance or the Chebyshev distance. In general, one can define a perceptibility metric $\rho : \mathcal{X} \times \mathcal{X} \rightarrow \mathbb{R}_{\geq 0}$ and a threshold $\epsilon > 0$ so that $\mathbf{x}'$ is considered imperceptible to $\mathbf{x}$ if $\rho(\mathbf{x}, \mathbf{x}') \leq \epsilon$.

\paragraph{Search problem formulation.} The process of finding an adversarial example is typically modeled as an optimization problem. For classification, the model $h$ outputs a \emph{logit vector} $\phi_h(\mathbf{x}) \in \mathbb{R}^K$ such that $y = \argmax_k \phi_h(\mathbf{x})_k$. To encourage the model to misclassify an input, one can define an \emph{adversarial loss} such as the margin loss:
\begin{align}
    \ell_\text{margin}(&\mathbf{x}, y; h) = \nonumber \\
    &\max\left( \phi_h(\mathbf{x})_y - \max_{k \neq y} \phi_h(\mathbf{x})_k + \kappa, 0 \right),
    \label{eq:margin_loss}
\end{align}
so that the model misclassifies $\mathbf{x}$ by a margin of $\kappa > 0$ when the loss is 0. The margin loss has been used in attack algorithms for image data~\cite{carlini2017towards}.

Given an adversarial loss $\ell$, the process of constructing an adversarial example can be cast as a constrained optimization problem:
\begin{align}
    \label{eq:adv_opt_hard}
    \min_{\mathbf{x}' \in \mathcal{X}} \enspace &\ell(\mathbf{x}', y; h) \quad \text{subject to} \; \rho(\mathbf{x}, \mathbf{x}') \leq \epsilon.
\end{align}
An alternative formulation is to relax the constraint into a soft constraint with $\lambda > 0$:
\begin{equation}
    \min_{\mathbf{x}' \in \mathcal{X}} \; \ell(\mathbf{x}', y; h) + \lambda \cdot \rho(\mathbf{x}, \mathbf{x}'),
\end{equation}
which can then be solved using gradient-based optimizers if the constraint function $\rho$ is differentiable.

\subsection{Text Adversarial Examples}

Although the search problem formulation in \autoref{eq:adv_opt_hard} has been widely applied to continuous data such as image and speech, it does not directly apply to text data because (1) the data space $\mathcal{X}$ is discrete, hence not permitting gradient-based optimization; and (2) the constraint function $\rho$ is difficult to define for text data. In fact, both issues arise when considering attacks against \emph{any} discrete input domain, but the latter is especially relevant for text data due to the sensitivity of natural language. For instance, inserting the word \emph{not} into a sentence can negate the meaning of the whole sentence despite having a token-level edit distance of 1.

\paragraph{Prior work.} Several attack algorithms have been proposed to circumvent these two issues, using a multitude of approaches. For attacks that operate on the character level, perceptibility can be approximated by the number of character edits, \emph{i.e.}, replacements, swaps, insertions and deletions~\cite{ebrahimi2017hotflip, li2018textbugger, gao2018black}. Attacks that operate on the word level adopt heuristics such as synonym substitution~\cite{samanta2017towards, zang-etal-2020-word} or replacing words by ones with similar word embeddings~\cite{alzantot2018generating, ren-etal-2019-generating, jin2020bert}. More recent attacks have also leveraged masked language models such as BERT~\cite{devlin-etal-2019-bert} to generate word substitutions by replacing masked tokens~\cite{garg-ramakrishnan-2020-bae, li2020contextualized, li-etal-2020-bert-attack}. Most of the aforementioned attacks follow the common recipe of proposing character-level or word-level perturbations to generate a constrained candidate set and optimizing the adversarial loss greedily or using beam search.

\paragraph{Shortcomings in prior work.} Despite the plethora of attacks against natural language models, their efficacy remains subpar compared to attacks against other data modalities. Both character-level and word-level changes are still relatively detectable, especially as such changes often introduce misspellings, grammatical errors, and other artifacts of unnaturalness in the perturbed text~\cite{morris-etal-2020-reevaluating}. Moreover, prior attacks mostly query the target model $h$ as a black-box and rely on zeroth-order strategies for minimizing the adversarial loss, resulting in sub-optimal performance.

For instance, BERT-Attack~\cite{li-etal-2020-bert-attack}---arguably the state-of-the-art attack against BERT---only reduces the test accuracy of the target model on the AG News dataset~\cite{zhang2015character} from $94.2$ to $10.6$. In comparison, attacks against image models can consistently reduce the model's accuracy to 0 on almost all computer vision tasks~\cite{akhtar2018threat}. This gap in performance raises the question of whether gradient-based search can produce more fluent and optimal adversarial examples on text data. In this work, we show that our gradient-based attack can reduce the same model's accuracy from $94.2$ to $2.5$ while being \emph{more semantically-faithful} to the original text.

\subsection{Other Attacks}

While most work on adversarial text attack falls within the formulation that we defined in \autoref{sec:background}, other notions of text adversarial attacks exist as well. One class of such attacks is known as a universal adversarial trigger---a short snippet of text that when appended to any input, causes the model to misclassify~\cite{wallace2019universal, song2020universal}. However, such triggers often contain unnatural combinations of words or tokens, and hence are very perceptible to a human observer.

Our work falls within the general area of adversarial learning, and many prior works in this area have explored the notion of adversarial example on different data modalities. While the most prominent data modality by far is image, adversarial examples can be constructed for speech~\cite{carlini2018audio} and graphs~\cite{dai2018adversarial, zugner2018adversarial} as well.
\section{Conclusion and Future Work}
\label{sec:conclusion}

We presented GBDA, a framework for gradient-based white-box attacks against text transformers. The main insight in our method is the search formulation of a parameterized adversarial distribution rather than of a single adversarial examples, which enables efficient gradient-based optimization and differentiable constraints such as perplexity and BERTScore. Our attack is highly effective on a variety of natural language tasks and model architectures, and attains state-of-the-art attack performance in both the white-box and black-box transfer attack settings.

\paragraph{Limitations.} One clear limitation of our method is the restriction to only token replacements. Indeed, our adversarial distribution formulation using the Gumbel-softmax does not trivially extend to token insertions and deletions. This limitation may adversely affect the naturalness of the generated adversarial examples. We hope that our adversarial distribution framework can be extended to incorporate a broader set of token-level changes.

In addition, the adversarial distribution $P_\Theta$ is highly over-parameterized. Despite most adversarial examples requiring only a few token changes, the distribution parameter $\Theta$ is of size $n \times V$, which is especially excessive for longer sentences. Future work may be able to reduce the number of parameters without affecting the optimality of generated adversarial examples.
\section{Experiments}
\label{sec:experiment}

In this section, we empirically validate our attack framework on a variety of natural language tasks. Code to reproduce our results is open sourced on GitHub\footnote{\url{https://github.com/facebookresearch/text-adversarial-attack}}.

\begin{table*}[t!]
    \centering
    \resizebox{\linewidth}{!}{
    \begin{tabular}{l|l|l}
\textbf{Attack} & \textbf{Prediction} & \textbf{Text} \\
\midrule
Original    &   Entailment  (83\%)  &   He found himself thinking in circles of worry and pulled himself back to his problem. \\ && He got lost in loops of worry, but snapped himself back to his problem. \\
\textcolor{blue}{GBDA} &   \textcolor{blue}{Neutral (95\%)}      &   He found himself thinking in circles of worry and pulled himself back to his problem. \\ && He got lost in loops of \textcolor{blue}{hell}, but snapped himself back to his problem. \\[0.5em]

Original    &   Contradiction (95\%)    &   You're the Desert Ghost. You're a living desert camel. \\
\textcolor{blue}{GBDA} &   \textcolor{blue}{Entailment (51\%)}       &   You're the Desert Ghost. You're a living desert \textcolor{blue}{animal}. \\[0.5em]

Original    &   Contradiction (98\%)    &   Pesticide concentrations should not exceed USEPA's Ambient Water Quality chronic criteria values where available. \\&& There is no assigned value for maximum pesticide concentration in water. \\
\textcolor{blue}{GBDA} &   \textcolor{blue}{Entailment (86\%)}       &   Pesticide concentrations should not exceed USEPA's Ambient Water Quality chronic criteria values where available. \\&& There is \textcolor{blue}{varying} assigned value for maximum pesticide concentration in water.

    \end{tabular}
    }
    \caption{\label{tab:examples}Examples of successful adversarial texts on the MNLI dataset
    }
\end{table*}

\subsection{Setup}

\paragraph{Tasks.} We evaluate on several benchmark text classification datasets, including \textbf{DBPedia}~\citep{zhang2015character} and \textbf{AG News}~\cite{zhang2015character} for article/news categorization, \textbf{Yelp Reviews}~\cite{zhang2015character} and \textbf{IMDB}~\cite{maas2011learning} for binary sentiment classification, and \textbf{MNLI}~\cite{williams2017broad} for natural language inference. The MNLI dataset contains two evaluation sets: matched (m.) and mismatched (mm.),
corresponding to whether the test domain is matched or mismatched with the training distribution.

\paragraph{Models.} We attack three transformer architectures with our gradient-based white-box attack: GPT-2~\cite{radford2019language}, XLM~\cite{lample2019cross} (using the en-de cross-lingual model), and BERT~\cite{devlin-etal-2019-bert}. For BERT, we use finetuned models from TextAttack~\cite{morris2020textattack} for all tasks except for DBPedia, where finetuned models are unavailable. For BERT on DBPedia and GPT-2/XLM on all tasks, we finetune a pretrained model to serve as the target model.

The soft constraints described in \autoref{sec:soft_constraints} utilizes a CLM $g$ with the same tokenizer as the target model. For GPT-2 we use the pre-trained GPT-2 model without finetuning as $g$, and for XLM we use the checkpoint obtained after finetuning using the CLM objective. For masked language models such as BERT~\cite{devlin-etal-2019-bert}, we train a causal language model $g$ on WikiText-103 using the same tokenizer as the target model.

\paragraph{Baselines.} We compare against several recent attacks on text transformers: TextFooler~\cite{jin2020bert}, BAE~\cite{garg-ramakrishnan-2020-bae}, and BERT-Attack~\cite{li-etal-2020-bert-attack}. All baseline attacks are evaluated on finetuned BERT models from the TextAttack library. Since these attacks only require black-box query access to the target model, we evaluate both our white-box attack on the finetuned BERT model, as well as transfer attack from the GPT-2 model for fair comparison. Please refer to \autoref{sec:quantitative} for details of both attack settings.

\paragraph{Hyperparameters.}
Our adversarial distribution parameter $\Theta$ is optimized using Adam~\cite{kingma2014adam} with a learning rate of $0.3$ and a batch size of $10$ for $100$ iterations.
The distribution parameters $ \Theta $ are initialized to zero except $ \Theta_{i, j} = C $ where $x_i = j$ is the $i$-th token of the clean input.
In practice we take $C \in {12, 15}$.
We use $ \lambda_{\mathrm{perp}} = 1$ and cross-validate $\lambda_{\mathrm{sim}} \in [20, 200]$ and $\kappa \in \{3, 5, 10\}$ using held-out data.

\subsection{Quantitative Evaluation}
\label{sec:quantitative}

\paragraph{White-box attacks.} We first evaluate the attack performance under the white-box setting. \autoref{tab:eval_whitebox} shows the result of our attacks against GPT-2, XLM (en-de), and BERT on different benchmark datasets. Following prior work~\cite{jin2020bert}, for each task, we randomly select $1000$ inputs from the task's test set as attack targets. After optimizing $\Theta$, we draw up to $100$ samples $\mathbf{z} \sim P_\Theta$ until the model misclassifies $\mathbf{z}$. The model's accuracy after attack (under the column ``Adv. Acc.'') is the accuracy evaluated on the last of the drawn samples.

Overall, our attack is able to successfully generate adversarial examples against all three models across the five benchmark datasets. The test accuracy can be reduced to below $10\%$ for almost all models and tasks. Following prior work, we also evaluate the semantic similarity between the adversarial example and the original input using the cosine similarity of Universal Sentence Encoders~\cite{cer2018universal} (USE). Our attack is able to consistently maintain a high level of semantic similarity to the original input in most cases (cosine similarity higher than $0.8$).

\paragraph{Transfer attacks.} We evaluate our attack against prior work under the black-box transfer attack setting. More specifically, for each model and task, we randomly select $1000$ target test inputs and optimize the adversarial distribution $P_\Theta$ on GPT-2. After optimizing $\Theta$, we draw up to $1000$ samples $\mathbf{z} \sim P_\Theta$ and evaluate them on the target BERT model from the TextAttack~\cite{morris2020textattack} library until the model misclassifies $\mathbf{z}$. This attack setting is strictly more restrictive than prior work because our query procedure only requires the target model to output a discrete label in order to decide when to stop sampling from $P_\Theta$, whereas prior work relied on a continuous-valued output score such as class probabilities.

\autoref{tab:eval_bert} shows the performance of our attack when transferred to finetuned BERT text classifiers. In almost all settings, GBDA is able to reduce the target model's accuracy to close to that of BERT-Attack or better within fewer number of queries. Moreover, the cosine similarity between the original input and the adversarial example is higher than that of BERT-Attack.

\begin{table}[t!]
\resizebox{\columnwidth}{!}{
\begin{tabular}{@{\ }l@{\ }c@{\ \ }c@{\ \ }c@{\ \ }c@{\ \ }c@{\ }}
\toprule
\textbf{Task} & \textbf{Clean Acc.} & \textbf{Attack Alg.} & \textbf{Adv. Acc.} & \textbf{\# Queries} & \textbf{Cosine Sim.} \\
\midrule
\multirow{3}{*}{\textbf{AG News}}   &\multirow{3}{*}{95.1}  &GBDA (ours)    & \bf{8.8}  & {\bf 107} & 0.69    \\
&                                                           &BERT-Attack    &10.6       & 213       & 0.63          \\
&                                                           &BAE            &13.0       & 419       & {\bf 0.75} \\
&                                                           &TextFooler     &12.6       & 357       & 0.57          \\
\midrule
\multirow{3}{*}{\textbf{Yelp}}  &\multirow{3}{*}{97.3}  &GBDA (ours)    & {\bf 2.6} & {\bf 43}  & 0.83  \\
&                                                       &BERT-Attack    & 5.1       & 273       & 0.77  \\
&                                                       &BAE            & 12.0      & 434       & {\bf 0.90} \\
&                                                       &TextFooler     & 6.6       & 743       & 0.74  \\
\midrule
\multirow{3}{*}{\textbf{IMDB}}  &\multirow{3}{*}{93.0}  &GBDA (ours)    & {\bf 8.5}    & {\bf 116} & 0.92  \\
&                                                       &BERT-Attack    & 11.4          & 454       & 0.86       \\
&                                                       &BAE            & 24.0          & 592       & {\bf 0.95} \\
&                                                       &TextFooler     & 13.6          & 1134      & 0.86        \\
\midrule
\multirow{3}{*}{\textbf{MNLI} (m.)}            & \multirow{3}{*}{84.6}   &  GBDA (ours)               & {\bf 2.3}/{\bf 10.8}               & 37/133 & 0.75/0.79                 \\
& & BERT-Attack  & 7.9/11.9               & {\bf 19}/{\bf 44} & 0.55/0.68                 \\
& & BAE         & 25.4/36.2             & 68/120    & \bf{0.88}/\bf{0.88} \\
& & TextFooler  & 9.6/25.3               & 78/152 & 0.57/0.65                 \\
\midrule
\multirow{3}{*}{\textbf{MNLI} (mm.)}            & \multirow{3}{*}{84.5}   &  GBDA (ours)              & {\bf 1.8}/{\bf 13.4}               & 30/159 &  0.76/0.80                 \\
& & BERT-Attack  & 7/13.7               & {\bf 24}/{\bf 43} & 0.53/0.69                 \\
& & BAE         & 19.2/30.3          &  75/110   & \bf{0.88}/\bf{0.88} \\
& & TextFooler  & 8.3/22.9               & 86/162 & 0.58/0.65                 \\
\bottomrule
\end{tabular}
}
\caption{Evaluation of black-box transfer attack from GPT-2 to finetuned BERT classifiers. Our attack is able exceed the attack performance of BERT-Attack and BAE, while maintaining a higher semantic similarity with fewer number of queries in most cases. Furthermore, our transfer attack does not require continuous-valued outputs, which all the baseline methods rely on.}
\label{tab:eval_bert}
\end{table}

We further evaluate our transfer attack against three finetuned transformer models from the TextAttack library: ALBERT~\cite{lan2019albert}, RoBERTa~\cite{liu2019roberta}, and XLNet~\cite{yang2019xlnet}. For this experiment, we use the same $\Theta$ optimized on GPT-2 for each of the target models. \autoref{tab:eval_blackbox} reports the performance of our attack after randomly sampling up to $1000$ times from $P_\Theta$. The attack performance is comparable to that of the transfer attack against BERT in \autoref{tab:eval_bert}, which means our adversarial distribution $P_\Theta$ is able to capture the common failure modes of a wide variety of transformer models. This result opens a practical avenue of attack against real world systems as the attacker requires very limited access to the target model in order to succeed.

\begin{figure*}
  \begin{minipage}{0.59\textwidth}
        \resizebox{\textwidth}{!}{
        \begin{tabular}{@{\ }l@{\ }c@{\ }c@{\ \ }c@{\ \ }c@{\ \ }c@{\ }}
        \toprule
        \textbf{Target Model}              & \textbf{Task} & \textbf{Clean\, Acc.} & \textbf{Adv.\, Acc.} & \textbf{\# Queries} & \textbf{Cosine Sim.} \\
        \midrule
        \multirow{3}{*}{\textbf{ALBERT}} & \textbf{AG News}             & 94.7                & 7.5                & 84 & 0.68                 \\
        & \textbf{Yelp}             & 97.5                & 5.9                & 76 & 0.79                 \\
        & \textbf{IMDB}             & 93.8                & 13.1               & 157 & 0.87                 \\
        \midrule
        \multirow{4}{*}{\textbf{RoBERTA}}       & \textbf{AG News}              & 94.7                & 10.7                & 130 & 0.67                 \\
        & \textbf{IMDB}             & 95.2                & 17.4                & 205 & 0.87                 \\
        & \textbf{MNLI} (m.)             & 88.1                & 4.1/15.1                & 63/179 & 0.69/0.76                 \\
        & \textbf{MNLI} (mm.)             & 87.8                & 3.2/15.9                & 51/189 & 0.69/0.78                 \\
        \midrule
        \multirow{3}{*}{\textbf{XLNet}}       & \textbf{IMDB}      & 93.8                & 12.1                & 149 & 0.87                 \\
        & \textbf{MNLI} (m.)             &  87.2               &  3.9/13.7               &  56/162 & 0.70/0.77                 \\
        & \textbf{MNLI} (mm.)             &  86.8               &  1.7/14.4               &  32/171 &  0.70/0.78                \\
        \bottomrule
        \end{tabular}
        }
        \captionof{table}{Result of black-box transfer attack from GPT-2 to other transformer models. Our attack is achieved by sampling from the same adversarial distribution $P_\Theta$ and is able to generalize to the three target transformer models considered in this study.}
        \label{tab:eval_blackbox}
  \end{minipage}
  \hfill
  \begin{minipage}{0.39\textwidth}
    \centering
    \includegraphics[width=\textwidth]{figures/transfer_vs_queries.pdf}
    \captionof{figure}{
    \label{fig:lam_sim_transfer}
    Effect of the parameter $\lambda_\mathrm{sim}$ on transfer attack success rate. Lower $\lambda_\mathrm{sim}$ produces more aggressive changes, but also generalizes better to different target models.}
    \end{minipage}
\end{figure*}

\subsection{Analysis}

\paragraph{Sample adversarial texts.} Table~\ref{tab:examples} shows examples of our adversarial attack on text.
Our method introduces minimal changes to the text, preserving most of the original sentence's meaning.
Despite not explicitly constraining replaced words to have the same Part-Of-Speech tag, we observe that our soft penalties make the adversarial examples obey this constraint.
For instance, in the first example of Table~\ref{tab:examples}, "camel" is replaced with "animal" and "no" with "varying".

\paragraph{Effect of $\lambda_{\mathrm{sim}}$.}
Figure \ref{fig:lam_sim_transfer} shows the impact of the similarity constraint on transfer attack adversarial accuracy for GPT-2 on AG News.
Each color corresponds to a different target model, whereas the color shade (from light to dark) indicates the value of the constraint hyperparameter: $\lambda_\mathrm{sim} = 50,20,10$.
A higher value of $\lambda_\mathrm{sim}$ reduces the aggressiveness of the perturbation, but also increases the number of queries required to achieve a given target adversarial accuracy.
This plot shows that there exists a trade-off between the similarity constraint, attack success rate, and query budget.

\paragraph{Impact of the fluency constraint.}
\autoref{tab:fluency_constraint} shows adversarial examples for GPT-2 on AG News, generated with and without the fluency constraint.
We fix all hyperparameters except for the fluency regularization constant $\lambda_\mathrm{lm}$, and sample successful adversarial texts from $P_\Theta$ after $\Theta$ has been optimized. It is evident that the fluency constraint promotes token combinations to form valid words and ensure grammatical correctness of the adversarial text. Without the fluency constraint, the adversarial text tends to contain nonsensical words.

\paragraph{Tokenization artifacts.}
Our attack operates entirely on tokenized inputs.
However, the input to the classification system is often in raw text form, which is then tokenized before being fed to the model.
Thus it is possible that we generate an adversarial example that, when converted to raw text, is not re-tokenized to the same set of tokens.

Consider this example: our adversarial example contains the tokens "jui-" and "cy", which decodes into "juicy", and is then re-encoded to "juic-" and "y".
In practice, we observe that these re-tokenization artifacts are rare: the "token error rate" is around $2\%$.
Furthermore, they do not impact adversarial accuracy by much: the re-tokenized example is in fact still adversarial. One potential mitigation strategy is to re-sample from $P_\Theta$ until the sampled text is stable under re-tokenization.
Note that all our adversarial accuracy results are computed after re-tokenization.

\paragraph{Runtime.}
Our method relies on white-box optimization and thus necessitates forward and backward passes through the attacked model, the language model and the similarity model, which increases the per-query time compared to black-box attacks that only compute forward passes.
However, this is compensated by a much more efficient optimization which brings the total runtime to $20s$ per generated example, on par with black-box attacks such as BERT-Attack~\cite{li-etal-2020-bert-attack}.

\begin{table*}[t!]
    \centering
    \resizebox{\linewidth}{!}{
    \begin{tabular}{l|l|l}
\textbf{Attack} & \textbf{Prediction} & \textbf{Text} \\
\midrule
Original    & World (99\%) & Turkey a step closer to Brussels The European Commission is set to give the green light later today \\ & & to accession talks with Turkey. EU leaders will take a final decision in December. \\
\textcolor{blue}{GBDA w/ fluency} & \textcolor{blue}{Business (100\%)} & Turkey a step closer to Brussels The \textcolor{blue}{eurozone Union} is set to give the green light later today to \\ & & accession talks with \textcolor{blue}{Barcelona}. EU leaders will take a final decision in December. \\
\textcolor{red}{GBDA w/o fluency} & \textcolor{red}{Business (77\%)} &  Turkey a step closer to \textcolor{red}{Uber} \textcolor{red}{Thecom} Commission is set to give the green light later today to \\ & & \textcolor{red}{accessrage negotiations} with Turkey. EU leaders will take a final decision in December. \\[0.5em]

Original    & Science (76\%) &  Worldwide PC Market Seen Doubling by 2010  NEW YORK (Reuters) - The number of personal \\ & & computers worldwide is expected to double to about 1.3 billion by 2010,  driven by explosive growth \\ & & in emerging markets such as China,  Russia and India, according to a report released on Tuesday by \\ & &  Forrester Research Inc. \\
\textcolor{blue}{GBDA w/ fluency} & \textcolor{blue}{Business (98\%)} &  Worldwide PC \textcolor{blue}{Index} Seen Doubling by 2010  NEW YORK (Reuters) - The number of personal \\ & &  \textcolor{blue}{consumers} worldwide is expected to double to about 1.3 billion by 2010,  driven by explosive growth \\ & & in emerging markets such as China,  Russia and India, according to a report released on Tuesday by \\ & &  Forrester Research Inc. \\
\textcolor{red}{GBDA w/o fluency} & \textcolor{red}{Business (96\%)} &  Worldwide PC Market Seen Doubling by \textcolor{red}{2010qua} NEW YORK ( \textcolor{red}{REUTERSrow} - The number of \\ & &  personal computers worldwide \textcolor{red}{pensions} expected to \textcolor{red}{doublearound} about 1.3 billion \textcolor{red}{audits investors},  \\ & &  driven by explosive growth in emerging markets such as \textcolor{red}{Chinalo ru} Russia and \textcolor{red}{Yug Holo} \\ & &  according to a report released \textcolor{red}{onTue} by  Forrester Research Inc. \\[0.5em]
    \end{tabular}
    }
    \caption{\label{tab:fluency_constraint}Examples of adversarial texts on AG News generated with and without the fluency constraint. Without the fluency constraint, the constructed adversarial text tends to contain more nonsensical token combinations.}
\end{table*}

\section{Appendix}
\label{sec:appendix}

\begin{figure}[t]
    \centering
    \includegraphics[width=0.99\linewidth,trim=30 21 30 34, clip]{figures/losses.pdf}
    \caption{
    \label{fig:optim_annealing}
    Losses across iterations.
    The adversarial distribution first focuses on fooling the classifier, and then improves the fluency of its samples.}
\end{figure}

\paragraph{Optimization behavior.}
Figure~\ref{fig:optim_annealing} shows the behavior of our adversarial distribution through optimization.
We also measure the entropy of the distribution: we can see that it first increases, and, after a few iterations, decreases.

redpajama_set_name: RedPajamaArXiv


text:
Aeschynomene gracilis är en ärtväxtart som beskrevs av Julius Rudolph Theodor Vogel. Aeschynomene gracilis ingår i släktet Aeschynomene och familjen ärtväxter. IUCN kategoriserar arten globalt som livskraftig. Inga underarter finns listade i Catalogue of Life.

Källor

Externa länkar

Ärtväxter
gracilis

redpajama_set_name: RedPajamaWikipedia


The more healthy you are, the more likely you are to get pregnant and deliver a healthy baby. Once you have a family, you want them to be healthy and active as well. Learn how to create a family environment focused on healthy eating and regular exercise. Teach your children from a young age how to choose healthy foods. See how much fun it can be to do fitness and strength training exercises with your spouse on a date night. By following a few simple ideas, your children will learn through your example how to live healthy lives. After all, isn't that what all parents want?
One of the best ways to inspire your spouse to exercise is to plan a date night around physical activity. Choosing something other than the usual dinner and a movie will help you form new traditions and lasting memories.Make sure you plan an activity that is fun and will be enjoyable for both you and your partner. No matter what activity you choose to do, be safe.


We provide 2d confirmation, gender and heartbeat scans for mommys to be!
Find out your baby's gender at 14+weeks! 5-10 minute session includes gender determination, measurement for gestational age/due date, heart rate and 2 B&W images.
Positive Pregnancy Test? We can confirm the presence of heartbeat at 8+weeks. 5-10 minute session includes 1 measurement for gestational age/estimated due date and 1 B&W print.
We can send a link to your smartphone of all the images captured during your session for an additional $5.


Contact Us | Rocky Mountain Environmental Ltd.
If you would like more information about our company and the products and services we offer, please complete the form below and we will contact you as quickly as possible. All fields marked with an asterisk (*) are compulsory.
Please list the Part #, Description and Quantity for the items you are interested in.


Blood tests – including a complete blood count, blood chemistries, kidney and liver function tests, and a urinalysis.
Spinal tap/lumbar puncture – a special needle is placed into the lower back into the spinal canal (the area around the spinal cord). The pressure in the spinal canal and brain can then be measured. A small amount of cerebral spinal fluid can be removed and sent for testing to determine if there is an infection or other problems. Cerebral spinal fluid is the fluid that bathes your child's brain and spinal cord.
Biopsy of primary tumor and/or metastatic lesions.
Stage 1 – involves a tumor that does not cross the midline of the body, is completely resectable and has not spread to other areas of the body. The lymph nodes on the same side of the body as the tumor do not have cancer cells present.
Stage 2A – involves a tumor that does not cross the midline of the body, but is not completely resectable. This stage of tumor has not spread to other areas of the body, and lymph nodes on the same side as the tumor do not have tumor cells present.
Stage 2B – involves a tumor that may or may not be completely resectable, has not spread to other areas of the body, but lymph nodes on the same side of the tumor have tumor cells present. Lymph nodes on the opposite side of the tumor must be negative for tumor cells in this stage of disease.
Stage 3 – involves a tumor that crosses the midline of the body, is not completely resectable, and lymph nodes are positive for tumor cells. This stage also includes a tumor that does not cross the midline, but the lymph nodes on the opposite side also contain tumor cells.
Stage 4 – involves a tumor that has metastasized (spread) to distant lymph nodes, bone marrow, liver, skin and/or other organs (except as defined in Stage 4S).
Stage 4S – involves a tumor that has metastasized to liver, skin and/or bone marrow (includes minimal bone marrow involvement; more extensive bone marrow involvement should be classified as Stage 4).


Revelo Resources Corp. today announced the completed acquisition of four properties from Altius Minerals Corp.'s 49 percent owned Chilean subsidiary BLC SpA, and the completion of a private placement with BLC for profits of $750,000.
"The conclusion of this deal shows that Revelo continues to grow its land position in northern Chile with high quality, 100 percent owned projects, and also continues to attract important, well-respected and serious investors to its shareholder base," Revelo President and CEO Tim Beale said. "Revelo remains committed to the consolidation of quality projects in Chile during this protracted downturn for the industry, with the expectation that these projects will, over time, attract well-financed and technically competent companies to further their exploration.
The private placement consists of 5 million units for proceeds of $750,000 at $0.15 per unit. The proceeds from the placement will be used to fund Revelo's exploration of the newly acquires properties and other activities.
"Our companies understand that we share a common, rather than competing, business objective," President and CEO of Altius Brian Dalton said. "That is to source high-quality and innovative projects that will be among the first to attract funding and development partners once funding inevitably returns to the exploration sector of this remarkably mineral endowed country. Our respective team member skills and landholdings bring many complements to each other. We look forward to a very positive Chilean collaboration with Revelo."


Renias refine "quality", "technology" and "person"rather than pursue expansion of scale, to become a company that responds to our customers with the highest product quality, outstanding first-class technical strength, service.
Folling our guide lines "Manufacturing with Safety and Trusting" Renias is creating an environment-friendly future for people and the for the planet.
Renias acquired "E-Mark" for automobile resin window safety glass, specified by UN Economic Commission.
Acquired factory certification for automotive safety glass specification stipulated by Japanese Ministry of Economy, Trade and Industry.
Acquired the number "DOT-1074" as a glazing material maker specified by NHTSA (National Traffic Safety Administration).
Acquired test certification of ANSI Z 26.1 - 1996 standard.


Return the MPI data type that should be used for receiving from neighbor in specified direction.
Return pointer to memory where received data is written to.
Return the MPI data type that should be used for sending to neighbor in specified direction.
Return pointer to data that should be send to neighbor in specified direction.


text:
Review: How Economics Shapes Science by Paula Stephan
Cureus	in General	 June 4, 2012 303 Words
www.cureus.com
Cohesive Mitigation School Strategies as Protective Factors Against the Increase of COVID-19 Cases
Cureus Conversations: Q&A With Cureus Co-Founder Alex Muacevic
Cureus Conversations: Q&A With Editor in Chief Dr. John Adler
Using Platelet-Rich Plasma (PRP)Therapy to Efficiently Reduce Beard Hair Loss
Since When is "Fast and Efficient" a Shortcoming?
Cureus on Does the Current Anti-science…
Kriegh Moulton on Does the Current Anti-science…
John R. Adler, Jr.,… on The Covid-19 Pandemic: Not a…
Mo on The Covid-19 Pandemic: Not a…
Venkataramana Kandi on Since When is "Fast and…
Cureus 2012 International Poster Competition
Cureus Authors in the News
Mobile Health Care
For those of us who haven't spent a career in academic medicine, we tend to think of research and publishing as driven by the quest for knowledge. Harvard University Press recently released a book by Paula Stephan (Professor of Economics at Georgia State University) focusing on how science – grants, research topics and, ultimately careers – is influenced by money to a much larger degree than most people realize.
In Stephan's discussion of collaboration in journal articles, for example, she makes the point that papers are increasingly written by teams of authors rather than a single author. This is due in part to the rising costs of research supplies, equipment and staff. About the only cost that has been decreasing is the cost of global communication which has added to collaborative endeavors. Increased collaboration also reflects the fact that researchers are increasingly narrowing and deepening their areas of specialty.
Interestingly, team-authored articles receive more citations than sole-authored articles in virtually all scientific fields; and in the US, scientific articles coauthored with a collaborator at another institution will have a higher impact factor than scientific articles from authors working together at the same institution.
In addition to these substantive points, Stephan also mentions a number of facts that are interesting and fun. What layman would know that "off the shelf" research mice sell for $17-$60 each, or that some mice predisposed to a certain disease can cost several thousand dollars a piece? How could a layman know the details of the IceCube Neutrino Observatory which sits under the South Pole and lists all 256 project members as authors of articles coming from the project.
How Economics Shapes Science by Paula Stephan, is very readable and fascinating for anyone with an interest in scientific and medical research and how they are influenced by money.
Paula Stephan
Cureus Cureus is redefining the way medical science is published, reviewed and shared.
Common Mistakes When Writing Peer Reviewed Papers
Forbidden Science – Addressing Behavior With Surgery
The Cureus Blog

redpajama_set_name: RedPajamaCommonCrawl


text:
La misión STS-97 es una misión realizada por el Transbordador espacial Endeavour. Su objetivo principal fue entregar e instalar paneles solares. También preparó el arribo del laboratorio espacial Destinity.

Tripulación
 Brent W. Jett, Jr. (3), Comandante
 Michael Bloomfield (2), Piloto
 Joseph R. Tanner (3), Especialista de misión
 Carlos Noriega (2), Especialista de misión -
 Marc Garneau (3), Especialista de misión -

 ( ) número de vuelos realizados incluyendo esta misión.

Parámetros de la misión
 Masa:
 Al despegue: 120,742 kg
 Al aterrizaje: 89,758 kg
 Carga: 7,906 kg
 Perigeo: 352 km
 Apogeo: 365 km
 Inclinación: 51.6°
 Período: 91.7 min

Acoplamiento con la ISS
 Acoplado: 2 de diciembre de 2000, 19:59:49 UTC
 Desacoplado: 9 de diciembre de 2000, 19:13:00 UTC
 Tiempo acoplado: 6 días, 23 h, 13 min, 11 s

Paseos espaciales
  Tanner y Noriega  - EVA 1
 EVA 1 Comienzo: 3 de diciembre de 2000 - 18:35 UTC
 EVA 1 Fin: 4 de diciembre - 02:08 UTC
 Duración: 7 horas, 33 minutos
  Tanner y Noriega  - EVA 2
 EVA 2 Comienzo: 5 de diciembre de 2000 - 17:21 UTC
 EVA 2 Fin: 5 de diciembre - 23:58 UTC
 Duración: 6 h, 37 min
  Tanner y Noriega  - EVA 3
 EVA 3 Comienzo: 7 de diciembre de 2000 - 16:13 UTC
 EVA 3 Fin: 7 de diciembre - 21:23 UTC
 Duración: 5 h, 10 min

Resumen de la Misión

Durante los 11 días de la misión, los objetivos primarios fueron completados, el cual instaló el primer set de paneles solares a la Estación Espacial Internacional. Lo astronautas realizaron tres EVAs durante el cual prepararon un puerto de acoplamiento para el arribo del laboratorio espacial Destinity, instalaron un dispositivo para medir la potencia eléctrica circundante en la estación, instalaron una cable de cámara fuera del módulo Unity, y transfirieron suministros, equipamiento y basura entre el equipo de la estación y el Endeavour.

En el Día 3, el comandante Brent Jett vinculó el Endeavour a la ISS mientras se encontraba 370 km sobre el noroeste de Kazajistán.

El chequeo exitoso de los trajes espaciales (EMU por sus siglas en inglés), las unidades SAFER (auxilio inmediato en el EVA), el Canadarm, el Sistema de Visión Espacial del Orbitador (OSVS en inglés) y el Sistema de Acoplamiento del Orbitador (ODS en inglés) fueron completados nominalmente. También, el la cámara del ODS fue instalada sin desalineamientos.

Al interior del Endeavour, el Especialista de misión Marc Garneau usó el Canadarm para mover el segmento P6 desde la bodega de carga, maniobrando en una posición de reposo de madrugada para calentar sus componentes. Los especialistas Joseph R. Tanner y Carlos Noriega se movieron a través del túnel de acoplamiento del Endeavour y abrieron la esclusa al puerto de acoplamiento de la ISS para dejar suministros y hardware en el umbral de la estación. En el día 4, la Expedición 1 entró por primera vez al módulo Unity para recoger lo dejado por la tripulación.

A las 9:36 a.m. EST el día 8 de diciembre, la tripulación dejó su primera visita a la Expedición 1 la cual está residiendo en la estación espacial. El 9 de diciembre la tripulación finalmente terminó sus tareas en la ISS. El Endeavour se despidió de la Expedición 1 a las 10:51 a.m. EST y cerraron las esclusas entre las dos naves. Después de 6 días, 23 h y 13 min, el transbordador se desacopló de la estación a las 2:13 p.m. EST. El desacoplamiento se produjo 378 km sobre el borde entre Kazajistán y China. El alejamiento final se produjo cerca del noroeste de Sudamérica.

La misión STS-97 es la 15.ª del Endeavour y la 101 del programa.

Véase también
 Transbordador espacial Endeavour
 Transbordador Espacial
 Lista de misiones tripuladas al espacio por programa

Enlaces externos

 Sumario de la misión

Programa del transbordador espacial
Misiones espaciales tripuladas
Naves y artefactos espaciales lanzados en 2000


text:
You are Here: Home » News » Roughyeds » Oldham RLFC launch campaign for 'Our League'
Oldham RLFC launch campaign for 'Our League'
The campaign is to help the club survive during the current pandemic.
At a time when, stricken by the pandemic, the national economy is taking a pounding -- and sport generally is fighting for its life -- Oldham RLFC has launched a campaign to get as many local people as possible signing up for 'Our League', the RFL's FREE membership app.
It features live streamed games, prizes, a full match centre, news, interviews and discounts on RFL material.
It costs not a penny to join, but the more membership figures the club gets, the more Roughyeds receive in central funding at a critical time when, like all clubs, income is well down.
Given that 'Our League' is FREE to fans but hugely beneficial to the club, Roughyeds are urging fans to sign up and to encourage neighbours, relatives, friends and workmates to follow suit.
Of the 25 Championship and League One clubs, Oldham currently lie in mid-table with 2,441 registered members and a 12th-place rating.
Dewsbury Rams, one place higher with 2,505 members, can be easily caught.
The flip side is that Whitehaven, one place and only seven members behind, are breathing down Oldham's neck.
Bradford Bulls are out in front, followed by Leigh, Widnes and Featherstone.
Barrow Raiders are the leading League 1 club in seventh spot overall and there's another League One club also higher than Roughyeds in Doncaster, who have 2,702 members.
Said the RFL's Mark Foster: "Membership numbers continue to grow because of what the app has to offer in terms of experience and engagement."
Subscription is FREE and the app is widely available via app stores and at rugby-league.com/ourleague.
As part of their application, Roughyeds fans need to say they support Oldham.
Said club chairman Chris Hamilton: "Financially, things are very tough for everyone but this is one way the club can benefit with no expense to anyone else. We would really appreciate the support of Oldhamers on this."


Rigas Farmaceitiska Fabrika JSC is the Northern European manufacturer of Herbal Medicinal products for the following dosage forms: liquids for internal use, liquids for external use, semi-solids. Our portfolio of finished products consists of various OTC medicines, food supplements and cosmetics. We focus on Liquid extracts, Tinctures, Sprays, Syrups, Gels, Creams, Herbal teas. RFF is GMP certified since 2004.


text:
Benefits of Breakfast in Weight Loss Don't Add Up
Nancy A. Melville
Despite long-held beliefs and widespread medical recommendations, breakfast does not appear to have an important role in weight loss after all, with a new meta-analysis showing no evidence that eating breakfast reduces daily caloric intake and weight gain is no worse among those who skip the meal.
"While breakfast has been advocated as the most important meal of the day in the media since 1917, there is a paucity of evidence to support breakfast consumption as a strategy to achieve weight loss, including in adults with overweight or obesity," the authors conclude.
"This systematic review of randomized controlled trials examining weight change in adults consuming or skipping breakfast found no evidence to support the notion that breakfast consumption promotes weight loss or that skipping breakfast leads to weight gain," they add.
In their meta-analysis, Katherine Sievert and colleagues with the School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia, evaluated data from 13 randomized controlled trials, including seven that examined the effect of eating breakfast on weight change (n = 486) and 10 that examined the effect on energy intake (n = 930).
The study was published today in BMJ.
Although there was some inconsistency between studies, overall there was a very small difference in weight favoring participants who skipped breakfast (mean difference, 0.44 kg).
Meanwhile, studies in which participants were assigned to eat breakfast showed they had a higher total daily energy intake than those allocated to skip breakfast (mean difference, 259.79 kcal/day), contrary to theories that not consuming breakfast leads to over-compensation later in the day.
"There was no evidence that skipping breakfast was associated with an increased total daily caloric intake," say Sievert and colleagues.
In an accompanying editorial, Tim Spector, MD, professor of genetic epidemiology at King's College, London, UK, notes that the "disadvantages of skipping breakfast have now been debunked by several randomized trials," which are examined in the current meta-analysis.
The conclusions were the same as in recent, largely ignored qualitative reviews, he adds, "namely, that no evidence supports the claim that skipping breakfast makes you gain weight or adversely reduces your resting metabolic rate."
Importantly, the overall data underscore that metabolism may simply be more of an individualized concept than has previously been realized, and a "one size fits all" approach may be futile, he says.
Low Quality Studies Included, Interpret Findings With Caution
In their article, Sievert and colleagues explain that much of the previous research linking consumption of breakfast with healthy weight is based on observational studies, but those studies may have important caveats in terms of participants' lifestyle.
"There are data to suggest that these findings on regular breakfast consumption in observational studies are reflective of a wider healthy lifestyle, in that individuals who are more health conscious and of higher socioeconomic status are more likely to eat breakfast as part of making healthy food choices," they explain.
The studies in the current analysis nevertheless had a variety of caveats of their own, including the fact that many either had a high or unclear risk of bias and follow-up periods were generally short, with a mean of 7 weeks for weight and 2 weeks for energy intake.
"As the quality of the included studies was mostly low, the findings should be interpreted with caution," the authors underscore.
Furthermore, with the analysis focusing on issues of weight management, conclusions cannot be made about the numerous other asserted health benefits of breakfast.
"Although eating breakfast regularly could have other important effects, such as improved concentration and attentiveness levels in childhood, caution is needed when recommending breakfast for weight loss in adults, as it could have the opposite effect."
Could the Gut Microbiome Play a Role?
In his editorial, Spector goes on to explain that evidence is increasingly pointing to the potential benefits of fasting intervals in weight loss.
And there may be a connection between the gut microbiome and timing as well as type of meals, he added. "The community of 100 trillion gut microbes have a circadian rhythm and vary in composition and function in fasting and fed states," he explains.
"Although this is a young discipline, some data suggest that microbial communities could benefit from short periods of fasting."
He concludes: "While waiting for guidelines to change, no harm can be done in trying out your own personal experiments in skipping breakfast."
BMJ. Published online January 30, 2019. Abstract, Editorial
For more diabetes and endocrinology news, follow us on Twitter and on Facebook.
Medscape Medical News © 2019 WebMD, LLC
Send comments and news tips to news@medscape.net.
Cite this: Benefits of Breakfast in Weight Loss Don't Add Up - Medscape - Jan 30, 2019.
Freelance writer, Medscape
Disclosure: Nancy A. Melville has disclosed no relevant financial relationships.
Can Overweight Docs Really Give Credible Weight Loss Advice?
Pros, Cons of Weight Loss Drugs; Jordan's Princess Tackles Cancer in the Arab World; and Doctors Embrace Corporate Jobs
Article Trending Clinical Topic: Weight Loss Drugs
NHS England Launches New Weight Management Services
Journal Article Aversion to Off-Label Prescribing in Clinical Pediatric Weight Management
Many Saw 'Meaningful' Weight Loss From 12-Week Online Program
Heparin Use in Deep Venous Thrombosis
Estimation of Fetal Weight
Neonatology Considerations for the Pediatric Surgeon
Total Joint Replacement Rehabilitation
Fast Five Quiz: Type 2 Diabetes and Obesity and Weight Management
16 Mnemonics All Docs Can Use
Physician Lifestyle Report 2015
Jan 27, 2023 This Week in Cardiology Podcast
Spanish Consensus Document on Severe Adult Asthma Updated
Cutting Carbs in Morning Equals Better Weight Loss, Waist Size
Weight-loss myths refuted in new review
b:curatedcuratedHasData : true


Surrounded yourself with a great team: No expedition is done completely solo – a reliable team is the most important foundation. Even free solo climber Alex Honnold needed a team to turn his pioneering El Capitan climb into an "Oscar"-winning reality. Being couch-bound, not being able to transport even a tea cup for two metres I had to make the important realisation that I will not make it back to the summits of this world all by myself. Thankfully i am surrounded by amazing people.
I cannot be grateful enough to my wonderful wife Esther, who has been helping and supporting me day in/day out, my daughter who has been a sunshine (almost) every day, my parents who have been helping me with tips and my friends who made sure that even with a handicapping injury one can have (some) social life.
An Optimistic Mindset: Stuck in a challenging situation it's sometimes hard to see the end of the tunnel. Sitting in a tent, freezing in a snow storm, eating miserable food it can feel so far away from home – the obvious question comes up: What am I doing here?
Taking one super painful tiny little step after the other (and being totally exhausted after just ten) the sheer idea of ever running the trails of Hong Kong seemed to be (and still is) a moonshot. But I forced myself to stay optimistic.
"The glass is half full"! I decided early on that being negative, depressed, sad etc about what happened will not make me recover faster. This is my situation, I have to accept it. There are tons of people that have it worse, I have great loving people around me, I will get through with this and I will be stronger afterwards!
Constant Learning: The few days I have been feeling vulnerable or sad about my situation I have used as opportunity to look inward. Why? What is happening? I have written countless pages to get to the bottom on how I ended up in this position- why did the accident happen? Where was the flaw in my decision making progress? It's too easy to look at things in hindsight – everything looks so clear and obvious (Why was I climbing there in the first place? Why did I not turn around)? But once I started to go deeper I realised that I had similar events in the past. They were merely close calls, so I just didn't register them. Hence the conclusion to alter my decision making process.
Can I say that it would never happen again: No – but do I feel better prepared, to not repeat the same decisions again. Absolutely!
The Discipline to do the hard thing: As I was lying in my bed, my foot swollen like an elephant a picture came to my mind, that had fascinated me as a boy. It was the one of Austria's tennis hero Thomas Muster crying out in pain, tied to a wooden rack, training on the court while recovering from a knee injury. Its a cry of grit, a cry of doing the hard part necessary to get back. Right after my accident taped that picture to my bedside table. Next to it a calendar – ticking off one day after the other.
At first it was the discipline to keep my foot totally free of any weight. What a difficult exercise, especially when one is used to move around daily. Once the healing process set it, it was the daily goal of exercising, of keeping the body moving. It's too early to tell, but I am convinced that the pain and suffering will pay off. And on the days when I felt lazy (and there were quite a few) I look at that photo – that cry of pain. Yes I can do that too!!
Most importantly having more time to spend with my wonderful daughter, who has definitely learned more German words and the last few weeks than ever before.
So while I have been shedding tons of tears during my physio (no joke – I have more pain now then after my fall) and I have finally reached the next level of mobility I know I will do it. I will come back!
This entry was posted in Adventure, Blog and tagged #adventure, climbing, recovery, travel. Bookmark the permalink.


I am calling upon the gifted and skilled users of the graphic forums to create for me a awesome signature.
Image(s) : Provided in the attachment, I tried to get the biggest resolution possible for the images. Text : My name. Extras (If any?) : If possible I'd like to have the font in a army/military font, to go with the military style of the pictures.
I pretty much have no requests other then the ones I mentioned above, the creator is free to let his imagination loose with the images I provided.
o.o first time ever done this kind of sig.
Wow Genbor_kreith that looks really good.
Thanks for your signature Gendor, unfortunately I can't use it because the forum rules only allow signatures sized 800X160.
Also if possible I'd really love it if you could add the Front wheel loader in the signature if possible.
nice work there. I'd just make the emblom & flag smaller to fit forum rules.


text:
Q: Giving custom variable to `hue` in sns.pairplot (Seaborn) I have the air quality(link here) dataset that contains missing values. I've imputed them while creating a dummy dataframe[using df.isnull()] to keep track of the missing values.
My goal is to generate a pairplot using seaborn(or otherwise - if any other simpler method exists) that gives a different color for the imputed values.
This is easily possible in matplotlib, where the parameter c of plt.plot can be assigned a list of values and the points are colored(but the problem is I can plot only against two columns and not a pairplot). A possible solution is to iteratively to create subplots against pairs of columns(which can make the code quite complicated!!)
However, in Seaborn (which already has the builtin function for pairplot) you are supposed to provide hue='column-name' which is not possible in this case as the missingness is stored in the dummy dataframe and need to retrieve the corresponding columns for color coding.
Please let me know how I can accomplish this in the simplest manner possible.

redpajama_set_name: RedPajamaStackExchange


text:
11-Year-Old Boy Helps Deliver Baby Brother [VIDEO]
Tonya Pendleton, BlackAmericaWeb.com
An 11-year-old boy had an opportunity to test out his aspirations to become a physician when he helped his mother deliver his baby brother. On Tuesday, Kenyarda Dukes realized she was in active labor and there was no time to get to a hospital. Her son James was the only person home at the time and he stayed cool under pressure.
WSBVTV.com reports:
"I called 911. They told me what to do. To get towels and wrap the baby up," he said.
James did just that, welcoming his new brother, Royal Dukes, into the world, weighing 6 pounds 9 ounces.
"Words can't express how proud I am. He is the epitome of when you tell your son that 'you're the man of the hous,e so you have to take care of those responsibilities,' and he did it to a 'T,'" the boys' father said.
James says he just did what he had to do.
11-Year-Old Boy Helps Deliver Baby Brother [VIDEO] was originally published on blackamericaweb.com
Babies	, birth	, brothers	, Marietta	, motherhood	, Siblings


Autonomous vehicles such as cars, trams, boats and trucks are cruising towards becoming a mainstream reality. To achieve full autonomy, onboard systems contain an array of complementing technologies. LIDAR, radar, ultrasonic, inertial sensors, cameras and digital maps are such technologies. However, they can only provide relative localisations. GNSS is the only technology which can provide an absolute position, and hence plays a critical role in any autonomous system. Conventional GNSS receivers however typically provide a positioning accuracy of approximately 5 metres (16 feet). With additional correction methods (using RTK, PPP) this can be improved significantly to centimetre level precision.
The more GNSS signals obtained from multiple constellations. Increasingly autonomous systems not only require increased accuracy but also reliability, integrity and availability.
Multiple GNSS signals from multiple systems are also indispensable when driving in more challenging environments such as cities, where buildings may block many GNSS satellites.
Finally, atmospheric conditions such as ionospheric scintillation may impact position accuracy and availability, which is also improved with use of multiple systems.
Septentrio's high precision GNSS technology with its high-accuracy positioning engine and AIM+ anti-jamming and anti-spoofing technology provides secure centimetre-level accuracy, multi-constellation availability and the reliability and integrity that is indispensable for a self-moving vehicle.
Septentrio mosaic, a multi-band, multi-constellation receiver in a low power surface mount module with a wide array of interfaces.
The AsteRx-m2a is a smaller-than-a-credit-card Heading and RTK receiver. It is ideal for UAS, INS integrations and other industrial applications that require high reliability in positioning combined with low power consumption.


text:
AMAA's Avedisian School Bids Farewell to its Graduates
Khoren and Shooshanig Avedisian School graduates at the 2019 Commencement Ceremony in Yerevan, on June 13
Principal Melanya Geghamyan
Doreen Bilezikian
Zaven Khanjian
Yervant Chekijian
YEREVAN—The 2019 Commencement Ceremony of the AMAA's Khoren and Shooshanig Avedisian School in Yerevan was very emotional. On June 13, teachers, parents, upper grade students, representatives of the AMAA and the Evangelical Church of Armenia, founders of the school, members of the school board, overseas guests, and representatives of various local and international organizations came together to bid farewell to the School's 31 graduates.
Principal Melanya Geghamyan greeted guests and expressed her heartfelt advice to the graduates. She thanked school benefactors Edward and Pamela Avedisian (who were unable to come to Yerevan this year), the Bilezikian family and others, including Zaven Khanjian, AMAA Executive Director/CEO, Harout Nercessian, AMAA Representative in Armenia, partner organizations, and the parents for believing in the vision of the Avedisian School. She thanked them for their trust, and for their continual support of the school.
Khanjian addressed the graduates and wished them much success in their journey to higher achievements, and advised them to walk in the right path and to build a "new and powerful" Homeland.
The keynote speaker was Doreen Bilezikian representing the Bilezikian family. Her message was profound and encouraging. She spoke about the dangers, difficulties, and obstacles in the crossroads of life, and advised the graduates not to fall apart, but to continue their road uninterrupted.
On behalf of the graduates, Anna Galstyan spoke in English.
The graduates then stood up with their homeroom teachers, K. Harutyunyan and N. Margaryan, as their names were called one by one to receive their certificates from Principal Geghamyan, AMAA Executive Director/CEO Zaven Khanjian, and Benefactor Yervant Chekijian. Then, 31 caps simultaneously took off in the air.
The warm atmosphere continued in the school's lobby, around the refreshments tables. Regardless of their spoken language, place of residence, profession and occupation, those gathered under the roof of the Avedisian School had much to say to each other.
"And let knowledge and love become the compass of your great voyage," stated Avedisian School Principal, Melanya Geghamyan.


text:
گروه‌های کاری
انتشارها
نشان تجاری
سیاست‌های حریم شخصی
The Node.js trademarks, service marks, and graphics marks are symbols of the quality, performance, and ease of use that people have come to associate with the Node.js software and project. To ensure that the Node.js marks continue to symbolize these qualities, we must ensure that the marks are only used in ways that do not mislead people or cause them to confuse Node.js with other software of lower quality. If we don't ensure the marks are used in this way, it cannot only confuse users, it can make it impossible to use the mark to protect against people who maliciously exploit the mark in the future. The primary goal of this policy is to make sure that this doesn't happen to the Node.js mark, so that the community and users of Node.js are always protected in the future.
At the same time, we'd like community members to feel comfortable spreading the word about Node.js and participating in the Node.js community. Keeping that goal in mind, we've tried to make the policy as flexible and easy to understand as legally possible.
The OpenJS Foundation has a perpetual license to use the Node.js marks. For more details on using the Node.js mark, please read the full policy. If you have any questions don't hesitate to email us.
Guidelines for the visual display of the Node.js mark are described in the Visual Guidelines.


text:
Wide Receiver #80
Danny Amendola News
Retires from NFL
Amendola opted to retire rather than return for his 14th NFL season, Adam Schefter of ESPN reports.
Analysis: Amendola received interest in free agency from multiple teams this offseason after recording 24 catches for 248 yards and three touchdowns with Houston in 2021, according to Schefter. However, the two-time Super Bowl champ with New England is apparently at peace with the course of his career, saying, "It was better than I could have ever imagined." Amendola accumulated 24 touchdowns and 6,212 receiving yards on 617 career catches over 13 seasons spent playing between the Rams, Patriots, Dolphins, Lions and Texans.
Jul 25, 2022 4:25 PM EST
Turns back clock
Amendola recorded seven receptions on eight targets for 113 yards and two touchdowns in Sunday's 28-25 loss to the Titans.
Analysis: Amendola was a substantial part of the offense thanks to the absence of Chris Conley (knee) and early exit of Brandin Cooks (knee). He took advantage of his opportunity and showed immediate rapport with Davis Mills, keying the Texans' comeback efforts. Amendola found the end zone from seven and 26 yards away, while also racking up big gains of 37 and 21 yards. The end result was his first game with over 100 yards since the 2019 season. Looking ahead, Amendola is set to hit free agency and could sign another short-term deal to serve as a depth receiver with the Texans or a new team.
Jan 9, 2022 8:25 PM EST
Back from COVID list
Amendola (undisclosed) was activated from the reserve/COVID-19 list Wednesday.
Analysis: Amendola landed on the COVID list last week and is now back on the active roster after missing one game. He missed the previous four games with a knee injury before entering the virus protocols, but he should be back on the field for the season finale this weekend.
On COVID list
Amendola (undisclosed) has been placed on the reserve/COVID-19 list.
Analysis: Until Amendola clears COVID protocols, Brandin Cooks, Nico Collins and Chris Conley (knee) are in line to serve as Houston's top available wide receivers. Amendola, who had been dealing with a knee issue and was recently activated off IR, last suited up in Week 12.
Dec 30, 2021 1:12 PM EST
Not on injury report
Amendola (knee) wasn't listed as injured on Wednesday's practice estimate.
Analysis: Amendola appears to have finally gotten past his knee injury. The veteran slot receiver's on track to suit up for the first time since Week 12 come Sunday in San Francisco.
Not playing Week 16
Amendola (knee) is inactive for Sunday's game against the Chargers.
Analysis: Though Amendola returned to practice Thursday and was reinstated from injured reserve Saturday, the Texans will apparently give him more time off to recover from the knee injury that sidelined him for the past three games. With Brandin Cooks (illness) also sidelined after landing on the reserve/COVID-19 list, Nico Collins, Chris Conley, Phillip Dorsett, Chris Moore and Jalen Camp will represent Houston's available options at receiver.
Dec 26, 2021 8:56 AM EST
Comes off IR
Amendola (knee) was activated off IR in advance of Sunday's game against the Chargers, Aaron Wilson of Sportstalk 790 Houston reports.
Analysis: With top wideout Brandin Cooks on the COVID-19 reserve list, Amendola's return comes at an opportune time. While it remains to be seen how much volume he'll be up for in Week 16, Amendola could pick up some targets out of the slot on Sunday.
Could play Sunday
Per coach David Culley, there's "a possibility" that Amendola (knee) could play Sunday against the Chargers, Brooks Kubena of the Houston Chronicle reports.
Analysis: Amendola has been designated to return from IR, but this nuggets suggests that he's not a lock to be activated for the contest, however. If he is though, Amendola could see immediate snaps out of the slot while bolstering a Week 16 Houston WR corps that may be without Brandin Cooks (reserve/COVID-19 list).
Designated to return from IR
Amendola (knee) has been designated to return from IR.
Analysis: The Texans now have a 21-day window in which to activate the slot man, but if he's able to demonstrate that he's ready for game action in the coming days, he could step into key role this Sunday against the Chargers in the event that Brandin Cooks isn't activated from the reserve/COVID-19 list. Amendola, who last suited up in Week 12, has recorded 17 catches (on 29 targets) for 135 yards and a TD in seven contests to date this season.
Dec 23, 2021 10:44 AM EST
Goes to injured reserve
The Texans placed Amendola (knee) on injured reserve Saturday, Aaron Wilson of Sportstalk 790 Houston reports.
Analysis: Amendola came out of the Texans' Week 12 loss to the Jets with a meniscus injury that required arthroscopic surgery Tuesday. With a recovery timeline of up to four weeks, this move was inevitable, and he won't be eligible return until a Dec. 26 game against the Chargers. While Amendola is in recovery mode, Brandin Cooks (illness), Nico Collins, Chris Conley (illness), Chris Moore and Davion Davis will comprise the Texans' group of wide receivers.
Dec 4, 2021 5:44 PM EST
Officially out Sunday
Amendola (knee) has been ruled out for Sunday's game against the Colts.
Analysis: Amendola sustained a meniscus injury during last week's loss to the Jets and was unable to practice ahead of Sunday's matchup. He's expected to miss up to four weeks, so the Texans could place the wide receiver on injured reserve to open up a spot on the active roster. Nico Collins, Chris Conley and Chris Moore could see an uptick in targets while Amendola is out.
Sustains knee injury
Amendola suffered a knee injury during Week 12's loss to the Jets and could miss up to four weeks, Brooks Kubena of the Houston Chronicle reports.
Analysis: Amendola underwent a scope of the knee Tuesday, and the Texans await the results. This will be the second time in 2021 that the 36-year-old slot receiver misses significant time. He was unavailable for four games early this season due to a hamstring injury. Despite missing those games, Amendola is currently tied for third with 29 targets, which means others will fill that void. Branding Cooks could receive more looks, but he's already well established as the team's No 1. Further down the depth chart, rookie Nico Collins, who's averaged 4.2 targets over six games since returning from his own injury, could see additional targets. Houston also has veterans Chris Conley and Chris Moore rostered. Moore was signed to the active roster after Amendola's first injury and remained on the roster since. He may be the primary fill-in at slot receiver.
Dec 1, 2021 6:17 AM EST
Five targets in loss
Amendola caught three of five targets for 23 yards in Sunday's 21-14 loss to the Jets.
Analysis: Amendola finished fourth among Houston's wideouts with 22 snaps but had an equal number of targets as did Brandin Cooks and Nico Collins, the Texans' top-two wideouts. One week after the 36-year-old wideout appeared to be headed for a reduced role, Amendola was a key figure as a receiver. Of course, it's a low-wattage passing attack, and Amendola's lone touchdown catch came in Week 1.
Nov 29, 2021 5:13 AM EST
Shrinking usage
Amendola did not catch his lone target and played 13 snaps in Sunday's 22-13 win over Tennessee.
Analysis: Amendola had the fewest number of snaps among the five wide receivers, which is a significant change. He had been among the top three in terms of playing time and targets prior to Houston's bye last week. That suggests the 36-year-old Amendola will play a reduced role over the final weeks of the regular season.
Another big target day
Amendola caught three of seven targets for 49 yards in Sunday's 17-9 loss to the Dolphins.
Analysis: Amendola remains a key part of the offense, having gathered 17 targets in the three games back from a thigh injury. The Texans have a bye Week 10, which could be an opportunity to recalibrate the offense. To have the 36-year-old Amendola sucking up targets on a 1-8 team seems pointless.
Nov 8, 2021 5:02 AM EST
Registers five targets in loss
Amendola caught two of five targets for three yards in Sunday's 38-22 loss to the Rams.
Analysis: In terms of production, Amendola's performance isn't much, but he somehow retains a significant part of the passing game when healthy. He was second among wide receivers in targets and snaps. In the two games since returning from a four-game absence due to a thigh injury, he's been targeted 10 times. That might be the Texans showcasing the 35-year-old receiver ahead of Tuesday's trade deadline.
Returns to action
Amendola caught three of five targets for 17 yards and returned one kickoff for 15 yards in Sunday's 31-5 loss to the Cardinals.
Analysis: Amendola returned to action Sunday after missing four games due to a thigh injury. He fit in behind Brandin Cooks and Nico Collins in terms of snap count and shared third wideout duties with Chris Conley. Amendola's five targets tied for third with Collins, behind David Johnson (seven) and Cooks (six). He and Tremon Smith both were added to the return team Sunday, the first game following the Texans' decision to release Andre Roberts. Amendola has been targeted 11 times over 53 snaps, the best target-to-snap ratio among the wideouts, but his injury history works against him. Plus, he's a 35-year-old (36 on Nov. 2) playing for a 1-6 team that should eventually start evaluating all of its younger players in the pipeline.
Oct 25, 2021 5:05 AM EST
Returns to full practice
Amendola (thigh) practiced fully Wednesday.
Analysis: Amendola is thus in line to return to action Sunday against the Cardinals and is in a position to reclaim a share of the Texans' slot duties in the process. In the two games he suited up for this season, Amendola caught all six of his targets for 43 yards and a TD. With rookie Davis Mills slated to start at QB at least one more game for Houston, Amendola profiles as a speculative Week 7 lineup option in deeper PPR formats while squaring off against 6-0 Arizona.
Oct 20, 2021 3:29 PM EST
To miss fourth straight
Amendola (thigh) is inactive Week 6 against the Colts.
Analysis: Amendola was able to log three practices on a limited basis this week, but that won't be enough for the 35-year-old to make his first appearance since Week 2. Look for Brandin Cooks, Chris Conley and Chris Moore to again lead the receiving corps in his stead, with a potential boost from the recently-activated Nico Collins.
Questionable against Colts
Amendola (thigh) is listed as questionable for Sunday's game against the Colts.
Analysis: Amendola was limited in all three practice sessions this week, so it's hard to get a read on his availability, especially after he also practiced in a limited fashion last week prior to sitting out. If he's able to suit up Sunday, it will be Amendola's first action since Week 2.
Oct 15, 2021 12:22 PM EST
News powered by
Latest Fantasy News
Cordale Flott: Plays in 11 games as rookie
Kayvon Thibodeaux: Comes on strong late in season
Micah McFadden: Works up to big role as rookie
Bryce Perkins: Struggles in limited action
John Wolford: Disappoints in two starts
Tevin Coleman: Gets call to active roster
Kyren Williams: Earns 44 total touches as rookie
Malcolm Brown: Minimal production in backup role
Kyle Rudolph: Returns to practice Friday
Evan Neal: Will play despite injury scare
Cheat Sheet Generator


text:
Boy, 16, is charged with murdering his mother and twin 13-year-old brothers who were found stabbed to death in their Alabama home
Scientists claim pathologist portrayed by Will Smith in 'Concussion' is a FRAUD who built his million-dollar career on distorted science after he published images of a healthy 50-year-old's brain in CTE study of NFL great 'Iron Mike'
What is it? Man captures video of spinning blue UFO 'cruising slow' over Utah
The stunning image of a UFO was captured on video last week, but is it an alien craft, or is there a simpler explanation?
Anthony Piceno, of American Fork, Utah, said he was drawn to the sight in the air because it was like nothing he had ever seen before.
His footage of the object later appeared on The Weather Channel.
Fact or fiction? Video captured this image of a blue UFO hovering over American Fork, Utah
The video shows the blue craft resembling a spinning top floating in the sky and moving very slowly.
Mr Piceno said: 'I looked up in the sky and, you know, it was just a different object, it was neon blue, red flashing lights'.
'I saw aliens': The Nanny star Fran Drescher claims she was abducted by extraterrestrials Beijing 'seven minutes from destruction' when a 2.5 ton satellite crashed to Earth at 300mph
He watched the object floating in the sky for a few minutes, observing how it was 'just cruising slow' in the sky.
But as with any UFO sighting, there are skeptics, who have their own ideas about what the flying object really was.
Lights in the sky: Anthony Piceno said he watched the object floating in the sky for a few minutes, observing how it was 'just cruising slow'
Lynn Hadfield, president of Utah Valley Aeromodelers, told The Weather Channel that he believes the lights are nothing more than radio-controlled planes in nearby Art Dye Park.
Mr Hadfield said: 'I believe it's an RC plane with lights... There have been a number of people who spot these around and wonder if they were UFOs'.
Chris Santacroce of Superfly Paragliding contends that Mr Piceno may have seen a paraglider in the area, although doing so at night is illegal.
The truth is out there: Mr Piceno points at the area in the sky where he saw the UFO
Questions: Skeptics say they have other ideas about what the object is, from a radio-controlled plane to a paraglider
Mr Santacroce told the network: 'If that was a paraglider over American Fork, then it wouldn't be the first time a paraglider flew at night with some lights on it'.
This is not the first time suspicious objects have been seen hovering in the sky in the area, as several people have reported close encounters in the past few months.
Last year, a top secret FBI memo detailing how police and army officers witnessed a UFO exploding over Utah was unearthed.
The cable, marked 'URGENT' and sent to then-FBI director J. Edgar Hoover, details how an army guard, a policeman and a highway patrol, who were all miles apart, each saw a UFO, which they said exploded over mountains near Logan, north of Salt Lake City.
Proof of (alien) life? A copy of the 1950 memo that recounts the discovery of flying saucers and aliens in New Mexico. The memo has been published on the FBI website
The memo is one of thousands of previously unreleased classified files that the bureau has made public in an online resource called The Vault.
Under the title 'Flying Discs', the document about the sightings in Utah said the three men each 'saw a silver coloured object high up approaching the mountains at Sardine Canyon' that 'appeared to explode in a rash of fire.'
It added: 'Several residents [reported] seeing what appeared to be two aerial explosions, followed by falling object.'
Documents show that an earlier UFO sighting had been investigated in Logan in September 1947.
It said numerous witnesses told the FBI they saw 'flying discs' in formation that were 'circling the city at a high rate of speed'.
Man captures video of spinning blue UFO 'cruising slow' over Utah
Kim Kardashian flaunts her hourglass figure in tight ensemble while sister Khloe doubles down on denim while shopping in Los Angeles