text:
\section{Introduction}
\label{sec:intro}
Recently task-oriented dialog systems have been attracting more and more research efforts \cite{gao2019neural,zhang2020recent},
where understanding user utterances is a critical precursor to the success of such dialog systems. While modern neural networks have achieved state-of-the-art results on language understanding (LU) \cite{wang2018bi,zhao2018improving,goo2018slot,liu2019cm,shah2019robust}, their robustness to changes in the input distribution is still one of the biggest challenges in practical use.

Real dialogs between human participants involve language phenomena that do not contribute so much to the intent of communication. As shown in Fig. \ref{fig:intro}, user expressions can be of high lexical and syntactic diversity when a system is deployed to users; typed texts may differ significantly from those recognized from voice speech; interaction environments may be full of chaos and even users themselves may introduce irrelevant noises such that the system can hardly get clean user input.

Unfortunately, neural LU models are vulnerable to these natural perturbations that are legitimate inputs but not observed in training data.
For example, \citet{bickmore2018patient} found that popular conversational assistants frequently failed to understand real health-related scenarios and were unable to deliver adequate responses on time.
Although many studies have discussed the LU robustness \cite{ray2018robust,zhu2018robust,iyyer2018adversarial,yoo2019data,ren2019generating,jin2020bert,he2020learning}, there is a lack of systematic studies for real-life robustness issues and corresponding benchmarks for evaluating task-oriented dialog systems.

In order to study the real-world robustness issues, we define the LU robustness from three aspects: \textit{language variety}, \textit{speech characteristics} and \textit{noise perturbation}. While collecting dialogs from deployed systems could obtain realistic data distribution, it is quite costly and not scalable since a large number of conversational interactions with real users are required.
Therefore, we propose an automatic method LAUG for \textbf{L}anguage understanding \textbf{AUG}mentation in this paper to approximate the natural perturbations to existing data. LAUG is a black-box testing toolkit on LU robustness composed of four data augmentation methods, including word perturbation, text paraphrasing, speech recognition, and speech disfluency.

We instantiate LAUG on two dialog corpora Frames \cite{el2017frames} and MultiWOZ \cite{budzianowski2018multiwoz} to demonstrate the toolkit's effectiveness. Quality evaluation by annotators indicates that the utterances augmented by LAUG are reasonable and appropriate with regards to each augmentation approach's target. A number of LU models with different categories and training paradigms are tested as base models with in-depth analysis. Experiments indicate a sharp performance decline in most baselines in terms of each robustness aspect. Real user evaluation further verifies that LAUG well reflects real-world robustness issues. Since our toolkit is model-agnostic and does not require model parameters or gradients, the augmented data can be easily obtained for both training and testing to build a robust dialog system.

Our contributions can be summarized as follows:
    (1) We classify the LU robustness systematically into three aspects that occur in real-world dialog, including language variety, speech characteristics and noise perturbation;
    (2) We propose a general and model-agnostic toolkit, \textit{LAUG}, which is an integration of four data augmentation methods on LU that covers the three aspects.
    (3) We conduct an in-depth analysis of LU robustness on two dialog corpora with a variety of baselines and standardized evaluation measures.
    (4) Quality and user evaluation results demonstrate that the augmented data are representative of real-world noisy data, therefore can be used for future research to test the LU robustness in task-oriented dialog\footnote{The data, toolkit, and codes are available at \url{https://github.com/thu-coai/LAUG}, and will be merged into \url{https://github.com/thu-coai/ConvLab-2} \cite{zhu2020convlab}.}.

\section{Robustness Type}

We summarize several common interleaved challenges in language understanding from three aspects, as shown in Fig. \ref{fig:type}:

\paragraph{Language Variety}
A modern dialog system in a text form has to interact with a large variety of real users. The user utterances can be characterized by a series of linguistic phenomena with a long tail of variations in terms of spelling, vocabulary, lexical/syntactic/pragmatic choice \cite{ray2018robust,jin2020bert,he2020learning,zhao2019data,ganhotra2020effects}.

\begin{figure}[!thb]
    \centering
    \begin{subfigure}{0.9\linewidth}\centering
    \includegraphics[width=\linewidth]{Intro1.pdf}
    \caption{Dataset construction}
    \end{subfigure}
    \begin{subfigure}{\linewidth}\centering
    \includegraphics[width=\linewidth]{Intro2.pdf}
    \caption{Real-world application}\label{fig:type}
    \end{subfigure}
    \caption{Difference between dialogs collected for training and those for real-world applications.}
    \label{fig:intro}
\end{figure}

\paragraph{Speech Characteristics}
The dialog system can take voice input or typed text, but these two differ in many ways. For example, written language tends to be more complex and intricate with longer sentences and many subordinate clauses, whereas spoken language can contain repetitions, incomplete sentences, self-corrections and interruptions \cite{wang2020data,park2019specaugment,wang2020multi,honal2003correction,zhu2018robust}.

\paragraph{Noise Perturbation}
Most dialog systems are trained only on noise-free interactions.
However, there are various noises in the real world, including background noise, channel noise, misspelling, and grammar mistakes \cite{xu2014targeted,li2020textat,yoo2019data,henderson2012discriminative,ren2019generating}.

\section{LAUG: Language Understanding Augmentation}
This section introduces commonly observed out-of-distribution data in real-world dialog into existing corpora. We approximate natural perturbations in an automatic way instead of collecting real data by asking users to converse with a dialog system.

To achieve our goals, we propose a toolkit \textit{LAUG}, for black-box evaluation of LU robustness. It is an ensemble of four data augmentation approaches, including Word Perturbation (WP), Text Paraphrasing (TP), Speech Recognition (SR), and Speech Disfluency (SD). Noting that LAUG is model-agnostic and can be applied to any LU dataset theoretically.
Each augmentation approach tests one or two proposed aspects of robustness as Table \ref{tab:check} shows. The intrinsic evaluation of the chosen approaches will be given in Sec. \ref{sec:setup}.

\begin{table}[h!tb]
    \centering
    \small
    \begin{tabular}{c|ccc}
    \hline
        Capacity & LV & SC & NP \\
    \hline
        Word Perturbation (WP) & $\surd$ & & $\surd$ \\
        Text Paraphrasing (TP) & $\surd$ & &\\
        Speech Recognition (SR) &  & $\surd$ & $\surd$ \\
        Speech Disfluency (SD) &  & $\surd$ & \\
    \hline
    \end{tabular}
    \caption{The capacity that each augmentation method evaluates, including Language Variety (LV), Speech Characteristics (SC) and Noise Perturbation (NP).}
    \label{tab:check}
\end{table}

\paragraph{Task Formulation}
Given the dialog context $X_t = \{x_{2t-m}, \dots, x_{2t-1}, x_{2t}\}$ at dialog turn $t$, where each $x$ is an utterance and $m$ is the size of sliding window that controls the length of utilizing dialog history, the model should recognize $y_t$, the dialog act (DA) of $x_{2t}$. Empirically, we set $m = 2$ in the experiment.
Let $\mathcal{U}, \mathcal{S}$ denote the set of user/system utterances, respectively. Then, we have $x_{2t-2i} \in \mathcal{U}$ and $ x_{2t-2i-1} \in \mathcal{S}$. The task of this paper is to examine different LU models whether they can predict $y_t$ correctly given a perturbed input $\tilde{X}_t$. The perturbation is only performed on user utterances.

\paragraph{Word Perturbation}

Inspired by EDA (\textit{Easy Data Augmentation}) \cite{wei2019eda}, we propose its semantically conditioned version, SC-EDA, which considers task-specific augmentation operations in LU. SC-EDA injects word-level perturbation into each utterance $x'$ and updates its corresponding semantic label $y'$.

\begin{table}[h!tb]
    \small
    \begin{tabular}{c | p{5.8cm}}
    \hline
        Original & I want to go to Cambridge . \\
        DA & attraction \{ inform (dest = Cambridge) \} \\
    \hline
        Syno. & I \textcolor{red}{wishing} to go to Cambridge . \\
        Insert & I \textcolor{red}{need} want to go to Cambridge . \\
        Swap & I \textcolor{red}{to want} go to Cambridge . \\
        Delete & I want \sout{\textcolor{red}{to}} go to Cambridge . \\
    \hline
        SVR & I want to go to \textcolor{red}{Liverpool} . \\
        DA & attraction \{ inform (dest = \textcolor{red}{Liverpool}) \} \\

    \hline
    \end{tabular}
    \caption{An SC-EDA example. Syno., Insert, Swap and Delete are four operations described in EDA, of which the dialog act is identical to the original one. SVR denotes \textit{slot value replacement}.}

    \label{tab:todeda}
\end{table}

Table \ref{tab:todeda} shows an example of SC-EDA. Original EDA randomly performs one of the four operations, including \textit{synonym replacement}, \textit{random insertion}, \textit{random swap} and \textit{random deletion}\footnote{See the EDA paper for details of each operation.}. Noting that, to keep the label unchanged, words related to slot values of dialog acts are not modified in these four operations.
Additionally, we design \textit{slot value replacement}, which changes the utterance and label at the same time to test model's generalization to \textbf{unseen entities}.
Some randomly picked slot values are replaced by unseen values with the same slot name in the database or crawled from web sources. For example in Table \ref{tab:todeda}, ``Cambridge'' is replaced by ``Liverpool'', where both belong to the same slot name ``dest'' (destination).

\textit{Synonym replacement} and \textit{slot value replacement} aim at increasing the language variety, while \textit{random word insertion/deletion/swap} test the robustness of noise perturbation. From another perspective, four operations from EDA perform an Invariance test, while \textit{slot value replacement} conducts a Directional Expectation test according to CheckList \cite{ribeiro2020beyond}.

\paragraph{Text Paraphrasing}
The target of text paraphrasing is to generate a new utterance $x' \neq x$ while maintaining its dialog act unchanged, i.e. $y' = y$.
We applied SC-GPT \cite{peng2020few}, a fine-tuned language model conditioned on the dialog acts, to paraphrase the sentences as data augmentation. Specifically, it characterizes the conditional probability $p_\theta(x|y) = \prod_{k=1}^{K} p_\theta (x_k | x_{<k}, y),$
where $x_{<k}$ denotes all the tokens before the $k$-th position. The model parameters $\theta$ are trained by maximizing the log-likelihood of $p_\theta$.

\begin{table}[thb]
    \centering
    \small
    \begin{tabular}{@{ }c@{ }|@{ }p{6.8cm}@{ }}
    \hline
        DA & train \textcolor{red}{*} \{ inform ( dest = Cambridge ; arrive = 20:45 ) \} \\
        Text & Hi, I'm looking for a train that is going to Cambridge and arriving there by 20:45, is there anything like that? \\
    \hline
        DA & train \{ inform ( dest = Cambridge ; arrive = 20:45 ) \} \\
        Text & Yes, to Cambridge, and I would like to arrive by 20:45. \\
    \hline
    \end{tabular}
    \caption{A pair of examples that consider contextual resolution or not. In the second example, the user omits to claim that he wants a train in the second utterance since he has mentioned this before.}
    \label{tab:paraphrase}
\end{table}

We observe that co-reference and ellipsis frequently occurs in user utterances. Therefore, we propose different encoding strategies during paraphrasing to further evaluate each model's capacity for \textbf{context resolution}. In particular, if the user mentions a certain domain \textit{for the first time} in a dialog, we will insert a ``*'' mark into the sequential dialog act $y'$ to indicate that the user tends to express without co-references or ellipsis, as shown in Table \ref{tab:paraphrase}. Then SC-GPT is finetuned on the processed data so that it can be aware of dialog context when generating paraphrases. As a result, we find that the average token length of generated utterances with/without ``*'' is 15.96/12.67 respectively after SC-GPT's finetuning on MultiWOZ.

It should be noted that slot values of an utterance can be paraphrased by models, resulting in a different semantic meaning $y'$. To prevent generating irrelevant sentences, we apply automatic value detection in paraphrases with original slot values by fuzzy matching\footnote{\url{https://pypi.org/project/fuzzywuzzy/}}
, and replace the detected values in bad paraphrases with original values. In addition, we filter out paraphrases that have missing or redundant information compared to the original utterance.

\paragraph{Speech Recognition} \label{sec:SR}
We simulate the speech recognition (SR) process with a TTS-ASR pipeline \cite{park2019specaugment}. First we transfer textual user utterance $x$ to its audio form $a$ using gTTS\footnote{\url{https://pypi.org/project/gTTS/}} \cite{oord2016wavenet}, a Text-to-Speech system. Then audio data is translated back into text $x'$ by DeepSpeech2 \cite{amodei2016deep}, an Automatic Speech Recognition (ASR) system. We directly use the released models in the DeepSpeech2 repository\footnote{\url{https://github.com/PaddlePaddle/DeepSpeech}} with the original configuration, where the speech model is trained on Baidu Internal English Dataset, and the language model is trained on CommonCrawl Data.

\begin{table}[!thb]
    \small
    \begin{tabular}{c|c|c}
    \hline
    Type & Original & Augmented \\
    \hline
    Similar sounds & leicester & lester\\
    Liaison & for 3 people & free people \\
    Spoken numbers & 13:45 & thirteen forty five\\
    \hline
    \end{tabular}
    \caption{Examples of speech recognition perturbation.}
    \label{tab:SR_types}
\end{table}

Table \ref{tab:SR_types} shows some typical examples of our SR augmentation. ASR sometimes wrongly identifies one word as another with similar pronunciation. Liaison constantly occurs between successive words. Expressions with numbers including time and price are written in numerical form but different in spoken language.

Since SR may modify the slot values in the translated utterances, fuzzy value detection is employed here to handle similar sounds and liaison problems when it extracts slot values to obtain a semantic label $y'$. However, we do not replace the noisy value with the original value as we encourage such misrecognition in SR, thus $y' \neq y$ is allowed. Moreover, numerical terms are normalized to deal with the spoken number problem. Most slot values could be relocated by our automatic value detection rules. The remainder slot values which vary too much to recognize are discarded along with their corresponding labels.

\paragraph{Speech Disfluency}\label{para:SD}
Disfluency is a common feature of spoken language. We follow the categorization of disfluency in previous works \cite{lickley1995missing,wang2020multi}: filled pauses, repeats, restarts, and repairs.

\begin{table}[h!tb]
    \small
    \begin{tabular}{@{~}c@{~}|@{~}p{6.2cm}@{~}}
    \hline
    Original & I want to go to Cambridge. \\
    \hline
    Pauses & I want to \textcolor{red}{um} go to \textcolor{red}{uh} Cambridge.\\
    Repeats & I\textcolor{red}{, I} want to go to\textcolor{red}{, go to}  Cambridge.\\
    Restarts & \textcolor{red}{I just} I want to go to Cambridge. \\
    Repairs & I want to go to \textcolor{red}{Liverpool, sorry I mean} Cambridge.  \\
    \hline
    \end{tabular}
    \caption{Example of four types of speech disfluency.}
    \label{tab:SD_types}
\end{table}

We present some examples of SD in Table \ref{tab:SD_types}.
Filler words (``um'', ``uh'') are injected into the sentence to present pauses.
Repeats are inserted by repeating the previous word.
In order to approximate the real distribution of disfluency, the \textit{interruption points} of filled pauses and repeats are predicted by a Bi-LSTM+CRF model \cite{zayats2016disfluency} trained on an annotated dataset SwitchBoard \cite{godfrey1992switchboard}, which was collected from real human talks.
For restarts, we insert \textit{false start terms} (``I just'') as a prefix of the utterance to simulate self-correction.
In LU task, we apply repairs on slot values to fool the models to predict wrong labels. We take the original slot value as \textit{Repair} (``Cambridge'') and take another value with the same slot name as \textit{Reparandum} (``Liverpool'').
An \textit{edit term} (``sorry, I mean'') is inserted between \textit{Repair} and \textit{Reparandum} to construct a correction. The filler words, restart terms, and edit terms and their occurrence frequency are all sampled from their distribution in SwitchBoard.

In order to keep the spans of slot values intact, each span is regarded as one whole word. No insertions are allowed to operate inside the span.
Therefore, SD augmentation do not change the original semantic and labels of the utterance, i.e. $y'=y$.

\section{Experimental Setup}
\label{sec:setup}

\begin{figure*}[!thb]
    \centering
    \begin{subfigure}{\linewidth}\centering
    \includegraphics[width=\linewidth]{CLS.pdf}
    \caption{Classification-based language understanding}\label{fig:cls}
    \end{subfigure}
    \begin{subfigure}{\linewidth}\centering
    \includegraphics[width=0.75\linewidth]{GEN.pdf}
    \caption{Generation-based language understanding}\label{fig:gen}
    \end{subfigure}
    \caption{An illustration of two categories of language understanding models. Dialog history is first encoded as conditions (not depicted here).}
    \label{fig:nlu}
\end{figure*}

\subsection{Data Preparation}
In our experiments we adopt Frames\footnote{As data division was not defined in Frames, we split the data into training/validation/test set with a ratio of 8:1:1.} \cite{el2017frames} and MultiWOZ \cite{budzianowski2018multiwoz}, which are two task-oriented dialog datasets where semantic labels of user utterances are annotated. In particular, MultiWOZ is one of the most challenging datasets due to its multi-domain setting and complex ontology, and we conduct our experiments on the latest annotation-enhanced version MultiWOZ 2.3 \cite{han2020multiwoz}, which provides cleaned annotations of user dialog acts (i.e. semantic labels).
The dialog act consists of four parts: domain, intent, slot names, and slot values. The statistics of two datasets are shown in Table \ref{tab:data_statistics}.
Following \citet{takanobu2020your}, we calculate overall F1 scores as evaluation metrics due to the multi-intent setting in LU.
\begin{table}[!thb]
    \small
    \centering
    \begin{tabular}{l|cc}
    \hline
    Datasets & Frames & MultiWOZ \\
    \hline
    \# Training Dialogs & 1,095 & 8,438\\
    \# Validation / Test Dialogs & 137 / 137 & 1,000 / 1,000\\

    \# Domains / \# Intents  & 2 / 12 & 7 / 5\\

    Avg. \# Turns per Dialog & 7.60 & 6.85 \\
    Avg. \# Tokens per Turn & 11.67 & 13.55 \\
    Avg. \# DAs per Turn & 1.87 & 1.66 \\
    \hline
    \end{tabular}
    \caption{Statistics of Frames and MultiWOZ 2.3. Only user turns $\mathcal{U}$ are counted here.}
    \label{tab:data_statistics}
\end{table}

The data are augmented with the inclusion of its copies, leading to a composite of all 4 augmentation types with equal proportion. Other setups are described in each experiment\footnote{See appendix for the hyperparameter setting of LAUG.}.

\begin{table}[h!tb]
    \centering
    \small
    \begin{tabular}{c|ccc|cc}
    \hline
        \multirow{2}{*}{Method}&
        \multicolumn{3}{c|}{Change Rate/\%}&\multicolumn{2}{c}{Human Annot./\%}\\\cline{2-4}\cline{5-6}
        & Char & Word & Slot &Utter.& DA\\
    \hline
        WP &17.9 &16.0 &36.3 &95.2 &97.0\\
        TP &60.3 &74.4 &13.3 &97.1 &97.7\\
        SR &7.9 &14.5 &40.8 &95.1 &96.7 \\
        SD &22.7 &30.4 &0.4 &98.8 &99.2 \\
    \hline
    \end{tabular}
    \caption{Statistics of augmented MultiWOZ data and their results of quality annotation. Automatic metrics include change rate of characters, words and slot values. Quality evaluation includes appropriateness at utterance level (Utter.) and at dialog act level (DA).
    \label{tab:aug_stat}
\end{table}

Table \ref{tab:aug_stat} shows the change rates in different aspects by comparing our augmented utterances with the original counterparts. We could find each augmentation method has a distinct effect on the data. For instance, TP rewrites the text without changing the original meaning, thus lexical and syntactic representations dramatically change, while most slot values remain unchanged. In contrast, SR makes the lowest change rate in characters and words but modifies the most slot values due to the speech misrecognition.

\subsection{Quality Evaluation}
To ensure the quality of our augmented test set, we conduct human annotation on 1,000 sampled utterances in each augmented test set of MultiWOZ. We ask annotators to check whether our augmented utterances are reasonable and our auto-detected value annotations are correct (two true-or-false questions). According to the feature of each augmentation method, different evaluation protocols are used. For TP and SD, annotators check whether the meaning of utterances and dialog acts are unchanged. For WP, changing slot values is allowed due to slot value replacement, but the slot name should be the same. For SR, annotators are asked to judge on the similarity of pronunciation rather than semantics. In summary, all the high scores in Table \ref{tab:aug_stat} demonstrate that LAUG makes reasonable augmented examples.

\begin{table*}[ht]
    \small
    \centering
    \begin{subtable}{\linewidth}\centering
    \begin{tabular}{c|c|c|cccc|c|cc}
    \hline
        Model &Train & Ori. & WP & TP & SR & SD & Avg.& Drop & Recov. \\
    \hline
        \multirow{2}{*}{MILU} & Original& 74.15 & 71.05 & 69.58 & 61.53 & 65.27 & 66.86 & -7.29 & /\\
        & Augmented& 75.78 & 72.49 & 71.96 & 64.76 & 70.92 & 70.03 & -5.75 & +3.17 \\
        \hline
        \multirow{2}{*}{BERT} & Original& 78.82 & 75.92 & 74.57 & 70.31 & 70.31 & 72.78 & -6.04 & /\\
        & Augmented& 78.21 & 76.70 & 75.63 & 72.04 & 77.34 & 75.43 & -2.78 & +2.65  \\
        \hline
        \multirow{2}{*}{ToD-BERT} & Original& \textbf{80.61} & 77.30 & 76.19 & 70.88 & 71.94 & 74.08 & -6.53 & /\\
        & Augmented& 80.37 & \textbf{77.32} & \textbf{77.26} & \textbf{72.54} & \textbf{79.04} & \textbf{76.54} & -3.83 & +2.46 \\
        \hline
        \multirow{2}{*}{CopyNet} & Original& 67.84 & 63.90 & 61.41 & 56.11 & 59.26  & 60.17 & -7.67 & /\\
        & Augmented& 69.35 & 67.10 & 65.90 & 60.98 & 67.71  & 65.42 & -3.93 & \textbf{+5.25} \\
        \hline
        \multirow{2}{*}{GPT-2} & Original& 78.78 & 74.96 & 72.85 & 69.00 & 69.19  & 71.50 & -7.28 & /\\
        & Augmented& 79.15 & 75.25 & 73.86 & 71.37 & 74.19 & 73.67 & -5.48 & +2.17\\
        \hline
    \end{tabular}
    \caption{Frames}
    \label{tab:frames}
    \end{subtable}
    \begin{subtable}{\linewidth}\centering
    \begin{tabular}{c|c|c|cccc|c|cc}
    \hline
        Model &Train & Ori. & WP & TP & SR & SD & Avg.& Drop & Recov. \\
    \hline
        \multirow{2}{*}{MILU} & Original& 91.33 &  88.26 & 87.20 & 77.98 & 83.67  & 84.28 & -7.05 & /\\
        & Augmented& 91.39 & 90.01 & 88.04 & 86.97 & 89.54 & 88.64 & -2.75 & +4.36 \\
        \hline
        \multirow{2}{*}{BERT} & Original & \textbf{93.40} & 90.96 & 88.51 & 82.35 & 85.98 & 86.95 & -6.45 &/\\
        & Augmented&  93.32 & 92.23 & 89.45 & 89.86 & 92.71 & 91.06 & -2.26 & +4.11 \\
        \hline
        \multirow{2}{*}{ToD-BERT} & Original & 93.28 & 91.27 & 88.95 & 81.16 & 87.18 & 87.14 & -6.14 &/\\
        & Augmented& 93.29 & \textbf{92.40} & 89.71 & \textbf{90.06} & \textbf{92.85} & \textbf{91.26} & -2.03 & +4.12 \\
        \hline
        \multirow{2}{*}{CopyNet} & Original & 90.97 & 85.25 & 87.40 & 71.06 & 77.66 & 80.34 & -10.63 &/\\
        & Augmented& 90.49 & 89.19 & 89.53 & 85.69 & 89.83 & 88.56 & -1.93 &\textbf{+8.22}\\
        \hline
        \multirow{2}{*}{GPT-2} & Original & 91.53 & 85.35 & 88.23 & 80.74 & 84.33 & 84.66 & -6.87 &/ \\
        & Augmented& 91.59 & 90.26 & \textbf{89.92} & 86.55 & 90.55 & 89.32 & -2.27 & +4.66\\
        \hline
    \end{tabular}
    \caption{MultiWOZ}
    \label{tab:multiwoz}
    \end{subtable}
    \caption{Robustness test results. Ori. stands for the original test set, WP, TP, SR, SD for 4 augmented test sets and Avg. for the average performance on 4 augmented test sets. The additional data in augmented training set has the same utterance amount as the original training set and is composed of 4 types of augmented data with equal proportion. Drop shows the performance decline between Avg. and Ori. while Recov. denotes the performance recovery of Avg. between training on augmented/original data (e.g., 88.64\%-84.28\% for MILU on MultiWOZ).}
    \label{tab:main}
\end{table*}

\subsection{Baselines}\label{sec:baseline}

LU models roughly fall into two categories: classification-based and generation-based models.
Classification based models \cite{hakkani2016multi,goo2018slot} extract semantics by intent detection and slot tagging. Intent detection is commonly regarded as a multi-label classification task, and slot tagging is often treated as a sequence labeling task with \textit{BIO format} \cite{ramshaw1999text}, as shown in Fig. \ref{fig:cls}. Generation-based models \cite{liu2016attention,zhao2018improving} generate a dialog act containing intent and slot values. They treat LU as a sequence-to-sequence
problem and transform a dialog act into a sequential structure as shown in Fig. \ref{fig:gen}.
Five base models with different categories are used in the experiments, as shown in Table \ref{tab:base_model}.

\begin{table}[h!tb]
    \centering
    \small
    \begin{tabular}{l@{}|ccc}
    \hline
        Model & Cls. & Gen. & PLM \\
    \hline
        MILU \cite{hakkani2016multi} & $\surd$ & &\\
        BERT \cite{devlin2019bert} & $\surd$ & & $\surd$ \\
        ToD-BERT \cite{wu2020tod} & $\surd$ &  & $\surd$ \\
        CopyNet \cite{gu2016incorporating} &  & $\surd$ & \\
        GPT-2 \cite{radford2019language} &  & $\surd$ & $\surd$\\
    \hline
    \end{tabular}
    \caption{Features of base models. Cls./Gen. denotes classification/generation-based models. PLM stands for pre-trained language models.}
    \label{tab:base_model}
\end{table}

To support a multi-intent setting in classification-based models, we decouple the LU process as follows: first perform domain classification and intent detection, then concatenate two special tokens which indicate the detected domain and intent (e.g. $[restaurant] [inform]$) at the beginning of the input sequence, and last encode the new sequence to predict slot tags. In this way, the model can address \textit{overlapping slot values} when values are shared in different dialog acts.

\begin{figure*}[h!tb]
    \centering
    \begin{subfigure}{0.5\linewidth}\centering
    \includegraphics[width=\linewidth]{BERT_new.pdf}
    \caption{BERT}
    \end{subfigure}%
    \begin{subfigure}{0.5\linewidth}\centering
    \includegraphics[width=\linewidth]{GPT2_new.pdf}
    \caption{GPT-2}
    \end{subfigure}
    \caption{Performance on MultiWOZ with different ratios of augmented training data amount to the original one. The total amount of training data varies but they are always composed of 4 types of augmented data with even proportion. Different test sets are shown with different colored lines.

    }
    \label{fig:augment}
\end{figure*}

\section{Evaluation Results}\label{sec:result}

\subsection{Main Results}

We conduct robustness testing on all three capacities for five base models using four augmentation methods in LAUG. All baselines are first trained on the original datasets, then finetuned on the augmented datasets. Overall F1-measure performance on Frames and MultiWOZ is shown in Table \ref{tab:main}. All experiments are conducted over 5 runs, and averaged results are reported.

Robustness for each capacity can be measured by performance drops on the corresponding augmented test sets. All models achieve some performance recovery on augmented test sets after trained on the augmented data, while keeping a comparable result on the original test set. This indicates the effectiveness of LAUG in improving the model's robustness.

We observe that pre-trained models outperform non-pre-trained ones on both original and augmented test sets.
Classification-based models have better performance and are more robust than generation-based models.
ToD-BERT, the state-of-the-art model which was further pre-trained on task-oriented dialog data, has comparable performance with BERT. With most augmentation methods, ToD-BERT shows slightly better robustness than BERT.

Since the data volume of Frames is far less than that of MultiWOZ, the performance improvement of pre-trained models on Frames is larger than that on MultiWOZ. Due to the same reason, augmented training data benefits the non-pre-trained models performance of  on Ori. test set more remarkably in Frames where data is not sufficient.

Among the four augmentation methods, SR has the largest impact on the models' performance, and SD comes the second. The dramatic performance drop when testing on SR and SD data indicates that robustness for speech characteristics may be the most challenging issue

Fig. \ref{fig:augment} shows how the performance of BERT and GPT-2 changes on MultiWOZ when the ratio of augmented training data to the original data varies from 0.1 to 4.0. F1 scores on augmented test sets increase when there are more augmented data for training. The performance of BERT on augmented test sets is improved when augmentation ratio is less than 0.5 but becomes almost unchanged after 0.5 while GPT-2 keeps increasing stably. This result shows the different characteristics between classification-based models and generation-based models when finetuned with augmented data.

\subsection{Ablation Study}

\paragraph{Between augmentation approaches}
In order to study the influence of each augmentation approach in LAUG, we test the performance changes when one augmentation approach is removed from constructing augmented training data. Results on MultiWOZ are shown in Table \ref{tab:between}.

\begin{table}[!th]
    \centering
    \small

    \begin{subtable}{\linewidth}\centering
    \begin{tabular}{c|>{\centering}p{0.65cm}|>{\centering}p{0.65cm}>{\centering}p{0.65cm}>{\centering}p{0.65cm}>{\centering}p{0.65cm}|c}
    \hline
        Train & Ori. & WP & TP & SR & SD & Avg. \\
    \hline
        Aug.& 91.39 & 90.01 & 88.04 & 86.97 & 89.54 & 88.64\\
    \hline
        -WP & 91.29 & \textbf{88.42} & 88.43 & 86.98 & 89.20 & 88.26 \\
        -TP & 91.55 & 90.15 & \textbf{87.81} & 86.82 & 89.42 & 88.55\\
        -SR & 91.23 & 90.13 & 88.30 & \textbf{77.90} & 89.51 & 86.46\\
        -SD & 91.56 & 90.24 & 88.60 & 86.78 & \textbf{83.96} & 87.40\\
    \hline
        Ori.& 91.33 &  88.26 & 87.20 & 77.98 & 83.67  & 84.28 \\
    \hline

    \end{tabular}
    \caption{MILU}
    \label{tab:between_MILU}
    \end{subtable}
    \begin{subtable}{\linewidth}\centering
    \begin{tabular}{c|>{\centering}p{0.65cm}|>{\centering}p{0.65cm}>{\centering}p{0.65cm}>{\centering}p{0.65cm}>{\centering}p{0.65cm}|c}
    \hline
        Train & Ori. & WP & TP & SR & SD & Avg. \\
    \hline
        Aug. &  93.32 & 92.23 & 89.45 & 89.86 & 92.71 & 91.06 \\
    \hline
        -WP & 93.23 & \textbf{90.94} & 89.42 & 89.93 & 92.82 & 90.78\\
        -TP & 93.08 & 92.24 & \textbf{88.62} & 89.80 & 92.62 & 90.82\\
        -SR & 93.43 & 92.30 & 89.50 & \textbf{83.48} & 93.07 & 89.59\\
        -SD & 93.11 & 92.15 & 89.44 & 90.00 & \textbf{85.22} & 89.20\\
    \hline
        Ori. & 93.40 & 90.96 & 88.51 & 82.35 & 85.98 & 86.95 \\
    \hline
    \end{tabular}
    \caption{BERT}
    \label{tab:between_BERT}
    \end{subtable}

    \caption{Ablation study between augmentation approaches for two models on MultiWOZ. Highlighted numbers denote the most sharp decline for each augmented test set.}
    \label{tab:between}
\end{table}

Large performance decline on each augmented test set is observed when the corresponding augmentation approach is removed in constructing training data. The performance after removing an augmentation method is comparable to the one without augmented training data. Only slight changes are observed without other approaches. These results indicate that our four augmentation approaches are relatively orthogonal.

\paragraph{Within augmentation approach}
Our implementation of WP and SD consist of several functional components. Ablation experiments here show how much performance is affected by each component in augmented test sets.

\begin{table}[!th]
    \centering
    \small
    \begin{subtable}{\linewidth}\centering
    \begin{tabular}{p{1.05cm}|p{0.6cm}p{0.75cm}|p{0.6cm}p{0.75cm}
    \hline
       Test & MILU & Diff. & BERT & Diff.\\% & Copy & Diff. \\
    \hline

        WP & 88.26 & / & 90.96 & /\\%& 85.25 & / \\
    \hline
        -Syno. & 88.90 & 0.64 & 91.27 & 0.48 \\%& 85.62 & 0.37 \\
        -Insert & 88.90 & 0.64 & 91.30 & 0.51 \\%& 85.87 & 0.62 \\
        -Delete & 88.97 & 0.71 & 91.20 & 0.41 \\%& 85.53 & 0.28 \\
        -Swap & 89.15 & 0.89 & 91.33 & \textbf{0.54} \\%& 85.80 & 0.55 \\
        -Slot & 89.45 & \textbf{1.19} & 91.30 & 0.51 \\%& 86.41 & \textbf{1.16}\\
    \hline
        Ori. & 91.33 & 3.05 & 93.40 & 2.61 \\%& 90.97 & 5.72\\
    \hline
    \end{tabular}
    \caption{Word Perturbation}
    \label{tab:eda_ablation}
    \end{subtable}
    \begin{subtable}{\linewidth}\centering
    \begin{tabular}{p{1.13cm}|p{0.6cm}p{0.75cm}|p{0.6cm}p{0.75cm}
    \hline
        Test & MILU & Diff. & BERT & Diff. \\%& Copy & Diff. \\
    \hline
        SD & 83.67 & / & 85.98 & / \\%& 65.42 & /\\
    \hline
        -Repair& 89.47 & \textbf{5.80} & 91.05 & \textbf{5.07}\\%& \textbf{12.67} &81.10 & \textbf{15.68}\\
        -Pause& 85.21 & 1.54 & 88.06 & 2.08 \\%& 71.10&5.68\\
        -Restart& 84.03 & 0.36 & 86.22 & 0.24 \\%&66.14 &0.72\\
        -Repeat& 83.64 & -0.03 & 85.68 & -0.30 \\%&65.32 &-0.10\\
    \hline
        Ori. & 91.33 & 7.66 &93.40 & 7.42\\%& 65.52&25.55 \\
    \hline
    \end{tabular}
    \caption{Speech Disfluency}
    \label{tab:disf_ablation}
    \end{subtable}
    \caption{Ablation study within two augmentation approaches. Models are trained on original training set. Highlight stands for the component with the most influence on model performance.}
    \label{tab:within}
\end{table}

Original EDA consists of four functions as described in Table \ref{tab:todeda}. Performance differences (Diff.) can reflect the influences of those components in Table \ref{tab:eda_ablation}. The additional function of our SC-EDA is slot value replacement. We can also observe an increase in performance when it is removed, especially for MILU. This implies a lack of LU robustness in detecting unseen entities.

Table \ref{tab:disf_ablation} shows the results of ablation study on SD. Among the four types of disfluencies described in Table \ref{tab:SD_types}, \textit{repairs} has the largest impact on models' performance. The performance is also affected by \textit{pauses} but to a less extent. The influences of \textit{repeats} and \textit{restarts} are small, which indicates that neural models are robust to handle these two problems.

\subsection{User Evaluation}
In order to test whether the data automatically augmented by LAUG can reflect and alleviate practical robustness problems, we conduct a real user evaluation.
We collected 240 speech utterances from real humans as follows: First, we sampled 120 combinations of DA from the test set of MultiWOZ. Given a combination, each user was asked to speak two utterances with different expressions, in their own language habits.
Then the audio signals were recognized into text using DeepSpeech2, thereby constructing a new test set in real scenarios\footnote{See appendix for details on real data collection.}. Results on this real test set are shown in Table~\ref{tab:user}.

\begin{table}[!htb]
    \centering
    \small
    \begin{tabular}{c|c|ccc}
    \hline
        Model & Train & Ori. & Avg. & Real \\
    \hline
        \multirow{2}{*}{MILU} & Original &91.33 & 84.28 & 63.55 \\
         & Augmented &91.39 & 88.64 & 66.77 \\
         \hline
         \multirow{2}{*}{BERT} & Original &93.40 & 86.95 & 65.22 \\
         & Augmented &93.32 & 91.06 & 69.12 \\
    \hline
    \end{tabular}
    \caption{User evaluation results on MultiWOZ. Ori. and Avg. have the same meaning as the ones in Table \ref{tab:main}, and Real is the real user evaluation set.}
    \label{tab:user}
\end{table}

The performance on the real test set is substantially lower than that on Ori. and Avg.,
indicating that real user evaluation is much more challenging. This is because multiple robustness issues may be included in one real case, while each augmentation method in LAUG evaluates them separately. Despite the difference, model performance on the real data is remarkably improved after every model is finetuned on the augmented data, verifying that LAUG effectively enhances the model's real-world robustness.

\subsection{Error Analysis}

\begin{table}[!thb]
    \centering
    \small
    \begin{tabular}{c|cc|cc}
    \hline
       \multirow{2}{*}{Error Type} & \multicolumn{2}{c|}{BERT Ori.} & \multicolumn{2}{c}{BERT Aug.}\\
       & Num & \% & Num & \% \\
    \hline
       Language Variety &21 & 43.8 &20 & 45.5 \\
       Speech Characteristics &14 & 29.2 &11 & 25.0 \\
       Noise Perturbation & 12& 25.0 &10 & 22.7 \\
       Others &14 & 29.2 &14 & 31.8 \\
    \hline
       Multiple Issues &12 & 25.0 & 11 & 25.0 \\
    \hline
    \end{tabular}
    \caption{Error analysis of BERT in user evaluation.}
    \label{tab:user_error}
\end{table}

Table \ref{tab:user_error} investigates which error type the model has made on the real test set by manually checking all the error outputs of BERT Ori. ``Others'' are the error cases which are not caused by robustness issues, for example, because of the model's poor performance. It can be observed that the model seriously suffers to LU robustness (over 70\%), and that almost half of the error is due to Language Variety. We find that this is because there are more diverse expressions in real user evaluation than in the original data. After augmented training, we can observe that the number of error cases of Speech Characteristics and Noise Perturbation is relatively decreased. This shows that BERT Aug. can solve these two kinds of problems better.
Noting that the sum of four percentages is over 100\% since 25\% error cases involve multiple robustness issues. This again demonstrates that real user evaluation is more challenging than the original test set\footnote{See appendix for case study.}.

\section{Related Work}
\label{sec:prior}

Robustness in LU has always been a challenge in task-oriented dialog.
Several studies have investigated the model's sensitivity to the collected data distribution, in order to prevent models from overfitting to the training data and improve robustness in the real world.
\citet{kang2018data} collected dialogs with templates and paraphrased with crowd-sourcing to achieve high coverage and diversity in training data.
\citet{dinan2019build} proposed a training schema that involves human in the loop in dialog systems to enhance the model's defense against human attack in an iterative way.
\citet{ganhotra2020effects} injected natural perturbation into the dialog history manually to refine over-controlled data generated through crowd-sourcing.
All these methods require laborious human intervention. This paper aims to provide an automatic way to test the LU robustness in task-oriented dialog.

Various textual adversarial attacks \cite{zhang2020adversarial} have been proposed and received increasing attentions these years to measure the robustness of a victim model. Most attack methods perform white-box attacks \cite{papernot2016crafting,li2019textbugger,ebrahimi2018hotflip} based on the model's internal structure or gradient signals. Even some black-box attack models are not purely ``black-box'', which require the prediction scores (classification probabilities) of the victim model \cite{jin2020bert,ren2019generating,alzantot2018generating}. However, all these methods address random perturbation but do not consider linguistic phenomena to evaluate the real-life generalization of LU models.

While data augmentation can be an efficient method to address data sparsity, it can improve the generalization abilities and measure the model robustness as well \cite{eshghi2017bootstrapping}.
Paraphrasing that rewrites the utterances in dialog has been used to get diverse representation and thus enhancing robustness \cite{ray2018robust,zhao2019data,iyyer2018adversarial}.
Word-level operations \cite{kolomiyets2011model,li2020textat,wei2019eda} including replacement, insertion, and deletion were also proposed to increase language variety.
Other studies \cite{shah2019robust,xu2014targeted} worked on the out-of-vocabulary problem when facing unseen user expression. Some other research focused on building robust spoken language understanding \cite{zhu2018robust,henderson2012discriminative,huang2019adapting} from audio signals beyond text transcripts. Simulating ASR errors \cite{schatzmann2007error,park2019specaugment,wang2020data} and speaker disfluency \cite{wang2020multi,qader2018disfluency} can be promising solutions to enhance robustness to voice input when only textual data are provided. As most work tackles LU robustness from only one perspective, we present a comprehensive study to reveal three critical issues in this paper, and shed light on a thorough robustness evaluation of LU in dialog systems.

\section{Conclusion and Discussion}
\label{sec:concl}
In this paper, we present a systematic robustness evaluation of language understanding (LU) in task-oriented dialog from three aspects: \textit{language variety}, \textit{speech characteristics}, and \textit{noise perturbation}. Accordingly, we develop four data augmentation methods to approximate these language phenomena. In-depth experiments and analysis are conducted on MultiWOZ and Frames, with both classification- and generation-based LU models. The performance drop of all models on augmented test data indicates that these robustness issues are challenging and critical, while pre-trained models are relatively more robust to LU.
Ablation studies are carried out to show the effect and orthogonality of each augmentation approach.
We also conduct a real user evaluation and verifies that our augmentation methods can reflect and help alleviate real robustness problems.

Existing and future dialog models can be evaluated in terms of robustness with our toolkit and data, as our augmentation model does not depend on any particular LU models. Moreover, our proposed robustness evaluation scheme is extensible. In addition to the four approaches in LAUG, more methods to evaluate LU robustness can be considered in the future.

\section*{Acknowledgments}
This work was partly supported by the NSFC projects (Key project with No.~61936010 and regular project with No.~61876096).
This work was also supported by the Guoqiang Institute of Tsinghua University, with Grant No.~2019GQG1 and 2020GQG0005.
We would like to thank colleagues from HUAWEI for their constant support and valuable discussion.

redpajama_set_name: RedPajamaArXiv


Bathtub Trap was published at January 5, 2019 at 5:04 pm. It is posted in the Bathtub category. Bathtub Trap is tagged with Bathtub Trap, Bathtub, Trap..
Bathtub Trap performs activities especially for office personnel who conduct work action at work. The office chair is not just as an easy method of fulfilling the requirements that really must be owned by any business / business entity involved for the reason that they are doing. On the basis of the functionality or usability chair comes with in deciding the graphic of the person in the placement and functionality of each, an important position, for instance of a chair for your director, ofcourse, should be used as director to his situation.
It's impossible right, chairs for team / personnel receive the BIG BOS. Besides a level with staff that is other later, additionally, it provides the impression that is not good for his control, what he explained later. A reprimand and sometimes even dismissal might be strike by us. Why must modified with Bathtub Trap in line with the situation or function? It is necessary in management to make it seem professional and also have specialist.
Independent of the features or desires an office chair likewise typically coordinated with the color of workplace interiors and also tastes personnel as well as a color which can be field your determination to work. Do not underestimate pick an office that is comfortable seats since you will find relaxed workplace chair could make you forget the amount of time in the work and your work's results additionally helps maximum in his function.


text:
Exclusive: Vulcan Productions, David Petersen give a "Lift" to dance doc
A dance program for underprivileged children operating out of the New York Theater Ballet is the focus of a documentary now in production from Vulcan Productions (Body Team 12, Ghost ...
By Barry Walsh
A dance program for underprivileged children operating out of the New York Theater Ballet is the focus of a documentary now in production from Vulcan Productions (Body Team 12, Ghost Fleet, The Ivory Game) and director David Petersen (Let the Church Say Amen) and his Beaufort 9 Films imprint.
Lift (working title) follows the dance program of the same name, presided over by artistic director Diana Byer and with American classical dancer Steven Melendez (pictured) acting as a mentor for several students. The film has been shot over the course of several years and follows various dancers as they endeavor to both hone their talents and contribute to their communities.
The Lift program was founded 30 years ago out of the New York Theater Ballet, and enrolls up to 30 disadvantaged children each year, providing partial or full scholarship assistance and other support. For 26 of those years, Byer faced uncertainty regarding how to keep the program afloat while also running her own ballet company, and, according to the producers, sacrificed her own financial security to ensure that the program could continue.
An upcoming shoot for the film will feature the performance of a new work choreographed by Melendez, Rapidly and Freely Without Control, which will be performed by soloist Victor Rosario, a recent addition to the New York City ballet's company and a prominent character in the doc.
"For the last eight years, this film has been a true labor of love, because we have had the chance to witness, over time, the astonishing development of each student through Steven's mentorship, and the transformative power of making this aristocratic art form their own by contributing gestures and movement to a dance that tells their story of stepping from the streets to the New York stage," Petersen said in a statement, adding: "In capturing this remarkable journey, the film shines a spotlight on the invisible story of underserved children in America."
"The influence and power of the Lift program is evident and unique," said Ruth Johnston, general manager for Vulcan Productions. "We are honored to work with the Lift team, and can't wait to share the story of this program, its people and their inspiring journeys with the world."
Exec producers on the project include Jody Allen, Ruth Johnston, Jannat Gargi and Rocky Collins for Vulcan Productions. The doc is also supported by FordFoundation | JustFilms and Catapult Film Fund.
Photo credit: Richard Termine
Beaufort 9 Films, David Petersen, Jannat Gargi, Jody Allen, Lift, Rocky Collins, Ruth Johnston, Vulcan Productions
Realscreen opens call for MIPTV Picks submissions
By Realscreen Staff
SXSW unveils 2020 documentary lineup
TCA: HBO orders Ronan Farrow doc, reveals doc slate for first half of 2020
Website // Author Page
Exclusive clip: Steven Cantor's "Ballet Now" for Hulu
Extra: Nat Geo links with Drive; Killer Tracks launches unscripted TV music label
TIFF '18 Exclusive doc clip: "Ghost Fleet"
PBS preps deep sea investigation "USS Indianapolis Live"
The Garden hires C4′s Nicola Brown, ups Ninder Billing
BBC preps "Rio Ferdinand: Being Mum And Dad" step family follow up
Zinc Media Group raises $4.6 million to accelerate UK, U.S. growth
Netflix opens French HQ, increases investment in country

redpajama_set_name: RedPajamaCommonCrawl


text:
NAB Show to Feature Prominent Legislators, Regulators and Staff
Rep. Greg Walden, Chairman of Influential House Subcommittee, to Make Remarks at Opening Session
WASHINGTON, D.C. -- The National Association of Broadcasters announced today that top government officials will address public policy issues related to media, entertainment and technology at NAB Show, to be held April 6-11, 2013 in Las Vegas. U.S. Representative Greg Walden, chairman of the Communications and Technology Subcommittee of the House Committee on Energy and Commerce, will discuss his career in broadcasting and legislative issues affecting the industry as one of the speakers at the NAB Show opening session on Monday, April 8.
Walden has represented Oregon's Second Congressional District since 1998. He also spent more than two decades as a radio station owner and is a licensed amateur radio operator (W7EQI). He uses his small business and technology experience as chairman of the Energy and Commerce Committee's Subcommittee on Communications and Technology. In November 2012, Walden's House Republican colleagues unanimously elected him to serve as chairman of the National Republican Congressional Committee (NRCC).
"NAB Show is a great opportunity to hear directly from government officials, regulators and policymakers, and to get up to speed on important public policy issues facing the industry today," said Dennis Wharton, NAB executive vice president of Communications. "You cannot underestimate the implications of legislative and regulatory initiatives in Washington."
Session topics include FCC implementation of television spectrum incentive auctions, retransmission consent, media ownership rules and content streaming regulations.
Sessions featuring government officials and/or focusing on telecommunications policy include (click on session titles for additional information):
Is '13' A Broadcaster's Lucky Number? The Government's Perspective, Monday, April 8, 10:30 a.m. This panel will feature John Branscome, Senate Commerce Committee Communications Counsel, and Neil Fried, House Energy and Commerce Committee Chief Counsel, among others.
Incentive Auctions of TV Spectrum � How and When They Will Impact Your Business, Monday, April 8, 2:30 p.m.
AM Band Revitalization Featuring FCC Commissioner Ajit V. Pai, Monday, April 8, 3:00 p.m.
Straight Talk from the Top � America's Communication Regulators, Tuesday, April 9, 10:30 a.m.
The Radio Regulatory Revue, Tuesday, April 9, 2:30 p.m.
Q&A with the Federal Communications Commission Chairman, Wednesday, April 10, 9:00 a.m.
EAS, Superstorm Sandy, and Accessible Emergency Warnings � A Discussion of Broadcasters' Role as "First Informers," Wednesday, April 10, 10:30 a.m.
Keep Calm & Carry On: Viewability, Retransmission Consent & Other Signal Carriage Issues, Wednesday, April 10, 10:30 a.m.
NAB Show, held April 6-11 in Las Vegas, is the world's largest electronic media show covering filmed entertainment and the development, management and delivery of content across all mediums. With more than 90,000 attendees from 151 countries and 1,600+ exhibitors, NAB Show is the ultimate marketplace for digital media and entertainment. From creation to consumption, across multiple platforms and countless nationalities, NAB Show is home to the solutions that transcend traditional broadcasting and embrace content delivery to new screens in new ways. Complete details are available at www.nabshow.com.
Click here to register as news media.


text:
Record Number of Guns Seized at Louisiana Airports
Erin McCarty Published: February 7, 2022
Shreveport Airport
I'm not sure if it's forgetfulness, or just ignorance, but more people tried to get guns through security at Louisiana airports in 2021 than in any previous year. The feds report the number of guns caught at Transportation Security Administration (TSA) checkpoints reached record levels across Louisiana in 2021.
In fact, the numbers show the New Orleans Airport (MSY) coming in ninth in the nation. 119 guns were stopped at the checkpoint at the Louis Armstrong Airport.
TSA Spokesperson Sari Koshetz says
This dangerous violation of federal regulations could mean a trip to jail instead of a vacation. And an accidental discharge in the checkpoint could have tragic results.
Shreveport makes the list too. Agents caught 17 guns at the checkpoint at Shreveport Regional Airport last year. 7 guns were caught in Lafayette during 2021, 6 were stopped in Baton Rouge, 5 at Alexandria and 4 each in Monroe and Lake Charles.
During 2021 in Louisiana 162 passengers brought guns to the TSA checkpoints compared to 85 in 2020 and 91 in 2019.
If you are stopped with a gun at the TSA checkpoints, you could face a fine of close to $14,000. That penalty can be implemented whether or not you get arrested.
Which Airports Seized the Most Guns Last Year?
Hartsfield-Jackson Atlanta International Airport (ATL) 507
Dallas/Fort Worth International Airport (DFW) 317
Houston's George Bush Intercontinental Airport (IAH) 245
Phoenix Sky Harbor International Airport (PHX) 196
The TSA says if you must bring a gun on a flight, it must be in checked luggage and you have to declare it when you check in. You are also instructed to unload the gun and have it packed in a locked and hard-sided piece of luggage.
You also have to know the gun laws at your destination.
You can also find out more about what is allowed on the "What Can I Bring?" page on TSA.gov.
10 Worst Performing Airports in Shreveport Area for On-Time Arrivals
Several airports in the Shreveport Bossier region are on the list of airports with the worst on-time record
Top 10 Performing Airports for On-Time Arrivals in Louisiana and Texas
Several airports in the Arklatex are among the best performing in Louisiana and Texas.
Top 10 Best Beaches Near Shreveport Bossier


Butcher Halloween Costume Your Complete Online Halloween Store for Butcher Halloween Costume - TOP Halloween deals, coupon codes, discount offers, and sales.
Need a little inspiration for your Halloween costume this year? Try this Popular Products: Pink Skeleton Gloves, Vampiro Halloween Hombre, Amazing Male Halloween Makeup, Halloween Disney Decorations, Kids Transformers Costume - compare prices and find the best Halloween Deals for shopping online.


text:
Mineral processing and extraction of metals are very energy-intensive processes, which are not exempted of producing large volumes of solid residues and wastewater, which also require energy to be further treated and disposed. Moreover, as the demand for metals increases, the metallurgical industry must rely on sources of materials with lower metal contents both from a primary (e.g., mineral ores) and/or secondary (e.g., slags, tailings, municipal waste) raw materials. Consequently, mining activities and waste recycling must evolve towards the development of more selective, efficient and environmentally friendly mineral and metal processing routes.

Mineral processing operations are needed firstly to concentrate the mineral phases of interest and reject the unwanted material physical or chemically associated to a defined raw material. The process, however, demand about 30 GJ/tonne of metal, which accounts about 29% of the total energy spent on mining in the USA. Meanwhile, pyrometallurgy is a significant producer of greenhouse gas emissions and harmful flue dust. Hydrometallurgy entails the consumption of large volumes of lixiviants such as H2SO4, HCl, KCN, NaCN which have poor selectivity. Moreover, despite the environmental concern and the use restriction imposed by some countries, cyanidation is still considered the prime process technology to recover gold from ores. Mercury is also used by artisanal miners in less economically developed countries to concentrate gold and silver from minerals, despite its obvious toxicity. Bio-hydro-metallurgy make use of living organisms, such as bacteria and fungi, and although this method demands only the input of  and  from the atmosphere, it requires low solid-to-liquid ratios and long contact times, which significantly reduces space-time yields.

Ionometallurgy makes use of non-aqueous ionic solvents such ionic liquids (ILs) and deep eutectic solvents (DESs), which allows the development of closed-loop flow sheet to effectively recover metals by, for instance, integrating the metallurgical unit operations of leaching and electrowinning. It allows to process metals at moderate temperatures in a non-aqueous environment which allows controlling metal speciation, tolerates impurities and at the same time exhibits suitable solubilities and current efficiencies. This simplify conventional processing routes and allows a substantial reduction in the size of a metal processing plant.

Metal extraction with ionic fluids
DESs are fluids generally composed of two or three cheap and safe components that are capable of self-association, often through hydrogen bond interactions, to form eutectic mixtures with a melting point lower than that of each individual component. DESs are generally liquid at temperatures lower than 100 °C, and they exhibit similar physico-chemical properties to traditional ILs, while being much cheaper and environmentally friendlier. Most of them are mixtures of choline chloride and a hydrogen-bond donor (e.g., urea, ethylene glycol, malonic acid) or mixtures of choline chloride with a hydrated metal salt. Other choline salts (e.g. acetate, citrate, nitrate) have a much higher costs or need to be synthesised, and the DES formulated from these anions are typically much more viscous and can have higher conductivities than for choline chloride. This results in lower plating rates and poorer throwing power and for this reason chloride-based DES systems are still favoured. For instance, Reline (a 1:2 mixture of choline chloride and urea) has been used to selectively recover Zn and Pb from a mixed metal oxide matrix. Similarly, Ethaline (a 1: 2 mixture of choline chloride and ethylene glycol) facilitates metal dissolution in electropolishing of steels. DESs have also demonstrated promising results to recover metals from complex mixtures such Cu/Zn and Ga/As, and precious metals from minerals. It has also been demonstrated that metals can be recovered from complex mixtures by electrocatalysis using a combination of DESs as lixiviants and an oxidising agent, while metal ions can be simultaneously separated from the solution by electrowinning.

Recovery of precious metals by ionometallurgy
Precious metals are rare, naturally occurring metallic chemical elements of high economic value. Chemically, the precious metals tend to be less reactive than most elements. They include gold and silver, but also the so-called platinum group metals: ruthenium, rhodium, palladium, osmium, iridium, and platinum (see precious metals). Extraction of these metals from their corresponding hosting minerals would typically require pyrometallurgy (e.g., roasting), hydrometallurgy (cyanidation), or both as processing routes.
Early studies have demonstrated that gold dissolution rate in Ethaline compares very favourably to the cyanidation method, which is further enhanced by the addition of iodine as an oxidising agent. In an industrial process the iodine has the potential to be employed as an electrocatalyst, whereby it is continuously recovered in situ from the reduced iodide by electrochemical oxidation at the anode of an electrochemical cell. Dissolved metals can be selectively deposited at the cathode by adjusting the electrode potential. The method also allows better selectivity as part of the gangue (e.g., pyrite) tend to be dissolved more slowly.

Sperrylite (PtAs2) and moncheite (PtTe2), which are typically the more abundant platinum minerals in many orthomagmatic deposits, do not react under the same conditions in Ethaline because they are disulphide (pyrite), diarsenide (sperrylite) or ditellurides (calaverite and moncheite) minerals, which are particularly resistant to iodine oxidation. The reaction mechanism by which dissolution of platinum minerals is taking place is still under investigation.

Metal recovery from sulfide minerals with ionometallurgy
Metal sulfides (e.g., pyrite FeS2, arsenopyrite FeAsS, chalcopyrite CuFeS2) are normally processed by chemical oxidation either in aqueous media or at high temperatures. In fact, most base metals, e.g., aluminium, chromium, must be (electro)chemically reduced at high temperatures by which the process entails a high energy demand, and sometimes large volumes of aqueous waste is generated. In aqueous media chalcopyrite, for instance, is more difficult to dissolve chemically than covellite and chalcocite due to surface effects (formation of polysulfide species,). The presence of Cl− ions has been suggested to alter the morphology of any sulfide surface formed, allowing the sulfide mineral to leach more easily by preventing passivation. DESs provide a high Cl− ion concentration and low water content, whilst reducing the need for either high additional salt or acid concentrations, circumventing most oxide chemistry. Thus, the electrodissolution of sulfide minerals has demonstrated promising results in DES media in absence of passivation layers, with the release into the solution of metal ions which could be recovered from solution.

During extraction of copper from copper sulfide minerals with Ethaline, chalcocite (Cu2S) and covellite (CuS) produce a yellow solution, indicating that [CuCl4]2− complex are formed. Meanwhile, in the solution formed from chalcopyrite, Cu2+ and Cu+ species co-exist in solution due to the generation of reducing Fe2+ species at the cathode. The best selective recovery of copper (>97 %) from chalcopyrite can be obtained with a mixed DES of 20 wt.% ChCl-oxalic acid and 80 wt.% Ethaline.

Metal recovery from oxide compounds with Ionometallurgy
Recovery of metals from oxide matrixes is generally carried out using mineral acids. However, electrochemical dissolution of metal oxides in DES can allow to enhance the dissolution up to more than 10 000 times in pH neutral solutions.

Studies have shown that ionic oxides such as ZnO tend to have high solubility in ChCl:malonic acid, ChCl:urea and Ethaline, which can resemble the solubilities in aqueous acidic solutions, e.g., HCl. Covalent oxides such as TiO2, however, exhibits almost no solubility. The electrochemical dissolution of metal oxides is strongly dependent on the proton activity from the HBD, i.e. capability of the protons to act as oxygen acceptors, and on the temperature. It has been reported that eutectic ionic fluids of lower pH-values, such as ChCl:oxalic acid and ChCl:lactic acid, allow a better solubility than that of higher pH (e.g., ChCl:acetic acid). Hence, different solubilities can be obtained by using, for instance, different carboxylic acids as HBD.

Outlook
Currently, the stability of most ionic liquids under practical electrochemical conditions is unknown, and the fundamental choice of ionic fluid is still empirical as there is almost no data on metal ion thermodynamics to feed into solubility and speciation models. Also, there are no Pourbaix diagrams available, no standard redox potentials, and bare knowledge of speciation or pH-values. It must be noticed that most processes reported in the literature involving ionic fluids have a Technology Readiness Level (TRL) 3 (experimental proof-of-concept) or 4 (technology validated in the lab), which is a disadvantage for short-term implementation. However, ionometallurgy has the potential to effectively recover metals in a more selective and sustainable way, as it considers environmentally benign solvents, reduction of greenhouse gas emissions and avoidance of corrosive and harmful reagents.

References

Metallurgy

redpajama_set_name: RedPajamaWikipedia


Inspired to hit the road?
Lose all track of time (and cell signal) along California's Redwood Coast. Movie buffs might recognize the region's scene-stealing backdrops from Hollywood blockbusters like Jurassic Park, E.T. and Star Wars. But the full wonder of California's 2000-year-old redwoods – some reaching 20 stories high – is impossible to capture on any screen. Look up at the forest canopy: that last 100ft of redwood growth marks 50 years since Redwood National Park was established in California's tree-hugging triumph over logging. And this year, in celebration of their 100th anniversary, Save the Redwoods League are offering free entry to over 40 redwood state parks every second Saturday of each month.
In 2018, another kind of greenery has been making headlines as California introduces the state-wide legalization of marijuana. But the draws of the Redwood Coast far surpass changes in this industry, inviting travelers to achieve the ultimate California mellow with its quirky shops, brewpubs, coffee roasters and oyster happy hours.
Home to a spirited arts community, an explosion of award-winning wineries and craft breweries and a socially responsible shopping district, Boise is what cool looks like before the rest of the world has figured it out. Fun festivals abound in Idaho's capital from downtown's Treefort Music Festival (hailed as the new alternative to SXSW) to the Boise Brew Olympics and Punk in Drublic – a beautiful marriage of punk rock and craft beer.
Once little more than a pit stop between Atlanta and Nashville, the nature-centric 'Noog has transformed itself into a bastion of elevated Southern living. Outdoor enthusiasts flock to Chattanooga for some of the best rock climbing in the country, myriad hiking and mountain biking trails and wild rides on the Ocoee River – one of America's top spots for whitewater rafting.
Foodies, hopheads and techies aren't far behind, either. Chattanooga's revitalized downtown – centered on the $20-million makeover of the city's signature train station into a multi-purpose nightlife and entertainment destination (including a world-class guitar museum) – is rife with pilgrimage-worthy New Southern cuisine, thirst-quenching breweries and techie-pleasing internet speeds. Meet the New South!
Space tourism is a rising star, with 2018 set to mark the launch of the world's first lunar tourism flight from SpaceX. Don't want to lose your life savings down a black hole? Visit the next best thing, Florida's Space Coast: home to the Kennedy Space Center and the setting for countless iconic launches including Apollo 8 – the world's first manned spacecraft to orbit the moon – which celebrates 50 years since blastoff in 2018.
View satellite launches from Cape Canaveral and Titusville or visit the new ATX (Astronaut Training Experience) at the Kennedy Space Center, where wannabe astronauts can go on a simulated mission to Mars. Continue your out-of-this-world adventures with a nighttime kayak in the bioluminescent waters around Merritt Island and observe nesting sea turtles on an eco-friendly tour.
Ready to go? Get to the heart of the USA with one of Lonely Planet's in-depth, award-winning guidebooks.
Set among steep hills with the bridge-strewn Ohio River swashing its edge, Cincinnati has always been a looker. Now beer, arts and savvy neighborhood development are giving it some swagger. The new Brewing Heritage Trail tells the beer story: how Cincy was a leading producer through the late 1800s, its citizens guzzling 2.5 times the national average. Today Rhinegeist and other modern beer makers have taken over the abandoned breweries, many of which are walkable in Over-the-Rhine, an old German neighborhood of ornate brick buildings, fresh eateries and funky shops.
2018 welcomes a new chapter for the city's artistic icons as the Music Hall celebrates its 140th birthday after undergoing massive renovations, and the Cincinnati Shakespeare Company settles into their new dynamic performance space.
River City has flipped from humdrum to happening, but the 'hey y'all' hospitality remains. Scott's Addition, once a gritty manufacturing district, thrums with microbreweries, cideries and buzzworthy restaurants, while the James River lures adventurers with whitewater rapids plus a new 52-mile bike trail along its banks.
Artistic highlights include downtown's bright murals, the whimsical Quirk Hotel (featuring unique design elements and its own art gallery) and innovative temporary exhibits at the Virginia Museum of Fine Arts. The American Civil War Museum – a recent consolidation of three separate Civil War sites – takes a multi-perspective look at Richmond's role as the capital of the Confederacy. One constant? Patrick Henry demanding liberty or death at reenactments every Sunday in summer at St John's Church.
The state of Kentucky is known for its rolling hills topped with stately horse farms, its fierce devotion to college basketball and, most importantly, its bourbon. The state's distilling heritage runs deep, and those looking for a taste should head straight to Kentucky Bourbon Country, the golden triangle between Louisville, Lexington and Elizabethtown where this seasoned spirit comes to life. You'll find an enticing network of the country's most well-known distilleries and top-notch restaurants with bourbon-inspired menus.
But this industry isn't so steeped in tradition that it forgets progress – craft distillers are opening their doors, long defunct bourbon districts are being revitalized, and in 2018 the Frazier Museum will be named the official starting point of the Kentucky Bourbon Trail.
Arches National Park's multicolored sandstone spans. Island in the Sky's Colorado River-carved vistas. Monument Valley's sky-piercing towers. Southeast Utah's major landmarks have been firm favorites among travelers for decades. Recently, however, lesser-known regional sites like the forested plateaus of the new Bears Ears National Monument have become hot-button news items due to political tussles in Washington, DC over protecting natural and cultural resources.
This vast quarter of the Beehive State holds many notable outdoor destinations, from the hotel-filled adventure town of Moab and uncrowded Capitol Reef to the atmospheric Ancestral Puebloan ruins of Hovenweep. Water has hewn the desert landscape here, carving the sandstone into startling forms such as the picturesque Natural Bridges and immense Lake Powell. This is quintessential Americana road-trip country.


Special thanks to Frank Farkas for this special report.
MIT's Sloan School was the setting for a "Gas Summit" on October 3rd, officially titled, "Common Goals, Uncommon Partners: Seeking Solutions to Reduce Methane Emissions." The conference was essentially a report on an ongoing collaboration between a coalition of environmental and environmental justice groups (including HEET, Clean Water Action, Mothers Out Front et al.) and both Eversource and Columbia Gas.
The conference takeaway and major surprise to those in the environmental community used to an adversarial relationship with the utilities, was that the coalition has gotten two large utilities to adopt the position that they share a common interest with the community in identifying and remedying all gas leaks emanating from their pipelines, and not just those with explosive potential.
Leveraged by recent state and municipal legislation that may force the companies to comply, the coalition partners seem to have convinced the two utilities of their responsibility for repairing their aging pipeline infrastructure so as to reduce methane emissions. What also appears to have motivated the companies to engage in the collaboration was the promise that, given the scientific/technical expertise the coalition brought to the table, the companies' cooperation would accelerate the development of a cost effective method for pinpointing the source of Grade 3 gas leaks, something the utilities had previously considered pie in the sky.
These leaks, which are defined in state law as "non-hazardous to persons or property," nevertheless pose a grave threat to the environment, as more recent state legislation reflects. Studies over the past several years have shown that Grade 3's spew large amounts of heat-trapping and plant-destroying methane into the atmosphere and into the soil. The worst of these leaks are the "gushers," and the first target of the collaboration, which on average constitute 7% of leaks but cause 50% of the greenhouse gas emissions.
Please set aside a chunk of time to watch the entire video. We think you'll agree that it is well worth it.
Let us know, after you view the video, if you think this model of collaboration may have application to the Berkshires and if so, your interest in being involved in a group brainstorming how to put it to work locally.


Guidepost Solutions serves as the U.S. government appointed monitor to oversee the implementation of specific compliance protocols for an international digital advertising company.
Guidepost Solutions was selected to design the intrusion alarm and video surveillance system for this 28,000 sq. ft. private high school located in San Francisco's Haight-Ashbury District.
Guidepost Solutions is serving as the consultant services for the districtwide electronic surveillance system upgrade.
Guidepost Solutions provided security consulting services that included video surveillance system recommendations and implementation utilizing the districts existing network infrastructure.
Guidepost Solutions completed a security technology master plan for School District U-46, located in the Chicago suburbs.
Guidepost Solutions advises financial technology company Neema on regulatory issues in connection with Neema's development of SOV, a digital currency that has been approved as the official legal tender, i.e. fiat currency, of the Republic of the Marshall Islands.
Guidepost Solutions provides consulting and compliance review services for this blockchain technology company which is regulated by the New York Department of Financial Services.
The company was preparing to enter the U.S. market and needed guidance with its anti-money laundering (AML) manual. Guidepost Solutions reviewed the manual and provided advice concerning expectations from U.S. regulators; standards for governance of AML programs; and the policies, procedures, and technology used to meet U.S. standards.
Guidepost Solutions was approved by the U.S. Department of Justice and Department of Housing and Urban Development to serve as Independent Compliance Consultant to a nonbank mortgage lender, following the lender's settlement with the government of a False Claims Act lawsuit.
Guidepost Solutions was hired by the Bank to conduct a review of its anti-money laundering and global sanctions policies and procedures.
Guidepost Solutions was retained by the Governor of the State of New York to review grants and approvals of certain programs and operations in the Buffalo Billion/Nano Economic Development Program for improper conduct, lobbying and undisclosed conflicts of interest.


Travelling – Join the discussion!
In a week I will be travelling back to Charleston and Maryland for some ministry partner development or "support raising" for those of you who aren't hip to the new lingo. (Yeah I just used the words hip and lingo in the same sentence!) I love visiting both places, Charleston has recently become one of my favorite places and Maryland will always have a big place in my heart especially Dc and yes I know DC is not in Maryland.
My question for you for this Friday is what places do you find your self visiting more often than others when you get a chance to get away, whether its for a weekend or a month?


These nail stickers can be very time consuming and tedious OR quick and simple – it really depends on how crazy you want to get with it! Keep it simple by just decorate your index or ring finger and it will instantly get plain nail polish to look chic!
1. Apply nail polish on your nails. We like to pick different color on our index or ring finger for fun. 2. Wait for the polish to dry. 3. Use tweezer and small scissors to cut the strip and decorate nails. KEY is to make sure the strip adheres to your nail completely. 4. Apply clear nail polish on top and apply second coat.


The Pearl Drum Rack is awesome the configuration possibilities are endless, the rack saves so much time setting your kit up which makes it perfect for live gigs and performances. The rack is durable, study and has a small footprint which allows more room and gets rid of the stands cluttering around your kit. All drummers who have bought this rack have never looked back, you wont be disappointed.


The commitment to social justice and service learning is an integral part of the Elms philosophy and mission and is infused throughout the curriculum. We seek to practice service learning in the way we treat one another in our school community as well as in the way we interact with our community at large. Our Campus Ministry team regularly organizes and advertises community service opportunities that are open to all students.
Grades 11 & 12 –remaining 20 hours for a total of 35 hours. (40 points) 10 of these hours must be done at an agency that provides "hands on" service with the poor or needy.


text:
Boat Registration
Auto Assistant App
Notice of Meetings
Minutes Full Commission
Selling Your Vehicle
Emission Cameras
Applicant must be a United States citizen or a legal permanent resident TCA 8-16-101.
Your residence or principal place of business must be in Sumner County.
Notary applications are accepted Monday through Friday 8:00 a.m. until 4:30 p.m. at the Gallatin location.
All applicants are required to appear in the Gallatin office to take the oath and pay the fee. We can notarize your application in the Clerk's office.
Notaries Public are elected by the County Commission to a four-year term. Your applications, bond and $12.00 fee must be received in the Gallatin office before the third Monday of the month.
A $10,000 original bond for the term must be submitted when you appear.
The Notaries Public can purchase a seal (rubber stamp) from most office supply companies.
Additional information: Notary FAQ's
CLICK HERE FOR NOTARY APPLICATION
What to bring when you appear before the Clerk: $12.00 fee, Notary Bond, and Application. Notary applications must be sumitted by noon on the date of the County Commission meeting. Meeting Dates.
ALL NOTARY APPLICANTS ARE REQUIRED TO APPEAR IN THE COUNTY CLERKS OFFICE TO HAVE THE OATH ADMINISTERED.
For questions please call our Notary Department 615-452-4063.
Search Notaries Public by name
@Tennessee Secretary of State
© Sumner County Government
355 N Belvedere Drive Gallatin, TN 37066
info@sumnertn.org
Visit Sumner County


Below are listed popular flight routes from airports in Chile to Colombo Bandaranaike in Sri Lanka. Only the cheapest flights found are shown. Use the form below to check the flight price for your dates.
Are you looking for flights from Colombo Bandaranaike to Chile or to other destinations? Please use our flight search form, or navigate to the guide for the country or the airport you would like to depart from.
More flights: Chile to Sri Lanka. No direct flights from Santiago Arturo Merino Benitez (SCL) to Colombo Bandaranayake (CMB), Sri Lanka found. Prices of flights to Colombo Bandaranayake sourced on Tuesday 24th July 2018 at 15:01.


text:
Q: CSS not working properly in IE So since the Wordpress plugin Hustle comes with a ton of bugs, I've implemented a fix provided by someone from the Hustle support forums. It changes the position of a clickable link and changes it's content to 'X'. (It's a button to close a pop-up.)
In Chrome, Firefox it works exactly how it should. The X appears on the top right corner of the pop-up, but in Internet Explorer it does not. Also by adding !important it won't work.
If you want to make the pop-up appear you have to scroll down a bit.
URL: www.melvinosenga.nl
CSS:
.inc_opt_never_see_again {
    position: absolute;
    visibility: hidden;
}

.inc_opt_never_see_again::before {
    content: "X";
    font-size: 20px;
    visibility: visible;
}

.inc-opt-close-btn.inc-opt-close-popup {
    display: none !important;
}

.wpoi-nsa {
    position: absolute;top: -35px;right: 5px;
}

redpajama_set_name: RedPajamaStackExchange


You may have heard whispers of the eclipse season upon us , or of this luna eclipse arriving with our full moon this weekend ? A time of strong energies and opportunities for renewal. Whether effected by the moon , the stars , our inner workings or the environment we find ourselves in , life is certainly full of challenges , upheval and constant change. Of this we know.
Giving ourselves little moments to stop , to breathe , to be with nature , to connect in with the flowers , these are the things that bring relief , comfort , strength. We don't always need to carve out big chunks of time for self care and reflection. In catching ourselves throughout the day with sweet little reminders to breathe , listen , allow , soften , become more present , we can make everyday easier , clearer , happier.
Flowers help so much with this. They are our friends , offering gentle healing essences to support us through. Using your sacredflower products during the day as little pick me ups , & reminders to breathe , soften , clear , strengthen. To just be for a moment . . . the flowers give us this , and for this l am so grateful.
Flowers , plants , trees , the sun , the moon , the big sky , the water , all of them are friends to me. l draw great nourishment from their endless giving and bountiful healing qualities. Turning our attention to nature as we move through the day is vital. A bit more time staring out the window , instead of at the screen. A few extra moments of appreciating the air we recieve because of trees , and the hues of green in the plants and the colours of the sky , the flowers . . .
At the moment the camillia flowers are blossoming strong here. Sometimes refered to as the rose of winter , she brings beautiful shades of pink and white and carries such a sweet , nurturing essence. l have sourced an organic camillia oil to use as a pure flower serum for your skin. Some of you have tried our ' anoint ' serum already. The camillia oil with rare & precious sacred flower essences & essential oils added , and l said I would mention it here when it was up perminantly on the website. So yes , its here ! If you want to find out more about this exeptional skin oil & divine anointing product see it on our home page.
Thanks so much for sharing this rambling , flower , earth love with me . . .
Please enjoy the discount code ' grateful ' for 20% off storewide , available to all of you , my valued flower community , until the full moon on this Saturday the 28th.


A new benchmarking report has found hotline incident reports for corporate fraud to be significantly on the rise, with a steady but solid increase recorded over the last two years.
'The 2013 Corporate Governance and Compliance Hotline Benchmarking Report' – compiled by governance and compliance consultant The Network Inc. and BDO Consulting – comprises data from over 624,046 incident reports from between 2008 to 2012 and notes an increase in incident reports in four out of the five organisational size categories, Compliance Week reports.
Companies with 5,000 to 10,000 employees sat atop the list with the highest rate of hotline incident report at 12.6 per cent, followed by companies with more than 50,000 employees at 12.1 per cent. In contrast, companies with 20,000 to 50,000 employees saw its incident rates decrease.
Personnel management emerged as the leading incident category with 46 per cent of reports that covered issues such as human resources and employee relations, followed by employment law with 15 per cent of incidents, corruption and fraud at 13 per cent and employment law violations at 12 per cent. The reporting of incidents related to fraud jumped to 23.6 per cent from 21.1 per cent in 2011, allegedly an all-time high for the Corporate Fraud Index since its induction in 2005.
In terms of specific industries, seven out of ten of the industry categories saw an increase in incident reports, the largest of which in the construction industry, and the largest decrease in the Finance, Insurance & Real Estate industry.


text:
RECEIPTS: Bill Gates's Foundation Just Paid for The Chinese Communist Party to Recruit Foreign Scientists.
Why isn't anyone stopping this?
by Natalie Winters
The Bill & Melinda Gates Foundation is helping fund the Chinese Communist Party's Ministry of Science and Technology, assisting the brutal regime in its efforts to lure foreign scientists in to boost China's scientific advancement, The National Pulse can reveal.
A recent, $100,000 grant from the Microsoft mogul's foundation was sent to the Foreign Talent Research Center of China's Ministry of Science and Technology in June, according to the organization's website.
The purpose of the cash is listed as organizing a forum on "pandemic preparedness and response," which would focus on "leveraging resources to improve global health and support disadvantaged populations who are disproportionately impacted by pandemic." The forum is affiliated with the Zhongguancun Forum, a Beijing-based technology conference sponsored by the Chinese Communist Party that counts high-level officials – including President Xi Jinping – as speakers.
The Gates Foundation's decision to fund the Foreign Talent Research Center of China's Ministry of Science and Technology comes amidst controversy over the likelihood of international collaboration between U.S. and Chinese researchers in Wuhan leading to the genesis of COVID-19. Since the outbreak of the virus, the Chinese Communist Party has stonewalled investigations into the origins of the virus and planted Western researchers with compromising ties to China in senior investigative roles.
Beyond COVID-19, the Chinese Communist Party has also come under fire for weaponizing its science and technology programs to lure Western scientists away from their home countries in order to facilitate Chinese advancement and military build-up. This program – commonly known as the Thousand Talents Plan – has led to several Department of Justice (DOJ) indictments of American researchers who routinely fail to disclose their financial ties to the Chinese Communist Party despite receiving U.S. taxpayer funds.
MUST READ: Corporates Have Begun 'Geoengineering' The Climate, With Basically No One's Consent.
Evidence of Gates aiding China.
The Ministry of Science and Technology's Foreign Talent Research Center appears to engage in similar conduct, as it is responsible for "bring[ing] in foreign talent," according to an outline of its missions.
The ministry "formulates and facilitates the implementation of plans for bringing in high-end foreign experts, develops mechanisms for pooling top-notch scientists and research teams from abroad, and provides services for foreign experts," it continues.
The unearthed grant comes amidst the Gates Foundation pouring millions of dollars into China, including to universities with ties to the Wuhan Institute of Virology.
Natalie Winters
Natalie Winters is freelance reporter.
Corporates Have Begun 'Geoengineering' The Climate, With Basically No One's Consent.
STUDY: Childlessness Leads to Liberalism, Support for Homosexuality, Abortion, and Promiscuity, while Parenthood Creates Conservatism and Traditional Values.
Gates Foundation Funds Clinton Initiative to Force China Reliance On Developing World.


The most unique one-of-a-kind gift for your family and friends!
We are using the most advanced printing technology (a giclée) and bringing together Brenda's beautiful painting and your family. Some of you have Brenda's personalized Christmas pieces. Many have requested that she create a piece to compliment the other personalized art selection. "In Honor of…" comes from Brenda's love of family and country and her belief that honor is due to so many wonderful people that we all have in our lives. Keep in mind that these prints are one-of-a-kind, created specifically for you.


text:
Internationally acclaimed celebrity hairstylist Amanda George is known for creating, changing and nurturing the color of some of the most famous women in Hollywood, including Charlize Theron, Victoria Beckham, Kate Hudson, Lindsay Lohan and Jenny McCarthy. One of the most sought after colorists in the world, her name is synonymous with both the most beautiful blondes and gorgeous brunettes.
Growing up in Great Britain, George's father worked in the film industry, and was frequently surrounded by people that made the movie stars glamorous. She developed a passion for the beauty industry and honed her skills in London's top salons.
In the early 1990's, George moved to Los Angeles to work alongside the best in the business. Quickly becoming one of the best kept secrets in hair, George's unrivaled coloring techniques put her in high demand, requiring her to travel once a month to New York to tend to her burgeoning East Coast clientele.
Earning the reputation for being one of the most celebrated colorists in the industry with a successful career spanning nearly two decades, George opened the doors to her own namesake salon in Beverly Hills in 2005, which gained fame and recognition for breeding some of the world's most expert stylists, its A-list clientele and luxurious atmosphere. Among her many projects, you can still find Amanda working behind the chair at her ROIL Salon in Beverly Hills.
George has received national acclaim in the world's leading fashion publications including Allure, Harper's Bazaar, Cosmopolitan and Vogue.
© 2018 Amanda George


text:
More Students from Underserved High Schools Go to College with Help from CAC Advisors
These are the four most important words for high school students and their college advisors to hear, founder and CEO of College Advising Corps (CAC) Dr. Nicole Hurd said at the organization's 10th annual summit last week in Washington, D.C.
This key message, along with advice and best practices were part of the many topics discussed at the four-day CAC advisor summit. Packed with motivational keynote addresses and interactive breakout sessions, the annual gathering of advisors from across the country is designed to help them succeed in executing CAC's mission: helping low-income, first-generation and underrepresented students enroll into college.
CAC advisors are not the typical school counselors; they are recent college graduates who serve one to two years at underserved high schools that have a severe deficit in school counselors. CAC partners with 24 universities and colleges across the country to recruit, train and place these graduates.
CAC's New Program in Arizona off to a Great Start
Jimena Morales of the University of Arizona Advising Corps was one of the advisors at the summit. She says a lot of her work involves helping students navigate the complex and daunting process of college admissions and securing financial aid.
One student who she worked with last year thought he would not be able to attend college because his parents were unable to afford it. Though he was initially hesitant, through Jimena's persistent advice and encouragement, he applied for and won a number of scholarships. But, most importantly, Jimena says, "I made him believe in himself and his potential."
CAC uses a unique near-peer model that helps advisors uniquely relate to students' experiences. More than just being close in age, many of the advisors, like Jimena, come from backgrounds similar to their students.
"I am a first-generation college student myself and I was able to go to college only because of a scholarship," Jimena says. "I know what my students are going through."
The University of Arizona Advising Corps launched last year with the support of ECMC Foundation. The Foundation also made investments to expand the Virginia Advising Corps, reaching underserved high schools in rural parts of Virginia; and to start a program in Florida by the 2017-18 academic school year.
Up Next for CAC: An Ambitious Growth Plan
Currently, College Advising Corps' 600 advisors work in 600 high schools across 15 states and assists more than 180,000 students. But, it is not stopping there. With an ambitious plan to scale, Dr. Hurd hopes to reach half of the 1.4 million low-income, high school seniors across the country within 10 years.
"If we are to have a vibrant democracy and working economy, college access and success is critical to scale," says Dr. Hurd. "We won't get to where we need to go as a country, an economy, and place of opportunity unless we increase the number of first-generation, low-income, underrepresented students going to college."
Scaling is only part of it; it is "critical to scale CAC with excellence and rigor," she says. CAC partners with Stanford University and Evaluation and Assessment Solutions for Education to evaluate the program. Results help them learn what's working, what isn't and what should be replicated. The researchers also explore and pilot new, innovative ideas, such as a parent texting service called "Parent Academy," e-advising and transportation vouchers for students.
Part of ECMC Foundation's investment in CAC is supporting research and evaluation. "The Foundation's funding allows us not to just scale in schools such as Arizona, Virginia, and Florida, but also to build out the learning lab that is all about creating knowledge to change the field," says Dr. Hurd.
Dr. Nicole Hurd was honored in October 2016 as a White House College Opportunity Champion for Change. Through her leadership, College Advising Corps grew from a pilot program in Virginia, where she founded it in 2005, to now the largest college access program in the country. Learn more about College Advising Corps.