text:
\subsubsection*{\bibname}}

\usepackage{graphicx}
\usepackage{amsthm}
\usepackage{math_cmd}
\usepackage{xcolor}
\usepackage{algpseudocode}
\usepackage{algorithm}
\usepackage{breakcites}
\usepackage{pifont}
\usepackage{diagbox}
\usepackage{multirow}
\usepackage{booktabs}
\usepackage{enumitem}
\usepackage[titletoc]{appendix}
\usepackage{hyperref}

\usepackage{natbib}
\begin{document}

\runningauthor{Bai, Zhang, Li, Henao, Wang, Carin}

\twocolumn[

\aistatstitle{Collaborative Anomaly Detection}

\aistatsauthor{ Ke Bai$^{*}$ \And Aonan Zhang \And Zhizhong Li \And Ricardo Henao }

\aistatsaddress{Duke University \And  Bytedance Inc.  \And SenseTime Research \And Duke University}

\aistatsauthor{ Chong Wang$^{**}$ \And  Lawrence Carin }
\aistatsaddress{  Apple Inc. \And   KAUST}
]

\input{abstract}
\input{introduction}
\input{related}
\input{cad}
\input{embedding}
\input{experiment}
\input{conclusion}

\bibliographystyle{apalike}

\section{Log-likelihood Ratio Estimation}

\subsection{Proof of Proposition 1 in Section 3.2.}

\begin{proposition}\label{prop:kl}
    \(p(\xv)=\sum_t m^{(t)}q^{(t)}(\xv)\) is optimal for minimizing the expected KL divergence
    \begin{align}\label{eq:min_kl}
        \min_{p(\xv)\in\mathcal{P}}\mathbb{E}[\operatorname{KL}(q^{(t)}|p)] =\min_{p(\xv)\in\mathcal{P}}\sum_t\mathbb{E}_{ q^{(t)}(\xv)}\left[ m^{(t)} \log \frac{q^{(t)}(\xv)}{p(\xv)}\right].
    \end{align}
\end{proposition}

\begin{proof}
Expanding the expectation over tasks. We get
\begin{align}
    \mathbb{E}[\operatorname{KL}(q^{(t)}|p)] = \sum_t m^{(t)} [\operatorname{KL}(q^{(t)}|p)].
\end{align}

We need to solve a minimization problem with constraints:
\begin{align}
     \min_{p(\xv)\in\mathcal{P}}\quad & \sum_t\mathbb{E}_{ q^{(t)}(\xv)}\left[ m^{(t)} \log \frac{q^{(t)}(\xv)}{p(\xv)}\right]\nonumber\\
     \text{s.t.}\quad & \int p(\xv) d \xv = 1.
\end{align}
With the Lagrange multiplier $\lambda$, we need to find function $p(\xv)$ to minimize the Lagrangian
\begin{align}
    \sum_t\mathbb{E}_{ q^{(t)}(\xv)}\left[ m^{(t)} \log \frac{q^{(t)}(\xv)}{p(\xv)}\right] + \lambda\left(\int p(\xv) d\xv  - 1\right).
\end{align}
After removing additive terms unrelated to $p(\xv)$, we get
\begin{align}
    \int\bigg( \sum_t  -m^{(t)}q^{(t)}(\xv) \log p(\xv) + \lambda p(\xv) \bigg) d\xv.
\end{align}

  In the calculus of variations, \(p(\xv)\) is varied by adding a function  \(\delta p(\xv)\) to it with \(\epsilon \) multiplier, where \(\epsilon \rightarrow 0\):
\begin{align}
    \int\bigg(\sum_t -m^{(t)} q^{(t)}(\xv) \log \big(p(\xv) + \epsilon \delta p(\xv)\big) + \lambda (p(\xv)+ \epsilon \delta p(\xv)) \bigg) d\xv .
\end{align}
The change in the value to first order in $\epsilon$ should be zero. By taking derivative w.r.t. $\epsilon$ and let $\epsilon = 0$, we get
\begin{align}
    \int\bigg( \sum_t  -m^{(t)}q^{(t)}(\xv) \frac{\delta p(\xv)}{p(\xv)}  + \lambda (\delta p(\xv)) \bigg) d\xv = 0.
\end{align}

The above equation is valid for any function $\delta p(\xv)$
\begin{align}
    \lambda -  \frac{1}{p(\xv)}\sum_t m^{(t)}q^{(t)}(\xv) = 0.
\end{align}

After reordering terms, we get
\begin{align}
\label{eq:final}
    p(\xv) = \frac{1}{\lambda}\sum_t m^{(t)} q^{(t)}(\xv).
\end{align}

Since $\int p(\xv) = 1$ and $\sum_t m^{(t)} = 1$, taking the integral over $\xv$ on both sides of Equation \ref{eq:final} permits $\lambda = 1$. Finally, we get
\begin{align}
    p(\xv) = \sum_t m^{(t)} q^{(t)}(\xv).
\end{align}

This concludes our proof.
\end{proof}

\subsection{Proof of the optimal solution to Equation 3}
Suppose we have two distributions \(q(\xv)\) and $p(\xv)$, their log-likelihood ratio $r(\xv)=\log(q(\xv)/p(\xv))$ can be estimated as follows.

First, we independently draw $N$ samples from $q(\xv)$ and label those data as positive: $\{\xv_i,y_i=1\}_{i=1}^N$. Similarly we draw $N$ samples from $p(\xv)$ as label those as negative: $\{\xv_i,y_i=-1\}_{i=N+1}^{2N}$. We combine all positive samples and negative samples into one dataset $D=\{\xv_i,y_i\}_{i=1}^{2N}$. In the limit of $N\rightarrow\infty$, the mixed distribution satisfies:
\begin{align}
p(y=1|\xv)=\frac{q(\xv)}{p(\xv)+q(\xv)}.
\end{align}
Then we train a binary classifier on the mixed dataset $D$. Suppose we are using a linear logistic regression model:
\begin{align}
\hat{p}(y\,|\,\mathbf{x};\,\bm\theta) = \sigma(y\cdot\bm\theta^\top\mathbf{x}),
\end{align}
where $\sigma$ is the sigmoid function:
\begin{align}
\sigma(z) = \frac{1}{1+e^{-z}}.
\end{align}
Then
\begin{align}
\log\frac{q(\xv)}{p(\xv)}&=\log\frac{p(y=1|\xv)}{1-p(y=1|\xv)}\nonumber\\
&\approx\log\frac{\hat{p}(y=1|\xv;\,\bm\theta)}{1-\hat{p}(y=1|\xv;\,\bm\theta)}\nonumber\\
&=\log \frac{\sigma(\bm\theta^\top\mathbf{x})}{\sigma(-\bm\theta^\top\xv)}\nonumber\\
&=\bm\theta^\top\mathbf{x}.

\end{align}
One can use a non-linear model implemented with a neural network $f(\xv,\bm\theta)$ to get a better estimation of the log-likelihood ratio.

\section{Conditional Normalizing Flow}
\subsection{Feature extractors.}

A good image generator implemented with normalizing flows is not necessary a good out-of-distribution detector, since normalizing flows tend to capture pixel correlations rather than semantics during training~\citep{kirichenko2020normalizing}. Moreover, normalizing flow models for image generation often requires a large number of parameters and long training time, which is not-applicable when the number of tasks is large.

To resolve above issues, we use pre-trained low-dimensional features instead of raw images to train normalizing flows. The features are extracted using unsupervised contrastive learning~\citep{chen2020simple} with a ResNet-18 as the backbone model. The training accuracy on the extracted features is 90.2\% with a linear classifier, demonstrating high feature quality. The unsupervised learning procedure guarantees that the extracted feature does not leak its label information while preserving image semantics.

\subsection{Two ways to implement conditional normalizing flows.}
In the following, we use \(\ev^{(t)}\) to represent task-specific information, which can involve in the prior \(\pi\) and the invertible mapping \(f\) as follows:
\[
    L = -\log q(\xv|\ev^{(t)}) = -\log\pi_{\ev^{(t)}}(f^{-1}_{\ev^{(t)}}(\xv)) - \log\left|\det\frac{\partial f^{-1}_{\ev^{(t)}}(\xv)}{\partial\xv}\right|.
\]
\begin{paragraph}{Task-specific priors}
For un-conditional normalizing flows, the prior $\pi$ is usually set as a centered isotropic Gaussian.
In our conditional case, each task \(t\) has its task-specific Gaussian prior \(\pi_{\ev_{prior}^{(t)}}\) with a uniformly sampled mean vector and an identity covariance matrix.
\end{paragraph}

\begin{paragraph}{Task-specific mappings}
  MAF uses Masked Autoencoder for Distribution Estimation (MADE) as its building blocks. The core idea behind MADE is autoregressive flow, which generates data recursively by
\[
    x_i = u_i \exp \alpha_i + \mu_i
\]
Where \(\mu_i = f(\xv_{1:i-1}),~\alpha_i = g(\xv_{1:i-1}) \). \(\uv\) is randomly generated from the previous layer or the prior distribution. In our MADE implementation, function \(f\) and \(g\) are both 4-layer autoregressive fully connected layer with masking. The task specific features are combined with the output of each MADE layer by scaling and biasing, i.e. \(f_{\ev^{(t)}} = f \odot \textit{tanh}(\ev_{scale}^{(t)}) + \textit{tanh}(\ev_{bias}^{(t)})\). The hyperbolic tangent function is used to stabilize the training, and it can be replaced with deep neural networks. \(\ev_{scale}^{(t)}\) and \(\ev_{bias}^{(t)}\) are trainable task embeddings.
These parameters are shared among different \(i\) in one layer. The same technique also applies to function \(g\). For more details on MAF, please refer to the origin paper \citet{germain2015made,papamakarios2017masked}.
\end{paragraph}

 Our implementation refers to two Github codebases \footnote{pytorch-normalizing-flows:  \url{https://github.com/karpathy/pytorch-normalizing-flows}.}$^{,}$\footnote{pytorch-flows:  \url{https://github.com/ikostrikov/pytorch-flows}}.

\section{Real scenarios for CAD}

\section{Training Details}
In this section, we give full descriptions of models in our experiments.

\subsection{MNIST and CIFAR10}

\begin{figure}[htb!]
\centering
    \includegraphics[width=.4\textwidth]{figures/mnist_hyp_framework.pdf}\vspace{-10pt}
    \caption{The framework we use to train MNIST and CIFAR10. $\xv$ denotes the input matrix. $\ev$ is the task embedding. FC represents the fully connected layer. The brackets followed by contains the dimension of its input and output. e\_dim is the dimension of the task embedding. The definition of c\_dim and the detail of feature extractor can be found in Table \ref{table:appendix_1}.}
\label{fig:appendix_1}
\bigskip
    \includegraphics[width=.3\textwidth]{figures/movielens_hyp_framework.pdf}\vspace{-10pt}
    \caption{The framework we use to train MovieLens 1M. The definition of f\_dim, e\_dim and the detail of neural network can be found in Table  \ref{table:appendix_2}.}
\label{fig:appendix_2}
\end{figure}

Figure \ref{fig:appendix_1} demonstrates the network structure of the likelihood ratio estimation model ($f(\xv,\ev,\bm\theta)$ in  Algorithm 1)
for MNIST and CIFAR10. We resize all MNIST figures from their original size $28\times28$ to $32\times32$ to align with CIFAR10 figure sizes. The data augmentations include random crop with padding (Pad the image to \(40\times40\) with zeros, then randomly choice a \(32\times32\) area) and random horizontal flip. We normalize pixels for each channel such that their mean equals to zero and variance equals to one.

The pre-embedding model uses a similar feature extractor, followed by a fully connected layer mapping from feature dimension c\_dim*8 to 256. Then $M_0$ fully connected output heads are used to predict the likelihood ratio for $M_0$ seed tasks.

\subsection{MovieLens 1M}
\subsubsection{Feature Extractor}
Unlike CIFAR10 and MNIST where each image has its raw pixel representation, the user feature is implicitly represented by the interaction between movies and users. We treat MovieLens as a click-through rating prediction dataset and use a deep factorization-machine~\citep{guo2017deepfm} to extract user embeddings \(\xv\). After training the factorization model, we use the trained embedding layer to extract features of each user.
During the training process, all users are visible to the model, so we can get the feature of each user. We implement this method using code from \citet{cao2019unifying}.

\subsubsection{Hyper-parameters}
Figure \ref{fig:appendix_2} demonstrates the network structure of the likelihood ratio estimation model ($f(\xv,\ev,\bm\theta)$ in  Algorithm 1) for MovieLens 1M dataset. The pre-embedding model uses the same network structure. The only difference is that we change the way to integrate task embeddings from adding them to the input to adding $M_0$ output heads in the pre-embedding model.

\begin{table}[h]
\centering
\caption{The network structure of feature extractor for CIFAR10 and MNIST. Each operation contains four parts: (kernel size, stride, padding size), number of output channel, batch normalization (BN) and activation function. In our setting, we use c\_dim = 64. The output size is a three-dimensional vector: (width, height, channel). The input channel is 3 for CIFAR10  and 1 for MNIST.}

\begin{tabular}{|c|l|c|}
\hline
Stage & \multicolumn{1}{c|}{Operation}              & Output Size  \\ \hline
Input &   \multicolumn{1}{c|}{Data Augmentation}       & (32,32,3 or 1)    \\ \hline
1     & (4, 2, 1), c\_dim, BN, ReLU  & (16, 16, c\_dim) \\ \hline
2     & (4, 2, 1), c\_dim*2, BN, ReLU  & (8, 8, c\_dim*2)   \\ \hline
3     & (5, 1, 0), c\_dim*4, BN, ReLU & (4, 4, c\_dim*4)  \\ \hline
4     & (4, 1, 0), c\_dim*8, BN,       & (1,1, c\_dim*8)   \\ \hline
\end{tabular}
\label{table:appendix_1}
\end{table}

\begin{table}[h]
\centering
\caption{The network structure of logistic regression for MovieLens 1M. Each operation contains three parts: (input size, output size) of fully connected layer, activation function and dropout rate. f\_dim represents the dimension of input data. e\_dim is the dimension of the task embedding. We use f\_dim=100, e\_dim = 16.}
\begin{tabular}{|c|l|}
\hline
Stage & \multicolumn{1}{c|}{Operation}                   \\ \hline
1     & (f\_dim + e\_dim, 32), ReLU, Dropout (0.5)              \\ \hline
2     & (32, 32), ReLU, Dropout (0.5)                            \\ \hline
3     & (32, 16), ReLU, Dropout (0.3)                            \\ \hline
4     & (16, 1), Linear , No Dropout                                               \\ \hline
\end{tabular}
\label{table:appendix_2}
\end{table}

\section{More results on MNIST}
Similar as Table 3 (left) in the main text for the CIFAR10 dataset, we study different task embedding initialization methods (see Section 5.2 in the main text for definitions of those task embeddings) on the MNIST dataset with two CAD settings where $k=4$ with $C_{10}^4=210$ tasks, and $k=5$ with $C_{10}^5=252$ tasks. We skip the setting with $k=2$ and $k=3$ since those settings are too simple such that every method we tried can get almost $100\%$ AUC\@. The results is shown in Table \ref{table:toy_result_mnist}.

\begin{table*}[h
  \caption

    Testing AUC\;(\(\times\)100) on MNIST.  Label embedding uses prior knowledge (ground truth label) so it has a natural advantage comparing with other methods (Note that the first method ``label embedding'' uses extra knowledge and is marked with *. The best results without extra knowledge are shown in bold.).

  }\label{table:toy_result_mnist}
  \begin{center}
    \begin{small}
      \begin{sc}

        \begin{tabular}{ccc}
          \toprule

          \multicolumn{1}{c|}{Embedding Init.~\textbackslash~K (\#tasks)}            & 4 (210)                    & 5 (252)      \\
          \midrule
          \multicolumn{1}{c|}{Label Embedding*}       & 99.88                      &99.67      \\
          \multicolumn{1}{c|}{Pseudo Label Embedding} & 97.88                      & 98.67  \\

          \multicolumn{1}{c|}{Random Initialization}  & 97.02                      & 86.80 \\
          \multicolumn{1}{c|}{Learned Embedding ($M_0 = 10$)}        & 98.88                      & 99.67      \\
          \multicolumn{1}{c|}{Learned Embedding ($M_0 = 64$)}        & \textbf{99.93}             & \textbf{99.90} \\

          \bottomrule
        \end{tabular}
      \end{sc}
    \end{small}

  \end{center}
\end{table*}

{
\small
\bibliographystyle{apalike}

\section{Collaborative Anomaly Detection (CAD)}\label{sec:cad}
\begin{figure*}
\scalebox{0.9}{
\begin{minipage}[t]{0.5\textwidth}
    \vspace{10pt}
    \begin{algorithm}[H]
        \begin{algorithmic}
            \Function {DataSampler}{$M,N$}
            \State Sample \(N\) task indices \(t_i\propto m^{(i)}\), where \(i\in[M]\).
            \State Sample \(N\) instances \(\xv_i\) given \(t_i\) as \(\xv_i\sim q^{(t_i)}(\xv)\).
            \State Sample \(N\) contrastive instances \(\tilde{\xv}_i\sim p(\xv)\).\\
            \quad \ \ \Return \(\mathbf{t}=(t_{1:N}), \xv=(\xv_{1:N}), \tilde{\xv}=(\tilde{\xv}_{1:N}).
            \)
            \EndFunction

            \State
            \Function {EstimateCLR}{$M,\ev$}
            \State Randomly initialize \(\thetav\).
            \State Randomly initialize \(\ev\!\!=\!\!(\!\ev^{(1)},\ldots,\ev^{(M)}\!)\) if \(\ev\!=\!\textproc{None}\).

            \While {not converge}
            \State \(\mathbf{t}, \xv, \tilde{\xv}\) = \Call{DataSampler}{$M$, $N = \textit{batch-size}$}
            \State Update parameters with gradients \(\nabla_{(\thetav,\ev)} L\), where
            \begin{align*}
                L = \frac{1}{N}\sum_{i\in [N]}\Big( & \log(1+\exp(-f(\xv_i,\ev^{(t_i)},\thetav)))              \\
                +                                   & \log(1+\exp(f(\tilde{\xv}_i,\ev^{(t_i)},\thetav)))\Big).
            \end{align*}
            \EndWhile
            \State \Return \(\thetav,\ev\).
            \EndFunction
        \end{algorithmic}
        \caption{Conditional likelihood ratio estimation.}\label{alg:mtocc}
    \end{algorithm}
\end{minipage}\hspace{40pt}
}
\begin{minipage}[t]{0.4\textwidth}
    \vspace{1pt}

    \centering
    \includegraphics[width=0.7\textwidth]{figures/lratio_demo1.pdf}\vspace{-8pt}

  \begin{minipage}[t]{1\textwidth}
    \caption{Colored dots are the samples. Each sample is represented by $1$-d age and gender embedding. (Top) Conditional density of three tasks, (Middle) Base distribution: population density. (Bottom) Likelihood ratio. red dots illustrate the positive sample and blue dots means negative }\label{fig:ll}
  \end{minipage}
\end{minipage}
\end{figure*}
In CAD, each item \(t\) forms a single task, with samples \(\vx\) from its own nominal distribution \(q^{(t)}(\xv)\) representing its exposed users.
Before proceeding to our main approach for CAD using conditional likelihood ratio estimation, we first present and analyze an alternative approach based on conditional density estimation, which will serve as a  baseline.

\subsection{CAD with conditional density estimation}

In this approach, the goal is to learn a conditional density function \(q(\xv|\ev^{(t)})\) to approximate \(q^{(t)}(\xv)\), and then identify anomalous \(\mathcal{A}^{(t)}\) as data whose density is below a threshold \(\mathcal{A}^{(t)}=\{\vx\in\mathcal{X}:q(\xv|\ev^{(t)})\leq \tau\}\).
Here \(\ev^{(t)}\) is a \emph{task embedding} vector for task \(t\).
Existing methods for conditional density estimation include Gaussian mixture models (GMM), kernel density estimators (KDE),
and those based on deep neural networks such as variants of variational auto-encoders (VAE)~\citep{pol2019anomaly}, generative adversarial networks (GAN)~\citep{mirza2014conditional}, and normalizing flows~\citep{abdelhamed2019noise}.
Section~\ref{sec:exp} will present experiment results on a variety of those methods.
Below we detail the normalizing flow, as it maximizes the exact likelihood function.

Conditional normalizing flows learn task-specific invertible mappings \(f_{\ev^{(t)}}(\zv):\mathcal{Z}\rightarrow\mathcal{X}\) from a simple task-dependent Gaussian distribution \(\pi_{\ev^{(t)}}(\zv)\) to the target distribution \(q(\xv|\ev^{(t)})\).
The loss function is the negative log-likelihood
\begin{align*}
        L & = -\log q(\xv|\ev^{(t)}) \\
        & = -\log\pi_{\ev^{(t)}}(f^{-1}_{\ev^{(t)}}(\xv)) - \log\left|\det\frac{\partial f^{-1}_{\ev^{(t)}}(\xv)}{\partial\xv}\right|
\end{align*}
In practice, flexible normalizing flows are implemented by stacking invertible layers \(f_{\ev^{(t)}}(\zv)=f^K_{\ev^{(t)}}\circ f^{K-1}_{\ev^{(t)}}\circ\ldots\circ f^1_{\ev^{(t)}}(\zv)\)~\citep{winkler2019learning,abdal2021styleflow}.
Our experiments suggest that conditional normalizing flows suffer from two limitations.
\(i\)) Estimating \(q^{(t)}(\xv)\) is hard since \(\xv\) lies in a high-dimensional space and there are often only a few samples available for each task.
Further, the estimated conditional density function may assign a high score for anomalous data due to the inductive bias of normalizing flows~\citep{nalisnick2018deep,kirichenko2020normalizing}.
\(ii\)) The performance of density estimation highly depends on the number of tasks.

To alleviate these issues, in Section~\ref{sec:cad_alg} we consider estimating the conditional likelihood-ratio. Then in Section~\ref{sec:task_embeddings} we propose to use seed tasks to initialize the task embeddings.

\subsection{CAD with conditional likelihood ratio estimation}\label{sec:cad_alg}

We consider introducing a \textit{base distribution} \(p(\xv)\) from which we can sample.
Then, CAD estimates the log-likelihood ratio \(r^{(t)}(\xv)\) as a proxy for the density \(q^{(t)}(\xv)\)
\begin{align}\vspace{-3pt}
    r^{(t)}(\xv)=\log \frac{q^{(t)}(\xv)}{p(\xv)}.\vspace{-3pt}
\end{align}

\paragraph{Base distribution selection}
As illustrated in Algorithm~\ref{alg:mtocc}, the base distribution resembles the negative samples in the one-class classification literature~\citep{steinwart2005classification}.
A good choice for the base distribution varies with the application; however, it should be informative enough to characterize the boundary between nominal and anomalous data.
In the context of recommendation systems, we choose the base distribution as the population exposure distribution \(p(\xv)=\sum_t m^{(t)}q^{(t)}(\xv)\), where \(m^{(t)}\) is the proportion of users exposed to task \(t\).
Let \(\mathcal{P}\) be a feasible domain of probability measures.
We can prove that
\begin{proposition}\label{prop:kl}
    \(p(\xv)=\sum_t\!m^{(t)}q^{(t)}(\xv)\) is optimal for minimizing the expected KL divergence

    \resizebox{.95\linewidth}{!}{
  \begin{minipage}{\linewidth}
  \begin{align}
\min_{p(\xv)\in\mathcal{P}}\mathbb{E}[\operatorname{KL}(q^{(t)}|p)] =\min_{p(\xv)\in\mathcal{P}}\sum_t\mathbb{E}_{q^{(t)}(\xv)} \left[ m^{(t)} \log \frac{q^{(t)}(\xv)}{p(\xv)} \right]
\end{align}
  \end{minipage}
}
\end{proposition}

Instead of using the standard density estimation defined on the Lebesgue measure \(\mathcal{R}^n\), our likelihood ratio is a Radon–Nikodym derivative \wrt \(p(\xv)\), which has two benefits.
First, the likelihood ratio corrects the bias of the density by the population.
For example, a moderately common sample \(\vx\) in one task \(t\), namely \(q^{(t)}(\xv)\approx 1\), should be marked as significant even when it is extremely rare in the population, \ie, \(p(\xv)\approx 0\).
Second, previous work on computational stability~\citep{friedman2001elements} shows that a smaller distance between two distributions leads to more accurate likelihood-ratio estimation.
Moreover, Proposition~\ref{prop:kl} shows that our choice of \(p(\xv)\) is the best option under the Kullback–Leibler (KL) divergence metric. The proof can be found in the appendix.

Figure \ref{fig:ll} demonstrates conditional density functions \(q^{(t)}(x)\) of three tasks. To learn the density functions is arguably difficult given the scarcity of data (first row). Instead, one can estimate conditional likelihood ratios with discriminative learning, where each task has its own positive samples and some negative samples (third row) drawn from the population density function (second row).
One can also observe that density functions have more modes than the likelihood ratio, thus harder to learn. This is because the population density p(x) in this case can smooth out modes in \(q^{(t)}(x)\) and result in simpler and smoother likelihood ratios.

\paragraph{Likelihood ratio estimation}

We learn the ratio \(r^{(t)}(\xv)\) by logistic regression.
Estimating the likelihood ratio for a single task \(t\) is considered first, and then we extend this to jointly learn all tasks.
For each task \(t\), we sample an observation from the nominal distribution \(\xv\sim q^{(t)}(\xv)\) as a positive example and pair it with a negative example drawn as \(\tilde{\xv}\sim p(\tilde{\xv})\).
Repeatedly doing so, we collect \(N\) contrastive pairs \((\xv_i,y_i=1)\) and \((\tilde{\xv}_i,\tilde{y}_i=-1)\).

When \(N\) goes to infinity, the optimal solution \(\thetav^*\) for the logistic regression loss
\begin{align}
    \label{eq:embed}
    L(\thetav) = & \frac{1}{N}\Big(\sum_{i\in [N]}\log(1+\exp(-f(\xv_i,\thetav)))\nonumber \\
    &+ \sum_{i\in [N]}\log(1+\exp(f(\tilde{\xv}_i,\thetav)))\Big),
\end{align}
guarantees that \(f(\xv,\thetav^*)=\log (q^{(t)}(\xv)/p(\xv))\) for any \(\xv\in\operatorname{supp}(q^{(t)})\)~\citep{huang2006correcting}. The proof is in the appendix.

To learn all tasks at once, assume the number of users exposed to item \(t\) is proportional to \(m^{(t)}\) and \(\sum_t m^{(t)}=1\).
We sample a random user \(\xv\) by first sampling her/his task assignment \(t\propto m^{(t)}\) and then sampling \(\xv\sim q^{(t)}(\xv)\).
The construction of contrastive pairs naturally extends from the single-task case.
In our implementation, \(f(\xv,\ev^{(t)},\thetav)\) is a deep neural network parameterized by \(\thetav\), and each task is characterized by a vector \(\ev^{(t)}\), \ie the task embedding.
We demonstrate our learning framework in Algorithm~\ref{alg:mtocc} as \(\textproc{EstimateCLR}(M,\ev)\), where \(M\) is the number of tasks and \(\ev\) stands for a task embedding initialization.
We use random initialization when setting \(\ev=\textproc{None}\), which is typically not the best choice in practice.
We will discuss better initialization strategies in Section~\ref{sec:task_embeddings}.

\section{Item Embeddings Initialization}\label{sec:init_embeddings}
Using random initialization for CTR models is typically not the best choice in practice.
\rh{why?}
In this section, we focus on \textit{learning} a tailored item embedding \(\widehat{\vv}_m\) that can be used in initialization of (\ref{eq:ctr}), given the pre-embedding model on impression data.
We seek item embeddings with the following properties:
\begin{enumerate}
    \item (\textbf{Correlation}) Similar to~\cite{meyerson2020traveling}, we find that randomly initialized item embeddings do not fully exploit item similarities during training, even with fine-tuning.
    To overcome this, the learned initialization should capture item similarity.
    \item (\textbf{Generalization}) The learned item embedding models can generalize to cold start items and new items which are absent when training the pre-embedding model.
    \item (\textbf{Efficiency}) Since each item \(m\) can be identified as a distribution \(q_m(\uv)\),
    a natural measurement of item similarity is to calculate their distributional distances.
    However, this requires an enormous number of samples to approximate all pair-wise distances.
    By contrast, the acquisition of learned item embeddings should be inexpensive.

\end{enumerate}

Suppose there are \(M\) items in total, in which $|\mathcal{V}^w|$ are warm start items that has at least \(T\) user clicks.
We propose to learn the item embedding \(\widehat{\vv}_m\) by first training a \emph{pre-embedding model} on warm start items, and then use the pre-embedding model to encode all items.
In practice, we find that randomly selecting \(M_0\ll |\mathcal{V}^w|\) \emph{seed items} from the warm start items are representative enough to capture correlations among all items.
This is the case, presumably, due to the redundancy among items where the inherent complexity of semantics may not be that big.
We study the choice of \(M_0\) in Section~\ref{sec:exp}.

\subsection{Train pre-embedding model with seed items}
Without loss of generality, seed items are identified as item \(1\) to \(M_0\).
We train a log-likelihood ratio task to get the pre-embedding model by calling \(\psi,\ev=\textproc{EstimateCLR}(M_0)\) in Algorithm~\ref{alg:mtocc}, where \(\psi\) denotes the parameters of the pre-embedding model (not related to the final CTR model).
The returned vector \(\ev=\ev_1,\ldots,\ev_{M_0}\) and the pre-embedding model \(\psi\) is sent to the next step.

\subsection{Obtaining task embeddings initializations}
Given the pre-embedding model \(\psi\), the following basis functions are constructed,
\begin{equation}\label{eq:response_vec}
    \vr(\uv)=\left(\vr_1(\uv), \ldots, \vr_{M_0}(\uv)\right),\nonumber
\end{equation}
where \(\vr_m(\uv)=\log(q_m(\uv)/p(\uv))\) is the log-likelihood ratio of the pre-embedding model \(\psi\).
The item embedding \(\widehat{\vv}_m\) is defined as a linear projection of \(q_m(\uv)\) onto these basis functions, \ie,
\begin{align}\label{eq:moh_embeeding_vec}
\widehat{\vv}_m = \left(\mathbb{E}_{q_m(\uv)}[\vr_1(\uv)],\ldots, \mathbb{E}_{q_m(\uv)}[\vr_{M_0}(\uv)]\right).
\end{align}
We independently draw \(N_m\) samples \(\uv_1,\ldots,\uv_{N_m}\) from \(q_m(\uv)\) to estimate \(\widehat{\vv}_m\), where each entry is estimated by \(\mathbb{E}_{q_m(\uv)}[\vr_{m^\prime}(\uv)]\approx\frac{1}{N_m}\sum_{n=1}^{N_m} f(\uv_n,\ev_{m^\prime},\psi)\).
This step only requires forward passing and does not involve any model updates.

After obtaining the learned item embedding initializations \(\widehat{\vv}=(\widehat{\vv}_1,\ldots,\widehat{\vv}_M)\), we can run the CTR model.
This completes the algorithm.

\subsection{Interpreting learned embeddings}

To informally explain why the item embedding initialization \(\widehat{\vv}_m\) in~(\ref{eq:moh_embeeding_vec}) captures item correlations, we treat \(q_m(\vu)\) and \(r_{m^\prime}(\uv)\) as infinite-dimensional vectors by varying \(\uv\) across its support.
Random-projection theory guarantees that \(\text{sim}(\widehat{\vv}_m,\widehat{\vv}_{m^\prime})\approx\text{sim}(q_m,q_{m^\prime})\), when \(M_0\rightarrow\infty\) with random regular basis functions \(\vr(\uv)\) and properly chosen similarity metrics~\cite{gottlieb2015nonlinear}.
In practice, naively generated random functions may not work well, since the random function must capture the difference of \(q_m\) and \(q_{m^\prime}\) on their support.
Likelihood ratios are more reasonable to cover all supports, while still being discriminative since they are learned by contrastive pairs on the population support \(\text{supp}(p(\uv))\).
In our experiments, we found the number of basis functions \(M_0\) can be much smaller than the total number of items \(M\) when tasks are correlated and redundant.
As we will show in experiments, the learned embedding can also generalize to even previously unseen items.

\section{Conclusion and Future Work}\label{sec:conclusion}
In this paper, we studied a series of correlated anomaly detection problems with exposure data.
The key to jointly learn these AD tasks is to share information (collaborate) across them by explicitly or implicitly sharing data among tasks.
Our study finds that using conditional likelihood ratio estimation with the learned task embeddings can efficiently perform CAD on MovieLens 1M with over thousands of tasks.
We further studied the CIFAR10 dataset with a massive number of correlated yet different tasks, demonstrating that the proposed combination is superior to other possible choices, when data for each task is scarce, and its generalization ability.

The current work has a few limitations, which can be studied in future work.
For example, the choice of seed tasks can be more selective than random picking.
A full-fledged CAD may adapt to streaming data where new users and items come in real-time.

\section{Task Embeddings Initialization}\label{sec:task_embeddings}

In this section, we focus on \textit{learning} a tailored task embedding \(\widehat{\ev}^{(t)}\) that can be used in initialization of Algorithm~\ref{alg:mtocc}.
We seek task embeddings with the following properties:
\begin{enumerate}
    \item (\textbf{Correlation})

    The learned initialization should capture task similarity.~\citep{meyerson2020traveling}
    \item (\textbf{Generalization}) The learned task embedding models can generalize to new correlated tasks.
    \item (\textbf{Efficiency})

    The acquisition of learned task embeddings should be inexpensive.
\end{enumerate}

Suppose there are \(M\) tasks in total.
We propose to learn the task embedding \(\widehat{\ev}^{(t)}\) by first selecting \(M_0\ll M\) \emph{seed tasks} to train a \emph{pre-embedding model}, and then use the pre-embedding model to encode all tasks.
In practice, we find that randomly selected seed tasks are representative enough to capture correlations among all tasks.
This is hypothetically due to the redundancy among tasks where the inherent complexity of semantics may not be that big.
We study the choice of \(M_0\) in Section~\ref{sec:mnist_cifar}.

\paragraph{Train pre-embedding model with seed tasks}
Without loss of generality, seed tasks are identified as task \(1\) to \(M_0\).
We train a log-likelihood ratio task to get the pre-embedding model by calling \(\psi,\ev=\textproc{EstimateCLR}(M_0,\textproc{None})\) in Algorithm~\ref{alg:mtocc}, where \(\psi\) denotes the parameters of the pre-embedding model (not related to the final conditional likelihood ratio estimation model).
The returned vector \(\ev\) is dropped,
and the pre-embedding model \(\psi\) is sent to the next step.

\paragraph{Obtaining task embeddings initializations}
Given the pre-embedding model \(\psi\), the following basis functions are constructed,
\begin{equation}\label{eq:response_vec}
    \vr(\vx)=\left(\vr^{(1)}(\vx), \ldots, \vr^{(M_0)}(\vx)\right),\nonumber
\end{equation}
where \(\vr^{(s)}(\xv)=\log(q^{(s)}(\xv)/p(\xv))\) is the log-likelihood ratio of the pre-embedding model \(\psi\).
The task embedding \(\widehat{\ev}^{(t)}\) is defined as a linear projection of \(q^{(t)}(\xv)\) onto these basis functions, \ie,
\begin{align}\label{eq:moh_embeeding_vec}
\widehat{\ev}^{(t)} = \left(\mathbb{E}_{q^{(t)}(\xv)}[\vr^{(1)}(\xv)],\ldots, \mathbb{E}_{q^{(t)}(\xv)}[\vr^{(M_0)}(\xv)]\right).
\end{align}
We independently draw \(N^{(t)}\) samples \(\xv_1,\ldots,\xv_{N^{(t)}}\) from \(q^{(t)}(\xv)\) to estimate \(\widehat{\ev}^{(t)}\), where each entry is estimated by \(\mathbb{E}_{q^{(t)}(\xv)}[\vr^{(s)}(\xv)]\approx\frac{1}{N^{(t)}}\sum_{n=1}^{N^{(t)}} f(\xv_n,\ev^{(s)},\psi)\).
This step only requires forward passing and does not involve any model updates.

After obtaining the learned task embedding initializations \(\widehat{\ev}=(\widehat{\ev}^{(1)},\ldots,\widehat{\ev}^{(M)})\), we can run the conditional likelihood ratio estimation as described in Section~\ref{sec:cad_alg}, where
 \(\theta,\ev=\textproc{EstimateCLR}(M,\widehat{\ev})\).
This completes the algorithm.

\subsection{Interpret the learned embeddings}

To informally explain why the task embedding initialization \(\widehat{\ev}^{(t)}\) in~(\ref{eq:moh_embeeding_vec}) captures task correlations, we treat \(q^{(t)}(\xv)\) and \(r^{(s)}(\xv)\) as infinite-dimensional vectors by varying \(\xv\) across its support.
Random-projection theory guarantees that \(\text{sim}(\widehat{\ev}^{(t)},\widehat{\ev}^{(t')})\approx\text{sim}(q^{(t)},q^{(t')})\), when \(M_0\rightarrow\infty\) with random regular basis functions \(\vr(\xv)\) and properly chosen similarity metrics~\citep{gottlieb2015nonlinear}.
In practice, naively generated random functions may not work well, since the random function must capture the difference of \(q^{(t)}\) and \(q^{(t')}\) on their support.
Likelihood ratios are more reasonable to cover all supports, while still being discriminative since they are learned by contrastive pairs on the population support \(\text{supp}(p(\xv))\).
In our experiments, we found the number of basis functions \(M_0\) can be much smaller than the total number of tasks \(M\) when tasks are correlated and redundant.
As we will show in experiments, the learned embedding can also generalize to previously unseen tasks.

\section{Experiments}\label{sec:exp}

We start with experiments on the MovieLens 1M dataset to demonstrate that \(i\)) CAD with conditional likelihood ratio estimation achieves much better results compared to conditional density estimation; and \(ii\)) task embedding initialization is crucial.
To further analyze these properties, we design a set of experiments on an image dataset (CIFAR10) with synthesized tasks.
Unless otherwise stated, all experiments are executed on a single NVIDIA Titan Xp GPU with 12,196M Memory.

\subsection{Experiments on MovieLens 1M}\label{subsec:movielens}

\paragraph{Problem setup}
The MovieLens 1M dataset contains approximately \(3,900\) movies and \(6,040\) users, together with over one million ratings~\citep{harper2015movielens},
hence \(3,900\) AD tasks in total, one for each movie.
We ignore all rating scores and only keep the binary information of whether movies are rated (exposed) or not.
We split users into training and testing sets with 8:2 ratio.
The goal for each AD task is to distinguish between ``common users'' (nominal) and ``fresh users'' (anomalous) in the testing set.

\input{tables/movielens_1m_results}

\paragraph{Feature pre-processing}
In MovieLens 1M data, there are four categorical labels for users: gender \((u_g)\), age \((u_a)\), occupation \((u_o)\), and zip-code \((u_z)\); and three categorical features for items: movie id, title, and genre.
We use a deep factorization-machine~\citep{guo2017deepfm} to extract user embeddings \(\xv\) (details in appendix) for training AD models.
The detailed four categorical labels \(u_{n,g},\;u_{n,a},\;u_{n,o},\;u_{n,z}\) for user \(\xv_n\) will be used for testing only.

\paragraph{Nominal and anomalous}
The definition of anomalous varies across domains~\citep{ruff2021unifying}.
We consider point anomaly in the context of recommendation systems, which aims to identify each anomalous user for an item \(t\).
An anomalous user is defined according to a specific user label.
Take the label ``age'' as an example.
Suppose an item \(t\) is exposed to a set of users \(\Xv^{(t)}=\{\xv^{(t)}_1,\ldots,\xv^{(t)}_{m^{(t)}}\}\), where the age of user \(\xv^{(t)}_n\) is \(u^{(t)}_{n,a}\). We identify the most frequent age category \(\hat{u}^{t}_a:=\arg\max_u \sum_{n=1}^{m^{(t)}}\mathbf{1}(u_{n,a}^{(t)}=u)\) as nominal and the least frequent age category \(\check{u}^{t}_a:=\arg\min_u \sum_{n=1}^{m^{(t)}}\mathbf{1}(u_{n,a}^{(t)}=u)\) as anomalous.
The AUC score is calculated on the test set by distinguishing between the nominal and anomalous users only.
Since there are multiple user labels, we may report multiple AUC scores, one for each user label.
In our experiments below, we test on two user labels: ``Age'' with 5 binned categories and ``Occupation'' with 21 categories.

\paragraph{Task filtering}
We remove tasks that are not eligible for AD experiments.
To be precise, we first remove tasks with fewer than 100 exposures.
Among the remaining tasks, we select \(50\%\) of them with the lowest user label histogram entropy, indicating nominal and anomalous are distinguishable.
There are 1,629 remaining tasks after the filtering.

\paragraph{Methods}
We study CAD with the proposed conditional likelihood ratio estimation method and two conditional density estimation methods, conditional Gaussian and conditional normalizing flows.
For conditional likelihood ratio estimation, we implement a 3-layer MLP, where task embeddings are concatenated to the input.
For Gaussian, we use a 3-layer MLP with two output heads to map the task embedding to Gaussian mean and variance.
For normalizing flows, we follow~\citep{papamakarios2017masked} to stack \(5\) blocks,
where each block is a stack of MADE~\citep{germain2015made} and BatchNorm.
For all methods, we train until their losses do not decrease, and select the best checkpoints on validation and report results on testing data.
Implementation details are put in
appendix.

The AUC results are shown in Table~\ref{table:movielens} (left).
Conditional density estimation completely fails in the CAD scenario, on both ``Age'' labels and on ``Occupation'' labels.
Normalizing flows are much more flexible than Gaussian, yet still not able to discriminate the anomalous.
This is because density estimation usually requires large data and model capacity for learning.
When conditioning on too many tasks, density estimation may fail to distinguish the complex correlations among tasks.
This phenomenon will be further inspected in Section~\ref{sec:mnist_cifar}.

\paragraph{Task embedding initialization}
In conditional likelihood ratio estimation, task embedding \(\ev^{(t)}\) plays a critical role for data sharing across tasks, since learning one task can enhance the learning of another related task with a similar task embedding.
We compare the following choices of task embedding initialization.
\(i\)) \textit{Graph Embedding}: use learned movie features through a knowledge graph base using KTUP~\citep{cao2019unifying} to initialize \(\ev^{(t)}\).
\(ii\)) \textit{Histogram Embedding}: when training on age labels, we calculate the histogram of users' age labels for each task \(t\): \(\{u^{(t)}_{1,a},\ldots,u^{(t)}_{m^{(t)},a}\}\) to initialize \(\ev^{(t)}\).
This is similarly done for occupations.
\(iii\)) \textit{Random Initialization}: use normal distribution to randomly initialize \(\ev^{(t)}\).
\(iv\)) \textit{Learned Embedding}: initialize \(\ev^{(t)}\) following the procedure in Section~\ref{sec:task_embeddings} with \(M_0=100\) seed tasks. We use the same training scheme for all initialization methods. The training detail can be found in the
appendix.

As shown in Table~\ref{table:movielens} (right), task embeddings with additional knowledge (marked with \(*\)) are generally superior over embeddings without such knowledge.
Learned embeddings are significantly better than random embeddings, and even comparable with graph embeddings, since they attempt to uncover the connections underlying over \(1,600\) tasks using seed tasks.

\paragraph{Downstream tasks}
 So far we have shown that our likelihood ratio can tell the anomaly data from the nominal one. It is a good measure of the familiarity between users and items. We could use this familiarity to diversify the recommendation (explore) or fit the user's current preference (exploit). After collecting feedback like users' reactions, satisfaction rate, or advertisements' revenue, we can re-estimate familiarity and do the next round of recommendations.
Even though this pipeline is nearly impossible to simulate given the current public dataset and evaluation metrics, we do believe that the familiarity between user and item is a necessary and meaningful signal to reflect the user behavior or item property. For more scenarios to use CAD, please check the Appendix.

\input{tables/cifar10_auc_results}
\input{tables/cifar10_embedding_results_v2}
\vspace{-5pt}
\subsection{Experimental studies on CIFAR10}\label{sec:mnist_cifar}

The difficulty of CAD stems from the large number of correlated tasks.
In this section, we synthesize AD tasks using the CIFAR10 dataset~\citep{Krizhevsky09learningmultiple} to gain further understandings of the two phenomena: \(i\)) why conditional likelihood ratio estimation leads to better CAD results; and \(ii\)) why the learned task embeddings work better than randomly initialized embeddings. To connect with previous recommendation setting, we can treat each task as a ``user'', the images in each task as ``items'' exposed to this user. The union of items in all tasks make up the whole dataset, which is in accordance with the recommendation that all the users share the one item set.

\paragraph{Problem setup}
Without further specification, the experiments below use the default training testing split for CIFAR10.
We further split out \(5,000\) samples out of \(60,000\) training samples as a validation set.
CIFAR10 has ten ground-truth categories.
We will select a subset of categories, called ``active categories'', to expose to a task.
For evaluation, given the task index, we select all the samples from its active category as the nominal data and treat the remaining data as anomalous.
We report the average AUC for all tasks on the test set.

\paragraph{CAD with increasing complexity}
We artificially create a sequence of experiments indexed by  \(1\leq k\leq 4\) with an increasing number of correlated tasks .
For each task in one experiment, we select \(k\) active categories out of ten.
We enumerate all possible categorical combinations, resulting in \(C_{10}^k\) tasks in total.
Each image is randomly exposed to only one task that is active for training.
The more tasks one has, the fewer samples each task contains.
We prepare four experiments with increasing complexity \(k=1,2,3,4\) on two scenarios: \(i\)) separately learning, all tasks are learned separately; \(ii\)) conditional learning, all tasks are learned jointly with information sharing.

We study CAD with two (conditional) density estimation methods: Gaussian and masked autoregressive flow (MAF,~\citep{papamakarios2017masked})
and compare with two modern AD methods: deep support vector data description (Deep SVDD,~\citep{ruff2018deep}) and anomaly detection with contrasting shifted instances (CSI,~\citep{tack2020csi}). Gaussian and MAF are implemented on 128d features extracted from an unsupervised model since they work better with high-level, dense features (\citep{kirichenko2020normalizing}).

Table~\ref{table:cifar10} (left) shows separately learning results.
There is a sharp performance drop for Gaussian when \(k\) grows since it can only capture one mode of data distribution.
MAF is more flexible than Gaussian, thus performs much better.
The performance of all methods (CSI, SVDD and LRatio) trained with raw images drops quickly when the number of tasks increases since fewer samples are assigned to each task.
Likelihood ratio estimation exploits samples from the base distribution, thus better discriminates nominal and anomalous both for extracted features and raw images.

Table~\ref{table:cifar10} (right) lists the conditional learning results.
Conditional Gaussian is generally as bad as separately trained Gaussian because of limited capacity.
MAF has enough capacity to learn a single task but suffers from large $k$.
Conditional likelihood ratio estimation performs well with the help of good task embedding and information sharing from negative samples.

\paragraph{Task embedding initialization}
We compare learned task embeddings with the following task embedding initialization approaches.

\vspace{-10pt}
\begin{enumerate}[itemsep=0em]
\item \textit{Label Embedding (Label Emb.)}: a binary length \(L\) vector \(\ev^{(t)}\) where \(e^{(t)}_\ell=1\) if and only if \(\ell\) is active for task \(t\). Prior knowledge of active tasks is required.
\vspace{-4pt}
\item \textit{Pseudo Label Embedding (Pseudo Emb.)}: we cluster training data with GMM with \(L\) components.
For each task \(t\), we calculate its empirical distribution on clusters as \(\ev^{(t)}\).\vspace{-4pt}
\item \textit{Random Initialization (Random Init.)}: initialize  \(\ev^{(t)}\) with  \(L\)-dimensional normal distributions.
\item \textit{Learned Embedding (Learned Emb.) (dim=\(M_0\))}: initialize task embeddings with \(M_0\) pre-selected tasks following the procedure in Section~\ref{sec:task_embeddings}.
\end{enumerate}
\vspace{-4pt}
\normalsize
\input{figures/cifar120}
We use the same architecture to integrate task embeddings, and use the training scheme for all initialization methods.
Details can be found in the
appendix.

As demonstrated in Table~\ref{table:toy_result} (left), CAD is generally harder when handling more tasks.
For two tasks \(t,t'\), label distribution of samples
can capture the similarities of tasks precisely, leading to good performance of \textit{label embedding}.
\textit{Pseudo label embedding} tries to approximate label embedding, but it does not perform well when the number of tasks increases dramatically  due to the limitation of GMM.
\textit{Learned embedding} with a moderate number of seed tasks can do much better than GMM, and it achieves almost the same AUC as label embeddings.
\textit{Random initialization} performs extremely poor even after fine tuning, indicting that task embedding initialization is critical in CAD.

\paragraph{Task generalization}
Surprisingly, the learned task embeddings can help identify unseen tasks fairly well without retraining or fine-tuning.
To demonstrate this, we use setting $k = m$ train the pre-embedding and conditional likelihood ratio estimation models.
Then we select another setting $k = n$ for testing.
For each testing task \(t\), its task embedding \(\ev^{(t)}\) comes from the \emph{existing} pre-embedding model.
We apply \(\ev^{(t)}\) to the \emph{existing} conditional likelihood ratio model for testing (active categories {\em vs}.\ the rest).

Table~\ref{table:toy_result} (right) shows the setup, where
\(m,n=2,3,4,5\) are selected as four exclusive sets of tasks (from easy to difficult) for training (indexed by each row) and testing (indexed by each column). We use (\(m\rightarrow n\)) to denote training on task set \(m\) and testing on task set \(n\).
There is a generalization from training tasks to testing tasks since the testing performance is much better than random guessing (AUC = 0.5). The generalization can be attributed to two factors.
First, learned embeddings capture the distribution over new task well. This will be further discussed in the next paragraph.
Second, conditional likelihood ratio estimates can adapt to unseen task embeddings and make reasonable predictions. In general, it is harder to generalize from an easy set of tasks to a difficult set of tasks (\eg, \(2\!\rightarrow\!5\)) than vice versa (\(5\!\rightarrow\!2\)). This is because the easy set of tasks cannot precisely characterize the fine-grained correlations among the set of difficult tasks.
However, this does not mean we can always train on hard tasks and expect to consistently achieve the best performance among all task sets since the result of \(3\!\rightarrow\!2\) is better than \(5\!\rightarrow\!2\), indicating the task set \(3\) is ``closer'' to task set \(2\) in the task semantic space, thus a better generalization performance.

\paragraph{Visualize generalization through task embeddings}
The learned task embeddings are generalizable because they can measure high-level semantics, thus express correlations among different tasks.
We pre-train a task embedding model with all \(252\) tasks where \(k=5\) out of \(10\) categories are active.
Then we use the task embedding model to encode another \(120\) unseen tasks with \(k=3\) active categories.
We find that the learned task embeddings can approximate similarities between those \(120\) new tasks even the pre-trained network has not seen them before.
Figure~\ref{fig:embedding_showcase}(a) shows the ground-truth similarity, measured by the number of overlapping categories between two tasks.
It turns out the similarity are mainly preserved by learned embeddings in (Figure~\ref{fig:embedding_showcase}(b)).
Learned embeddings are robust, since even a few sub-sampled indices (64 indices in Figure~\ref{fig:embedding_showcase}(c) and 10 indices in Figure~\ref{fig:embedding_showcase}(d)) can approximately preserve task similarities.

\section{conditional likelihood ratio estimation}

We start with experiments on the MovieLens 1M dataset to demonstrate that \(i\)) CAD with conditional likelihood ratio estimation achieves much better results compared to conditional density estimation; and \(ii\)) task embedding initialization is crucial.
To further analyze these properties, we design a set of experiments on an image dataset (CIFAR10) with synthesized tasks.
Unless otherwise stated, all experiments are executed on a single NVIDIA Titan Xp GPU with 12,196M Memory.

\subsection{Experiments on MovieLens 1M}\label{subsec:movielens}

\paragraph{Problem setup}
The MovieLens 1M dataset contains approximately \(3,900\) movies and \(6,040\) users, together with over one million ratings~\cite{harper2015movielens},
hence \(3,900\) AD tasks in total, one for each movie.
We ignore all rating scores and only keep the binary information of whether movies are rated (exposed) or not.
We split users into training and testing sets with 8:2 ratio.
The goal for each AD task is to distinguish between ``common users'' (nominal) and ``fresh users'' (anomalous) in the testing set.

\paragraph{Feature pre-processing}
In MovieLens 1M data, there are four categorical labels for users: gender \((u_g)\), age \((u_a)\), occupation \((u_o)\), and zip-code \((u_z)\); and three categorical features for items: movie id, title, and genre.
We use a deep factorization-machine~\cite{guo2017deepfm} to extract user embeddings \(\xv\) (details in appendix) for training AD models.
The detailed four categorical labels \(u_{n,g},\;u_{n,a},\;u_{n,o},\;u_{n,z}\) for user \(\xv_n\) will be used for testing only.

\paragraph{Nominal and anomalous}
The definition of anomalous varies across domains~\cite{ruff2021unifying}.
We consider point anomaly in the context of recommendation systems, which aims to identify each anomalous user for an item \(t\).
An anomalous user is defined according to a specific user label.
Take the label ``age'' as an example.
Suppose an item \(t\) is exposed to a set of users \(\Xv^{(t)}=\{\xv^{(t)}_1,\ldots,\xv^{(t)}_{m^{(t)}}\}\), where the age of user \(\xv^{(t)}_n\) is \(u^{(t)}_{n,a}\). We identify the most frequent age category \(\hat{u}^{t}_a:=\arg\max_u \sum_{n=1}^{m^{(t)}}\mathbf{1}(u_{n,a}^{(t)}=u)\) as nominal and the least frequent age category \(\check{u}^{t}_a:=\arg\min_u \sum_{n=1}^{m^{(t)}}\mathbf{1}(u_{n,a}^{(t)}=u)\) as anomalous.
The AUC score is calculated on the test set by distinguishing between the nominal and anomalous users only.
Since there are multiple user labels, we may report multiple AUC scores, one for each user label.
In our experiments below, we test on two user labels: ``Age'' with 5 binned categories and ``Occupation'' with 21 categories.

\paragraph{Task filtering}
We remove tasks that are not eligible for AD experiments.
To be precise, we first remove tasks with fewer than 100 exposures.
Among the remaining tasks, we select \(50\%\) of them with the lowest user label histogram entropy, indicating nominal and anomalous are distinguishable.
There are 1,629 remaining tasks after the filtering.

\paragraph{Methods}
We study CAD with the proposed conditional likelihood ratio estimation method and two conditional density estimation methods, conditional Gaussian and conditional normalizing flows.
For conditional likelihood ratio estimation, we implement a 3-layer MLP, where task embeddings are concatenated to the input.
For Gaussian, we use a 3-layer MLP with two output heads to map the task embedding to Gaussian mean and variance.
For normalizing flows, we follow~\cite{papamakarios2017masked} to stack \(5\) blocks,
where each block is a stack of MADE~\cite{germain2015made} and BatchNorm.
For all methods, we train until their losses do not decrease, and select the best checkpoints on validation and report results on testing data.
Implementation details are put in
appendix.

The AUC results are shown in Table~\ref{table:movielens} (left).
Conditional density estimation completely fails in the CAD scenario, on both ``Age'' labels and on ``Occupation'' labels.
Normalizing flows are much more flexible than Gaussian, yet still not able to discriminate the anomalous.
This is because density estimation usually requires large data and model capacity for learning.
When conditioning on too many tasks, density estimation may fail to distinguish the complex correlations among tasks.
This phenomenon will be further inspected in Section~\ref{sec:mnist_cifar}.

\paragraph{Task embedding initialization}
In conditional likelihood ratio estimation, task embedding \(\ev^{(t)}\) plays a critical role for data sharing across tasks, since learning one task can enhance the learning of another related task with a similar task embedding.
We compare the following choices of task embedding initialization.
\(i\)) \textit{Graph Embedding}: use learned movie features through a knowledge graph base using KTUP~\cite{cao2019unifying} to initialize \(\ev^{(t)}\).
\(ii\)) \textit{Histogram Embedding}: when training on age labels, we calculate the histogram of users' age labels for each task \(t\): \(\{u^{(t)}_{1,a},\ldots,u^{(t)}_{m^{(t)},a}\}\) to initialize \(\ev^{(t)}\).
This is similarly done for occupations.
\(iii\)) \textit{Random Initialization}: use normal distribution to randomly initialize \(\ev^{(t)}\).
\(iv\)) \textit{Learned Embedding}: initialize \(\ev^{(t)}\) following the procedure in Section~\ref{sec:task_embeddings} with \(M_0=100\) seed tasks. We use the same training scheme for all initialization methods. The training detail can be found in the appendix.

As shown in Table~\ref{table:movielens} (right), task embeddings with additional knowledge (marked with \(*\)) are generally superior over embeddings without such knowledge.
Learned embeddings are significantly better than random embeddings, and even comparable with graph embeddings, since they attempt to uncover the connections underlying over \(1,600\) tasks using seed tasks.

\input{tables/movielens_1m_results}

\section{Problem Formulation}\label{sec:problem}

Given a collection of users \(\mathcal{U}=\{\uv_1,\ldots,\uv_N\}\) and items \(\mathcal{V}=\{\vv_1,\ldots,\vv_M\}\), an event is an interaction between one user and one item. This paper studies two types of binary events: clicks and impressions. In the following, we denote clicks with a $N$-by-$M$ matrix $\mathbf{Y}$, where each entry $y_{nm}\!=\!1$ represents user $n$ clicks item $m$, $y_{nm}\!=\!0$ otherwise. We denote $\mathcal{Y}^+=\{(n,m):y_{nm}=1\}$ as a set of user-item pairs with clicks and $\mathcal{Y}^-=\{(n,m):y_{nm}=0\}$ stands for no-click pairs. When focusing on a single item $m$, we use $\mathcal{Y}^+_m=\{n:y_{nm}=1\}$ to denote the set of users who click item $m$. We use similar notations $z$, $\mathbf{Z}$, and $\mathcal{Z}$ for impressions. Note that a user cannot click on an item without viewing it, thus $\mathcal{Y}^+\subseteq\mathcal{Z}^+$.

The goal of a CTR model is to predict the click probability given user and item embeddings together with context information \(\cv\) (user profile, user's five last clicked items, item targeting, {\em etc}.) as
\begin{align}
p(y_{nm}=1|\uv_n,\vv_m,\cv).
\label{eq:ctr}
\end{align}
We do not assume there is rich context information, as in real scenarios context may be incomplete or missing.
Modern deep CTR models can accurately predict items with abundant user clicks. We denote those as \textit{warm start} items $\mathcal{V}^w=\{m:|\mathcal{Y}_m^+|\geq T_Y, |\mathcal{Z}_m^+|\geq T_Z\}$, where $T_Y$, $T_Z$ is a threshold determined by domain knowledge. In real scenarios, user clicks are sparse for most items. The extreme case consists of \textit{cold start} items $\mathcal{V}^c=\{m:|\mathcal{Y}_m^+|=0, |\mathcal{Z}_m^+| = 0 \}$, for which item embeddings \(\vv_m\) are hard to learn. This paper focuses on improving CTR predictions for items that are not warm (that is, $|\mathcal{Y}_m^+|= 0$ and $|\mathcal{Z}_m^+|\geq 0$), with special attention on cold start items. The idea is to learn the correlation between cold start items and those easy-to-learn warm start items. To achieve this, we encode item correlations through their impression data with a pre-embedding model on warm start items. Before training the CTR model, for each item (including warm start and cold start items) we collect its user impressions to evaluate its similarity to a few pre-selected ``seed items". These similarity scores are transformed into a vector to initialize the item embedding for succeeding CTR training. We summarize our training pipeline in Figure~\ref{fig:embedding_init}. The main technical challenge is to learn a similarity score that reflects the distance between items' impression data distributions, which will be discussed in the next section.

Compared to existing methods for cold start initialization such as WMF~\cite{hu2008collaborative} and DropoutNet~\cite{volkovs2017dropoutnet}, our method has the following advantages. First, it only requires forward passes to initialize cold item embeddings. Second, it does not require context information when impression data is present. Third, it works consistently for both warm start and cold start items for any CTR model so one can seamlessly integrate it to existing systems.

\section{Introduction}

User-item exposures constitute weakly labeled data when compared to strong labels such as clicks, converts ({\em e.g.}, user purchase items), and ratings.
For instance, a user may browse an item without buying it or people may watch a video without leaving a rating.
Though these situations generally do not indicate user preferences toward specific items or movies, the data generated by interactions {\em without action\/} is of interest because it is a reflection of user behavior.
For example, the familiarity between the item and a user, with which
the recommendation system could increase the recommendation diversity and thus improve the user experience~\citep{kunaver2017diversity}.

\begin{figure}[t]
  \centering
\scalebox{0.8}{
  \begin{minipage}[c]{0.48\textwidth}
    \includegraphics[width=\textwidth]{figures/movie_showcase10.pdf}
  \end{minipage}
  }\hfill
  \begin{minipage}[c]{0.48\textwidth}
    \caption{\
       Each movie (task) has audiences with two real-valued features: age and gender.
       The goal is to learn a function for each movie to distinguish between nominal and anomalous audiences in the test set. The third column is an illustration of density estimation over the second column exposure data given the movie. The number under each cartoon portrait on the fourth column is its estimated density, describing the familiarity of this user to the movie.

    }\label{fig:showcase}
  \end{minipage}

    \vspace{-10pt}

\end{figure}

We formulate this familiarity estimation problem as a collection of correlated anomaly detection (AD) tasks.
Figure~\ref{fig:showcase} illustrates the setting using movie recommendation as an example.
Assume that each movie is exposed to audiences with two features: age and gender;
our goal is to determine which users in the test set are ``common'' (\ie nominal) and which users are ``fresh'' (\ie anomalous) for each movie.
In row one of Figure~\ref{fig:showcase}, the movie ``Avengers'' is mostly exposed to young adults, hence preschoolers and the elderly are anomalous to it.
Alternatively, ``Big Hero 6'' is commonly exposed to young audiences, so preschoolers are nominal to it.

Note that the ``age'' and ``gender'' features are for illustration  and evaluation only. These privacy data may not be available in reality. What's more, the items features are usually more complex, {\em e.g.} image or text feature extracted from the pre-trained neural network. Simple statistics like counting over categories are not applicable.

If each movie is treated as a single AD task, then different movies (tasks) have user groups with different exposure patterns.
Given that tasks share the same user pool, we will consider all tasks collectively instead of individually.

There is a plethora of research on the single-task AD problem.
One can estimate the density \(q(\xv)\) of nominal data or a proxy for it~\citep{polonik1995measuring,tsybakov1997nonparametric}, and then set a threshold for the density function to discriminate nominal and anomalous data.
One can also learn a mapping of nominal data from a high-dimensional space to a compact region concentrated around a pre-defined centroid, using kernel functions~\citep{scholkopf2001estimating,tax2004support} or deep neural networks~\citep{ruff2018deep}.
Moreover, outlier exposures are informative to classify nominal {\it vs.} anomalous data~\citep{hendrycks2018deep}.
There are also semi-supervised~\citep{ruff2019deep}, self-supervised~\citep{hendrycks2019using} and fully supervised~\citep{ruff2020rethinking} approaches.

Yet none of the aforementioned methods are tractable to solve all tasks at once when the number of tasks is large.
For example, density-based AD methods seek to estimate density functions for each movie as shown in the third column of Figure~\ref{fig:showcase}.
However, it is inefficient to learn these conditional densities independently, due to the scarcity of exposure data for each task and the linear growth in computation costs with the number of tasks.

We propose Collaborative Anomaly Detection (CAD) to address all tasks simultaneously, where an important observation is that having a massive number of tasks facilitates the training of single tasks.
In CAD, all tasks first learn their initial \emph{task embeddings} to encode the task correlations.
Then, to get the anomaly detection result, we apply the conditional likelihood ratio estimation for all tasks, where the ratio is between the distribution of exposure data from the current task and the distribution of data from all tasks.
We compared our method with baselines such as conditional density estimation with Gaussian models or normalizing flows~\citep{rezende2015variational}.
Our study reveals: \(i\)) likelihood ratio estimation achieves better AD results since it benefits from the full usage of all available data, while being more efficient than density estimation when data are scarce; and
\(ii\)) task embedding initialization is important to effectively exploit task correlations, and we utilize a warm-start strategy with the initial task embedding learned from a small subset of tasks.
With these properties, CAD scales better and
easily adapt to new tasks with limited exposure data. Our problem is motivated from recommendation system, the multi-task anomaly detection setting and the proposed methods can be applied to any other domains.

The remainder of the paper is organized as follows.
Section~\ref{sec:related} briefly reviews related work on AD\@.
Section~\ref{sec:cad} introduces CAD with density estimation and likelihood ratio estimation conditioning on task embeddings.
In Section~\ref{sec:task_embeddings} we discuss how to initialize task embeddings at low cost, and present the entire training procedure.
Empirical results on MovieLens 1M in Section~\ref{sec:exp} show the large efficiency and accuracy gains of CAD relative to naively applying AD for each task.
We also create synthesized tasks on CIFAR10, and test different CAD methods to further study their efficiency and generalization.
Concluding remarks are presented in Section~\ref{sec:conclusion}.

\section{Related Work}\label{sec:related}
Modern AD research has seen significant progress with the utilization of deep learning, where previously common feature mapping implementations such as kernel methods~\citep{scholkopf2001estimating,tax2004support} are replaced by deep networks~\citep{ruff2018deep}.
Deep AD methods either aim to estimate the density of nominal data~\citep{an2015variational,chen2017outlier,huang2019inverse, zisselman2020deep}, or to classify between nominal and anomalous data in semi-supervised, self-supervised or supervised settings~\citep{hendrycks2018deep,hendrycks2019using,ruff2019deep, ruff2020rethinking}.

CAD is a special case of conditional AD, where anomalous data are studied in a context such as time~\citep{gupta2013outlier}, space~\citep{schubert2014local}, or graph structures~\citep{akoglu2015graph}.
CAD studies anomalous data in the context of each item in recommendation systems.
The large number of items and complex latent structures among them make CAD considerably more challenging.

\iffalse
\subsection{Likelihood ratios as proxies of densities}
Estimating a target density \(q(\xv)\) is often challenging, especially when \(\xv\) is in a high-dimensional space, or when there is not sufficient data.
Instead, we estimate the likelihood ratio between \(q(\xv)\) and a {\it base distribution\/} \(p(\xv)\) from which we can sample. In general, the choice of base distribution \(p(\xv)\) significantly impacts the stability and performance.

The idea of replacing densities with likelihood ratios has been broadly used for unnormalized distribution sampling~\citep{li2019adversarial} and variational inference~\citep{mescheder2017adversarial}.
~\citep{ren2019likelihood} uses likelihood ratio estimation between the original and noisy nominal data, to diminish the effect of background for uncertainty estimation.
In the latter, the base distribution results from perturbing samples from the original distribution.
We use a likelihood ratio to detect abnormal data, using a different base distribution.

\begin{wrapfigure}[20]{r}{0.5\linewidth}
  \begin{center}
    \includegraphics[width=0.8\linewidth]{figures/architecture_v3.pdf}
    \caption{Two typical multi-task learning architectures.
    (a) Multiple output heads (MOH) split tasks in the output layer using task-specific heads $\boldsymbol{\phi}^{(t)}$ for each task $t$. (b) Each task $t$ is specified via an embedding $\ev^{(t)}$. There are also options that combine task embeddings to the neural network layers.
    }\label{fig:baseline_framework}
  \end{center}
    \vspace{-5mm}
\end{wrapfigure}

\section{Preliminaries on conditioning}\label{sec:multitask}
\fi

\section{Log-likelihood Ratio Estimation}

\subsection{Proof of Proposition 1 in Section 3.2.}

\begin{proposition}\label{prop:kl}
    \(p(\xv)=\sum_t m^{(t)}q^{(t)}(\xv)\) is optimal for minimizing the expected KL divergence
    \begin{align}\label{eq:min_kl}
        \min_{p(\xv)\in\mathcal{P}}\mathbb{E}[\operatorname{KL}(q^{(t)}|p)] =\min_{p(\xv)\in\mathcal{P}}\sum_t\mathbb{E}_{ q^{(t)}(\xv)}\left[ m^{(t)} \log \frac{q^{(t)}(\xv)}{p(\xv)}\right].
    \end{align}
\end{proposition}

\begin{proof}
Expanding the expectation over tasks. We get
\begin{align}
    \mathbb{E}[\operatorname{KL}(q^{(t)}|p)] = \sum_t m^{(t)} [\operatorname{KL}(q^{(t)}|p)].
\end{align}

We need to solve a minimization problem with constraints:
\begin{align}
     \min_{p(\xv)\in\mathcal{P}}\quad & \sum_t\mathbb{E}_{ q^{(t)}(\xv)}\left[ m^{(t)} \log \frac{q^{(t)}(\xv)}{p(\xv)}\right]\nonumber\\
     \text{s.t.}\quad & \int p(\xv) d \xv = 1.
\end{align}
With the Lagrange multiplier $\lambda$, we need to find function $p(\xv)$ to minimize the Lagrangian
\begin{align}
    \sum_t\mathbb{E}_{ q^{(t)}(\xv)}\left[ m^{(t)} \log \frac{q^{(t)}(\xv)}{p(\xv)}\right] + \lambda\left(\int p(\xv) d\xv  - 1\right).
\end{align}
After removing additive terms unrelated to $p(\xv)$, we get
\begin{align}
    \int\bigg( \sum_t  -m^{(t)}q^{(t)}(\xv) \log p(\xv) + \lambda p(\xv) \bigg) d\xv.
\end{align}

  In the calculus of variations, \(p(\xv)\) is varied by adding a function  \(\delta p(\xv)\) to it with \(\epsilon \) multiplier, where \(\epsilon \rightarrow 0\):
\begin{align}
    \int\bigg(\sum_t -m^{(t)} q^{(t)}(\xv) \log \big(p(\xv) + \epsilon \delta p(\xv)\big) + \lambda (p(\xv)+ \epsilon \delta p(\xv)) \bigg) d\xv .
\end{align}
The change in the value to first order in $\epsilon$ should be zero. By taking derivative w.r.t. $\epsilon$ and let $\epsilon = 0$, we get
\begin{align}
    \int\bigg( \sum_t  -m^{(t)}q^{(t)}(\xv) \frac{\delta p(\xv)}{p(\xv)}  + \lambda (\delta p(\xv)) \bigg) d\xv = 0.
\end{align}

The above equation is valid for any function $\delta p(\xv)$
\begin{align}
    \lambda -  \frac{1}{p(\xv)}\sum_t m^{(t)}q^{(t)}(\xv) = 0.
\end{align}

After reordering terms, we get
\begin{align}
\label{eq:final}
    p(\xv) = \frac{1}{\lambda}\sum_t m^{(t)} q^{(t)}(\xv).
\end{align}

Since $\int p(\xv) = 1$ and $\sum_t m^{(t)} = 1$, taking the integral over $\xv$ on both sides of Equation \ref{eq:final} permits $\lambda = 1$. Finally, we get
\begin{align}
    p(\xv) = \sum_t m^{(t)} q^{(t)}(\xv).
\end{align}

This concludes our proof.
\end{proof}

\subsection{Proof of the optimal solution to Equation 3}
Suppose we have two distributions \(q(\xv)\) and $p(\xv)$, their log-likelihood ratio $r(\xv)=\log(q(\xv)/p(\xv))$ can be estimated as follows.

First, we independently draw $N$ samples from $q(\xv)$ and label those data as positive: $\{\xv_i,y_i=1\}_{i=1}^N$. Similarly we draw $N$ samples from $p(\xv)$ as label those as negative: $\{\xv_i,y_i=-1\}_{i=N+1}^{2N}$. We combine all positive samples and negative samples into one dataset $D=\{\xv_i,y_i\}_{i=1}^{2N}$. In the limit of $N\rightarrow\infty$, the mixed distribution satisfies:
\begin{align}
p(y=1|\xv)=\frac{q(\xv)}{p(\xv)+q(\xv)}.
\end{align}
Then we train a binary classifier on the mixed dataset $D$. Suppose we are using a linear logistic regression model:
\begin{align}
\hat{p}(y\,|\,\mathbf{x};\,\bm\theta) = \sigma(y\cdot\bm\theta^\top\mathbf{x}),
\end{align}
where $\sigma$ is the sigmoid function:
\begin{align}
\sigma(z) = \frac{1}{1+e^{-z}}.
\end{align}
Then
\begin{align}
\log\frac{q(\xv)}{p(\xv)}&=\log\frac{p(y=1|\xv)}{1-p(y=1|\xv)}\nonumber\\
&\approx\log\frac{\hat{p}(y=1|\xv;\,\bm\theta)}{1-\hat{p}(y=1|\xv;\,\bm\theta)}\nonumber\\
&=\log \frac{\sigma(\bm\theta^\top\mathbf{x})}{\sigma(-\bm\theta^\top\xv)}\nonumber\\
&=\bm\theta^\top\mathbf{x}.

\end{align}
One can use a non-linear model implemented with a neural network $f(\xv,\bm\theta)$ to get a better estimation of the log-likelihood ratio.

\section{Conditional Normalizing Flow}
\subsection{Feature extractors.}

A good image generator implemented with normalizing flows is not necessary a good out-of-distribution detector, since normalizing flows tend to capture pixel correlations rather than semantics during training~\citep{kirichenko2020normalizing}. Moreover, normalizing flow models for image generation often requires a large number of parameters and long training time, which is not-applicable when the number of tasks is large.

To resolve above issues, we use pre-trained low-dimensional features instead of raw images to train normalizing flows. The features are extracted using unsupervised contrastive learning~\citep{chen2020simple} with a ResNet-18 as the backbone model. The training accuracy on the extracted features is 90.2\% with a linear classifier, demonstrating high feature quality. The unsupervised learning procedure guarantees that the extracted feature does not leak its label information while preserving image semantics.

\subsection{Two ways to implement conditional normalizing flows.}
In the following, we use \(\ev^{(t)}\) to represent task-specific information, which can involve in the prior \(\pi\) and the invertible mapping \(f\) as follows:
\[
    L = -\log q(\xv|\ev^{(t)}) = -\log\pi_{\ev^{(t)}}(f^{-1}_{\ev^{(t)}}(\xv)) - \log\left|\det\frac{\partial f^{-1}_{\ev^{(t)}}(\xv)}{\partial\xv}\right|.
\]
\begin{paragraph}{Task-specific priors}
For un-conditional normalizing flows, the prior $\pi$ is usually set as a centered isotropic Gaussian.
In our conditional case, each task \(t\) has its task-specific Gaussian prior \(\pi_{\ev_{prior}^{(t)}}\) with a uniformly sampled mean vector and an identity covariance matrix.
\end{paragraph}

\begin{paragraph}{Task-specific mappings}
  MAF uses Masked Autoencoder for Distribution Estimation (MADE) as its building blocks. The core idea behind MADE is autoregressive flow, which generates data recursively by
\[
    x_i = u_i \exp \alpha_i + \mu_i
\]
Where \(\mu_i = f(\xv_{1:i-1}),~\alpha_i = g(\xv_{1:i-1}) \). \(\uv\) is randomly generated from the previous layer or the prior distribution. In our MADE implementation, function \(f\) and \(g\) are both 4-layer autoregressive fully connected layer with masking. The task specific features are combined with the output of each MADE layer by scaling and biasing, i.e. \(f_{\ev^{(t)}} = f \odot \textit{tanh}(\ev_{scale}^{(t)}) + \textit{tanh}(\ev_{bias}^{(t)})\). The hyperbolic tangent function is used to stabilize the training, and it can be replaced with deep neural networks. \(\ev_{scale}^{(t)}\) and \(\ev_{bias}^{(t)}\) are trainable task embeddings.
These parameters are shared among different \(i\) in one layer. The same technique also applies to function \(g\). For more details on MAF, please refer to the origin paper \citet{germain2015made,papamakarios2017masked}.
\end{paragraph}

 Our implementation refers to two Github codebases \footnote{pytorch-normalizing-flows:  \url{https://github.com/karpathy/pytorch-normalizing-flows}.}$^{,}$\footnote{pytorch-flows:  \url{https://github.com/ikostrikov/pytorch-flows}}.

\section{Real scenarios for CAD}

\section{Training Details}
In this section, we give full descriptions of models in our experiments.

\subsection{MNIST and CIFAR10}

\begin{figure}[htb!]
\centering
    \includegraphics[width=.4\textwidth]{figures/mnist_hyp_framework.pdf}\vspace{-10pt}
    \caption{The framework we use to train MNIST and CIFAR10. $\xv$ denotes the input matrix. $\ev$ is the task embedding. FC represents the fully connected layer. The brackets followed by contains the dimension of its input and output. e\_dim is the dimension of the task embedding. The definition of c\_dim and the detail of feature extractor can be found in Table \ref{table:appendix_1}.}
\label{fig:appendix_1}
\bigskip
    \includegraphics[width=.3\textwidth]{figures/movielens_hyp_framework.pdf}\vspace{-10pt}
    \caption{The framework we use to train MovieLens 1M. The definition of f\_dim, e\_dim and the detail of neural network can be found in Table  \ref{table:appendix_2}.}
\label{fig:appendix_2}
\end{figure}

Figure \ref{fig:appendix_1} demonstrates the network structure of the likelihood ratio estimation model ($f(\xv,\ev,\bm\theta)$ in  Algorithm 1)
for MNIST and CIFAR10. We resize all MNIST figures from their original size $28\times28$ to $32\times32$ to align with CIFAR10 figure sizes. The data augmentations include random crop with padding (Pad the image to \(40\times40\) with zeros, then randomly choice a \(32\times32\) area) and random horizontal flip. We normalize pixels for each channel such that their mean equals to zero and variance equals to one.

The pre-embedding model uses a similar feature extractor, followed by a fully connected layer mapping from feature dimension c\_dim*8 to 256. Then $M_0$ fully connected output heads are used to predict the likelihood ratio for $M_0$ seed tasks.

\subsection{MovieLens 1M}
\subsubsection{Feature Extractor}
Unlike CIFAR10 and MNIST where each image has its raw pixel representation, the user feature is implicitly represented by the interaction between movies and users. We treat MovieLens as a click-through rating prediction dataset and use a deep factorization-machine~\citep{guo2017deepfm} to extract user embeddings \(\xv\). After training the factorization model, we use the trained embedding layer to extract features of each user.
During the training process, all users are visible to the model, so we can get the feature of each user. We implement this method using code from \citet{cao2019unifying}.

\subsubsection{Hyper-parameters}
Figure \ref{fig:appendix_2} demonstrates the network structure of the likelihood ratio estimation model ($f(\xv,\ev,\bm\theta)$ in  Algorithm 1) for MovieLens 1M dataset. The pre-embedding model uses the same network structure. The only difference is that we change the way to integrate task embeddings from adding them to the input to adding $M_0$ output heads in the pre-embedding model.

\begin{table}[h]
\centering
\caption{The network structure of feature extractor for CIFAR10 and MNIST. Each operation contains four parts: (kernel size, stride, padding size), number of output channel, batch normalization (BN) and activation function. In our setting, we use c\_dim = 64. The output size is a three-dimensional vector: (width, height, channel). The input channel is 3 for CIFAR10  and 1 for MNIST.}

\begin{tabular}{|c|l|c|}
\hline
Stage & \multicolumn{1}{c|}{Operation}              & Output Size  \\ \hline
Input &   \multicolumn{1}{c|}{Data Augmentation}       & (32,32,3 or 1)    \\ \hline
1     & (4, 2, 1), c\_dim, BN, ReLU  & (16, 16, c\_dim) \\ \hline
2     & (4, 2, 1), c\_dim*2, BN, ReLU  & (8, 8, c\_dim*2)   \\ \hline
3     & (5, 1, 0), c\_dim*4, BN, ReLU & (4, 4, c\_dim*4)  \\ \hline
4     & (4, 1, 0), c\_dim*8, BN,       & (1,1, c\_dim*8)   \\ \hline
\end{tabular}
\label{table:appendix_1}
\end{table}

\begin{table}[h]
\centering
\caption{The network structure of logistic regression for MovieLens 1M. Each operation contains three parts: (input size, output size) of fully connected layer, activation function and dropout rate. f\_dim represents the dimension of input data. e\_dim is the dimension of the task embedding. We use f\_dim=100, e\_dim = 16.}
\begin{tabular}{|c|l|}
\hline
Stage & \multicolumn{1}{c|}{Operation}                   \\ \hline
1     & (f\_dim + e\_dim, 32), ReLU, Dropout (0.5)              \\ \hline
2     & (32, 32), ReLU, Dropout (0.5)                            \\ \hline
3     & (32, 16), ReLU, Dropout (0.3)                            \\ \hline
4     & (16, 1), Linear , No Dropout                                               \\ \hline
\end{tabular}
\label{table:appendix_2}
\end{table}

\section{More results on MNIST}
Similar as Table 3 (left) in the main text for the CIFAR10 dataset, we study different task embedding initialization methods (see Section 5.2 in the main text for definitions of those task embeddings) on the MNIST dataset with two CAD settings where $k=4$ with $C_{10}^4=210$ tasks, and $k=5$ with $C_{10}^5=252$ tasks. We skip the setting with $k=2$ and $k=3$ since those settings are too simple such that every method we tried can get almost $100\%$ AUC\@. The results is shown in Table \ref{table:toy_result_mnist}.

\begin{table*}[h
  \caption

    Testing AUC\;(\(\times\)100) on MNIST.  Label embedding uses prior knowledge (ground truth label) so it has a natural advantage comparing with other methods (Note that the first method ``label embedding'' uses extra knowledge and is marked with *. The best results without extra knowledge are shown in bold.).

  }\label{table:toy_result_mnist}
  \begin{center}
    \begin{small}
      \begin{sc}

        \begin{tabular}{ccc}
          \toprule

          \multicolumn{1}{c|}{Embedding Init.~\textbackslash~K (\#tasks)}            & 4 (210)                    & 5 (252)      \\
          \midrule
          \multicolumn{1}{c|}{Label Embedding*}       & 99.88                      &99.67      \\
          \multicolumn{1}{c|}{Pseudo Label Embedding} & 97.88                      & 98.67  \\

          \multicolumn{1}{c|}{Random Initialization}  & 97.02                      & 86.80 \\
          \multicolumn{1}{c|}{Learned Embedding ($M_0 = 10$)}        & 98.88                      & 99.67      \\
          \multicolumn{1}{c|}{Learned Embedding ($M_0 = 64$)}        & \textbf{99.93}             & \textbf{99.90} \\

          \bottomrule
        \end{tabular}
      \end{sc}
    \end{small}

  \end{center}
\end{table*}

redpajama_set_name: RedPajamaArXiv


MADRID, Spain -- Argentine forward Lionel Messi will need surgery on a foot injury sustained in Sunday's 3-1 Primera Liga win over Real Zaragoza that will sideline him for up to three months.
"Messi has broken the fifth metatarsal in his left foot. He will need surgery and the estimated time out will be three months," Barcelona confirmed on their Web site.
The 19-year-old international limped off in the 23rd minute of Sunday's game after tackles from Zaragoza midfielders Alberto Zapater and Albert Celades in a frenetic opening period.
Messi will miss Barca's remaining Champions League Group A matches against Levski Sofia and Werder Bremen. The European champions are third in the group.
He will also been unable to compete in December's World Club Cup in Japan.
Messi's loss is a major blow to the Primera Liga leaders with Cameroon striker Samuel Eto'o still recovering from knee surgery and not back until the new year.
Barca were dealt a further blow later on Monday when striker Javier Saviola joined their lengthening injury list.
"Saviola has torn a muscle in his left thigh. We predict a recovery period of between four and six weeks," said a club spokesman.
The 24-year-old Argentine scored Barca's third goal against Zaragoza after coming on as a second-half substitute.
The pressure will now be on Ronaldinho, Eidur Gudjohnsen, Ludovic Giuly and Santi Ezquerro to cover Barca's three front positions into the new year.
Added to his problems in attack, coach Frank Rijkaard has Brazilians Edmilson and Luciano Belletti out for around 10 days, while midfielder Xavi has not yet returned to action since sustaining a hairline fracture of his tibia.
Messi's absence is a major blow for Barca following-on from Samuel Eto'o's injury.


ENDS WITH 2.0 AND BEGINS WITH KABALI!
Superstar Rajinikanth has begun shooting for Shankar's monumental project 2.0 (Enthiran 2). The first schedule of 2.0 which kick-started last Thursday (17th December) is expected to be wrapped up in a few days.
Rajinikanth is likely to join the sets of Kabali once again by the first week of January 2016. Looks like Superstar ends the year with 2.0 and then begins the new year with Kabali. Reports suggest that close to a month of shoot is still left and the post production work will begin by February.
The audio launch date will supposedly be announced once the shoot is completed. The makers are apparently confident of releasing the film in April 2016.
People looking for online information on 20, Enthiran 2, Kabali, Rajinikanth, Superstar will find this news story useful.


Since 1990, Centurion American has developed well over 25,000 single-family lots in dozens of premier communities surrounding North Texas. We have demonstrated the ability to effectively work with investors, landowners, financial institutions, and vendors to acquire over 20,000 acres of land inventory, resulting in a diverse mix of developments varying in size and scope. Delivering award-winning communities with impeccable amenities such as parks, golf courses, water park themes, and hiking and biking trails, Centurion has successfully designed master-planned communities that have been recognized across the real estate industry.
Recreational facilities are at the heart of many of the communities we've developed in Texas and Colorado. We work closely with local municipalities, commercial developers, and public school systems as part of the overall master plan. Our established relationships with the area's top builders allow us to implement the latest design concepts ranging from upscale, luxury homes in secluded neighborhoods to affordable housing communities for first-time home buyers.
Centurion has set the standard in purchasing and developing land in prime locations with the right combination of natural land settings, strong job growth, good school systems, and access to local community shopping. Over the past 25 years, it is this proven track record of achievements that has consistently created excellent value and ROI for our partners, investors, builders, and homeowners.


text:
Transaction Rumors & Proposals
Astros interested in Lohse or Garza
Did you just find a juicy new offseason trade or free agent rumor? DON'T POST IN THIS FORUM! Please click on this banner and then "Add Rumor". Thanks!
By Ulice Payne,
July 1, 2015 in Transaction Rumors & Proposals
Ulice Payne
The Houston Chronicle reports the Astros have at least preliminary interest in Lohse or Garza.
http://reviewingthebrew.com/2015/06/30/milwaukee-brewers-rumors-astros-have-interest-in-kyle-lohse-and-matt-garza/
I'm sure the Cardinals already knew this.
Don't see Garza going anywhere even if he gets any interest. No team will trade for the whole contract and the Brewers aren't the kind of team to throw in money.
I could see the Brewers throwing in enough to get a trade done. I don't see them throwing in enough to get a decent prospect back.
As long as we have enough minor league arms with enough innings left on them to get us through this year, one of Lohse/Garza has to go, but I don't think I want both to go unless everyone is healthy.
pogokat
If the Brewers are really rebuilding, getting rid of these two and getting back as much money as possible without losing any prospects is crucial. The team has to strip down as close to nil as it can as quick as it can to be competitive sooner.
Mudville9
Well played...
Formerly Uecker Quit Usingers
CheezWizHed
Brewer Fanatic Contributor
Boy, I've not been paying attention...
I was wondering why the Astros would want either of these pitchers. Then I saw they are in first place in their division. Nice turn around for that franchise. I knew they had a good young core, but I'm surprised to see them put it together so fast.
craigharmann
Exactly this. The Astros have no need for garbage. Keuchel is a stud, McCullers has held his own, McHugh has been up and down and Valasquez has been great for a guy jumping from AA. Feldman is coming back and he's a 4/5 at best. They've got the parts to go for Cueto or Kazmir (and I think he's the one ultimately, because he's from Houston), but if they're going for it this year, which I hope they do, they need good pitchers, not middle of the road pitchers who are either underperforming or on the downside of their career.....or both.
I saw a rumor that there was interest in Gomez by the Astros. I'm not quite sure why they would do that, because Marisnick, Springer and Rasmus are all capable of playing CF. And most of the time, those three have comprised the outfield. They either need to get Production from Carter or Singleton, and that will go a long way offensively. Correa coming up has been amazing. He's as good, if not better than advertised.
SHIP THEM ASAP EVEN IF WE GET A LOW LEVEL PROSPECT
superfly
Carter is a bum currently, but has upside. He's the kind of guy we should be targeting if they really do want a guy like Garza or Lohse. Sure he'll never be anything special, but he's a guy with power that can fill the void at 1B for a couple years until the long term solution is ready. Maybe we catch lightning in a bottle and he figures it all out too.
Singleton is better yet, but I don't know the Stros well enough to know how fed up they are with him. He's a possible long term 1B solution at 23 years old.
Carter can either crush the ball, or spectacularly strike out. Branyanesque in that way. I've seen it firsthand. Singleton was basically handed first last year, and couldn't hit his way out of a paper bag. I'm hoping he figures it out. I just can't see the Astros being remotely interested in Lohse or Garza. They're looking for winners, not gutty gamers or gutsy battlers.
Chris Carter has at least a 100 OPS+ all of the last 3 seasons....far far from a bum. Strikeout galore, but still puts up some numbers.
That being said I don't see adding him with a growing salary. Wouldn't make a lot of sense on a rebuilding team.
The problem is that McCullers and Velasquez are going to run into innings limits in August, especially Velasquez since he was hurt and missed half the season last year and the first month of this year. Then what?
The appeal of Lohse is that they won't have to give up much to get him. I don't know that they mortgage the future this year, with their core being so young. Garza being signed for two more years is a non-rental, unlike Cueto and Kazmir, who could be gone after the season.
Mr Southpaw
How about J.D. Davis for Lohse?
JohnBriggs12
Astros just put George Springer on DL with broken wrist. Maybe the Brewers could package Lohse with Parra and get back a decent AA pitcher, say lefty Brian Holmes. I think the only way they can dump some of Lohse's salary is to include him in a package deal.
I just saw that. If there's anyone on the Brewers roster I would want to have on the Astros to replace George, it would be Gomez. But they've got some good young guys and Rasmus and Marisnick are almost back, so I bet for the short term, they'll stick with the guys they've got.
Twitter: @ryanclarence
One of Lohse or Garza needs to be gone by time Wily is back. Jungmann has been the best and most consistent pitcher, then Fiers, Wily and Jimmy are locked in.
Proud member since 2003 (geez ha I was 14 then)
FORMERLY BrewCrewWS2008 and YoungGeezy don't even remember other names used
bill hAll Star
I wonder if they are willing to play Gattis or Carter out there anymore. Gattis has logged 20 innings in the OF this year and Carter has not played any.
LF of either Gattis or Carter with Marisnick/Rasmus CF platoon and Tucker/Santana RF platoon is serviceable, I guess. Parra would probably help but I'm not sure how much.
jerichoholicninja
A coat rack with a glove taped to it could play a passable LF in Minute Maid.
I think Parra would be a great get for them. Gomez is going to cost more than they should be willing to give up at this point in their rebuild.
Gomez is going to cost more than they should be willing to give up at this point in their rebuild.
Rebuild? Last time I checked, the Astros are leading the AL west, halfway through the season. I wouldn't say they're in a rebuild. In fact, when it comes to prospects, they're loaded. Daz Cameron, the 37th pick in the draft, was ranked high by many scouts. They're stocked. They could give up pieces. And the Brewers need a ton of pieces. The Brewers need a trade with the Astros more than the Astros need a trade with the Brewers.
They haven't won anything yet, hence, rebuild. They are on the upswing. There is no reason to start trading young guys at this point.
Khris Davis would be the perfect left fielder for the Astros. That short wall would help his bad arm. Package him with Lohse or Garza and get a decent return.
The Astros don't need a LF. Springer plays RF. He's out.they've got Tucker and Rasmus to play left. In the minors they have L.J. Hoes and Alex Pressley (who was called up to replace Springer). Domingo Santana has been good, too. He's another young guy. They've got plenty of guys. They need another top flight starter and production for first base. They should get that from Carter or Singleton.
I've said it before, Houston would be quite dumb to trade away any of their prized young prospects this very first year of seeing success. None from their top 15 team prospects should be available in trade. Their FAs to be are Colby Rasmus, Chad Qualls if option isn't picked up, Roberto Hernandez, Tony Sipp, and Joe Thatcher. Then you are looking at Scott Feldman, Luis Valbuena, and Jason Castro the following season. Neshek has to turn in to Broxton to not pick up his option.
There isn't One single AS in that group the team will be losing. To make any sort of quality trades, they will be losing AS potential in return for rentals? The owner should fire the GM if he ends up trading away any top 10 team prospect at complete lack of future vision. Unless its for Multiple years of a quality SP but certainly not a rental.
Yes. I could see them trading for a pitcher, but I could just as much see them stand pat. However, they've got a lot of guys at AA and AAA that could lead to 40 man issues come the offseason, so if they can package a couple for lower level guys and/or a stud pitcher, it's possible.
CheeseheadInQC
Francisco Castro
Matthew Trueblood
RedStickBrew
bjkrautk
jjfanec

redpajama_set_name: RedPajamaCommonCrawl


Other solutions for general audio and video problems are listed in the Fix common audio and video issues article. Düşüncelerinizi paylaşmak için oturum açın. Start now ^Back to top Was this page helpful?
Bu özellik şu anda kullanılamıyor. Adobe Support > Flash Player > Common problems Common problems Search Flash Player User Guide > On this page 1. Click Manage add-ons. Why Won't Adobe Flash Player Work On My Mac Click Manage Website Settings.
You can also uninstall Flash altogether on Mac and Windows by following those links, but you'll save yourself a lot of potential time and trouble--and reap the same benefits--by making strategic, Adobe Flash Player Not Working Mac ClearTechInfo 221.302 görüntüleme 2:08 How To Download & Install Adobe Flash Player in Mozilla Firefox - Süre: 4:19. jobs4mind.com - Work at Home Job Portal 32.349 görüntüleme 2:42 How to fix crashes from adobe flash player in google chrome browser - Süre: 3:27. https://helpx.adobe.com/flash-player/kb/installation-problems-flash-player-windows.html Still having problems?
ESRepair 457.895 görüntüleme 13:58 Download Adobe Flash Player & Java FREE - Windows xp/vista/7/8 - Süre: 8:14. Adobe Flash Player Application Initialization Error At least, until Occupy Flash--and its friends at Facebook and elsewhere--finally have their way. Follow Follow UsOn Youtube Don't miss out on WIRED's latest videos. Ne edi be yea srɔ alesi naɖe articlewo gɔme na SUMO la, meɖekuku dze egɔme le afii.
The point is clear, though: Flash is officially more trouble than it's worth. Or select Run and type dxdiag.exein the text entry box that appears. Flash Player Not Working Chrome If you would like to view your driver version number, select Graphics and Displays.
Submit your problem report, including the Graphics and system information file, to Adobe byfollowing these steps. Flash Player Not Working Windows 10 Applies to: Flash Player 1.
A green line splits the player in two. If Flash content does not work, acts strangely, causes errors, or if you see a notice that you need to update Flash (for instance, at Mozilla's Plugin Check) even after installing A black or white box replaces the video. my site Here we go, broken down by browser.
Caution: Adobe's download page may include a checkbox for optional software (such as Google Chrome or McAfee Security Scan) that is selected by default. Bu videoyu Daha Sonra İzle oynatma listesine eklemek için oturum açın Ekle Oynatma listeleri yükleniyor... Lis Tutos 485 görüntüleme 3:20 How to install adobe flash player web plug-in - Süre: 2:40. Chrome Plugins Change regionUnited States (Change) Choose your region Selecting a region changes the language and/or content on Adobe.com.
Ame nyui siawo kpeɖe miaŋu mie ŋlɔ article sia: AliceWyman, Chris_Ilias, philipp, underpass, novica, Tonnes, Verdi, scoobidiver, Swarnava, Joergen, ideato, upwinxp, pollti, jsavage, Artist, marksc. If you are experiencing video playback issues in Flash Player, your video adapter driver software may have incompatibility issues. When the download has finished, close Firefox. dig this We check the ClearTechInfo.com Tutorial / Blog comments and forum postings every day.
Type the description of your problem in the search box on theFlash Player forumto search online discussions. Woha ateŋu ana kpeɖeŋu - di alesi na wɔe. Submit Thank You Invalid Email Follow UsOn Twitter 9 hours This brewery pumps out 5,000 cold ones an hour bit.ly/2iDZfNN Follow Follow UsOn Facebook Don't miss our latest news, features and To check if your installation of Flash is up to date, visit Mozilla's Plugin Check page.


text:
Q: Check if a particular string is contained in a list of strings I'm trying to search a string to see if it contains any strings from a list,
var s = driver.FindElement(By.Id("list"));
var innerHtml = s.GetAttribute("innerHTML");

innerHtml is the string I want to search for a list of strings provided by me, example
 var list = new List<string> { "One", "Two", "Three" };

so if say innerHtml contains "One" output Match: One

A: You can do this in the following way:
int result = list.IndexOf(innerHTML);

It will return the index of the item with which there is a match, else if not found it would return -1.
If you want a string output, as mentioned in the question, you may do something like:
if (result != -1)
    Console.WriteLine(list[result] + " matched.");
else
    Console.WriteLine("No match found");

Another simple way to do this is:
string matchedElement = list.Find(x => x.Equals(innerHTML));

This would return the matched element if there is a match, otherwise it would return a null.
See docs for more details.

A: You can do it with LINQ by applying Contains to innerHtml for each of the items on the list:
var matches = list.Where(item => innerHtml.Contains(item)).ToList();

Variable matches would contain a subset of strings from the list which are matched inside innerHtml.
Note: This approach does not match at word boundaries, which means that you would find a match of "One" when innerHtml contains "Onerous".

A: foreach(var str in list)
{
    if (innerHtml.Contains(str))
    {
        // match found, do your stuff.
    }
}

String.Contains documentation

A: For those who want to serach Arrray of chars in another list of strings
List WildCard = new() { "", "%", "?" };
List PlateNo = new() { "13eer", "rt4444", "45566" };
if (WildCard.Any(x => PlateNo.Any(y => y.Contains(x))))
Console.WriteLine("Plate has wildchar}");

redpajama_set_name: RedPajamaStackExchange