text:
\subsection{Existing OD Algorithms and Scalability}
\label{subsec:od_algorithms}

Outlier detection (also known as anomaly detection) is a key machine learning task that aims to find data points that deviate from a general distribution \cite{aggarwal2015outlier}.
As shown in Table \ref{table:algorithms}, key OD algorithms can be grouped into four categories:
 (\textit{i}) proximity-based algorithms that rely on measuring sample similarity including $k$NNOD \cite{Angiulli2002fast}, ABOD \cite{kriegel2008angle},
 COF \cite{DBLP:conf/pakdd/TangCFC02},
 LOF \cite{Breunig2000}, and LOCI \cite{DBLP:conf/icde/PapadimitriouKGF03}; (\textit{ii}) statistical approaches including
 KDE \cite{DBLP:conf/sdm/SchubertZK14},
 HBOS \cite{goldstein2012histogram}, and COPOD \cite{DBLP:conf/icdm/LiZBIH20};
 (\textit{iii}) ensemble-based methods that build a collection of detectors for aggregation like LODA \cite{pevny2016loda},
 and LCSP \cite{zhao2019lscp}; and (\textit{iv}) linear models such as PCA \cite{shyu2003novel}.

Importantly, many OD algorithms suffer from scalability issues \cite{zhao2021suod}. For example, Table \ref{table:algorithms} shows that various proximity-based OD algorithms have time and space complexities that are at least $O(n^2)$---they all require estimating and storing pairwise distances among all $n$ samples.
The high time complexities of many OD algorithms limit their applicability in real-world applications that require either real-time responses (e.g., financial fraud detection) or the concurrent processing of millions of samples.
As shown in Table~\ref{table:algorithms}, {\sc TBOD}\xspace optimizes nearly all the existing OD algorithms mentioned.

\subsection{DNN Infrastructure and Acceleration}
\label{subsec:dl-infrastructure}

Deep neural networks have dramatically improved the accuracy of artificial intelligence systems across numerous fields \cite{goodfellow2016deep}. Its success is fueled by recent advances in both hardware and software~\cite{lecun20191}. Specifically, DNN systems depend on tensor operations that can often be parallelized and executed in small batches. These operations are well-suited for GPUs, especially as a single GPU nowadays often has many more cores than a single CPU; while GPU cores are not as general purpose as CPU cores, they suffice in executing the tensor operations of deep learning. Moreover, the maturity of DNN programming interfaces such as PyTorch \cite{paszke2019pytorch} and TensorFlow \cite{abadi2016tensorflow} makes developing machine learning models easy with a wide range of GPUs.

With the above considerations, we build {\sc TBOD}\xspace using the DNN ecosystem, taking advantage of its established hardware acceleration and software accessibility. This design choice also opens the opportunity for unifying classical OD algorithms (see \S \ref{subsec:od_algorithms}) and DNN-based OD algorithms on the same platform---this emerging direction has gained increasing attention in OD research \cite{ruff2019deep}.

\subsection{Outlier Detection Systems}
\label{subsec:od-sys}

Over the years, comprehensive OD systems on CPUs that cover a diverse group of algorithms have been developed in different programming languages, including ELKI Data Mining \citep{Achtert2010visual} and
RapidMiner \citep{Hofmann2013rapidminer} in Java, and PyOD \citep{DBLP:journals/jmlr/ZhaoNL19} in Python. Among these, PyOD is the state-of-the-art (SOTA) with deep optimization including just-in-time compilation and parallelization. It is widely used by both academia and industry works, with hundreds of citations \cite{pyodcite} and millions of downloads per year \cite{pyodpepy}. Recently, the PyOD team proposed an acceleration system called SUOD to further speed up the training and prediction in PyOD with a large collection of heterogeneous OD models \cite{zhao2021suod}. Specifically, SUOD uses algorithmic approximation and efficient parallelization to reduce the computational cost and therefore runtime. There are also other parallel systems designed for specific (family of) OD algorithms---Parallel Bay and Parallel LOF can scale up proximity-based OD algorithms in multi-core computer systems \cite{lozano2005parallel, oku2014parallel}.

Other than improving system efficiency on CPUs, there are efforts to use GPUs for fast OD calculations \cite{DBLP:journals/tpds/AngiulliBLS16,DBLP:journals/sigkdd/LealG18}. These approaches rely on exploring the characteristics of a specific OD algorithm for GPU acceleration. This limits their generalization to a wide collection of OD algorithms. Furthermore, none of them has multi-GPU support, leading to limited scalability. To our best knowledge, there is no existing comprehensive GPU-based OD system.
Therefore, we consider the SOTA comprehensive system, PyOD, as the primary baseline.

\subsection{Definition and Problem Formulation}
A comprehensive OD system implements a collection of OD models $\mathcal{M}=\{M_1,...,M_m\}$ such that given a user-specified OD model $M \in \mathcal{M}$ and input data $\mathbf{X} \in \mathbbm{R}^{n \times d}$ without ground truth labels (rows of $\mathbf{X}$ index data points, while columns index features), the system outputs outlier scores $\mathbf{O} := M(\mathbf{X}) \in \mathbbm{R}^{n}$, which should roughly be deterministic
irrespective of the underlying system (higher values in $\mathbf{O}$ correspond to data points that are more likely to be considered outliers; thresholding on outlier scores can be used to determine which points are outliers). Given the hardware configuration $\mathcal{C}$, e.g., CPU, RAM, and GPU (if available), the system performance can be measured in \textit{efficiency} (both time and space complexities).

\subsection{System Overview}
As a reminder from Section~\ref{sec:intro},
{\sc TBOD}\xspace is a comprehensive OD system as outlined in Fig.~\ref{fig:overview}. For an outlier detection task, {\sc TBOD}\xspace abstracts it into a combination of predefined tensor operators via the proposed \textit{programming model} for direct GPU acceleration (\S \ref{sec:model_abstraction}). Notably, the predefined tensor operators are optimized with \textit{provable quantization} that achieves faster computation and smaller memory usage while provably maintaining operator accuracy (\S \ref{sec:quantization}). To overcome the limit of a single GPU, we further introduce \textit{automatic batching} and \textit{multi-GPU} support in {\sc TBOD}\xspace (\S \ref{sec:batching}).

\subsection{Algorithmic Abstraction}
\label{subsec:algo_abstract}

The key idea of our programming model is to decompose existing OD algorithms into a set of low-level {\em basic tensor operators} (BTOs), which can directly benefit from GPU acceleration. On top of these BTOs, we introduce higher-level OD operators called {\em functional operators} (FOs) with richer semantics. Consequently, OD algorithms can be constructed as combinations of BTOs and FOs.

Fig.~\ref{fig:operators} shows the hierarchy of {\sc TBOD}\xspace's programming abstraction in a bottom-up way, with increasing dependency: 8 BTOs are first constructed as the foundation of {\sc TBOD}\xspace (shown at the bottom in gray), while 6 FOs are then created on top of them (shown in the middle). Finally, OD algorithms and key functions on the top of the figure can be assembled by BTOs and FOs. In other words, {\sc TBOD}\xspace uses a tree-structured dependency graph, where the BTOs (as leaves of the tree) are fully independent for deep optimization, and all the algorithms depending on these BTOs can be collectively optimized. Clearly, this abstraction process reduces the repetitive implementation and optimization effort, and improves system efficiency and generalization. Additionally, it also facilities fast prototyping and experimenting with new algorithms.

\subsection{Building Complete Algorithms}
\label{subsec:complete_algorithm}

It is easy to build an end-to-end OD application by constructing its {\em computation graph} using FOs and BTOs. For instance, angle-based outlier detection (ABOD) is a classical OD algorithm \cite{kriegel2008angle}, where each sample's outlier score is calculated as the average cosine similarity of its neighbors.
Fig.~\ref{fig:abstraction_examples}(a) highlights an implementation of ABOD that uses an FO called \texttt{$k$NN} to obtain a list of neighbors for each sample and then applies another FO called \texttt{cosine sim.} for calculating cosine similarity. Note that the FOs are also built as combinations of BTOs. For example, \texttt{$k$NN} is implemented as calculating pairwise sample distance using \texttt{cdist} and then identifying the $k$ ``neighbors" with smallest distances using \texttt{topk}. Additionally, Fig.~\ref{fig:abstraction_examples}(b) shows the computation graph of copula-based outlier detection (COPOD) \cite{DBLP:conf/icdm/LiZBIH20}.

\begin{figure}[t]
\centering
\subfloat[Abstraction of ABOD]{%
  \includegraphics[clip,width=0.5\columnwidth]{figures/abod.pdf}%
}
\subfloat[Abstraction of COPOD]{%
  \includegraphics[clip,width=0.5\columnwidth]{figures/copod.pdf}%
}
\vspace{-1em}
\caption{Examples of building complex OD algorithms with FO and BTO conveniently.}
\label{fig:abstraction_examples}
\vspace{-1em}
\end{figure}

\subsection{$(1+\epsilon)$-property for Rounding Error Calculation}

Let $\mathbb{F}$ denote the set of 64-bit floating-point numbers. For $x,y\in\mathbb{F}$, we define the floating-point operation ``$\circledast$'' as $x \circledast y \triangleq \text{fl}(x *y)$, where  $* \in \{+, -, \times, /\}$ and $\text{fl}(\cdot)$ refers to the IEEE 754 standard for rounding a real number to a 64-bit floating-point number \cite{kahan1996ieee}. For example, $\oplus$ is floating-point addition and $\otimes$ is floating-point multiplication.
The standard technique for calculating the rounding errors in floating-point operations is the $(1+\epsilon)$-property \cite{DBLP:journals/pacmpl/Lee0A18}, which is formally defined as follows.
\begin{theorem}[Theorem 3.2 of~\citealt{DBLP:journals/pacmpl/Lee0A18}]
Let $x,y\in\mathbb{F}$, and $*\in\{+,-,\times,/\}$. Suppose that $|x*y|\le\max\mathbb{F}$. Then when we compute $x*y$ in floating-point, there exist multiplicative and additive error terms $\delta\in\mathbb{R}$ and $\delta'\in\mathbb{R}$ respectively such that
\begin{equation}
x\circledast y=(x*y)(1+\delta)+\delta',\quad\text{where }|\delta|\le\epsilon,|\delta'|\le\epsilon'.
\end{equation}
In the above equation, $\epsilon$ and $\epsilon'$ are constants that do not depend on $x$ or $y$. For instance, when working with 64-bit floating-point numbers, $\epsilon=2^{-53}$ and $\epsilon'=2^{-1075}$.
\label{theorem:quantization}
\end{theorem}

As discussed by \citet[Section~5]{DBLP:journals/pacmpl/Lee0A18}, this property can be further simplified when the exact result of the floating operation is not in the so-called ``subnormal'' range: the addictive error term $\delta'$ can be soundly removed, leading to a simplified $(1+\epsilon)$-property:
\begin{equation}
    x \circledast y = (x * y)(1+\delta),
    \quad
    \text{where }|\delta|\le\epsilon.
\label{equ:epsilon}
\end{equation}

\subsection{Provable Quantization in {\sc TBOD}\xspace}
\label{subsec:apply_pq}
{\sc TBOD}\xspace applies provable quantization for an applicable operator $r(\cdot)$ with input $x$ in three steps: (\textit{i}) input quantization (\textit{ii}) low-precision evaluation and (\textit{iii}) exactness verification.
In a nutshell,
the input is first quantized into lower precision, and then the operator is evaluated in the lower precision, where $r(\cdot)$ is evaluated as $r_q(x)$. To verify the exactness of the quantization, we calculate the rounding error $\text{Err}(r_q(x))$ by the simplified $(1+\epsilon)$-property in eq. \eqref{equ:epsilon} and then check whether the result of $r(\cdot)$ may change with the rounding error. If it passes the verification, then we use quantization; otherwise, we use the original precision of $x$.

\textbf{Remark.}
Not all the operators are applicable for provable quantization---they need meeting the following criteria: (1) the output values are not in a floating-point representation and (2) the performance gain in low-precision evaluation is larger than the overhead of verification. Based on these criterion, we mark the operators that are generally applicable for provable quantization by $\bigstar$ in Fig.~\ref{fig:operators}.

\subsection{Case Study: Neighbors Within Range}
\label{subsec:nwr}
We show the usage of provable quantization in {\sc TBOD}\xspace on neighbors within range (\texttt{NWR}, one of the FOs of our programming model), a common step in many OD algorithms, e.g., LOF \cite{DBLP:conf/sigmod/BreunigKNS00} and LOCI \cite{DBLP:conf/icde/PapadimitriouKGF03}. \texttt{NWR} identifies nearest neighbors within a preset distance threshold (usually a small number), which may be considered as a variant of $k$ nearest neighbors. More formally, given an input tensor $\mathbf{X}:=[X_1, X_2, ...,X_n] \in \mathbb{R}^{n \times d}$ ($n$ samples and $d$ dimensions) and the  distance threshold $\phi$, \texttt{NWR} first calculates the pairwise distance among each sample via the \texttt{cdist} operator, yielding a distance matrix $\mathbf{D} \in \mathbb{R}^{n\times n}$, where $\mathbf{D}_{i,j}$ is the pairwise distance between $X_i$ and $X_j$. Then, each pairwise distance in $\mathbf{D}$ is compared with $\phi$, and \texttt{NWR} outputs the indices of samples where $\mathbf{D}_{i,j} \leq \phi$. As the pairwise distance calculation in \texttt{NWR} requires $O(n^2)$ space, provable quantization can provide significant GPU memory savings.

\texttt{NWR} meets the criterion we outline in \S \ref{subsec:apply_pq}, where eq.~\eqref{equ:epsilon} can be applied to estimate the rounding errors. Recall that the pairwise Euclidean distance between two samples is
\begin{equation}
\mathbf{D}_{ij} = \norm{X_i-X_j}^2_2=\norm{X_i}_2^2+\norm{X_j}_2^2-2X_i^T X_j.
\label{eq:dist-decomp}
\end{equation}
We shall compute this in floating-point. Importantly, in our analysis to follow, the ordering of floating-point operations matters in determining rounding errors. To this end, we calculate distance using the right-most expression in eq.~\eqref{eq:dist-decomp} (note that we do \emph{not} first compute the difference $X_i-X_j$ and then compute its squared Euclidean norm). When calculating the first term in the RHS of eq.~\eqref{eq:dist-decomp} via floating-point operations, we get
\begin{align*}
    &\text{fl}(\norm{X_i}_2^2)\\
    &\!=\!(X_{i,1}\otimes X_{i,1})\oplus(X_{i,2}\otimes X_{i,2})\oplus\cdots\oplus(X_{i,d}\otimes X_{i,d})\\
    &\!=\!(X_{i,1}^2(1\!+\!\delta_1))\oplus(X_{i,2}^2(1\!+\!\delta_2))\oplus\cdots\oplus(X_{i,d}^2(1\!+\!\delta_d)),
\end{align*}
where the second equality uses eq.~\eqref{equ:epsilon} and we note that the errors $\delta_1,\dots,\delta_d$ across the floating-point multiplications (for squaring) need not be the same (in fact, these need not be the same across samples $i=1,2,\dots,n$ but we omit this indexing to keep the equation from getting cluttered).

Next, by defining $x_{\max}\triangleq\max_{i\in\{1,\dots,n\},k\in\{1,\dots,d\}}|X_{i,k}|$ and recalling from Theorem~\ref{theorem:quantization} that each of $\delta_{1},\delta_{2},\dots,\delta_{d}$ above is at most $\epsilon$, we get
\begin{align*}
 & \text{fl}(\norm{X_i}_2^2)\\
 & \!=\!(X_{i,1}^2(1\!+\!\delta_1))\oplus(X_{i,2}^2(1\!+\!\delta_2))\oplus\cdots\oplus(X_{i,d}^2(1\!+\!\delta_d))\\
 & \!\le\!\underbrace{[x_{\max}^{2}(1+\epsilon)]\oplus[x_{\max}^{2}(1+\epsilon)]\oplus\cdots\oplus[x_{\max}^{2}(1+\epsilon)]}_{d\text{ terms added via floating-point addition}}\\
 & \!\le\!d\cdot x_{\max}^{2}(1+\epsilon)^{1+\lceil\log_{2}d\rceil},
\end{align*}
where for the last step, $\log_2 d$ shows up since summation of $d$ elements in lower-level programming languages is implemented in a divide-and-conquer manner that reduces to $\lceil\log_2 d\rceil$ operations (there is still a ``$1+$'' term in the exponent for the floating-point multiplication/squaring). The rounding error can be bounded as follows:
\begin{align*}
\text{Err}(\|X_{i}\|_{2}^{2}) & =\|X_{i}\|_{2}^{2}-\text{fl}(\|X_{i}\|_{2}^{2})\\
 & \le d\cdot x_{\max}^{2}-\text{fl}(\|X_{i}\|_{2}^{2})\\
 & \le\big|d\cdot x_{\max}^{2}-\text{fl}(\|X_{i}\|_{2}^{2})\big|\\
 & \le d\cdot x_{\max}^{2}[(1+\epsilon)^{1+\lceil\log_{2}d\rceil}-1].
\end{align*}
This same analysis can be used to bound the floating-point errors of the other terms in eq.~\eqref{eq:dist-decomp}. Overall, we get
\begin{equation}
    \text{Err}(\mathbf{D}_{ij}) \leq 4d \cdot  x_{\text{max}}^2 [(1+\epsilon)^{1+\lceil\log_2 d\rceil+2} - 1],
\label{eq:nwr-check}
\end{equation}
where the ``$+2$'' shows up in the exponent due to the addition and subtraction in the RHS of~\eqref{eq:dist-decomp} that we compute in floating-point.

Inequality~\eqref{eq:nwr-check} provides a numerical way for checking whether a single entry $\mathbf{D}_{ij}$ is within the range of $\phi$ as $|\mathbf{D}_{ij}-\phi| > \text{Err}(\mathbf{D}_{ij})$. More conveniently, we could scale the input  $\mathbf{X}$ into the range of $[0, 1]$ before the distance calculation \cite{raschka2015python}, so that $x_{\text{max}} \leq 1$ and the implementation complexity can be further reduced. With this treatment, a large amount of GPU memory can be saved in \texttt{NWR} operations (see \S \ref{subsec:prov_quant_exp} for results).

\subsection{Trivial Batching}

\textbf{Operators without global dependency.} An operator can be directly executed in small batches when it is either: (\textit{i}) \textit{sample-independent}: the estimation of each sample is independent, or (\textit{ii}) \textit{feature-independent}: the contribution of each feature is independent. Fig.~\ref{fig:trivial_batching} shows that when either of these conditions holds, the computations of an operator can be directly partitioned into multiple batches by splitting along the sample or feature  dimension, and the operator is then executed on the batches for final aggregation. For instance, \texttt{topk} is a sample-independent operator. For an input tensor $\mathbf{X} \in \mathbb{R}^{n\times d}$, {\tt topk} outputs the indices of the largest $k$ values for each sample (i.e., row), resulting in an output tensor $\mathbf{I}_{\mathbf{X}}:= \texttt{topk}(\mathbf{X}, k) \in \mathbb{R}^{n\times k}$. Therefore, we could split $\mathbf{X}$ into batches with full features, each with $b$ samples (i.e.,, $\{\mathbf{X}_{1}, \mathbf{X}_{2}, ...\}  \in \mathbb{R}^{b\times d}$). As another example, \texttt{Histogram} outputs the frequencies of each feature's values, which is feature-independent. Thus, we could partition $\mathbf{X}$ into blocks of $b$ features, e.g., $\{\mathbf{X}_{1}, \mathbf{X}_{2}, ...\} \in \mathbb{R}^{n\times b}$.

\begin{figure}[t]
\centering
  \includegraphics[width=0.48\textwidth]{figures/row_dependency.pdf}
  \vspace{-2em}
  \caption{Direct batching with independence assumption create batches along the sample or feature index.}
 \label{fig:trivial_batching}
\vspace{-1em}
\end{figure}

\textbf{Operators with global dependency.}
Not all operators satisfy the independence structures discussed above. For example, \texttt{SVD} performs singular value decomposition. It is neither sample- nor feature-independent and cannot be directly batched.

Using our algorithmic abstraction (see \S \ref{sec:model_abstraction}), {\sc TBOD}\xspace provides customized batching solutions for a small number of BTOs with global dependency. For instance, as shown in Fig.~\ref{fig:cdist_simple}, {\sc TBOD}\xspace uses the approach introduced in~\citet{neeb2016distributed} to batch  \texttt{cdist}, where an input dataset is first split by samples, and then {\sc TBOD}\xspace calculates the pairwise distance of \textit{each pairs of split} in batches.

\begin{figure}[t]
\centering
  \includegraphics[width=0.48\textwidth]{figures/cdist_simple.pdf}
  \vspace{-2em}
  \caption{Customized batching solution for cdist in {\sc TBOD}\xspace.}
 \label{fig:cdist_simple}
\vspace{-1em}
\end{figure}

\subsection{Sequential Batching and Operator Fusion}
\label{subsec:operator_fusion}
\textbf{Simple concatenation}. Since BTOs are independent from each other, executing a sequence of BTOs in batches is straightforward, i.e., simply feed the output of a batch operator as an input to another one. For example, \texttt{$k$NN} (see Fig.~\ref{fig:operators}) finds the $k$ nearest neighbors by first calculating pairwise distances of input samples via the \texttt{cdist} BTO and then returns the index of $k$ items with the smallest distance via the \texttt{topk} BTO.
Therefore, batching for \texttt{$k$NN} is achieved by running \texttt{cdist} and \texttt{topk} sequentially, where each uses automatic batching and the output of the former is the input of the latter.

\textbf{Operator fusion}. Although the simple concatenation discussed above is straightforward, a closer look unlocks deeper optimization opportunities in automatic batching with a sequence of operators. Notably, the output of \texttt{$k$NN} is the indices of the $k$ nearest neighbors of an input dataset, where the pairwise distance generated by \texttt{cdist} is only used in an intermediate step but not returned.
If we could prevent moving this large distance matrix between operators (i.e., \texttt{cdist} and \texttt{topk}), space efficiency can be improved.
In deep learning systems, {\em operator fusion} is a common optimization technique to fuse multiple operators into a single one in a computational graph \cite{DBLP:conf/nips/NeubigGD17,DBLP:conf/iclr/LooksHHN17,DBLP:conf/osdi/WangZGMTZLRCJ21}.
Inspired by this, {\sc TBOD}\xspace opportunistically fuses a sequence of operators in automatic batching whenever possible. Fig.~\ref{fig:knn_direct_fusion} compares simple concatenation (subfigure a) and operator fusion (subfigure b) on \texttt{$k$NN}. Specifically, the latter executes the \texttt{topk} BTO on the \texttt{cdist} BTO's individual batches separately rather than running \texttt{topk} on the full distance matrix outputted by \texttt{cdist}. Note that the global $k$ nearest neighbors (of the full dataset) can be identified from the $k$ local neighbors candidates from batches in the final aggregation, so the result is still exact. This prevents moving the entire $n \times n$ distance matrix between operators, which often causes OOM.

\begin{figure}[t]
\subfloat[Simple concatenation: \texttt{topk} is invoked on the full result of \texttt{cdist}---we need to move the distance matrix $\mathbf{D}$.]{%
  \includegraphics[clip,width=\columnwidth]{figures/knn_direct.pdf}%
}

\vspace{-1em}
\subfloat[Operator fusion: \texttt{topk} is directly invoked on the batch result of \texttt{cdsit}, preventing the move of distance matrix $\mathbf{D}$.] {%
  \includegraphics[clip,width=\columnwidth]{figures/knn_fusion.pdf}%
}
\vspace{-1em}
\caption{The comparison of automatic batching for $k$NN between simple concatenation and operator fusion. The latter has better scalability by not creating and moving large distance matrix $\mathbf{D}$}
\label{fig:knn_direct_fusion}
\vspace{-1em}
\end{figure}

\subsection{Multi-GPU Support}
With automatic batching, we address the memory  limitation of a single GPU by making {\sc TBOD}\xspace compatible with multiple GPUs, an idea that has shown significant efficiency improvements in applications like graph neural networks \cite{DBLP:conf/mlsys/JiaLGZA20}.
Intuitively, if there is only one GPU, {\sc TBOD}\xspace iterates across multiple batches sequentially and aggregates the results.
When multiple GPUs are available, we could achieve better performance by executing OD computations concurrently on multiple GPUs.
Specifically, {\sc TBOD}\xspace first applies automatic batching to an underlying task---multiple subtasks are created and assigned to available GPUs.
{\sc TBOD}\xspace creates a {\em subprocess} for each available GPU to execute the assigned subtasks and a {\em shared global container} to store the results returned from each GPU.
Since we equally distribute subtasks across GPUs, we deem the runtime of each GPU is close.
Once all the subtasks are complete, the final output is generated by aggregating the results in the global container. For example, automatically batching \texttt{cdist} in Fig.~\ref{fig:cdist_simple} leads to 4 subtasks, each of which calculates the pairwise distances for a pair of splits (denoated as blue blocks in the figure).
Each of the four available GPUs executes an assigned subtask and sends the \texttt{cdist} results to the global container. Finally, the \texttt{cdist} result on the entire dataset is obtained by aggregating the intermediate results in the global container.

Note that the multi-GPU execution is at the operator level (e.g., \texttt{cdist}). To prevent the overhead caused by creating subprocesses repetitively, {\sc TBOD}\xspace leverages the operator fusion descried in \S \ref{subsec:operator_fusion} whenever possible.

\subsection{Implementation and Experiment Environment}
{\sc TBOD}\xspace is implemented on top of PyTorch \cite{paszke2019pytorch}. We extended PyTorch in the following aspects to support efficient OD. First, we implemented a set of BTOs and FOs (see Fig.~\ref{fig:operators}) for fast tensor operations in OD. Second, for operators that support batching (as described in \S\ref{sec:batching}), we created batched versions of them to improve scalability. Additionally, we enabled specialized multi-GPU support in {\sc TBOD}\xspace by leveraging PyTorch's \texttt{multiprocessing}.

All the experiments were performed on an Amazon EC2 cluster with an Intel Xeon E5-2686 v4 CPU, 61GB DRAM, and an NVIDIA Tesla V100 GPU. For the multi-GPU support evaluation, we extend it to multiple NVIDIA Tesla V100 GPUs with the same CPU node.

\subsection{Datasets, Baselines, and Evaluation Metrics}
\textbf{Datasets.} Appendix Table \ref{table:data} shows the six real-world benchmark datasets used in this study, which are widely evaluated in OD research \cite{zhao2019lscp,ruff2019deep,DBLP:conf/icdm/LiZBIH20,zhao2021suod}. Given the limited size of real-world OD datasets, we use the data generation function in PyOD to create larger synthetic datasets to evaluate the scalability of {\sc TBOD}\xspace.

\textbf{Baselines.} As discussed in \S \ref{sec:related_work}, there is no existing GPU system that covers a diverse group of OD algorithms. Therefore, we use the SOTA CPU-based OD system PyOD \cite{DBLP:journals/jmlr/ZhaoNL19} as a baseline, which is deeply optimized with just-in-time compilation and parallelization. Additionally, we compare to a direct implementation in PyTorch when appropriate (as a GPU baseline).

\textbf{OD algorithms and operators.} Throughout the experiments, we compare the performance of five representative OD algorithms across different systems (see \S \ref{subsec:od_algorithms}):
proximity-based algorithms including LOF, ABOD, and $k$NNOD; statistical method HBOS, and linear model PCA. We also provide an operator level analysis to demonstrate the effectiveness of certain techniques.

\textbf{Evaluation metrics.} Since {\sc TBOD}\xspace and the baselines do not involve any approximation, the output results are exact and consistent across systems. Therefore, we omit the accuracy evaluation, and compare the wall-clock time and GPU memory consumption as measures of time and space efficiency.

\begin{figure*}[t]
\centering
\subfloat[LOF (quadratic)]{%
  \includegraphics[clip,width=0.45\columnwidth]{figures_exp/scalability_LOF.pdf}%
}
\subfloat[ABOD (quadratic)]{%
  \includegraphics[clip,width=0.41\columnwidth]{figures_exp/scalability_ABOD.pdf}%
}
\subfloat[HBOS (linear)]{%
  \includegraphics[clip,width=0.41\columnwidth]{figures_exp/scalability_HBOS.pdf}%
}
\subfloat[$k$NNOD (quadratic)]{%
  \includegraphics[clip,width=0.41\columnwidth]{figures_exp/scalability_KNN.pdf}%
}
\subfloat[PCA (linear)]{%
  \includegraphics[clip,width=0.41\columnwidth]{figures_exp/scalability_PCA.pdf}%
}

\caption{Scalability plot of the selected algorithms in {\sc TBOD}\xspace, where the actual runtime (in black) is close to the expected runtime (in red).}
\label{fig:scalability}
\vspace{-1em}
\end{figure*}

\subsection{End-to-end Speed Evaluation: {\sc TBOD}\xspace vs PyOD}
\label{subsec:end_2_end}
We first present the runtime comparison between {\sc TBOD}\xspace and PyOD using two real datasets (fashion-mnist and cifar-10; see  Fig.~\ref{fig:full_system_evaluation}) and two larger synthetic datasets (see Appendix Fig.~\ref{fig:full_system_synthetic}).
The results show that {\sc TBOD}\xspace is on average 11.90$\times$ faster than PyOD on the five benchmarks (14.80$\times$, 21.05$\times$, 7.43$\times$, 11.21$\times$, and 5.04$\times$ speed-up on LOF, $k$NNOD, ABOD, HBOS, and PCA, respectively).
For proximity-based algorithms, a larger speed-up is observed for datasets with a higher number of dimensions: LOF and $k$NNOD are 28.09$\times$ and 38.93$\times$ faster on cifar-10 with 3,072 features. This is expected as GPUs are well-suited for dense tensor multiplication, which is essential in proximity-based methods. Separately, a larger improvement can be achieved for HBOS and PCA on datasets with larger sample sizes. For instance, HBOS is 11.83$\times$ faster on Synthetic 1 (100,000 samples), while the speedup is 17.16$\times$ on Synthetic 2 (200,000 samples). This is expected as HBOS treats each feature independently for density estimation on GPUs, so a large number of samples with a small number of features should yield a large speed-up. In summary, all the OD algorithms tested are significantly faster in {\sc TBOD}\xspace than in the SOTA PyOD system, with the precise amount of speed improvement varying across algorithms.

\begin{figure*}[t]
\centering
\subfloat[\texttt{Neighbor within range} (real-world datasets)]{%
  \includegraphics[clip,width=\columnwidth]{figures_exp/nwr_quant_real.pdf}%
}
\subfloat[\texttt{Neighbor within range} (synthetic datasets)]{%
  \includegraphics[clip,width=\columnwidth]{figures_exp/nwr_quant_sync.pdf}%
}\\
\vspace{-0.5em}
\caption{GPU memory consumption comparison between using provable quantization (16-bit and 32-bit) and the full precision (64-bit). Clearly, provable quantization leads to significant memory consumption saving on \texttt{nwr} and \texttt{topk} (see Appendix Fig~.).}
\label{fig:quantization_evaluation}
\end{figure*}

\subsection{Scalability of {\sc TBOD}\xspace}
\label{subsec:scalability}
We now gauge the scalability of {\sc TBOD}\xspace on datasets of varying sizes, including ones larger than fashion-mnist and cifar-10.
In Fig.~\ref{fig:scalability}, we plot {\sc TBOD}\xspace's runtime with five OD algorithms on the synthetic datasets with sample sizes ranging from 50,000 to 1,500,000 (all with 200 features). To the best of our knowledge, none of the existing OD systems can handle datasets with more than a million samples within a reasonable amount of time \cite{aggarwal2015outlier,zhao2021suod}, as most of the OD algorithms are associated with quadratic time complexity.
Fig.~\ref{fig:scalability} shows that the five OD algorithms in {\sc TBOD}\xspace tested can process million-sample datasets within an hour, and the actual runtime is close to the expected runtime.
Thus, {\sc TBOD}\xspace provides a scalable approach for enabling OD in many real-world applications.

\subsection{Provable Quantization}
\label{subsec:prov_quant_exp}
As discussed in \S \ref{sec:quantization}, provable quantization in {\sc TBOD}\xspace can optimize the operator memory usage while provably preserving correctness (i.e., no accuracy degradation). To demonstrate its effectiveness, we compare the GPU memory consumption of two applicable operators, \texttt{neighbor within range (nwr)} and \texttt{topk}, with and without provable quantization. Multiple real-world and synthetic datasets are used in the comparison (see Appendix Table \ref{table:data}), where synthetic datasets' names, such as ``10k" and ``1M", denote their sample sizes. We deem the 64-bit floating-point precision as the ground truth, and evaluate the provable quantization results in 32- and 16-bit floating-point.

The results demonstrate that provable quantization always leads to memory savings. Specifically, Fig.~\ref{fig:quantization_evaluation}  shows \texttt{nwr} with provable quantization on average saves 71.27\% (with 16-bit precision) and 47.59\% (with 32-bit precision) of the full 64-bit precision GPU memory.
Similarly, Fig.~\ref{fig:quantization_topk} in Appendix shows that \texttt{topk} with provable quantization saves 73.49\% (with 16-bit precision) and 49.58\% (with 32-bit precision) of the full precision memory. Regarding the runtime comparison, operating in lower precision may also lead to an edge. Table \ref{table:quantization} shows the operator runtime comparison between using provable quantization (in 16-bit and 32-bit precision) and using the full 64-bit precision. It shows that provable quantization in 16-bit precision is faster than the computations in full precision in most cases.
This empirical finding can be attributed to lower-precision operations typically being faster, and this speed improvement outweighing the overhead of post verification. Note that 32-bit provable quantization often takes the longest time---it is slower than 16-bit provable quantization in operation, and the post verification overhead makes it slower than 64-bit operation. These results demonstrate the effectiveness of using provable quantization in lower precision, especially in 16-bit, leading to both memory and computation time savings.

\begin{table*}[t]
\centering
	\caption{Comparison of operator runtime (in seconds) with provable quantization (i.e., 16-bit and 32-bit) and without quantization (i.e., 64-bit). The best model is highlighted in bold (per column), where provable quantization in 16-bit outperform in most cases.
	\vspace{.25em}\footnotesize
\begin{tabular}{l|llllll|llllll}
        \toprule
       \multicolumn{7}{c}{\texttt{nwr}}   & \multicolumn{6}{c}{\texttt{topk}}   \\
       \midrule
\textbf{Prec.}       & mammog. & mnist & musk & 10k  & 20k  & 30k & cifar-10 & f-mnist & speech & 1M & 2M & 5M \\
\midrule
\textbf{16-bit} & 2.66        & \textbf{0.54}  & \textbf{0.06} & \textbf{0.87} & \textbf{3.02} & \textbf{6.56}  & 0.31     & 0.09          & 0.0054 & \textbf{0.58}  & \textbf{1.16}   & \textbf{2.90}  \\
\textbf{32-bit} & 2.92        & 1.51  & 0.09 & 2.57  & 10.09  & 22.51 & 0.32     & 0.1           & 0.0048 & 0.70   & 1.40  & 3.52  \\
\textbf{64-bit} & \textbf{2.59}        & 1.22  & 0.09 & 2.09 & 8.21  & 18.52 & \textbf{0.26}     & \textbf{0.07}          & \textbf{0.0024} & \textbf{0.58}  & 1.17  & 2.93 \\
\bottomrule
\end{tabular}
	\label{table:quantization}
\vspace{-1em}
\end{table*}

\subsection{Automatic Batching}
\label{subsec:batching_exp}
To evaluate the effectiveness of automatic batching, we compare the runtime of multiple BOs and FOs under (\textit{i}) an Numpy implementation on a CPU \cite{harris2020array}, (\textit{ii}) a direct PyTorch implementation without batching \cite{paszke2019pytorch}, and (\textit{iii}) {\sc TBOD}\xspace's implementation with batching.

Table \ref{table:batching} compares the three implementations of key operators in OD systems. Clearly, {\sc TBOD}\xspace with automatic batching achieves the best balance of efficiency and scalability, leading to 7.22$\times$, 17.46$\times$, and 11.49$\times$ speedups compared to a highly optimized NumPy implementation on CPUs. {\sc TBOD}\xspace can also handle more than 10$\times$ larger datasets where the direct PyTorch implementation faces out of memory (OOM) errors. {\sc TBOD}\xspace is only marginally slower than PyTorch when the input dataset is small (see the first row of each operator). In this case, batching is not needed and {\sc TBOD}\xspace is equivalent to PyTorch; {\sc TBOD}\xspace is slightly slower due to the overhead of {\sc TBOD}\xspace deciding whether or not to enable automatic batching.

\begin{table}[t]
\centering
    \vspace{-0.5em}
	\caption{Operator runtime comparison among implementations in NumPy (no batching), PyTorch (no batching) and {\sc TBOD}\xspace (with automatic batching); the most efficient result is highlighted in bold per row. Automatic batching in {\sc TBOD}\xspace prevents out of memory (OOM) errors yet shows great efficiency.}
	\vspace{.25em}\footnotesize
    \begin{tabular}{ll|lll}
    \toprule
    \textbf{Operator} & \textbf{Size} & \textbf{NumPy} & \textbf{PyTorch} & {\sc TBOD}\xspace \\
    \midrule
    \texttt{topk}              & 10,000,000       & 7.88          & \textbf{1.08} & 1.09           \\
    \texttt{topk}              & 20,000,000      & 15.77         & OOM           & \textbf{2.44}  \\
    \texttt{topk}              & 100,000,000      & OOM           & OOM           & \textbf{10.83} \\
    \midrule
    \texttt{intersect}         & 20,000,000      & 1.99          & \textbf{0.12} & 0.14           \\
    \texttt{intersect}         & 100,000,000      & 11            & \textbf{0.63} & \textbf{0.63}  \\
    \texttt{intersect}         & 200,000,000      & 21.65         & OOM           & \textbf{2.11}  \\
    \midrule
    \texttt{$k$NN}               & 50,000         & 20.15         & \textbf{0.27} & 0.28           \\
    \texttt{$k$NN}               & 200,000        & 194.43        & OOM           & \textbf{19.65} \\
    \texttt{$k$NN}               & 400,000        & 818.32        & OOM           & \textbf{71.22} \\
    \bottomrule
    \end{tabular}
	\label{table:batching}
\vspace{-1em}
\end{table}

\subsection{Multi-GPU Results}
\label{subsec:multi_gpu}
We now evaluate the scalability of {\sc TBOD}\xspace on multiple GPUs on a single compute node.
Specifically, we compare the run time of three compute-intensive OD algorithms (i.e., LOF, ABOD, and $k$NNOD) with 1,2,4, and 8 NVIDIA Tesla V100 GPUs.

Fig.~\ref{fig:multi_gpu} shows that for three OD algorithms tested, {\sc TBOD}\xspace can achieve nearly linear speed-up with more GPUs---the GPU efficiency is mostly above 90\%.
For instance, the $k$NNOD result shows that using 2, 4, and 8 GPUs are $1.91 \times$, $3.73 \times$, and $7.34 \times$ faster than the single-GPU performance. As a comparison, using 2, 4, and 8 GPUs with ABOD and LOF are  $1.85 \times$, $3.63 \times$, $7.14 \times$ faster and $1.91 \times$, $3.70 \times$, $7.27 \times$ faster, respectively. First, it is inevitable that there is a small overhead in multi-processing, causing the speed-up to not exactly be linear. Second, the minor efficiency difference between $k$NNOD and ABOD is due to most OD operations in the former being executed on multiple GPUs, while the latter involves several sequential steps that have to be run on CPUs. To sum up, {\sc TBOD}\xspace can leverage multiple GPUs efficiently to process large datasets.

\begin{figure}[t]
\centering
\subfloat{%
  \includegraphics[clip,width=\columnwidth]{figures_exp/lof_multi.pdf}%
}

\subfloat{%
  \includegraphics[clip,width=\columnwidth]{figures_exp/abod_multi.pdf}%
}

\subfloat{%
  \includegraphics[clip,width=\columnwidth]{figures_exp/knn_multi.pdf}%
}\\
\vspace{-1em}
\caption{Runtime comparison of using different numbers of GPUs (\textbf{top}: LOF; \textbf{middle}: ABOD; \textbf{bottom}: $k$NNOD). {\sc TBOD}\xspace can efficiently leverage multiple-GPUs for faster OD.}
\label{fig:multi_gpu}
\vspace{-1em}
\end{figure}

\section{Datasets}
\label{appendix:datasets}
Table \ref{table:data} shows the six real-world benchmark datasets used in this study, which are widely evaluated in OD research \cite{zhao2019lscp,ruff2019deep,DBLP:conf/icdm/LiZBIH20,zhao2021suod}. Given the limited size of the real-world datasets, we use the data generation function in PyOD to create larger synthetic datasets as well.

\begin{table}[!htb]
\centering
	\caption{Real-world benchmark datasets used in the experiments}
	\vspace{.25em}\footnotesize
	\begin{tabular}{l | r  r  r  r }
		\toprule
		\textbf{Dataset} & \textbf{Pts (\textbf{\textit{n}})} & \textbf{Dim (\textbf{\textit{d}})} & \textbf{\% Outlier}\\
		\midrule
		mnist         & 7603  & 100  & 9.21  \\
		musk          & 3062  & 166  & 3.17  \\
        mammography   & 11183 & 6    & 2.32 \\
        speech        & 3686  & 400  & 1.65 \\
        f-mnist & 60000 & 784  & 10    \\
        cifar-10      & 60000 & 3072 & 10    \\
		\bottomrule
	\end{tabular}
	\label{table:data}
\end{table}

\section{Additional Experimental Results}

\subsection{End-to-end Evaluation on Synthetic Datasets}

As discussed in \S \ref{subsec:end_2_end}, we compare the runtime of {\sc TBOD}\xspace and PyOD on two real-world datasets (fashion-mnist and cifar-10, see  Fig.~\ref{fig:full_system_evaluation}) and two larger synthetic datasets (see Appendix Fig.~\ref{fig:full_system_synthetic}).
The results show that {\sc TBOD}\xspace is faster than PyOD on all five algorithms tested for both real-world and synthetic datasets.

\begin{figure}[t]
\centering
\subfloat[Synthetic dataset 1 (100,000 samples with 200 features) ]{%
  \includegraphics[clip,width=\columnwidth]{figures_exp/100000_200.pdf}%
}

\subfloat[Synthetic dataset 2 (200,000 samples with 200 features) ]{%
  \includegraphics[clip,width=\columnwidth]{figures_exp/200000_200.pdf}%
}
\caption{Runtime comparison between PyOD and {\sc TBOD}\xspace in seconds for two synthetic datasets. See Fig. \ref{fig:full_system_evaluation} for results on real-world datasets. {\sc TBOD}\xspace outperforms PyOD in all experiments.}
\label{fig:full_system_synthetic}
\end{figure}

\subsection{Provable Quantization with \texttt{topk}}
\begin{figure}[t]
\centering
\subfloat[\texttt{topk} (real-world datasets)]{%
  \includegraphics[clip,width=\columnwidth]{figures_exp/topk_quant_real.pdf}%
}

\subfloat[\texttt{topk} (synthetic datasets)]{%
  \includegraphics[clip,width=\columnwidth]{figures_exp/topk_quant_sync.pdf}%
}

\caption{GPU memory consumption comparison between using provable quantization (16-bit and 32-bit) and full precision (64-bit). Clearly, provable quantization leads to significant memory consumption savings for both operators.}
\label{fig:quantization_topk}
\end{figure}

As discussed in \S \ref{subsec:prov_quant_exp}, the results demonstrate that provable quantization always leads to memory savings. Specifically, Figure.~\ref{fig:quantization_topk} shows that \texttt{topk} with provable quantization reduces memory usage by 73.49\% (with 16-bit precision) and 49.58\% (with 32-bit precision) compared to using full precision.

\section{Introduction}
\label{sec:intro}
\input{01intro}

\section{Background and Related Work}
\label{sec:related_work}
\input{02related_work}

\section{Overview}
\label{sec:system_overview}
\input{03system_overview}

\section{Programming Model}
\label{sec:model_abstraction}
\input{041model_abstraction}

\section{Provable Quantization}
\label{sec:quantization}
\input{042quantization}

\section{Automatic Batching and Multi-GPU Support}
\label{sec:batching}
\input{043batching}

\section{Experimental Evaluation}
\label{sec:experiments}
\input{05experiment}

\section{Conclusion}
\label{sec:conclusion}
\input{06conclusion}

\clearpage
\newpage

\section{Introduction}

Introduce the current landscape of OD algorithms and corresponding systems. They are slow (\updated{efficiency-wise}) and infeasible (\updated{scalability-wise}) on large, high-dimensional datasets. \\

\noindent \textbf{Challenges}:
\begin{itemize}
    \item {\bf Algorithmic complexity.} The supports for a variety of OD algorithms. Some algorithms cannot be represented as tensor algebra computations.
    \item {\bf Scalability} large memory requirements, difficult to run on GPUs with limited memory. Justify the difference from other ML algorithms
    \item {\bf Performance/efficiency} Computationally intensive.
\end{itemize}

The limitations of existing general/acceleration/distributed systems of OD.
\begin{enumerate}
    \item Comprehensive system: CPU +multi-process. (slow and hard-to-scale)
    \item Specific fast OD algorithms (limited generalization)
    \begin{enumerate}
        \item Distributed algorithm
        \item GPU acceleration (single GPU only and hard to generalize)
    \end{enumerate}
    \item Deep learning algorithms: lack the support of leading classical OD algorithms
\end{enumerate}

\noindent \textbf{Observations}: (1) classical OD faces the efficiency and scalablity issue given the existing systems (2) existing systems are hard to generalize (3) classical OD and DL-based OD algorithms are split, where research opportunities may rise. For instance, applying classical OD algorithms on the representation generated by deep neural networks.\\

\noindent \textbf{By acknowledging the rise of deep learning ecosystems and the success of GPU acceleration, we propose {\sc TBOD}\xspace to address these challenges. It abstracts classical OD algorithms into basic tensor operations, which can be efficiently accelerated by GPU and easy to generalize. {\sc TBOD}\xspace also comes with a set of scalability instruments to handle large datasets}, including automatic batching, provable quantization, and multi-GPU support.

Our technical contribution includes:

\begin{enumerate}
    \item \textbf{First GPU-based OD system}. {\sc TBOD}\xspace abstracts complex OD algorithms into basic tensor operators (TBO) for acceleration on GPUs. It is \updated{x} times faster than the leading CPU system and \updated{y} times than general GPU implementations.
    \item \textbf{Efficiency and scalability enhancement}. We leverages provable quantization, automatic batching, and multi-GPU support to improve efficiency and scalability of {\sc TBOD}\xspace while provably maintaining algorithmic correctness, \yz{along with additional methods like algorithmic approximation.}

    \item \textbf{Automated computation path forecasting}. We automate the decision on key hyperparameters, e.g., batch size, to improve the overall performance.
    \item \textbf{Extensibility and integration}: open-sourced {\sc TBOD}\xspace allows for easy exploration of new OD algorithms, and it also provides a unified platform to easily combine classical and deep learning OD algorithms.
\end{enumerate}

\begin{figure}[!ht]
     \centering
     \begin{subfigure}[b]{0.22\textwidth}
         \centering
         \includegraphics[width=\textwidth]{example-image-a}
         \caption{Efficiency}
         \label{fig:efficiency}
     \end{subfigure}
     \begin{subfigure}[b]{0.22\textwidth}
         \centering
         \includegraphics[width=\textwidth]{example-image-a}
         \caption{Scalability}
         \label{fig:scalablity}
     \end{subfigure}
        \caption{\label{fig:results}Insert a figure (or two sub figures) here to show efficiency and scalability (handle large amout of data).}
\end{figure}

\section{Background and Related Work}

\subsection{Scalable Outlier Detection}
Describe why scalability is a huge issue in OD (due to proximity estimation).
Briefly mention which algorithms are suffered from scalability. Cite the included algorithms here.\\

\noindent List latest works on scalable OD, e.g., distributed OD and SUOD \cite{zhao2021suod}. Point out the difference. Refer section 2.2 in SUOD to further elaborate. \yz{describe some baselines here.}\\

\noindent \yz{Highlight there is no comprehensive OD system leveraging GPU.}

\subsection{General Deep Learning Infrastructures}
\textbf{Discuss the emergence of deep learning infrastructure}, both software and hardware. DL can be easily parallelized, run in batch, and mainly matrix operations, which is a perfect use of GPUs for acceleration. \\

\noindent There are DL-based outlier detection, and the latest research showing the benefit of hybrid models \cite{ruff2019deep}. {\sc TBOD}\xspace provides a unified platform for both flavors for easy combination. \\

\noindent \textbf{(Not a key component) Briefly describe Hammingbird (HB)?} \cite{DBLP:conf/osdi/NakandalaSYKCWI20}, and list the key difference from {\sc TBOD}\xspace:
\begin{enumerate}
    \item {\sc TBOD}\xspace is a fully independent system designed for both training and prediction, while HB is for prediction only and depends on traditional ML pipeline, i.e., scikit-learn \cite{DBLP:journals/jmlr/PedregosaVGMTGBPWDVPCBPD11}.
    \item {\sc TBOD}\xspace is fully unsupervised, while HB contains supervised models.
    \item {\sc TBOD}\xspace emphasize scalability via approximation, while HB does not.

\end{enumerate}

\subsection{Major Challenges and Assumptions}
We will decrease the number of questions.
\begin{enumerate}
    \item \updated{(\S \ref{sec:efficiency})} Given there are many OD algorithms with complex and different mechanism. How to design a unified system that can efficiently accelerate most of them?
    %

    %
    \item \updated{(\S \ref{sec:scalability})} By acknowledging that GPUs have limited memory (\yz{add a citation}), how can we improve the system scalability to handle large amount of data?

    \item \updated{(\S \ref{sec:automl}) How to automate the process with the {\sc TBOD}\xspace? which technique to use?}
\end{enumerate}

\subsection{Definition and Problem Formulation}
Unsupervised distributed OD tasks in {\sc TBOD}\xspace come with:

\begin{itemize}
\item outlier detection model $M$.
\item data $\mathbf{X} \in \mathbbm{R}^{n \times d}$ without ground truth labels.
\item $t$ available GPUs.
\item (optional) batch size $b$ for selected algorithms.
\end{itemize}

\textbf{The goal is to return the forecasted outlier scores on $\mathbf{X}$ by $M$ by leveraging $t$ GPUs efficiently}. The system design is driven by \textit{efficiency} and \textit{scalability}.\\

\noindent \textbf{Definition of \textit{efficiency}}: {\sc TBOD}\xspace should be faster than its CPU-based counterpart, e.g., PyOD, on a factor of \updated{k}?\\

\noindent \textbf{Definition of \textit{scalability}}: {\sc TBOD}\xspace should be able to handle on the same degree of amount of data as its CPU-based counterpart, e.g., PyOD.\\

\section{High-level System Overview}
\
In this section, we discuss the high-level system description. Specifically, we will lay out how the following sections are organized. General ideas: OD algorithms are abstracted into basic building blocks, and each building block is optimized for GPU and enhance with scalability instruments.

\yz{Specifically, we want to state the purpose of each component briefly. }

\begin{figure}[!htp]\label{fig:flowchart}
  \includegraphics[width=0.5\textwidth]{figures/flowchart_placeholder.png}
  \caption{Placeholder}
\end{figure}

\section{OD Model Abstraction}
\label{sec:model_abstraction}

\textbf{Motivation}: (1) there are many OD algorithms and not all can be easily converted into tensor operations; (2) it is intractable to implement all OD algorithms and (3) we want to support the future OD algorithm .

Consequently, we propose to abstract complex OD algorithms into tensor-based operators (TBO). (1) this improves system efficiency and generalization by eliminating repetitive components. (2) this makes the combination/interaction of operators accessible, facilitating trying new research methods.

\subsection{Algorithm Abstraction and Modularization}

\begin{figure*}[!htp]\label{fig:operators}
  \includegraphics[width=0.9\textwidth]{figures/operators.png}
  \caption{Placeholder (add deep learning operators as well)}
\end{figure*}

\subsubsection{Basic OD Operators}
Discuss kNN, topk, ...

\subsubsection{Deep Learning Feature Extractors}
Discuss the basic feature extractors in this approach

\subsubsection{Additional Design Considerations}

It is noted that GPUs are mostly suited for tensor operations, e.g., matrix multiplication, other than the calculations needing random data access, e.g., classical tree structures. This characteristic leads to the following design considerations:

\begin{enumerate}
    \item list some additional operations (histogram operations)
    \item memory saving by calculating quadratic distance other than cdist \cite{albanie2019euclidean}
    \item \yz{Not all the algorithms are supported yet in the current version, e.g., tree-based models. }
\end{enumerate}

\subsection{Use TBOs to Build Complete Algorithms}
Show one or two examples of using TBOs to build a complete algorithms (choose a non-trivial one possibly)

\subsection{Unified Design for Classical and Deep Learning based OD}
Discuss why and how to combine different types of OD operators. We could replicate some research works in the Experiment section.

\section{Provable Quantization}

\textbf{Motivation}: in OD scenarios, some operations are suited to be conducted in lower precision while yielding exact results. This technique helps to reduce the GPU memory consumption.

\subsection{Theoretical Background}
\cite{DBLP:journals/pacmpl/Lee0A18} to discuss how provable quantization is designed.

\textbf{Procedure}: (1) calculation in lower precision (2) post verification

\subsection{Case Study: Neighbors within Range}
\subsection{Case Study: topK}

Be sure to introduce these in the model abstraction section.

\section{Automatic Batching}
\textbf{Motivation}: to overcome the limits of GPU memory, we propose to make BTOs batchable. With automatic batching, we gain better scalablity with large datasets.

TBOs are categorized into \textit{trivial batching} and \textit{non-trivial batching}.

\subsection{Trivial Batching}
Simply split by the samples, e.g., topK.

\subsection{Non-trivial Batching}
\label{subsec:non-trivial-batching}
Discuss the challenges of batching for certain algorithms (why are they challenging). Global dependency, e.g., kNN and PCA. We could demonstrate different versions of kNN (simple distance batching to returning nearest neighbors).

\section{Multi-GPU Extension}
\textbf{Motivation}: it is possible to speed the system computation by leveraging multiple GPU, especially for deep learning operators.

\subsection{Basic OD Operators}
Multi-process---each process is responsible for one GPU.

\subsection{Deep Learning Feature Extractors}
Unclear yet. Maybe torch native DDP.

\section{Experiments and Results}
Discuss experiment settings.
We design experiments to answer the following questions:
\begin{enumerate}
    \item Efficiency comparison (exactly solution): General system comparison with the baseline systems on multiple datasets (both real-world and synthetic)
    \item Scalability discussion: (i) the effect of each component (like ablation) and (ii) the full system with larger and larger datasets (iii) exact vs. approximate solution
    \item Unified classical and DL OD algorithms with {\sc TBOD}\xspace
\end{enumerate}

\subsection{Datasets, Baselines, and Evaluation Metrics}

\textbf{Baselines}:
\begin{enumerate}
    \item (CPU) Python Outlier Detection (PyOD) Toolbox \cite{DBLP:journals/jmlr/ZhaoNL19}: leading Python library featured with various optimization (e.g., just-in-time compilation) and parallelization
    \item (CPU) SUOD \cite{zhao2021suod}: a scalable acceleration for training a large number of heterogeneous OD.
    \item (GPU) PyTorch implementation of certain algorithms. \yz{Similar to ReHighlight there is no comprehensive OD system leveraging GPU.}
\end{enumerate}

\noindent \textbf{Algorithms}: select three to four representative OD algorithms (different types). \noindent \updated{At which level should we conduct the comparison? BTO level or OD models level?} For full system comparison, OD algorithm level makes more sense, while it is ok to compare at the BTO level for specific techniques.\\

\noindent \textbf{Evaluation}: for exact results, no need to compare accuracy but time only. \\

\noindent \textbf{Hardware}: \updated{should we test under different hardware? Maybe we should fix to a specific type of GPU and then extend the number of it for multi-GPU extension.}

\noindent \textbf{Presentation order}. First show the effect of the full system and then each technique (the order can be reverted).

\begin{figure}[!htp]
  \includegraphics[width=0.5\textwidth]{figures/quant_time.png}
  \caption{Quantization Placeholder}
  \label{fig:provable_quant}
\end{figure}

\subsection{Full System Evaluation}
All the applicable techniques are enabled.

\subsubsection{Efficiency Evaluation}

\textbf{Goal}: showing {\sc TBOD}\xspace is much more efficient than the baseline systems (when all of them can handle the dataset without the issue of out of memory/out of time)

\noindent \textbf{Datasets}: from small to medium OD benchmark datasets

Table 1: how much time does {\sc TBOD}\xspace take in comparison to other baselines? This can be a multi-group bar plot. x-axis is the datasets, and y is the elapsed time (different algorithms can be different color bars).

\noindent \textbf{Analysis}: we could run pairwise statistical tests to show {\sc TBOD}\xspace is significantly faster.

\subsubsection{Scalability Evaluation}

\textbf{Goal}: showing {\sc TBOD}\xspace can handle much larger datasets the baseline systems.

\noindent \textbf{Datasets}: select large deep learning datasets (cifar 10) and also synthetic OD datasets

Table 1: how much time does {\sc TBOD}\xspace take in comparison to other baselines? This can be a multi-group bar plots. For each plot (corresponding to one dataset), x-axis is the OD algorithms, and y is the elapsed time.

\noindent \updated{Multi-GPU is another angle for scalability.}

\subsubsection{Combining Classical OD with DL Operators}. Show 1 or 2 examples of combining classical OD and DL operators. Showing the unified system is more efficient (faster) then building each of them separately in two systems (CPU for classical OD and GPU for DL).

\subsection{Effect of Provable Quantization}
\textbf{Goal}: provable quantization could reduce peak GPU memory consumption and the elapsed time.

\noindent \textbf{Process}:show that the \textbf{peak GPU memory usage} and \textbf{elapsed time} under different precision (16, 32, and 64 bits) and varying data sizes shown in Fig. \ref{fig:provable_quant}. Since our quantization is provable (exact), we do not need accuracy.
\begin{enumerate}
    \item Neighbors within range
    \item if space allows, we could add intersection1d and topk
\end{enumerate}

\subsection{Evaluation of Automatic Batching}
\noindent All the basic operators come with this technique, while it makes sense to show we could handle larger datasets for non-trivial algorithms.

\noindent \textbf{Comparison}: (1) CPU implementation (2) GPU implementation (no batching) and (3) GPU batching in {\sc TBOD}\xspace.

\begin{figure}[!htp]
  \includegraphics[width=0.45\textwidth]{figures/batching.png}
  \caption{Batching Placeholder}
  \label{fig:batching}
\end{figure}

\subsection{Single GPU vs. Multi-GPU}
\textbf{Goal}: showing the multi-GPU version can lead to better efficiency (not necessarily scalability).

\noindent \textbf{Process}: \updated{Our current operators are not cuda-heavy. The results are not too impressive. Should we demonstrate this on some deep learning operators instead?} If so, we will maily compare the execution time of a specific DL operator under \textbf{varying size of data samples} and \textbf{the number of GPUs (same mode)}.

\subsection{(Less important) Evaluation of Additional Techniques}
\textbf{Goal}: if space allows, we could add the results of some approximate solutions and automated computational path prediction.

\clearpage

\bibliographystyle{ACM-Reference-Format}

\section{Introduction}
\label{sec:intro}
\input{01intro}

\section{Background and Related Work}
\label{sec:related_work}
\input{02related_work}

\section{Overview}
\label{sec:system_overview}
\input{03system_overview}

\section{Programming Model}
\label{sec:model_abstraction}
\input{041model_abstraction}

\section{Provable Quantization}
\label{sec:quantization}
\input{042quantization}

\section{Automatic Batching and Multi-GPU Support}
\label{sec:batching}
\input{043batching}

\section{Experimental Evaluation}
\label{sec:experiments}
\input{05experiment}

\section{Conclusion}
\label{sec:conclusion}
\input{06conclusion}

redpajama_set_name: RedPajamaArXiv


text:
My Favorite Teacher Was an Ironworker
Rick Taylor
Publisher: AuthorHouse Pages: 348 Price: (paperback) $17.72 ISBN: 9781463405489 Reviewed: July, 2012 Author Website: Visit »
Dr. Rick Taylor's cathartic, inspiring memoir, punctuated by his youngest son Scott's optimistic world-view, is an excellent guide to living life as well as possible. Funny, sad and thought provoking, Taylor's recollections revolve around the discovery that life's lessons can be learned from those we least suspect — in this case, Taylor's precocious son Scott.
Growing up in 1950 Surry, Maine, Taylor felt uneasy on Sunday nights riding his bicycle past a nearby graveyard to his friend's house to watch television. However, years later, after diving into a raging Spokane River to pull 3-year-old Scott to safety, those childhood fears were washed away. Taylor discovered that he only feared the things that would hurt the ones he loved.
He also learned from his young son to take sports less seriously. Though a competitive Little Leaguer, Scott's attitude was that winning isn't everything; he would congratulate the opposing team, whether his team had won or lost. In relationships, Scott taught his father that after the romance has faded, "girlfriends" can become "girl friends."
Taylor, a medical doctor, writes that Scott, eventually standing six-foot-five with huge arms, never thought of being a teacher. Like his grandfather, he loved "real man" work — building bridges, buildings, etc. As a medical intern, Taylor expected to learn from his colleagues. But through Scott, he discovered that if you pay attention to all kinds of people, you will learn amazing things about life.
In telling his story, Taylor is a little heavy-handed relating heart-wrenching moments. Misspellings (The Beattles, Thru), and other mechanical errors will be a slight distraction to discerning readers. And while most of his stories ring true, Taylor admits his memoir "is a combination of fiction and reality."
Nonetheless, the lessons here are compelling. In a bittersweet reflection at the end, Taylor laments his graduate course work in life will not be complete until he follows Scott, 29, (who died in an accident) into the next phase of life's journey.
Also available in hardcover and ebook.

redpajama_set_name: RedPajamaCommonCrawl


text:
Q: Error during program installation using Synaptic Package manager I tried to download and install the Pixillion Image Converter Software from the Synaptic Package Manager. It downloaded and started to install. Then the message below appeared.

W: Download is performed unsandboxed as root as file '/var/cache/apt/archives/partial/python3-problem-report_2.20.4-0ubuntu4.8_all.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)

I am using Lubuntu 17.04 and Mozilla Fiefox

redpajama_set_name: RedPajamaStackExchange


text:
When Being a 'Karen' Turns Criminal? Central Park Dog-Walker Could Be Prosecuted
Elura NanosMay 26th, 2020, 9:14 am
Another day, another white woman called the police on a black man for a nonsensical reason. This time, Amy Cooper was walking her dog in a wooded area of Central Park called "the Ramble." As Cooper strolled through the park with her new dog off-leash, she encountered 57-year-old science editor Christian Cooper (no relation) and the two argued about the park's rule that dogs be leashed at all times.
According to Christian Cooper's later Facebook post, Amy Cooper refused to put her dog on a leash, saying, "The dog runs are closed. He needs his exercise."
After further argument, Christian Cooper said, "Look, if you're going to do what you want, I'm going to do what I want, but you're not going to like it." His threat? To give the dog some treats. Christian Cooper, an avid birdwatcher, says he often carries dog treats for precisely these occasions. According to his account of the incident, he pulled the treats out and Amy Cooper immediately began screaming "don't you touch my dog!" Christian Cooper began filming.
Here's what he wrote on Facebook about the incident:
Central Park this morning: This woman's dog is tearing through the plantings in the Ramble.
ME: Ma'am, dogs in the Ramble have to be on the leash at all times. The sign is right there.
HER: The dog runs are closed. He needs his exercise.
ME: All you have to do is take him to the other side of the drive, outside the Ramble, and you can let him run off leash all you want.
HER: It's too dangerous.
ME: Look, if you're going to do what you want, I'm going to do what I want, but you're not going to like it.
HER: What's that?
ME (to the dog): Come here, puppy!
HER: He won't come to you.
ME: We'll see about that…
I pull out the dog treats I carry for just for such intransigence. I didn't even get a chance to toss any treats to the pooch before Karen scrambled to grab the dog.
HER: DON'T YOU TOUCH MY DOG!!!!!
That's when I started video recording with my iPhone, and when her inner Karen fully emerged and took a dark turn…
Christian Cooper's sister later shared the video:
Oh, when Karens take a walk with their dogs off leash in the famous Bramble in NY's Central Park, where it is clearly posted on signs that dogs MUST be leashed at all times, and someone like my brother (an avid birder) politely asks her to put her dog on the leash. pic.twitter.com/3YnzuATsDm
— Melody Cooper (@melodyMcooper) May 25, 2020
In the video, Amy Cooper can be heard repeatedly saying that she is going to call police and "tell them there's an African American man threatening my life."
When asked why he chose to film the encounter, Christian Cooper said, "We live in an age of Ahmaud Arbery where black men are gunned down because of assumptions people make about black men, black people, and I'm just not going to participate in that."
But could this woman actually be prosecuted for her phone call? She sure could. In New York, making a false report is a class A misdemeanor.
Under Section 240.50, a person is guilty if she, "Initiates or circulates a false report" of something that didn't actually happen, or, "gratuitously reports to a law enforcement officer or agency…an allegedly impending occurrence of an offense or incident which in fact is not about to occur." So, Amy Cooper's near-hysterical call to police that an African American man was threatening her life, and threatening her dog appears to fit squarely within the confines of the false reporting statute. Cooper has spoken to several news outlets and apologized for her behavior; those apologies, particularly when taken together with the video, tend to confirm that her reporting of the incident was false.
She said that she was afraid, not a racist.
"I think I was just scared," she said. "When you're alone in the Ramble, you don't know what's happening. It's not excusable, it's not defensible."
The penalty for class A misdemeanors in New York is up to one year in jail, and a fine up to $1,000.
Although Amy Cooper has not been charged for making a false report, she has already faced some consequences. Animal Rescue has reportedly taken possession of her dog.
The Animal Rescue has taken possession of Amy Cooper's dog. pic.twitter.com/t8izZX9xhi
— chris evans (@notcapnamerica) May 26, 2020
She has also been placed on administrative leave by her employer, investment company Franklin Templeton.
In response to an incident involving an employee on May 25th, Franklin Templeton issued the following statement. pic.twitter.com/8f2lMwK0r5
— Franklin Templeton (@FTI_US) May 26, 2020
Amy Cooper said her "entire life is being destroyed right now."
Later Tuesday, Franklin Templeton the employee involved in the Central Park "incident" was fired.
Following our internal review of the incident in Central Park yesterday, we have made the decision to terminate the employee involved, effective immediately. We do not tolerate racism of any kind at Franklin Templeton.
"We do not tolerate racism of any kind at Franklin Templeton," the company said.
Eliza Orlins, a decade-long public defender turned Manhattan DA candidate, commented that this incident could have gone a very different way if Christian Cooper had not recorded it.
What happens: White lady calls the cops on Black man. Cops believe her. He gets arrested and then arraigned. Outrageous bail gets set. His family cannot afford to buy his freedom. He gets sent to Rikers Island, where he sits for any number of days, months, or years. (2/4)
— Eliza Orlins (@elizaorlins) May 25, 2020
People are sitting on Rikers Island on the basis of nothing more than an accusation (like in this video!) with the added risk of contracting COVID-19. So, we can call out these "Karens," but also we need a Manhattan DA who is committed to real reform. I will be that DA. (4/4)
Editor's note: story was updated to reflect that Amy Cooper was fired.
[screengrab via Christian Cooper]
This is an opinion piece. The views expressed in this article are those of just the author.
Amy cooperCentral ParkChristian Cooperdog walkerKaren
Elura Nanos - Columnist & Trial Analyst, Law & Crime; CEO Lawyer Up
Elura is a columnist and trial analyst for Law & Crime. Elura is also a former civil prosecutor for NYC's Administration for Children's Services, the CEO of Lawyer Up, and the author of How To Talk To Your Lawyer and the Legalese-to-English series. Follow Elura on Twitter @elurananos
More Stories by Elura Nanos


Feral Illustrations is about the love for fashion illustration, live drawings and animals. The illustrations vary in size and explore many different mediums, for example oil, charcoal, pencil drawings, digital and print.
Visit the Feral Illustrations website.


The first two-thirds of this gallery is part of the Art Nouveau collection from a spectacular exhibit of Zsolnay porcelain in Pécs, at the Zsolnay Cultural Center, which is still under construction. These pieces are my favorites, so I processed them first, but the older and more classic Zsolnay is also exquisite; unfortunately, shooting conditions were pretty difficult (almost impossible, in fact) in that part of the gallery (lighting, reflections), so I haven't included nearly as much of it. Perhaps another visit is in order.
Zsolnay Porcelain Manufacturing was established in 1853 and over the years has produced outstanding works of art. It's still operating today, although certainly not on the scale of the past. Zsolnay is headquartered in Pécs, where it also has a museum located in the Zsolnay family's former home. For photos of the museum collection, see https://pbase.com/helenpb/zsolnaymuseum.
I have inherited a Zsolnay Pecs what I would describe as a pot pouri bowl. It is cream in colour with beautiful turqoise beads and gold gilding. Marks on base are 1811 with the numbers 48 underneath. Anyone give me a date of manufacture ?
Just found out we have two vases in the family made by the factory about 1890. Fascinating to find out more about them and the porcelain factory.
A fantastic collection, perfectly captured and presented. V.
Wonderful works of Art, superbly emphasized by your excellent images, Helen. V.
Wonderful presentation of beautiful exhibits.
Beautiful photographs of this exquisite collection! Can't wait to see more!


SKU: 8097. Categories: Footbags, New Products, Plastic Filled, Unique.
World Footbag is pleased to announce the arrival of an awesome new footbag to our line up. The Record Setter footbag is virtually identical to the custom made footbag that Guinness world record consecutive kicks champion Ted Martin used to rally off 63,326 non-stop kicks over the course of nearly nine hours!
The Record Setter is a hand stitched 62 panel footbag, made with soft but durable cowhide leather. This old school beauty is filled with plastic pellets, and weighs in at 53 grams. Slightly larger than the average footbag with a diameter of 2.5 inches this footbag was manufactured under the direction of world record holder Ted Martin to match the design, size and feel of the actual footbag he used to set the world record.
Ted wholeheartedly endorses this footbag and hopes you will agree that this is one of the best footbags you have ever kicked!


Just for you! Water Resistant Patio Dining Set Cover best design By Red Barrel Studio. Water Resistant Patio Dining Set Cover very well made, sleek and simple. Complete your living room furniture with a modern Water Resistant Patio Dining Set Cover. Its cute sturdy, attractivce and it looks expensive and a good value for the money. Water Resistant Patio Dining Set Cover is one of the most homey, cozy, nice look and exotic Water Resistant Patio Dining Set Cover especially for the price and made of superior products. Great quality, easy to assemble, delivery on time and in good condition. Water Resistant Patio Dining Set Cover is good merchandise at fair prices and amazing free shipping. Guarantee damaged claim by offering to send parts or to keep the item at a discounted price. Great buy would definitely recommend. Shop with our low-price guarantee and find great deals on ##ptitle# and more!. Reading the reviews helped you purchase.
Within this fast evolving realm of competitors and development, it has been a undeniable fact that people barely get enough time to spend in their homes, taking care of themselves. It is mandatory for us that people spend the vast majority of the times time in office, performing our responsibility. If the workplace does not have in proper tools and facilities for the workers, the employees get severe health issues. Likewise, the decorum and the tools from the offices are supplied immense importance. Nowadays, the majority of the equipments are selected inside a careful method so as to ensure the employees comfort. One particular important item in the office may be the ergonomic desk office chairs. As most of the offices have turned out to be smart, which means they depend more about machine than hard physical work, ergonomic desk workplace chairs are especially designed for such workplaces. They are the seats used for using the offices having pc work stations. These have been developed specifically chairs that help within the reduction of pressures on the a number of nerves from the spinal wires, thereby limiting the chance of muscle and nerve illnesses. Also they assist in maintaining the right body posture. It aligns the forearms, wrists, back, throat and also the head in the correct jobs, thereby reducing lower back pain, work associated musculoskeletal disorders and other this kind of diseases. And when you just looking for a seat to unwind, then think about one of the reclining chair from your list. In this article we will talk about the very best fifteen greatest ergonomic desk chairs to be able to provide the visitors with the correct understanding to ensure that when they buy such chairs, it might be simple to allow them to choose the most appropriate one.
This set includes a 1 remaining arm sofa set, two armless couch sets and one corner sofa established. This provides enough room to accommodate your family and friends. The fabric is 100% polyester for sufficient comfort and durability. The advantage of this sofa set may be the matching and mixing of chairs within the space within the room for any perfect shape. And also allows for fitting even in little areas. It takes just mild putting together.Additionally, it functions push soft cushions for maximum comfort. You might want to do this set. It works nicely.
When it comes to purchasing living room furniture, leather-based is always a smart option. Not only does it look good with many styles, but its extremely durable (its the ideal material for a household with children or animals) and its extremely-easy to clean, as well. The downside of leather Quinnipiac Camping Hammock? It may have a higher cost than material, micro-fiber or faux leather Quinnipiac Camping Hammock. This reclining loveseat stays a budget-pleasant option because of 1 guru trick: its sitting area is upholstered with leather, while the attributes are padded with increased affordable fake leather-based. That means you receive the appear and feel of a complete leather loveseat with no hefty price tag. Use the lever around the loveseats equip to kick back and take it easy on its high-denseness foam filling, and consider the money held on. This specific loveseat posseses an added reward: expert set up will come in numerous areas of the nation for the next fee.
This couch is appreciated for its unique design and comfort. Its impressive flare-arm couch with chairs offers you optimum room for stretching and using your living space. So generally the modern couch chaise increases the beauty of this established. The set also includes pillow leading arms padded back again and chairs for optimum comfort and ease. The couch includes a corner-obstructed wood frame that provides the sofa sufficient sturdiness. The sofa measures 89 inches wide by 62 inches in depth by 4 in . in height making it ideal for medium dimension areas. You may want to do small putting together eg from the legs.
This reclining loveseat takes the second spot because its an excellent inexpensive, super comfortable choice that can fit in with most house decorations styles. Its solid wood body is padded with circles (for many spring) and high-denseness froth (for many soft), and padded with brownish fake leather-based which makes it an ultra-flexible furniture piece. Combine it with the person cave for any cozy destination to view the game (and dont worry about any spillsits vinyl fabric furniture makes cleanup easy) or perhaps your living room with a few ornamental throw cushions for that perfect place to cozy up following a long day. Real-life clients adore this reclining loveseat because its comfy, durable, simple to assemble 1 reviewer even noted that they could put together two units in under 45 minutes! and shipping in the merchant is easy.
This fits you if all you need is a beautiful coach. It is designed with a flared frame and pillow leading amrests inside a awesome cobblestone grey. Now, it has tough foam soft cushions which are wrapped up in inside a polyester furniture to create the specified comfort. It has a durable part blocked body that adds to the sturdiness. Also, feet are in a fake wooden finish. It has an remarkable gray color that suits with any decoration. It measures 89 W by 39 Deb x 40 They would therefore large enough to accommodate you together with your family/buddies. More to the point, it arrives fully put together. This saves the agony of getting to put together the established.
This plush group of city-stylish couches is created to provide an adorned domain. In fact, even your modest areas may have you obliged for an immaculate movie night. Lay him lower to obtain a completely altered mattress. This avant-garde person futon mattress is accessible in different tones. This incorporates some vibrant shades out of control for example Lemon, Light Azure, and Light Crimson. Place them in your inventive studies or loft to assess the exuberance. It is also available in decreasing tones for example dark, darkish and dark. These nuances, once again, print a respectable aspect of the upper course. With the breaking element of the back, you can rest to your preferred position. This couch looks rich with tufted fabric consistency upholstery. The good thing is that it is personalized with hypo-allergenic filling up. A conclusive result provides a comfortable rest.
This is the subsequent sofa on our checklist. It's additional comfortable and very ideal for little rooms or attic living. It is padded in polyester fabric, prospects inset control keys that give a stylish diamond-tufted style. It is constructed of long lasting materials and also the thighs are made of durable wood to increase its durability. The loveseat has an espresso stained wooden thighs and non-marking feet caps. It offers an appropriate foam cushioning and polyester fabric furniture that makes it very luxurious. It features a longue spot that provides an exceptional room for resting.
Puffy, overstuffed loveseat cushions are not your lifestyle? This lying loveseat from GDF has sharper, solution outlines than your traditional reclining loveseat, which makes it an elegant accessory for a middle-century contemporary, contemporary, modern or modern type of house. And, at just 46.46 inches in width, its easy to squeeze into a smaller space, like apartments, dens or office spaces. Obtainable in several color and material options, such as grilling with charcoal material, navy blue fabric and standing micro-fiber, its easy to find the perfect complement for your home decorations, and also the durability your family requirements for daily use. The days are gone of excellent-looking, but uncomfortable furniture you will no longer have to sacrifice comfort and ease for design. This lying loveseat features an oversized, gentle, plush-filled seat cushioning with sufficient space for two or sufficient room for you to spread out and relax.
Does your room show space requirements? Are you inclined to throw a slumber celebration in the mild of the lack of room to rest? This DHP Futon mattress could be ideal for you in cases like this. This Futon couch by DHP includes the functionality of a couch bed with a contemporary and elegant look. Put it in your family room to achieve an additional sleepy mattress at night. A micro-fiber floor that took off in the centre will be a ideal mixture. This unique couch comes with a tapestry with gleaming stainless hair and legs. Bringing together these outcomes together inside a comfy popular design couch. You may choose to get it in Faux Leather-based, Velvet or Bed linen. An element that is worth specifying is its back design. Considering the different seats, DHP promotes the progres from the couch for your comfort and ease levels. You can gain levels to have an computer animated babble, or for silent shifting picture evening.
British folded equip couches are additional comfy thanks to the large, soft cushions behind and for the seats. Reduced hands are comfy for laying and also the exposed thighs bare this couch style from sensation overweight or impacting. It really came from in the change of the century and can be associated with something typical of the English countryside. Differentiating qualities generally include low uncovered thighs, soft, generous cushions, a good back again and recessed arms. Though it may be regarded as a classic design, it is also a perfect addition to a contemporary room thanks to its comfy and flexible look.
Copyright © Water Resistant Patio Dining Set Cover By Red Barrel Studio in Hammocks All right reserved.


Welcome to my web site which has evolved from me combining my career as an Artist and Photographer. My passion for photography evolved from my profession as a painter and my fascination with light, colour and weather patterns. This knowledge has been transferred to my photographic images and with over 20 years experience I have specialised in landscape and sea photography.
Living in North Devon I have been lucky enough to be exposed to some of the most spectacular landscapes and scenery. Through this I have become totally inspired by my surroundings and have endeavoured to encapsulate visual harmony, the dramatic changing coastline and the wonderful environment surrounding me. Thus creating images which radiate the atmosphere of the moment, evoking that moment in time and which arouse a sense of energy and attraction.
Gradually over time I have developed a unique style which I have been able to personalise by becoming a printer and therefore I am in control of the photography process from start to finish. Creating images which are transferred to paper and canvas.
This passion for my work has also led me to lead photography workshops, which tend to be on a one to one basis, so that clients can develop their own individual style and technique, whilst offering a dedicated, professional service.
The products range from prints which are treated as fine art prints and not photographic reproductions to canvases and limited editions , which all exude exceptional quality . I also produce a wonderful selection of greeting cards and postcards of the area which are very successful.
My work has enhanced the living and working environments of many establishments and my craft has been used for corporate sales, as well as on an indivdual basis . I have worked for a range of diverse and prestigious clients including the NHS, BAA and P & O , where I have over 1800 images of North Devon displayed on the ship The Ventura. Whilst interior designers have used the beauty of the area to complement the decor and bring a bit of rural living into busy, working environments.
I have one of the largest photographic libraries of North Devon and many more of North Cornwall and Exmoor. Therefore I have had a large number of my landscape images used in publicity materials to advocate the beauty and amenities within the area. Such work has also been published in supplements; magazines, newspapers and featured articles, as well as local councils and tourism organisations and operators using local North Devon images.
I also supply and specialise in stock images of North Devon and supply many images to further enhance local web sites and tourist ventures. A place I feel privileged to live and work in.


If you're hungry for networking, make sure you're at this month's Sunset Reception.
The Sunset Reception is a monthly get-together orchestrated by the Greater Oklahoma City Chamber, hosted at a Chamber member's business (if you're a member, hosting one of these can be a great way to promote your business! Contact Dalena West for more information). This month's Sunset Reception will be at the Lago International Buffet & Bar.


text:
The Change Challenge
Training Matters
Women & Work
Egos, emotions, and errors
Jan 16 2006 by Dan Bobinski Print This Article
It's sad, really. After so much research on the benefits of Emotional Intelligence, too many managers and leaders continue to ignore the facts. They're stuck in their old patterns of intimidation and coercion, demoralizing employees and creating attitudes of grudging compliance.
Consider this interaction that occurred recently at a high-level meeting:
Manager: Wouldn't it be a good idea if we got input from our more experienced technicians on how to implement this new solution?
Senior Executive: I'm not going to waste my time with that. I'm the bottom line authority here, and if I say 'do it,' I expect the project to get done. Now go get this project implemented.
According to the manager that related this scenario to me, the senior executive was popping his veins. He was truly upset that anyone would challenge his right to dictate what was going to happen.
Emotional Intelligence means being intelligent about the emotions, values, and expectations of all concerned and making the best decision based on those observations.
Using this definition, the senior executive in the above example showed no emotional intelligence whatsoever. He firmly believed that simply because he held a senior position, he could throw his ego (and his title) around and people should do what he wanted.
What people want is to have input into important decisions
Granted, the executive certainly had the right to make such a decision. In fact, such an approach can be quite efficient; but it's rarely effective. The long term consequences can have a tremendously negative impact. People can become apathetic or grudgingly compliant toward what they're being told to do. What people want - especially those with solid knowledge about their profession - is to have input into important decisions.
Unfortunately, some leaders and managers don't care. They don't think about how people might lose commitment and stop investing their best efforts. They don't realize that people want to give input and feel like their opinions matter.
The result? Wasted resources. Missed opportunities. Less effective results.
Consider Marie, a mid-level supervisor for a national hotel chain. She's been with her company seven years, and each year her performance review noted that she'd gone over and above the call of duty in providing exceptional customer service.
Unfortunately, after tearing some ligaments during a horse-riding accident, Marie needed surgery. Upon informing her boss that she might need to be off work for six weeks, Marie received a shock. "I don't know if I can hold your job that long," her boss blurted out. No attempt was made to ask if Marie thought she could contribute while she was out. Just a blanket statement that her job might be in jeopardy.
"It's hard devoting yourself to people with power issues," Marie told me. "They have to be in control all the time." She went on to tell me that she'd had a plan to help through telecommuting, but after her manager bowled her over, she felt like giving up.
Essentially, it's a huge mistake to ignore the intellect and motivations of people. The idea that employees can't contribute to problem-solving is an error. A manager's or leader's ego may insist on the right to dictate actions, but the true impact of that often creates more damage than the action seeks to solve.
The point to remember is Emotional Intelligence is learnable. All that needs to happen is for managers and leaders to see the benefit of doing so.
Perhaps a financial benefit might provide motivation for some. Consider two managers who didn't get along. Although the company was profitable, egos reigned and production wasn't optimal. When company leaders and managers underwent training in Emotional Intelligence, it sunk in, albeit slowly. Six months after the program, one of the leaders approached me with the following:
"It used to be that I didn't care what happened in [the other person's] department. But after the training I realized that we simply had different work styles. So I chose to work with this guy instead of against him, and together we now make decisions that save the company more than $20,000 per day."
Twenty thousand per day? Even if they do that only five days a year, the payoff is huge.
What it takes is managers seeing that ideas and solutions other than their own need to be considered. They have to set aside their egos. They need to want to learn. And they have to be willing to choose new patterns of employee interaction.
It's possible. I see it all the time. And it's the intelligent thing to do.
Engagement & MotivationTeam Working
Dan Bobinski
Daniel Bobinski teaches teams and individuals how to use emotional intelligence and how to create high impact training. He�s also a best-selling author, a popular speaker, and he loves helping teams and individuals achieve workplace excellence
Good change, bad change
Outside and inside the box
The old management style is repeated in our political leadership. We can see that for many, power over others is the ultimate drug. We have experienced politicians who hang on to power at any cost and in doing so, sacrifice the principles that may have originally motivated them to seek office. I often wonder how adults can be surprised when children use bully behaviour. It would a surprise if they didn't because adults in powerful positions model that behaviour for them. Many children learn about 'power over others' from abusive parents. The other extreme is overly permissive parents who refuse to set limits on their children's behaviour. This early training sets the tone for our future managers and political leaders. Teaching social responsibility, impulse control, problem solving, reality testing and empathy, a few of the emotional intelligence components would make a positive difference in parenting and in setting a new standard for business and political leadership.
Donna Chevrier Canada
New on M-I.com
Ten habits of bad management
The mortar in a project's wall
The myth of change management
New year questions for team leaders
Why �how are you?� is such an important question
Ideas are fragile (handle with care)
How to take expert advice
The group excluded from diversity programs
Re-skill, re-deploy
A conversation with Dr Rainer Zitelmann
Feelings vs reasoning
Leaders who can read collective emotions are more effective
Why it's good to bring your emotions to work
Emotional intelligence or emotional maturity?
Ten secrets of Emotional Intelligence
Sign up for a monthly round-up of the best from M-I.com. We won't waste your time with spam and we'll never share your details without your permission.
More by Dan
Engaging change: six tips for surviving and thriving
So you're a manager. Now what?
Training is a profit center, not an expense
The price of poor listening
Believe, and help others see why they should, too
Is learning part of your strategic plan?
How to get your ideas adopted
Leveraging the gift of crisis
Building a championship business
Is communication the problem, or teamwork?
Why culture is more important than strategy
Office politics: playing the game
Extreme Trust: Honesty as a Competitive Advantage
Don Peppers and Martha Rogers
Dont Be A Cant - A Manual for Happiness
Frank Bastow
You're Not Listening: What You're Missing and Why It Matters
Kate Murphy
GRACE: A Leader's Guide To A Better Us
John Baldoni
Copyright © 2000 - 2020 Management-Issues.com, except where otherwise noted. | Powered by SedaSoft


text:
How To Get Out Of The Developer Room In Fallout 4 – TheGamer
Wondering how to get into the secret developer room in Fallout 4? Here's what you need to know.
Bethesda's second outing as the developers of this acclaimed RPG franchise is full of items, locations, and secrets to discover throughout its sprawling map. Indeed, in Fallout 4, this search for new weapons, armor, and crafting materials can often be part of the fun.
RELATED: Fallout 4: How To Get A Fat Man And Install The MIRV Modification
However, sometimes, as you journey across the Commonwealth, you want to get an item quickly. Luckily for players who are looking for this option, the developers equipped the PC version of the game with a secret developer room that contains every item to be found throughout the entire game.
For PC players, it is a relatively simple endeavor to access the secret developer room. Simply press your ~ (tilde) key to access your console commands. Once the game's console commands have been brought up, type "COC QASMOKE". Following this, you will be transported to what looks like a rundown old town square and have instant access to every item in Fallout 4.
The developer room has every single item available in Fallout 4. The area features a number of chests containing different types of items, from consumables to scrap and all the game's holotapes.
On top of every building material and item, you will also be able to access every weapon and piece of armor, even special and quest-specific ones. The room even features a series of power armor stands, so you can see what your new outfit will look like before taking it back to the commonwealth.
Once you have filled your inventory with rare and powerful items, there will come a time when you want to leave the developer room. Leaving is essentially the same as entering, press your tilde key, and this time type "COC SANCTUARYEXT" and you will be transported back to the main map, and specifically to Sanctuary.
Unfortunately, there is no built-in way to access a developer room in the console version of Fallout 4, this is primarily down to the lack of console commands in this version of the game. However, since Bethesda added the functionality to download mods on console, a number of different, fan-made, "developer" rooms have become available. Simply access the Mods section and search for it there.
NEXT: Fallout 4: Things You Should Never Do
An avid writer and gamer, Henry has worked for publications such as News UK and is now writing for TheGamer and GameRant. Henry is looking to share his passion for this great medium with others.


How to factory reset UTSTARCOM C1161? How to wipe all data in UTSTARCOM C1161? How to bypass screen lock in UTSTARCOM C1161? How to restore defaults in UTSTARCOM C1161?
The following tutorial shows all method of master reset UTSTARCOM C1161. Check out how to accomplish hard reset by hardware keys and Vendor settings. As a result your UTSTARCOM C1161 will be as new and your core will run faster.


text:
Cal MacAninch (born 24 November 1963) is a Scottish actor, who is known for portraying the character of DI John Keenan in police drama HolbyBlue on BBC1 (from 2007–08). Other notable appearances were his roles as Mr Thackeray in the ITV period drama, Mr Selfridge, and Henry Lang in Downton Abbey, but he has played many leading roles in British television and film over the last 30 years.

Biography
MacAninch played Rowan Collins in Series 4 of the ITV drama series Wild at Heart. He returned to the show for its fifth series as a main cast member. He also played Tauren, a sorcerer in the BBC television series Merlin in the episode "To Kill the King". He starred in the 2001 BBC miniseries The Best of Both Worlds with Alice Evans. He appeared as Henry Lang, a valet, in the second series of Downton Abbey.

Family, personal life
MacAninch and his wife, actress Shauna Macdonald, have three daughters. MacAninch is a keen marathon runner.

Radio

Television

web|url=https://www.bbc.co.uk/mediacentre/mediapacks/mayflies|title=Mayflies - Meet the cast and creatives behind the drama that 'discovers the joy and the costs of love' |website=bbc.co.uk/mediacentre|date=19 December 2022|accessdate=20 December 2022}}</ref>

Film

References

External links

 Works of Robert Burns, as read by MacAninch

1963 births
Living people
Scottish male radio actors
Scottish male television actors
Male actors from Glasgow

redpajama_set_name: RedPajamaWikipedia