text:
\section{Introduction: Exception Mining for Relational Data}
Exception mining is an important data analysis task in many domains. For relational data, exception mining supports outlier detection, where statistical deviations are viewed as due to a node or entity being genuinely exceptional, rather than due to statistical noise in the data. Statistical approaches to unsupervised exception/outlier detection are based on a generative model of the data~\cite{aggarwal2013}.
The generative model represents normal behavior. An individual object is deemed an outlier if  the model assigns sufficiently low likelihood to generating it.
Following the well-established Exceptional Model Mining framework \cite{Duivesteijn2016}, we propose a new method for extending statistical  outlier detection to the case of object-relational data using a novel likelihood-ratio comparison for generative probabilistic models.

The object-relational data model is one of the main data models for structured data~\cite{Koller1997}. The main
characteristics of objects that we utilize in this paper are the following. (1) {\em Object Identity.} Each object has a unique identifier that is the same across contexts. For example, a player has a name that identifies him in different matches. (2) {\em Class Membership.} An object is an instance of a class, which is a collection of similar objects. Objects in the same class share a set of attributes. For example, van Persie is a player object that belongs to the class striker, which is a subclass of the  class player. Note that this use of the term ``class'' is different from the machine learning sense of ``class'' as a prediction target. (3) {\em Object Relationships.} Objects are linked  to other objects. Both objects and their links have attributes. A common type of object relationship is a component relationship between a complex object and its parts.
For example, a match links two teams, and each team comprises a set of players for that match. A difference between relational and vectorial data is therefore that an individual object is characterized not only by a list of attributes, but also by its links and by attributes of the object linked to it. We refer to the substructure comprising this information as the {\em object data}. Equivalent terms are ``egonet'' from network analysis \cite{Akoglu2015} and ``interpretation'' \cite{Maervoet2012}. Relational outlier detection aims to identify objects whose data differ from the general population or class. Our approach to this problem leverages statistical-relational model discovery, as follows.

\paragraph{Approach}
A class-model Bayesian network (BN) structure is learned with data for the entire population. The nodes in the BN represent attributes for links, of multiple types, and attributes of objects, also of multiple types. To learn the BN model, we apply techniques from statistical-relational learning, a  recent field that combines AI and machine learning \cite{SRL2007,Schulte2012,Domingos2009}.
Given a set of parameter values and an input database, it is possible to compute a {\em class model likelihood} that quantifies how well the BN fits the object data. The class model likelihood uses BN parameter values {\em estimated from the entire class data.} This  is a relational extension of the standard log-likelihood method for i.i.d. vectorial data, which uses the likelihood of a data point as its outlier score.
 While the class model likelihood is a good baseline score, it can be improved by comparing it to {\em the object model likelihood}, which uses BN parameter values {\em estimated from the object data.}
The {\em model log-likelihood ratio} (LR) is the log-ratio of the object model likelihood to the class model likelihood. This ratio quantifies how the probabilistic associations that hold in the general population deviate from the associations in the object data substructure.
While the
likelihood ratio discriminates relational outliers better than the class model likelihood alone, it can be improved further by applying two transformations: (1) a mutual information decomposition, and (2) replacing log-likelihood differences by log-likelihood distances. We refer to the resulting novel score as the {\em log-likelihood distance}.
\paragraph{Evaluation} \label{sec:eval} Our code and datasets are available on-line at \cite{url}.
Our performance evaluation follows the design of previous outlier detection studies~\cite{Gao2010,aggarwal2013},
where the methods are scored against a test set of known outliers.
We use three synthetic and two real-world datasets, from the UK Premier Soccer League and the Internet Movie Database (IMDb). On the synthetic data we have known ground truth. For the real-world data, we use a one-class design, where one object class is designated as normal and objects from outside the class are the outliers. For example, we compare goalies as outliers against the class of strikers as normal objects.
On all datasets, the log-likelihood distance metric achieves the best detection accuracy compared to baseline methods.

We also offer case studies where we assess whether individuals that our score ranks as highly unusual in their class are  indeed unusual.
The case studies illustrate that our outlier score is {\em easy to interpret}, because the Bayesian network provides a sum decomposition of the data distributions by features. Interpretability is very important for users of an outlier detection method as there is often no ground truth to
evaluate outliers suggested by the method.

\paragraph{Related Work} Section~\ref{sec:related} discusses the relationship to related work in detail. Our approach applies the exceptional model mining (EMM) framework~\cite{Duivesteijn2016} to multi-relational data. Figure~\ref{fig:emm} illustrates the EMM schema. \begin{figure}[t]
\centering
\includegraphics[width=0.4\textwidth]{emm.pdf}
\caption{A general schema for Exceptional Model Mining  for propositional data
\label{fig:emm}}
\end{figure}
The EMM framework leverages the extensive work on model learning in machine learning for exception mining: A subgroup is exceptional to the extent that a model learned from data for the subgroup deviates from a model learned for the general population. A computational method for measuring this extent is called a quality measure; we also refer to it as an outlierness metric. For a given model type, finding an appropriate quality measure for quantifying exceptionality is the main research question in EMM. The EMM framework allows us to leverage the extensive work on statistical-relational model learning for exception mining in multi-relational data. Compared to previous EMM models, the novelty of our work is as follows. 1) EMM has so far been developed only for propositional i.i.d. data, not relational data. Accordingly EMM has not been applied with SRL models. 2) In the propositional i.i.d. setting, each object is represented by a single data row, and it is meaningless to learn a model for a single object. Instead, EMM is applied to identify exceptional subgroups of objects. With relational data, each object is represented by its own dataset (egonet, interpretation), and it is meaningful to apply EMM to identify single exceptional objects.
Compared to previous relational outlier detection work, our model-based approach is novel in that it neither summarizes the object data by a feature set (as in the Oddball system, see \cite{Akoglu2015}) nor looks for rules that exceptional objects violate (e.g. \cite{Maervoet2012}).

\paragraph{Contributions} Our main contributions may be
 summarized as follows.

\begin{enumerate}
	\item The first EMM approach to outlier detection for structured data that is based on a probabilistic model.
	\item A new outlier score based on a novel model likelihood comparison, the log-likelihood distance.
	\end{enumerate}

\paragraph{Paper Organization} We review background about Bayesian networks for relational data. Then we describe how we apply the EMM framework to multi-relational data. We introduce a novel log-likelihood distance outlier score as the quality or outlierness metric. After presenting the details of our approach, we review related work. Empirical evaluation compares model-based and aggregation-based approaches to relational outlier detection, with respect to three synthetic and three real-world problems.

\section{Background: Bayesian Networks for Relational Data}
We adopt
 the Parametrized Bayes net (PBN) formalism \cite{Poole2003} that combines Bayes nets with logical syntax for expressing relational concepts. EMM is an inclusive framework and can in principle be applied with other SRL models, such as Markov Logic networks \cite{Domingos2009}. We worked with PBNs because i) they offer the most scalable structure learning methods \cite{Schulte2012a} to support our larger datasets, and ii) the PBN conditional probability parameters can be easily interpreted, which means that the resulting exceptionality metrics can be easily interpreted  (see Section~\ref{sec:eval} below).

\subsection{Bayesian Networks}

A {\bf Bayesian Network (BN)} is a directed acyclic graph (DAG) whose nodes comprise a set of random variables \cite{Pearl1988}. Depending on context, we interchangeably refer to the nodes  and variables of a BN. Fix a set of variables $\bs{V} = \{\feature_{1},\ldots,\feature_{n}\}$.
The possible values of $\feature_{i}$ are enumerated as $\{\v_{i1},\ldots,\v_{i\states_{i}}\}$. The notation $P(\feature_{i} = \v)\equiv P(\v)$ denotes the probability of variable $\feature_{i}$ taking on value $\v$. We also use the vector notation $P(\bs{V} = \set{\v}) \equiv P(\set{\v})$ to denote the joint probability that each variable $\feature_{i}$ takes on value $\set{\v}_{i}$.

The conditional probability parameters of a Bayesian network specify the distribution of a child node given an assignment of values to its parent node. For an assignment of values to its nodes, a BN defines the joint probability as the product of the conditional probability of the child node value given its parent values, for each child node in the network. This means that the log-joint probability can be {\em decomposed} as the node-wise sum

\begin{equation} \label{eq:bn}
\ln P(\bs{V} = \set{\v};B,\bs{\parameter}) = \sum_{i=1}^{n} \ln \theta(\set{\v}_{i}|\set{\v}_{\mathbf{pa}_{i}})
\end{equation}

where $\set{\v}_{i}$ resp. $\set{\v}_{\mathbf{pa}_{i}}$ is the assignment of values to node $\feature_{i}$ resp. the parents of $\feature_{i}$ determined by the assignment $\set{\v}$.
To avoid difficulties with $\ln(0)$, here and below we assume that joint distributions are positive everywhere. Since the parameter values for a Bayes net define a joint distribution over its nodes, they therefore entail a marginal, or unconditional, probability for a single node. We denote the \textbf{marginal probability} that node $\feature$ has value $\v$ as $P(\feature = \v;B,\bs{\parameter}) \equiv \theta(\v)$.

\paragraph{Example.} Figure~\ref{fig:bns} shows an example of a Bayesian network and associated joint and marginal probabilities.
\begin{figure}[t]
	\centering
	\includegraphics[width=0.4\textwidth]
	{bnexamples-edited}
	\caption{Example of joint and marginal probabilities computed from a toy Bayesian network structure. The parameters were estimated from the  Premier League dataset. (Top): A class model Bayesian network $B_{\class}$ for all teams with class parameters $\bs{\parameter}_{\class}$. (Bottom): The same Bayesian network structure with object parameters $\bs{\parameter}_{o}$ learned for Wigan Athletics ($T = WA$). Our model-based methods outlier scores compare the data likelihood of the class parameters and the object parameters.

		\label{fig:bns}
		}
\end{figure}

\begin{figure}[t]
\centering
\includegraphics[width=0.4\textwidth]{emm-srl.pdf}
\caption{The EMM approach for statistical-relational models. The model class we utilize in this paper are Parametrized Bayesian networks, with a log-linear likelihood function. As outlierness metrics we consider the standard Kullback-Leibler divergence, and a novel divergence introduced in this paper.
\label{fig:flow}}
\end{figure}

\subsection{Relational Data}

 We adopt a functor-based notation for combining logical and statistical concepts~\cite{Poole2003,Kimmig2014}.
A functor is a function or predicate symbol. Each functor has a set of values (constants) called the \textbf{domain} of the functor. The domain of a \textbf{predicate} is $\{\mathit{T},\mathit{F}\}$. Predicates are usually written with uppercase Roman letters, other terms with lowercase letters.
A predicate of arity at least two is a \textbf{relationship} functor. Relationship functors specify which objects are linked. Other functors represent \textbf{features} or \textbf{attributes} of an object or a tuple of objects (i.e., of a relationship).
A \textbf{population} is a set of objects.
A \textbf{term} is of the form $f(\sigma_{1},\ldots,\sigma_{k})$ where $f$ is a functor
and each $\sigma_{i}$ is a first-order variable or a constant denoting an object. A term is \textbf{ground} if it contains no first-order variables; otherwise it is a first-order term. In the context of a statistical model, we refer to first-order terms as \textbf{Parametrized Random Variables} (PRVs) \cite{Kimmig2014}.
A \textbf{grounding} replaces each first-order variable in a term by a constant; the result is a ground term. A grounding may be applied simultaneously to a set of terms.  A relational database $\D$ specifies the values of all ground terms, which can be listed in data tables.

Consider a joint assignment
$P(\bs{V} = \set{\v})$ of values to a set of PRVs $\bs{V}$. The {\em grounding space} of the PRVs is the set of all possible grounding substitutions, each applied to all PRVs in $\bs{V}$. The {\em count} of groundings that satisfy the assignment with respect to a database $\D$ is denoted by $\#_{\D}(\bs{V} = \set{\v})$. The \textbf{database frequency} $P_{\D}(\bs{V} = \set{\v})$ is the grounding count divided by the number of all possible groundings.

\emph{Example.} \label{sec:example}
The Opta dataset represents information about premier league data
(Sec.~\ref{sec:real-world-data}).
The basic populations are teams, players, matches, with
corresponding first-order variables $\it{T}, \it{P}, \it{M}$. Table~\ref{table:data} specifies values for some ground terms. The first three column headers show first-order variables ranging over different populations. The remaining columns represent features. Table~\ref{table:counts} illustrates grounding counts. Counts are based on the 2011-2012 Premier League Season. We count only groundings $(\it{team},\it{match})$ such that $\it{team}$ plays in $\it{match}$. Each team, including Wigan Athletics, appears in 38 matches. The total number of team-match pairs is $38 \times 20 = 760$.

\begin{table}[htbp]
\caption{Sample Population Data Table (Soccer). \label{table:data}}
\centering
\resizebox{0.5\textwidth}{!}{
\begin{tabular}{|c|c|l|c|c|c|c|}
\hline
 \multicolumn{1}{|l|}{MatchId \it{M}} &
 TeamId \it{T} & PlayerId \it{P}& \multicolumn{1}{l|}{First\_goal(\it{P},\it{M})}
& \multicolumn{1}{l|}{TimePlayed(\it{P},\it{M})} &
\multicolumn{1}{l|}{ShotEff(\it{T},\it{M})}&result(\it{T},\it{M}) \\ \hline
 117 & WA & McCarthy & 0 & 90 & 0.53&\it{win}\\ \hline
 148 & WA & McCarthy & 0 & 85 & 0.57&\it{loss}\\ \hline
 15 & MC & Silva & 1 & 90 & 0.59&\it{win}\\ \hline
  \ldots& \ldots &\ldots&\ldots&\ldots&\ldots&\\
\end{tabular}}
\caption{Sample Object Data Table, for team $\it{T} = \it{WA}$. \label{table:individual}}
\resizebox{0.5\textwidth}{!}{
\begin{tabular}{|c|c|l|c|c|c|c|}
\hline
 \multicolumn{1}{|l|}{MatchId \it{M}} &
 TeamId $\it{T} = \it{WA}$ & PlayerId \it{P}& \multicolumn{1}{l|}{First\_goal(\it{P},\it{M})}
& \multicolumn{1}{l|}{TimePlayed(\it{P},\it{M})} &
\multicolumn{1}{l|}{ShotEff(\it{WA},\it{M})}&result(\it{WA},\it{M}) \\ \hline
 117 & WA & McCarthy & 0 & 90 & 0.53&\it{win}\\ \hline
 148 & WA & McCarthy & 0 & 85 & 0.57&\it{loss}\\ \hline
   \ldots& WA &\ldots&\ldots&\ldots&\ldots&\\
\end{tabular}}
\end{table}

A novel aspect of our paper is that we learn model parameters for specific objects as well as for the entire population.
The appropriate \textbf{object data table} is formed from the population data table by restricting the relevant first-order variable to the target object.
For example, the object database for target Team $\it{Wigan Athletic}$,
forms a subtable of the data table of Table~\ref{table:data} that contains only rows where
TeamID = $\it{WA}$; see Table~\ref{table:individual}. In database terminology, an object database is like a view centered on the object. The object database is an individual-centered representation \cite{Flach1999a}.

\begin{table}
	\caption{Example of Grounding Count and Frequency in Premier League Data, for the conjunction $\it{passEff(T,M)=hi}, shotEff(T,M)=hi, Result(T,M)=win$.\label{table:counts}}
	\centering
	\resizebox{0.5\textwidth}{!}{
		\begin{tabular}{|c|c|c|}
			\hline
			Database&Count or $\#_{D}(\bs{V} = \set{\v})$&Frequency or $P_{D}(\bs{V} = \set{\v})$\\\hline
			Population&76& $76/760=0.10$\\\hline
			Wigan Athletics&7&$7/38=0.18$\\\hline

		\end{tabular}}
	\end{table}

\subsection{Bayesian Networks for Relational Data}

A \textbf{Parametrized Bayesian Network Structure} (PBN) is a Bayesian network structure  whose nodes are PRVs.
The relationships and features in an object database define a set of nodes for Bayes net learning; see Figure~\ref{fig:bns}.

\subsubsection{Model Likelihood for Parametrized Bayesian Networks}

A standard method for applying a generative model assumes that the generative model represents normal behavior since it was learned from the entire population. An object is deemed an outlier if the model assigns sufficiently low likelihood to generating its features \cite{Cansado2008}. This likelihood method is an important baseline for our investigation.
Defining a likelihood for relational data is more complicated than for i.i.d. data, because an object is characterized not only by a feature vector, but by an object  database.
We employ the relational pseudo log-likelihood \cite{Schulte2011}, which can be computed as follows for a given Bayesian network  and database.

\begin{equation} \label{eq:likelihood}
\it{LOG}(\D,B,\bs{\parameter}) =   \sum_{i=1}^{n}\sum_{j=1}^{\states_{i}} \sum_{\mathbf{pa}_{i}}\P_{\D}(\v_{ij},\mathbf{pa}_{i})\ln \theta(\v_{ij}|\mathbf{pa}_{i})
\end{equation}

Equation~\eqref{eq:likelihood} represents the standard BN log-likelihood function for the object  data~\cite{Campos2006}, except that parent-child instantiation counts are standardized to be proportions \cite{Schulte2011}. The equation can be read as follows.

\begin{enumerate}
\item For each parent-child configuration,
use the conditional probability of the child given the parent.
\item Multiply the logarithm of the conditional probability by the database frequency of the parent-child configuration.
\item Sum this product over all parent-child configurations and all nodes.
\end{enumerate}

Schulte proves that the maximum of the pseudo-likelihood ~\eqref{eq:likelihood} is given by the empirical database frequencies \cite[Prop.3.1.]{Schulte2011}. In all our experiments we use these maximum likelihood parameter estimates.

{\em Example.} The family configuration \begin{displaymath} \it{passEff(T,M)=hi}, shotEff(T,M)=hi, Result(T,M)=win\end{displaymath} contributes one term to the pseudo log-likelihood for the BN of Figure~\ref{fig:bns}. For the population database, this term is $0.1 \times \ln(0.44) =-0.08 $. For the  Wigan Athletics database, the term is $0.18 \times \ln(0.44) =-0.14 $.

\section{EMM for Relational Data} \label{sec:metrics}

This section describes our approach to applying the EMM framework to relational data, using the following notation.
\begin{itemize}
\item $\D_{C}$ is the database for the entire class of objects; cf. Table~\ref{table:data}. This database defines the \textbf{class distribution} $P_{C} \equiv P_{\D_{\class}}$.
\item $\D_{o}$ is the restriction of the input database to the target object; cf. Table~\ref{table:individual}. This database defines the \textbf{object distribution} $P_{o} \equiv P_{\D_{o}}$.
\item $B_{C}$ is a model (e.g., Bayesian network) learned with $\D_{\mathcal{P}}$ as the input database; cf. Figure~\ref{fig:bns}(a).
\item $\bs{\parameter}_{C}$ resp. $\bs{\parameter}_{o}$ are parameters learned for $B_{C}$ using $\D_{\class}$ resp. $\D_{o}$ as the input database.
\end{itemize}

Figure~\ref{fig:flow} illustrates these concepts and the system flow for computing an outlierness score. First, we learn a Bayesian network $B_{C}$ for the entire population using a previous learning algorithm (see Section~\ref{sec:methods} below). We then evaluate {\em how well the class model fits the target object data.} For vectorial data, the  standard model fit metric
is the log-likelihood of the target {\em datapoint}. For relational data, the counterpart is the relational log-likelihood \eqref{eq:likelihood} of the target {\em database}:

\begin{equation} \label{eq:loglikelihood-score}
\it{LOG}(\D_{o},B_{C},\bs{\parameter}_{C}).
\end{equation}

While this
is a good baseline outlier score, it can be improved by considering scores based on the likelihood ratio, or {\bf log-likelihood difference}:

\begin{equation} \label{eq:log-diff}
\it{LR}(\D_{o},B_{C},\bs{\parameter}_{o}) \equiv \it{LOG}(\D_{o},B_{C},\bs{\parameter}_{o}) - \it{LOG}(\D_{o},B_{C},\bs{\parameter}_{C}).
\end{equation}

The log-likelihood difference compares  how well the class-level parameters fit the object data, vs. how well the object parameters fit the object data. In terms of the conditional probability parameters, it measures how much the log-conditional probabilities in the class distribution differ from those in the object distribution. Note that this definition applies only for relational data where an individual is characterized by a substructure rather than a ``flat'' feature vector. Assuming maximum likelihood parameter estimation, $\it{LR}$ is equivalent to the Kullback-Leibler divergence between the class-level and object-level parameters~\cite{Campos2006}.
While the $\it{LR}$ score provides more outlier information than the model log-likelihood, it can be improved further by two transformations as follows. (1) Decompose the joint probability into a single-feature component and a mutual information component. (2) Replace log-likelihood differences by log-likelihood distances. The resulting score is the \textbf{log-likelihood distance} ($\mid$), which is the main novel score we propose in this paper. Formally it is defined as follows for each feature $i$. The total score is the sum of feature-wise scores. Section~\ref{sec:divergence-examples}
 below provides example computations.

\begin{equation} \label{eq:log-dist}
\begin{array}{l}

		\mid_{i} =\sum_{j=1}^{\states_{i}} P_{o}(\v_{ij}) \left|\ln \frac{\theta_{o}( \v_{ij})}{\theta_{C}( \v_{ij})}\right|+\\
		\sum_{j=1}^{\states_{i}} \sum_{\mathbf{pa}_{i}}
		P_{o}( \v_{ij},\mathbf{pa}_{i})
		\left|\ln \frac{\theta_{o}( \v_{ij}|\mathbf{pa}_{i})}{\theta_{o}(\v_{ij})} - \ln \frac{\theta_{C}( \v_{ij}|\mathbf{pa}_{i})}{\theta_{C}(\v_{ij})}\right|. \end{array}
		\end{equation}

The first sum is the \textbf{single-feature} component, where each feature is considered independently of all others. It computes the expected log-distance with respect to  the singe feature value probabilities between the object and the class models.
The second $\mid$ sum is the \textbf{mutual information component}, based on the mutual information among all features. It computes the expected log-distance between the object and the class models with respect to the mutual information of feature value assignments.
Intuitively, the first sum measures how the models differ if we treat each feature in isolation. The second sum measures how the models differ in terms of how strongly parent and child features are associated with each other.

\subsection{Motivation}
The motivation for the mutual information decomposition is two-fold.

\noindent
(1) {\em Interpretability}, which is very important for outlier detection. The single-feature components are easy to interpret since they involve no feature interactions. Each parent-child local factor is based on the average relevance of parent values for predicting the value of the child node, where relevance is measured by $\ln (\theta(\v_{ij}|\mathbf{pa}_{i})/\theta(\v_{ij}))$. This relevance term  is basically the same as the widely used lift measure \cite{Tuffery2011}, therefore an intuitively meaningful quantity. The $\mid$ score compares how relevant a given parent condition is in the object data with how relevant it is in the general class.

\noindent
(2) {\em Avoiding cancellations.} Each term in the log-likelihood difference \eqref{eq:log-diff} decomposes into a relevance difference and a marginal difference:

\begin{equation} \label{eq:decompose}
\ln \frac{\theta_{o}( \v_{ij}|\mathbf{pa}_{i})}{\theta_{C}( \v_{ij}|\mathbf{pa}_{i})} = \ln \frac{\theta_{o}(\v_{ij}|\mathbf{pa}_{i})}{\theta_{o}(\v_{ij})} - \ln \frac{\theta_{C}( \v_{ij}|\mathbf{pa}_{i})}{\theta_{C}(\v_{ij})} + \ln \frac{\theta_{o}( \v_{ij})}{\theta_{C}( \v_{ij})}.
\end{equation}

These differences can have different signs for different child-parent configurations and cancel each other out; see Table~\ref{table:Formula}. Taking distances as in Equation~\ref{eq:log-dist} avoids this undesirable cancellation. Since our goal is to assess the distinctness of an object, {\em we do not want differences to cancel out.} The general point is that averaging differences is appropriate when considering costs, or utilities, but not appropriate for assessing the distinctness of an object. For instance, the average of both vectors (0,0) and (1,-1) is 0, but their distance is not.

\subsection{Comparison Outlier Scores} Our lesion study compares our log-likelihood distance  $\mid$ score to baselines that are defined by omitting a component of $\mid$. In this section we define these scores.

  The scores increase in sophistication in the sense that they apply more transformations of the log-likelihood ratio.
More sophisticated scores provide more information about outliers.
Table~\ref{table:Formula} defines local feature scores; the total score is the sum of feature-wise scores. All metrics are defined such that {\em a higher score indicates a greater anomaly.} The metrics are as follows.

\begin{LaTeXdescription}
\item[Feature Divergence $\it{FD}$] is the first  component of the $\mid$ score. It considers each feature independently (no feature correlations).
\item[Log-Likelihood Score \it{LOG}] is the standard model-based outlier detection score using data likelihood.
\item[Log-Likelihood Difference \it{LR}] is the log-likelihood difference~\eqref{eq:log-diff} between the class-level and object-level parameters.
\item[Log-Likelihood Difference with absolute value $|\it{LR}|$] replaces differences in $\it{LR}$ by distances.
\item[Log-Likelihood Difference with decomposition $\it{LR}^{+}$] applies a mutual information decomposition to $\it{LR}$.
\end{LaTeXdescription}

	\begin{table}
		\caption{Baseline Outlier Scores for Bayesian Networks
			\label{table:Formula}}
		\resizebox{0.5\textwidth}{!}{
			\begin{tabular}{|l|l|} \hline
					Method & Formula\\
					\hline

							$\it{FD}_{i}	$&	$\begin{array}{l}\sum_{i=1}^{n}\sum_{j=1}^{\states_{i}} P_{o}( \v_{ij}) \left|\ln \frac{\theta_{o}( \v_{ij})}{\theta_{C}( \v_{ij})}\right|\end{array}	$\\ \hline

						$-\it{LOG}_{i}$& $  \begin{array}{l} -\sum_{i=1}^{n}\sum_{j=1}^{\states_{i}} \sum_{\mathbf{pa}_{i}} P_{o}( \v_{ij},\mathbf{pa}_{i})\ln \theta_{C}( \v_{ij}|\mathbf{pa}_{i})\end{array}$ \\ \hline
					$\it{LR}_{i}$&$\begin{array}{l}  \sum_{j=1}^{\states_{i}} \sum_{\mathbf{pa}_{i}} P_{o}( \v_{ij},\mathbf{pa}_{i})\ln \frac{\theta_{o}(  \v_{ij}|\mathbf{pa}_{i})}{\theta_{C}( \v_{ij}|\mathbf{pa}_{i})}.  \end{array}$\\	\hline
					$|\it{LR}_{i}|$& $\begin{array}{l}  \sum_{j=1}^{\states_{i}} \sum_{\mathbf{pa}_{i}} P_{o}( \v_{ij},\mathbf{pa}_{i})|\ln \frac{\theta_{o}(\v_{ij}|\mathbf{pa}_{i})}{\theta_{C}( \v_{ij}|\mathbf{pa}_{i})}|. \end{array}$ \\	\hline
					$\it{LR}_{i}^{+}$&$\begin{array}{l}  \sum_{j=1}^{\states_{i}} P_{o}( \v_{ij}) \ln \frac{\theta_{o}( \v_{ij})}{\theta_{C}( \v_{ij})}+
					\\ \sum_{j=1}^{\states_{i}} \sum_{\mathbf{pa}_{i}}
					P_{o}( \v_{ij},\mathbf{pa}_{i})
					\ln \frac{\theta_{o}( \v_{ij}|\mathbf{pa}_{i})}{\theta_{o}(\v_{ij})} - \ln \frac{\theta_{C}( \v_{ij}|\mathbf{pa}_{i})}{\theta_{C}(\v_{ij})}.  \end{array}$ \\ \hline

			\end{tabular}
		}
	\end{table}

\begin{figure*}[htbp]
	\centering
	\resizebox{0.85\textwidth}{!}{
		\includegraphic
		{figures/figure1kddCropped.pdf}
	}
	\caption{Illustrative Bayesian networks. The networks are not learned from data, but hand-constructed to be plausible for the soccer domain. (a) High Correlation: Normal individuals exhibit a strong association between their features, outliers no association. Both normals and outliers have a close to uniform distribution over single features.

		(b) Low Correlation: Normal individuals exhibit no association between their features, outliers have a strong association. Both normals and outliers have a close to uniform distribution over single features.

		(c) Single Attributes: Both normal and outlier individuals exhibit a strong association between their features. In normals, 90\% of the time, feature 1 has value 0. For outliers, feature 1 has value 0 only 10\% of the time.

		\label{fig:synthetic-bns}}
\end{figure*}

\section{Examples} \label{sec:divergence-examples}

We provide three simple examples with only two features that illustrate the computation of the outlier scores. They are designed so that outliers and normal objects are easy to distinguish, and so that it is easy to trace the behavior of an outlier score.
The examples therefore serve as thought experiments that bring out the strengths and weaknesses of model-based outlier scores.
Figure~\ref{fig:synthetic-bns} describes the BN representation of the examples. Table~\ref{table:example} provides the computation of the scores. For intuition, we can think of a soccer setting, where each match assigns a value to each attribute $F_i, i =  1,2$ for each player.
Scores for the $F_{2}$ feature are computed conditional on $F_{1} = 1$. Expectation terms are computed first for $F_{2} = 1$, then $F_{2} = 0$.

The single feature distributions are uniform, so the feature component $\mid_{1}$
		is 0 for each node in both examples.
The table illustrates the undesirable cancelling effects in $\it{LR}$. In the high correlation scenario~\ref{fig:synthetic-bns}(a), the outlier object has a lower probability than the normal class distribution of $\it{Match\_Result} = 0$ given that $\it{Shot\_Efficiency} = 1$. Specifically, 0.5 vs. 0.9. The outlier object exhibits a higher probability $\it{Match\_Result} = 1$ than the normal class distribution, conditional on $\it{Shot\_Efficiency} = 1$; specifically, 0.5 vs. 0.1. In line 1, column 2 of Table~\ref{table:example}  the log-ratios $\ln(0.5/0.9)$ and $\ln(0.5/0.1)$ therefore have different signs. In the low correlation scenario~\ref{fig:synthetic-bns}(b), the cancelling occurs in the same way, but with the normal and outlier probabilities reversed.
The cancelling effect is even stronger for attributes with more than two possible values.

\begin{table}[hbpt]
	\caption{Example Computation of different outlier scores.

	 }
		\label{table:example}

	\centering
	\resizebox{0.5\textwidth}{!}{
		\begin{tabular}{|c|p{3cm}|p{6cm}|l|}
			\hline
			Score&$F1=1$ Computation&$F2|F1=1$ Computation&Result\\ \hline
			$\it{LR}$&\begin{tabular}{p{5cm}} $1/2 \ln(0.5/0.5)=0 $\end{tabular}&$1/4\ln(0.5/0.9)+ 1/4\ln(0.5/0.1)$&0.36\\ \hline

			$|\it{LR}|$&$0$ (no parents)&\begin{tabular}{p{5cm}}$1/4 |\ln(0.5/0.5)-\ln(0.9/0.5)|+1/4 |\ln(0.5/0.5)-\ln(0.1/0.5)|$\end{tabular}&0.79\\ \hline
			$\it{FD}$&$|\ln(0.5/0.5)|=0$&\begin{tabular}{p{5.5cm}}$1/2|\ln(0.5/0.5)| + 1/2|\ln(0.5/0.5)|$\end{tabular}&0\\ \hline
			$$\mid$$&$0+0$&$0.79+\it{FD}$

			&0.79

			\\ \hline
		\end{tabular}}

		\resizebox{0.5\textwidth}{!}{
			\begin{tabular}{p{8cm}}
				Table V(a): High Correlation Case, Figure~\ref{fig:synthetic-bns}(a).

			\end{tabular}}

			%
			\centering
			\resizebox{0.5\textwidth}{!}{
				\begin{tabular}{|l|p{3cm}|p{6cm}|l|}
					\hline
					Score&$F1=1$ Computation&$F2|F1=1$ Computation&Result\\ \hline
					$\it{LR}$&$1/2\ln(0.5/0.5)=0$&$0.5 \cdot 0.9 \ln(0.9/0.5)+ 0.5 \cdot 0.1 \ln(0.1/0.5)$&0.26\\ \hline

					$|\it{LR}|$&$0$ (no parents) &$0.5 \cdot 0.9 |\ln(0.9/0.5)-\ln(0.5/0.5)|+ 0.5 \cdot 0.1 |\ln(0.1/0.5)-\ln(0.5/0.5)| $&0.50\\ \hline
					$\it{FD}$&$|\ln(0.5/0.5)|=0$&$1/2|\ln(0.5/0.5)| + 1/2|\ln(0.5/0.5)|$&0\\ \hline
					$\mid$&$0+0$&$0.5+\it{FD}$

					&0.5\\ \hline
				\end{tabular}}
				\resizebox{0.5\textwidth}{!}{
					\begin{tabular}{p{8cm}}
						Table V(b): Low Correlation Case. Figure~\ref{fig:synthetic-bns}(b).

					\end{tabular}}
				\end{table}

				%

				%

				%

				%

\section{Related Work} \label{sec:related}
Outlier detection is a densely researched field, for a survey please see~\cite{aggarwal2013}.
Figure~\ref{fig:novelty} provides a tree picture of where our method is situated with respect to other outlier detection methods and other data models.
Our method falls in the category of {\em unsupervised} statistical model-based approaches. To our knowledge, ours is the first model-based method tailored for object-relational data. Like other model-based approaches, it detects {\em global outliers.} Aggarwal \cite{aggarwal2013} defines a global outlier to be a data point that notably deviates from the rest of the population. We review relevant approaches from different data models, the most common atomic object model---where data is represented by vectors---and structured data models. Akoglu et al. provide an excellent recent survey of outlier detection in relational models~\cite{Akoglu2015}.\\

\textit{a) Attribute Vector Data Model:}
 By far most work on outlier detection considers atomic objects with flat feature vectors.
This leads to an impedance mismatch:
The required input format for these outlier detection methods is a single data matrix, not a structured dataset. For example, one cannot provide a relational database as input. This mismatch is not simply a question of choosing a file format, but instead reflects a different underlying data model: complex objects with both attributes and component objects vs. atomic objects with attributes only.
It is possible to ``flatten'' structured data by converting it to unstructured feature vectors, for instance by using aggregate functions.
We evaluated the aggregation approach in this paper by applying three standard methods for outlier detection.

Work on atomic contextual  outliers \cite{Tang2013} is like ours in that it considers the distinctness of a target individual from a reference class. A reference class is not specified for each object,
but is constructed as part of outlier detection.
Our work could be combined with a class discovery approach by providing a score of how informative the inferred classes are.
\begin{figure}
	\centering
	\includegraphics[width=0.5\textwidth] {NoveltyDiagram}
	\caption{A tree structure for related work on outlier detection for structured data. A path specifies an outlier detection problem, the leaves list major approaches to the problem. Approaches in italics appear in experiments.
		\label{fig:novelty}}
\end{figure}

\textit{b) Structured Data Models:} We discuss related techniques in three types of structured data models: SQL (relational), XML (hierarchical), and OLAP (multi-dimensional).

For relational data, many outlier detection approaches aim to discover rules that represent the presence of anomalous associations for an individual or the absence of normal associations \cite{Maervoet2012,Gao2010}. The survey by \cite{Novak2009} unifies within a general rule search framework related tasks such as exception mining, which looks for associations that characterize unusual cases, subgroup mining, which looks for associations  characterizing important subgroups, and contrast space mining, which looks for differences between classes. Another rule-based approach uses Inductive Logic Programming techniques \cite{Angiulli2007}.
While local rules are informative, they are not based on a global statistical model and do not provide a single outlier score for each individual.

A latent variable approach in information networks ranks potential outliers in reference to the latent communities inferred by network analysis \cite{Gao2010}. Our model aggregates information from entities and links of different types, but does not assume that different communities have been identified.

Koh {\em et al.}~\cite{Koh2008} propose a method for hierarchical structures represented in XML document trees. Their aim is to identify feature outliers, not class outliers as in our work. Also, they use aggregate functions to convert the object hierarchy into feature vectors. Their outlier score is based on local correlations, and they do not construct a model.

The multi-dimensional data model defines numeric measures for a set of dimensions.
The differences in the two data models mean that multi-dimensional outlier detection models~\cite{Sarawagi1998} do not carry over to object-relational outlier detection. (1) The object data model allows but does not require any numeric measures. In our datasets, all features are discrete. Nor do we assume that it is possible to aggregate numeric measures to summarize lower-level data at higher levels.
(2) In scoring a potential outlier object, our method considers other objects {\em both} below and above the target object in the component hierarchy. OLAP exploration methods consider only cells below or at the same level as the target cell. For example, in scoring a player, our method would consider features of the player's team.
Also, the $\mid$ outlier score of an object is not determined by the outlier scores of its components, in contrast to the approach of Sarawagi {\em et al.}.
(3) Our approach models a joint distribution over features, exploiting correlations among features. Most of the OLAP-based methods consider only a single numeric measure at a time, not a joint model.

Statistical data cleaning methods are related to outlier detection, in that erroneous data may be detected as outliers (e.g., \cite{de2016bayeswipe}). Nonetheless, these data cleaning methods differ from our work in several ways. 1) Although they often originate in the database community, they are usually developed only for single-table propositional data, not relational data. (An exception is the ERACER system \cite{Mayfield2010}.) 2) Our work assumes that the data is (mainly) correct, and identifies exceptional identities for the given data. 3) Data cleaning methods focus on unusual values or tuples (e.g. a mistaken rating for a movie by a user), not exceptional subdatabases or egonets.

				\section{Experimental Design}
All the experiments were performed on a 64-bit Centos machine with 4GB RAM and an Intel Core i5-480 M processor. The likelihood-based outlier scores were computed with SQL queries using JDBC, JRE 1.7.0. and MySQL Server version 5.5.34.
We describe the datasets used in our experiments.

				\subsection{Synthetic Datasets}

				We generated three synthetic datasets with normal and outlier players using the distributions represented in the three Bayesian networks of Figure~\ref{fig:synthetic-bns}.

				Each player participates in 38 matches, similar to the real-world data. The main goal of designing synthetic experiments is to test the methods on  easy to detect outliers. Each match assigns a value to each feature $F_i, i =  1,2$ for each player.
				\begin{LaTeXdescription}
				\item\textbf{High Correlation} See Figure~\ref{fig:synthetic-bns}(a).
				\item\textbf{Low Correlation} See Figure~\ref{fig:synthetic-bns}(b).
				\item\textbf{Single features} See Figure~\ref{fig:synthetic-bns}(c).
				\end{LaTeXdescription}

				%
				We used the $\it{mlbench}$ package in $\it{R}$ to generate synthetic features in matches, following these distributions for 240 normal players and 40 outliers. We followed the real-world Opta data in terms of number of normal and outlier individuals. The scores are used to rank all 280 players.
				\subsection{Real-World Datasets} \label{sec:real-world-data}
				Data tables are prepared from Opta data~\cite{opta-original} and IMDb~\cite{IMDb-original}. Our datasets and code are available on-line~\cite{bib:jbnsite}.
				\paragraph{Soccer Data}
The Opta data were released by Manchester City.
It lists box scores, that is, counts of all the ball actions within each game by each player, for the 2011-2012 season.
For each player in a match, our data set contains eleven player features.
For each team in a match, there are five features computed as player feature aggregates, as well as the team formation and the result (win, tie, loss).
There are two relationships, $\it{Appears\_Player}(\P,M)$, $\it{Appears\_Team}(\T,M)$.

				\paragraph{IMDB Data}
The Internet Movie Database (IMDB) is an on-line database of information related to films, television programs, and video games.
The IMDB website offers a dataset containing information on cast, crew, titles, technical details and biographies into a set of compressed text files.
We preprocessed the data like \cite{Peralta2007} to obtain a database with seven tables: one for each population and one for the three relationships $\it{Rated}(\it{User},\it{Movie})$, $\it{Directs}(\it{Director},\it{Movie})$, and $\it{ActsIn}(\mathit{Actor},\it{Movie})$.

					In real-world data, there is no ground truth about which objects are outliers. To address this issue, we employ a one-class design: we learn a model for the class distribution, with data from that class only. Then we rank all individuals from the normal class together with all objects from a contrast class treated as outliers, to test whether an outlier score recognizes objects from the contrast class as outliers.

					Table~\ref{MetricComputation} shows the normal and contrast classes for three different datasets.  In-class outliers are possible, e.g. unusual strikers are still members of the striker class. Our case studies describe a few in-class outliers. In the soccer data, we considered only individuals who played more than 5 matches out of a maximum 38.

										%

					\begin{table}
							\caption{Outlier/normal Objects in Real-World Datasets. }
							\label{MetricComputation}
						\centering
						\resizebox{0.4\textwidth}{!}{
							\begin{tabular}{|c|c|c|c|}
								\hline
								Normal&\#Normal&Outlier&\#Outlier\\\hline
								Striker & 153 & Goalie&22\\ \hline
								Midfielder & 155 & Striker&74\\ \hline
								Drama & 197 & Comedy&47\\ \hline

							\end{tabular}}

						\end{table}

						\subsection{Methods Compared}
						\label{sec:methods}
						We compare two types of approaches, and within each approach several outlier detection methods. The first approach evaluates the likelihood-based outlier scores described in Section~\ref{sec:metrics}. For relational Bayesian network structure learning we utilize the previous learn-and-join algorithm (LAJ), which is
				a state-of-the-art BN structure learning method for relational data \cite{Schulte2012}. The LAJ algorithm employs an iterative deepening strategy, which can be described as as search through a lattice of table joins. For each table join, different BNs are learned and the learned edges are propagated from smaller to larger table joins. 	For a full description, complexity analysis, and learning time measurements, please see \cite{Schulte2012}. 	We used the implementation of the LAJ algorithm due to its creators \cite{bib:jbnsite}.

						The second approach first ``flattens" the structured data into a matrix of feature vectors, then applies standard matrix-based outlier detection methods. We refer to such methods as \textbf{aggregation-based}

						(cf. Figures~\ref{fig:novelty}). For example, this was the approach taken by Breunig {\em et al.} for identifying anomalous players in sports data \cite{Breunig2000}. Following their paper, for each continuous feature in the object data, we use the average over its values, and for each discrete feature, we use the occurrence count of each feature value in the object data. Aggregation
						tends to lose information about correlations.

Our experiments address the empirical question of whether this loss of information affects outlier detection.

						We evaluated three standard matrix-based outlier detection methods: Density-based $\it{LOF}$~\cite{Breunig2000}, distance-based $\it{KNNOutlier}$~\cite{Ramaswamy2000} and subspace analysis $\it{OutRank}$~\cite{Muller2012}.

						These represent common, fundamental  approaches for vectorial data.

						Like $\mid$, subspace analysis is sensitive to correlations among features.
						We used the available implementation of all three data matrix methods from the state of the art data mining software \textit{ELKI} \cite{Elke2013}. We used \textit{PRO-CLUS} as the clustering function for $\it{OutRank}$, recommended by~\cite{Muller2012}.

						%

						\section{Empirical Results}

						We present results regarding computational feasibility,

						predictive performance, and case studies.

						\subsubsection{Computational Cost of the $\mid$ Score.}
						Table~\ref{table:LearningTime} shows that the computation of the $\mid$ value for a given target object is feasible. On average, it takes a quarter of a minute for each soccer player, and one minute for each movie. This includes the time for parameter learning from the object database.
Learning the class model BN takes longer, but needs to be done only once for the entire object class.
						{\em The BN model
						provides a crucial
 low-dimensional representation of the

						distribution information in the data.} Table~\ref{table:Number of Parameters} compares the number of terms required to compute the $\mid$ score in the BN representation to the number of terms in an unfactored representation with one parameter for each joint probability.

						\begin{table}[htbp]

							\begin{subtable}

								\centering
									\caption{Time (min) for computing the $\mid$ score.

				\label{table:LearningTime}}
								\resizebox{0.4\textwidth}{!}{
									\begin{tabular}{|c|l|l|} \hline
										Dataset& Class Model

										& Average per Object  \\ \hline
										Strikers vs. Goalies&4.14&0.25\\ \hline
										Midfielder vs. Goalies &4.02&0.25 \\ \hline
										Drama vs. Comedy &8.30&1.00\\ \hline
									\end{tabular}
								}

							\end{subtable}

							\begin{subtable}

								\centering
									\caption{The Bayesian network representation decreases the number of terms required for computing the $\mid$ score.

										\label{table:Number of Parameters}}
								\resizebox{0.4\textwidth}{!}{
									\begin{tabular}{|c|l|l|}
										\hline
										Dataset & \begin{tabular}{c} \#Terms \\Using BN\end{tabular}  & \begin{tabular}{c} \#Terms \\ without Using BN
										\end{tabular}\\ \hline
										Strikers vs. Goalies & 1,430&114,633,792\\ \hline
										Midfielders vs. Goalies & 1,376&43,670,016\\ \hline
										Drama vs. Comedy & 50,802&215,040,000\\ \hline
									\end{tabular}
								}

							\end{subtable}

													\end{table}

						\subsubsection{Detection Accuracy} We follow the evaluation design of~\cite{Gao2010} and
	make each baseline methods detect the same percentage of  objects as outliers:

					Sort the outlier scores obtained by the three baseline methods in descending order, and take the top $r$ percent as outliers. Then we use \textbf{precision}, a.k.a. \textbf{true positive rate} as the evaluation metric which is the percentage of correct ones in the set of outliers identified by the algorithm. As in~\cite{Gao2010}, we set the percentages of outlier to be 1\% and 5\%. In the one-class design, precision measures how many members of the outlier class were correctly recognized. We also report some AUC measurements \cite{aggarwal2013}, which aggregate precision values at different percentage cutoffs.\footnote{Our $\mid$ score performs the best also with other metrics such as recall, to a similar degree.}

						\paragraph{Likelihood-Based Methods}

						Table~\ref{table:Method Comparison} shows the $\it{Precision}$ values for each probabilistic ranking. Our $\mid$ score achieves the top score on each dataset. On the synthetic data, $\mid$ and $|\it{LR}|$ are the only scores with 100\% precision at 1\% and 5\%. This confirms the value of using distances rather than differences.
						While it ought to be easy to distinguish the outliers, Table~\ref{table:Metric Comparison} shows that {\em $\mid$  is the only score that achieves perfect detection}, that is AUC = 1.0.

		\begin{table}
																																\caption{Precision of outlier scores in different datasets.
						\label{table:Method Comparison}}
												\resizebox{0.5\textwidth}{!}{
												\begin{tabular}{|l|c|c|c|c|c|c|c|c|c|}\hline
													Dataset& percentage&\multicolumn{5}{|c|}{Model-based models }&\multicolumn{3}{|c|}{Aggregation-based models}\\
														\hline
																& &\mid&$|\it{LR}|$&\it{LR}&\it{FD}&\it{LOG}&\it{LOF}&$\it{OutRank}$	&\it{KNNOutlier}\\
																	\hline
															    \multirow{2}{*}{High-Correlation}& 1\%& \textbf{1.00}& \textbf{1.00}& 0.73& 0.47& 0.91& 0.11& 0.53& 0.48\\
															    &5\%&\textbf{1.00}&\textbf{1.00}&0.85&0.65&0.95&0.22&0.50&0.65\\
														    \hline																				    	 \multirow{2}{*}{Low-Correlation}& 1\%& \textbf{1.00}& \textbf{1.00}& 0.87& 0.14& 0.93& 0.10& 0.00& 0.06\\
												    	    &5\%&\textbf{1.00}&\textbf{1.00}&0.90&0.25&0.95&0.25&0.10&0.14\\
													    	    \hline
																\multirow{2}{*}{Single-Feature}& 1\%& \textbf{1.00}& \textbf{1.00}& 0.39& 0.53& 0.81& 0.46& \textbf{1.00}& 0.51\\
															&5\%&\textbf{1.00}&\textbf{1.00}&0.55&0.62&0.92&0.55&\textbf{1.00}&0.54\\
															\hline
																\multirow{2}{*}{Striker-Goalie}& 5\%& \textbf{0.57}& 0.27& 0.22& 0.51& 0.36& 0.19& 0.47& 0.42\\
																	&15\%&\textbf{0.63}&0.36&0.31&0.58&0.40&0.32&0.50&0.52\\
																	\hline
															\multirow{2}{*}{Midfielder-Striker}& 1\%&\textbf{ 0.49}& 0.42& 0.25& 0.41& 0.46& 0.29& 0.44& 0.16\\
															&5\%&\textbf{0.52}&0.48&0.39&0.44&0.50&0.38&0.48&0.35\\
												\hline  																				\multirow{2}{*}{Drama-Comedy}&1\%& \textbf{0.44}& 0.38& 0.39& 0.15& 0.22& 0.29& 0.07& 0.014\\
															&5\%&\textbf{0.47}&0.45&0.44&0.40&0.28&0.36&0.17&0.20\\
																\hline

												\end{tabular}	}

						\end{table}

			\begin{table}
				\centering
				\caption{\textit{AUC} of $\mid$ vs. $|LR|$.

					\label{table:Metric Comparison}}
				\resizebox{0.5\textwidth}{!}{
					\begin{tabular}{|l|l|l|l|l|l|l|}\hline
						Score & High-Cor. &Low-Cor.&Single-F.&Striker&Midfielder&Drama\\ \hline
						\mid&1.00&1.00&1.00&0.89&0.66&0.70\\\hline
						$|\it{LR}|$&0.95&0.95&0.89&0.61&0.64&0.65\\\hline

					\end{tabular}}

				\end{table}
								\paragraph{Aggregation-Based Methods vs. \mid}
								Table~\ref{table:Method Comparison} shows the precision values for aggregation-based methods compared to $\mid$. {\em Our $\mid$ score outperforms all aggregation-based methods on all datasets}, except for a tie with $\it{OutRank}$(ProClus) on the relatively easy problem of distinguishing strikers from goalies.
								The performances of aggregation-based methods are most like that of the probabilistic score $\it{FD}$, which does not consider the
								correlation among the features. This finding reflects the fact that aggregation tends to lose information about correlations.

								The aggregation-based methods achieve their highest performance on the Strikers vs. Goalies dataset. In this dataset action count features such as ShotsOnTarget, ShotEfficiency point to strikers and the feature SavesMade points to goalies. Therefore, outliers in this dataset are easy to find by considering features in isolation.

								%
								%
								%

								\subsubsection{Case Studies} For a case study, we examine three top outliers as ranked by $\mid$, shown in Table~\ref{table:CaseStudy}.
								The aim of the case study is to provide a qualitative sense of the outliers indicated by the scores. Also, we illustrate how the BN representation leads to an interpretable ranking.
							Specifically, we employ a {\em feature-wise decomposition} of the score combined with a {\em drill down} analysis:

\begin{enumerate}
\item Find the node $\feature_{i}$ that has the highest $\mid_{i}$ divergence score for the outlier object.
\item Find the parent-child combination that contributes the most to the $\mid_{i}$ score for that node.
\item Decompose the $\mid$ score for the parent-child combination into feature and mutual information component.
\end{enumerate}

								We present strong associations---indicated by the $\mid$'s mutual information component---in the intuitive format of association rules.

								\paragraph{Strikers vs. Goalies}

								%
								In real-world data, a rare object may be a {\em within-class outlier}, i.e., highly anomalous even within its class. In an unsupervised setting without class labels, we do not expect an outlier score to distinguish such an in-class outlier from outliers outside the class.
								An example is the striker Edin Dzeko. He is a highly anomalous striker who obtains
								the top $\mid$ divergence score among both strikers and goalies. His $\mid$ score is highest for the Dribble Efficiency feature. The highest $\mid$ score for that feature occurs when Dribble Efficiency is low, and its parents have the following values: Shot Efficiency high, Tackle Efficiency medium. Looking at the single feature divergence,

								we see that Edin Dzeko is indeed an outlier in the Dribble Efficiency subspace: His dribble efficiency is low in 16\% of his matches, whereas a randomly selected striker has low dribble efficiency in 50\% of their matches. Thus, Edin Dzeko is an unusually good dribbler. Looking at the mutual information component of $\mid$, i.e., the parent-child correlations, for Edin Dzeko the confidence of the rule
								$$\it{ShotEff} = \it{high}, \it{TackleEff} = \it{medium}\rightarrow \it{DribbleEff} = \it{low}$$ is 50\%, whereas in the general striker class it is $38\%$.

								%

								\paragraph{Midfielders vs. Strikers}

								For the single feature score, Robin van Persie is recognized as a clear striker because of the $\it{ShotsOnTarget}$ feature. It makes sense that strikers shoot on target more often than midfielders. Robin van Persie  achieves a high number of shots on targets in $34\%$ of his matches, compared to $3\%$ for a random midfielder. The mutual information component shows that he also exhibits  unusual correlations. For example,
								the confidence of the rule
								$$\it{ShotEff} = \it{high}, \it{TimePlayed} = \it{high} \rightarrow \it{ShotsOnTarget} = \it{high}$$
								is 70\% for van Persie, whereas for strikers overall it is 52\%.

The most anomalous midfielder is Scott Sinclair. His most unusual feature is $\it{DribbleEfficiency}$: For feature divergence, he achieves a high dribble efficiency $50\%$ of the time, compared to a random midfielder with $30\%$.

								\paragraph{Drama vs. Comedy}

								The top outlier rank is assigned to the within-class outlier $\it{Brave Heart}$. Its most  unusual feature is  $\it{ActorQuality}$: In a random drama movie,  $42\%$ of actors have the highest quality level 4, whereas for $\it{Brave Heart}$ $93\%$ of actors achieve the highest quality level.

								The  $\mid$ score identifies the comedies  $\it{Blues Brothers}$ and $\it{Austin Powers}$ as the top out-of-class outliers.

								In a random drama movie,  $49\%$ of actors have casting position 3, whereas for $\it{Austin Powers}$ $78\%$ of actors have this casting position, and for $\it{Blues Brothers}$ $88\%$ of actors do.
								\begin{table*}
									\centering
									\caption{Case study for the top outliers returned by the log-likelihood distance score \mid
										\label{table:CaseStudy}}
									\resizebox{0.8\textwidth}{!}{
										\begin{tabular}{|l|l|l|l|l|l|l|l|} \hline
											\multicolumn{8}{|c|}{Strikers (Normal) vs. Goalies (Outlier)}\\
											\hline
											PlayerName&Position&$\mid$ Rank&$\mid$ Max Node&$\mid$ Node Score&$\it{FD}$ Max feature Value& Object Probability& Class Probability \\ \hline
											Edin Dzeko&Striker&1&DribbleEfficiency&83.84&DE=low&0.16&0.5 \\ \hline
											Paul Robinson&Goalie&2&SavesMade&49.4&SM=Medium&0.3&0.04\\ \hline
											Michel Vorm&Goalie&3&SavesMade&85.9&SM=Medium&0.37&0.04\\ \hline
											\multicolumn{8}{|c|}{Midfielders (Normal) vs. Strikers (Outlier)}\\
											\hline
											PlayerName&Position&$\mid$ Rank&$\mid$ Max Node&$\mid$ Node Score&$\it{FD}$ Max feature Value& Object Probability& Class Probability \\ \hline
											Robin Van Persie& Striker&1&ShotsOnTarget&153.18&ST=high&0.34&0.03 \\ \hline
											Wayne Rooney& Striker&2&ShotsOnTarget&113.14&ST=high&0.26&0.03\\ \hline
											Scott Sinclair&Midfielder&6&DribbleEfficiency&71.9&DE=high&0.5&0.3\\ \hline
											\multicolumn{8}{|c|}{Drama (Normal) vs. Comedy (Outlier)}\\
											\hline
											MovieTitle&Genre&$\mid$ Rank&$\mid$ Max Node&$\mid$ Node Score& $\it{FD}$ Max feature Value& Object Probability& Class Probability \\ \hline
											Brave Heart&Drama&1&ActorQuality&89995.4&a\_quality=4&0.93&0.42\\ \hline
											Austin Powers&Comedy&2&Cast\_Position&61021.28&Cast\_Num=3&0.78&0.49\\ \hline
											Blue Brothers&Comedy&3&Cast\_Position&24432.21&Cast\_num=3&0.88&0.49\\ \hline
										\end{tabular}
									}
								\end{table*}

\section{Conclusion} We presented a new approach for applying Bayes nets to object-relational outlier detection, a challenging and practically important topic for machine learning. This approach follows the general framework of Exceptional Model Mining~\cite{Duivesteijn2016}, and applies it to multi-relational data. The key idea is to learn one set of parameter values that represent class-level associations, another set to represent object-level associations, and compare how well each parametrization fits the relational data that characterize the target object. The classic metric for comparing two parametrized models is their log-likelihood ratio; we refined this concept to define  a new relational log-likelihood distance metric via two transformations:  (1) a mutual information decomposition, and (2) replacing log-likelihood differences by log-likelihood distances. This metric combines a single feature component, where features are treated as independent, with a correlation component that measures the deviation in the features' mutual information.

In experiments on three synthetic and three real-world outlier sets, the log-likelihood distance achieved the best detection accuracy. The alternative of converting the structured data to a flat data matrix via aggregation had a negative impact.
Case studies showed that the log-distance score leads to easily interpreted rankings.

								There are several avenues for future work.  (i) A limitation of our current approach is that it ranks potential outliers, but does not set a threshold for a binary identification of outlier vs. non-outlier. (ii) Our divergence uses expected L1-distance for interpretability, but other distance scores like L2 could be investigated as well. (iii) Extending the expected L1-distance for continuous features is a useful addition.

								In sum, outlier metrics based on model likelihoods are a new type of structured outlier score for object-relational data.  Our evaluation indicates that this model-based score provides informative, interpretable, and accurate rankings of objects as potential outliers.

				\section*{Acknowledgement}
This work was supported by a Discovery Grant from the National Sciences and Engineering Council of Canada. We are indebted to Peter Flach for referring us to the EMM framework.

								\bibliographystyle{abbrv}

redpajama_set_name: RedPajamaArXiv


I have been a fan of antique vehicles since I was a kid. I spent all of my free time in the garage learning everything I could about fixing cars. I still work on friends and relatives cars, as well as my own. Now, I mostly work on RV camping trailers and motor homes. I have been a technician for 42 years and an RV technician for 18 years. I love the challenge of trouble shooting RV problems. We are a family owned and operated company.
I've never met a car I didn't like, but my personal favorites are 57 Chevy Belair and 68 Pontiac GTO.
We repair awnings, slide toppers, slide outs, electrical, heat and a/c, water heaters, refrigerators, toilets, roofing, converters, wheel bearings, brakes, plumbing, and much more.
Copyright © 2018 Trailer Time RV Service - All Rights Reserved.


text:
The Cure Play Out è un VHS dei Cure, uscito nel dicembre del 1991.

Il video
Le telecamere seguono la band durante i mesi di gennaio e febbraio 1991 nelle loro apparizioni televisive e nelle prove domestiche, registrandone canzoni e intermezzi di relax e chiacchiere tra i membri. Sono documentati un concerto eseguito sotto il falso nome di "Five Imaginary Boys", in cui vengono presentate tre canzoni dal nuovo album Wish in uscita l'anno seguente, il concerto londinese per il "Great British Music Weekend", gli show televisivi "Jonathan Ross Show" e "MTV Unplugged", in cui viene eseguito un altro inedito proveniente da Wish, e l'esibizione ai "BRIT Awards".

Tracce
Day One, T&C II Club, London, 17th January 1991
 Wendy Time
 The Big Hand
 Away
 Let's Go to Bed
 A Strange Day
 Day Two - Wembley Arena - London - The Great British Music Weekend - 19th January 1991
 Pictures of You
 Fascination Street
 Lullaby
 A Forest
 Day Three - The Jonathan Ross Show 23rd January 1991 Various snips including Hello I Love You - Just Like Heaven - The Walk - Bland With an Edge
 Harold and Joe
 Day Four - E-zee Hire Rehearsal Studio - London - 22nd January 1991
 The Blood
 The Walk
 Day Five - MTV "Unplugged" - 24nd January 1991
 Just Like Heaven
 A Letter to Elise
 If Only Tonight We Could Sleep
 Boys Don't Cry
 Day Six - Dominion Theatre - London - Rehearsals for "The Brit Awards" - 9th February 1991
 Day Seven - Dominion Theatre - London - "The Brit Awards" - 10th February 1991
 Never Enough

Formazione
 Robert Smith - chitarra, voce
 Simon Gallup - basso
 Porl Thompson - chitarra, tastiere
 Boris Williams - batteria
 Perry Bamonte - chitarra

redpajama_set_name: RedPajamaWikipedia


text:
You are here: Home / Action & Adventure / Huntress Initiate (Huntress Clan Saga Book 1) by Jamie Davis
Huntress Initiate (Huntress Clan Saga Book 1) by Jamie Davis
Free Dec. 21-25
How did they expect her to play the game if she didn't even understand the rules?
Abandoned as an infant, Quinn Faust is about to discover she's not just a normal human being.
Bounced from foster home to foster home, she came to rely on her street smarts, martial arts skills, and the tiny silver amulet that was the only link to her lost family.
Hired by a video game company to test new Virtual Reality gear to train soldiers, Quinn quickly learns there's more to the game than they told her.
Now strange happenings reveal a world hidden in the shadows, the existence of supernatural creatures, magic and mystical powers.
Does Quinn have what it takes to claim the birthright of long-lost Hunter clans?
Learn more about the writer. Visit the Author's Website.
Get This Action & Adventure Book From Amazon.
Visit the Author's Facebook Fan Page.
Visit the Author's Twitter page.
Jamie Davis is a nurse, retired paramedic, author, and nationally recognized medical educator who began teaching new emergency responders as a training officer for his local EMS program. He loves everything fantasy and sci-fi and especially the places where stories intersect with his love of medicine or gaming.
Jamie lives in a home in the woods in Maryland with his wife, three children, and dog. He is an avid gamer, preferring historical and fantasy miniature gaming, as well as tabletop games. He writes LitRPG, GameLit, urban, and contemporary paranormal fantasy stories, among other things. His Future Race Game rules were written to satisfy a desire to play a version of the pod races from Star Wars episode 1.
He loves hearing from readers and going to cons and events where he meets up with fans. Reach out and say "hi."

redpajama_set_name: RedPajamaCommonCrawl


text:
Recommending new books each month to spark lively conversation.
Reading Group Choices selects discussible books and suggests discussion topics for reading groups.
Featured Books Recommended Books New Books Favorites Giveaways Blog Store All Books
Searching is easy! Type in a keyword, phrase, or exact wording and you will be directed to the results. You can also click on "view all books" beneath the search box to run a more advanced search, and to scroll through all of the books in our database.
View All Books in Database
THE GOOD SISTER
Drusilla Campbell
Roxanne Callahan has always been her younger sister's caretaker. Now married, her happiness is threatened when beautiful and emotionally unstable Simone, suffering from crippling postpartum depression, commits an unforgivable crime for which Roxanne comes to believe she is partially responsible. In the glare of national media attention brought on her sister, Roxanne fights to hold her marriage together as she is drawn back into the pain of her troubled past and relives the fraught relationship she and Simone shared with their narcissistic mother. At the same time, only she can help Simone's nine year old daughter, Merell, make sense of the family's tragedy.
Roxanne Callahan has always been her younger sister's caretaker. Now married, her happiness is threatened when beautiful and emotionally unstable Simone, suffering from crippling postpartum depression, commits an unforgivable crime for which Roxanne comes to believe she is partially responsible. In the glare of national media attention brought on her sister, Roxanne fights to hold her marriage together as she is drawn back into the pain of her troubled past and relives the fraught relationship she and Simone shared with their narcissistic mother. At the same time, only she can help Simone's nine year old daughter, Merell, make sense of the family's tragedy. Cathartic, lyrical, and unflinchingly honest, The Good Sister is a novel of four generations of women struggling to overcome a legacy of violence, lies and secrecy, ultimately finding strength and courage in their love for each other.
less …
About Drusilla Campbell
Drusilla Campbell is the author of three critically acclaimed novels: Wildwood, The Edge of Sky, and Blood Orange. Before she started school she had crossed the Pacific Ocean three times. In her twenties she lived in Europe and Central America. Today she's happy to stay at home in San Diego with her husband, the attorney and poet Art Campbell, two rescued dogs, and three horses.
"This portrait of the inner life of a woman whose psychotic state led her to believe that killing her children and herself would have been best for all of them should be on everyone's book club list."—Publishers Weekly
1. Many themes run through The Good Sister. One of these is codependency, the kind of unhealthy relationship that can tie friends and family together in knots of emotional misery. In large part The Good Sister is about Roxanne's attempts to establish an independent life apart from her sister. Why is this so difficult for her? What obstacles real and imagined stand between her and this goal?
2. Roxanne loves her sister and feels responsible for her, and Simone is excessively dependent upon Roxanne. Is Roxanne dependent upon Simone? In what ways? What events from each of their childhoods led to this enmeshed relationship? When and by what means might it have been terminated much earlier in their lives? Why didn't that happen?
3. Although she feels her mother abandoned her by leaving her with Gran, Roxanne has many happy memories around the three years she spent at the ranch. It has been said that our memories both sustain and burden us. How is this true in Roxanne's case?
4. The Good Sister is the story of four generations of women, each of whom has been abandoned in some way. How does the theme of abandonment run through the story and influence the lives of the characters?
5. Merell slips into and out of rooms like a little ghost, unseen by the adults around her who are too much involved in their own lives. Roxanne was unseen by her mother except in her role of caretaker. Simone is slipping from depression into psychosis and Merell believes she is the only one who sees her mother's pain.Is it true that we often do not see clearly the people closest and most important to us? Why is that?
6. There are two "good sisters" in this book: Roxanne and Merell. How does Merell's development mirror Roxanne's? How do we see the pattern of behavior playing out in Merell in contrast to Roxanne?
7. Do you think there is often one sibling who does everything possible to make the family happy and peaceful? If you were in Roxanne's shoes, would you have stopped your life to help your sister in need?
8. Elizabeth believes that we sometimes serve as "angels" to the people who need us. Have you ever known this to be true?
9. Roxanne knows that without Elizabeth's friendship she might never have found the courage to move away from home and take the first steps toward an independent life. Was it Elizabeth's example that enabled Roxanne or something else? Is it possible that we look for in friends those qualities we know or sense are lacking in ourselves?
10. While postpartum depression is quite common, postpartum psychosis is – blessedly – rare and does not occur in women who are not psychologically damaged in some way. Dr. Balch says in her testimony that in Simone's case, a "perfect storm" of factors were involved. What were those? And does knowing why Simone tried to kill her children help you to feel sympathetically toward her?
11. Did reading The Good Sister make you think of real mothers in the news who endangered or killed their children because of postpartum depression, such as Andrea Yates? Do you feel sympathy towards them, or do you see them as monsters? Do you think the media treats these women fairly? Do you think there is still a stigma against women suffering from postpartum depression in society?
12. Simone is torn between her fear of life and her longing to regain the feeling she had when she sailed aboard the Oriole. How do these conflicting desires play out on the day she decides to take her daughters to the marina on Shelter Island?
13. We apply the adjective good to being a mother and sister. What does it mean to be a good mother, a good sister?
14. Johnny thought Simone would be a good wife. What caused him to make this egregious misjudgment? Is it possible that with no children or perhaps only Merell, Simone might have lived up to his expectation?
15. Throughout the book Roxanne represses her anger, but in the final chapter Simone's psychiatrist tells her it is time for her to be brave. What does he mean? What does he want Roxanne to do, why is this difficult and why does he believe it will help Simone?
16. Will this family ever be whole again? What forces work in its favor and what pulls against it?
January Book Selections
Browse Featured Books
Books we highly recommend.
Browse Recommended Books
Browse New Books
Also, don't forget to register for any of our January Giveaways!
Sign Up For Reading Group Choices
Shop our Annual Print Editions
Reading Group Choices features a curated collection of the latest book recommendations for your reading group. learn more
©2021 Reading Group Choices. All rights reserved.


Did you know that you should use paraphrases for IELTS Speaking questions on your exam?
This seems like a small detail, but, if you're aiming for a 7 or higher, every strategic way to arrive there helps.
Firstly, paraphrasing the question is important for vocabulary. If you simply copy the question in the beginning of your answer, it shows a lack of lexical flexibility and knowledge.
Thus, utilizing other words to express the topic is an excellent way to show the examiner that you have a range of vocabulary.
In the examples below, note how sometimes I paraphrase the question, and, at other times, I instead get directly to my specific answer.
I don't have a job at the moment, but I am studying to be an engineer.
I frequently enjoy eating little bits, here and there.
Describe a time you waited in line for a long time.
On one memorable occasion, I chose to stand in a queue to get concert tickets, and I was stuck there for over 4 hours.
Describe a problem you had while travelling.
One time I was on a flight from Amsterdam to Paris, and it was a nightmare.
What are the advantages and disadvantages of taking public transport?
Do you believe that there should be more laws regarding advertising?
To my mind, the government should have more federal rules for those creating ad content.
Don't repeat the question- use your own words!
Share your own paraphrases in the comments section below!


We're excited to share our strategic vision. We created it in 2014 and it holds strong as our vision moving forward.
Big Picture Big agriculture and globalization results in a loss of biodiversity, environmental crises, reduced food and community resilience, increased world hunger, and an over-reliance on industrial food systems. To create regional, small-scale, organic food systems made up of farmers, home growers, and conscious eaters resulting in a thriving food shed that exemplifies diversity, resiliency, and community.
Community Our food and farming heritage and culture are fragmented by the agribusiness agenda and our communities have little cohesion with regards to interdependence, skill-sharing, or celebration of food and growing. The above interest steamrolls into a movement of sustainable organic growing and eating that leads to a robust, vibrant, secure, viable, and resilient food region.
The average age of farmers is 58.
Traditional methods of farming and knowledge transfer have been lost.
Barriers to farming are many: access to land, capital, training, support, etc.
There is strong development pressure on NC farmland.
To increase the number of viable organic farms in our region.
To increase the number of community leaders who advocate for resilient food systems and organic farming.
Farmers are a celebrated as a central component to a vibrant community and as such are thriving in every way.
To make it easier to enter farming as an organic grower than any other cultural approach.
Gardeners & Homesteaders of all scales The industrialization of food and growing leaves the average person out of touch with the basics of food, kitchen, and garden literacy. The loss of ancestral knowledge and of a local food community leaves people disconnected, disempowered, and insecure.
To increase the number of people who are successfully growing on a home-scale.
To increase public and community support for home growing.
Home growers become a confident, empowered, and excited population working towards collective food sovereignty and security, who are gaining life skills and enjoying delicious food.
Organic Living & Eating Misinformation and manipulation by our current food system are leaving eaters sick and ill-informed.
To increase the number of people who shop and eat locally and organically.
To increase the number of people who are preparing & cooking food at home.


We have responded to Defra's consultation on amending food labelling laws in the event that the UK leaves the EU in a 'no deal' scenario. Read our response below.
1. Duration of the transition: Proposal 6 months.
IFST suggests a transition phase of 12-18 months.
The proposed changes will have a significant impact for businesses and require high numbers of labels to be re-designed and printed. The planned extension to COOL for primary ingredients - April 2020, will extend COOL further across the food industry.
When labels are re-designed several steps need to be followed: Redesign : Compliance check : Review : Printing (non UK in many cases): Supply Chain : Stock management.
Due to the high volume of labels to be changed there could be bottle necks at various stages of the process and resource issues given a 6-month time frame. We would recommend at least a 12-month time frame; Ideally 12- 18 months aiming to capture in "one hit" COOL additional labelling requirements i.e. one timeframe for Brexit linked changes and COOL revisions.
The EU has developed extensive science-based regulation providing a greater level of food safety for the consumer than many other global regions. Therefore we feel it would be in the consumer interest to be informed if a food is coming from within the EU i.e. 3 categories for the labelling: UK ; EU ; non UK-EU or similar wording.


text:
Ghana begins 2007 at 28th
By gfa
LISTEN JAN 21, 2007
Ghana has began 2007 at the 28th position as the first FIFA/Coca-Cola World Ranking of the new year is released.
After ending last year's ranking in the same position, Ghana managed to increase its point build up by a further 12 thus bagging 824 points.
Ghana, after a relative success in 2006 has rather seen a decline in rank due to her non-participation in any competitive qualifying campaign.
And this has affected the nation's position on the ranking table – dropping five places down the pecking order after the World Cup to the 28th place.
The decline can be explained as a result of the revision in the criteria adopted by FIFA to rank teams.
According to FIFA, country's who do not participate in any of the competitive major tournaments or qualifying series at the senior level drop valuable points.
The four-time African champions by virtue of being host of the 2008 African Cup of Nations are not taking part in the continent's qualifying campaign.
However on the African continent, Ghana is still placed 6th, incidentally that was the same position it held as at the last ranking schedule in December 2006.
Nigeria have started the year as Africa's top side. Cameroon, Ivory Coast, Egypt and Guinea follow in that order.
After a limited number of international matches played so far in the New Year, no significant changes have occurred on the log.
But devaluation of older results in the ranking's calculation means some has affected some positions.
January's ranking took into account only 31 new matches - all of them friendlies and most involving Asian teams – that largely influenced the calculation for the month.
Sixteen were played at the end of 2006, the remainder since the turn of the year.
Nevertheless, while Brazil, Italy, Argentina and France continue to lead the way, Germany (up one) edge ahead of England into fifth place.


text:
Le Tournoi d'ouverture du championnat de Bolivie 2022 est la quarante-huitième édition du championnat de première division en Bolivie.

Déroulement du tournoi
Les 16 équipes sont divisées dans deux groupes où elles se rencontrent deux fois en match aller et match retour, plus deux autres matchs, pour un total de 16 rencontres.

Les quatre premiers de chaque groupe se retrouvent dans un tournoi à élimination directe en match aller et retour des quarts de finale aux demi-finales, la finale est jouée en un seul match sur terrain neutre. Le vainqueur est désigné champion du tournoi d'ouverture et se qualifie pour la Copa Libertadores 2023, le vice-champion  se qualifie pour la Copa Sudamericana 2023.

Les clubs participants

Compétition
Le barème utilisé pour établir le classement est le suivant :
 Victoire : 3 points
 Match nul : 1 point
 Défaite : 0 point

Classement

Groupe A

|valign=top align=left width=50%|

|}

Groupe B

|valign=top align=left width=50%|

|}

Tableau final

Le vainqueur est désigné champion du tournoi d'ouverture et se qualifie pour la Copa Libertadores 2023, le vice-champion se qualifie pour la Copa Sudamericana 2023.

Bilan de la saison

Voir aussi

Références

Liens internes
 Championnat de Bolivie de football

Liens externes

2022
Bolivie
Sport en Bolivie en 2022


text:
Nantis indefessus är en fiskart som först beskrevs av Mirande, Aguilera och Maria De Las Mercedes Azpelicueta 2004.  Nantis indefessus ingår i släktet Nantis och familjen Characidae. Inga underarter finns listade i Catalogue of Life.

Källor

Strålfeniga fiskar
indefessus


text:
Home Features Favorite Rides Arizona Motorcycle Rides: Florence, Phoenix, Tucson
Arizona Motorcycle Rides: Florence, Phoenix, Tucson
Mike Henry
story and photography by Mike Henry
[Arizona Motorcycle Rides: Florence, Phoenix, Tucson originally published as a Favorite Ride in the April 2008 issue of Rider magazine]
Most motorcyclists dream of that once-in-a-lifetime trek to the Arctic Circle, or even participating in an Iron Butt Rally, but few of us ever have a chance to experience such an epic adventure. Like most mortals, I have responsibilities. In my case, a wife, a two-year-old daughter and a six-day-a-week job, all of which limit my riding to day trips. (I also enjoy commuting, but that's another story).
I'll spend days, or even weeks, planning a dawn-to-dusk ride, which is a little obsessive I'll admit, but to my mind countless people spend more time on less noble endeavors. I like to use free mapping sites on the Internet, plotting my way across countless miles, thinking of all of the great people I'll meet along the way. Of course, I have a laminated map in the window of my tankbag, discolored by the sun from lack of really long rides.
I planned one such trip this year, and even recruited my 64-year-old father for a leg of the ride. It didn't take much prodding before he agreed to meet me for a few hours of riding and a meal stop. My dad rides a Harley-Davidson Electra Glide most of the time, but on this day, he rented a Buell Ulysses from Top Spoke Rentals in Scottsdale, Arizona.
World's finest V-Strom 650 by the Tom Mix Memorial near Florence.
I threw a leg over my Suzuki DL650 V-Strom, and hit the tarmac at 8 a.m. My route took me from downtown Tucson, north on AZ 77 through Florence, and up Highway 60 to Florence Junction where I was to meet my dad. The 42 miles from Catalina to Florence is usually, well, dull. On this day, there was a horrific crosswind blowing from east to west. My V-Strom was more like a control surface on a C-130 than a motorcycle, as I fought to keep the thing in my own lane. My wife's acupuncturist says "wind is bad for the body." No wonder she gave me that look when I told her I ride a motorcycle. The speed limit was 65, however, which helped the time pass.
The roadside is littered with small turnouts and picnic tables, and at least one historical landmark. I stopped for a few minutes at the Tom Mix Memorial. I'd heard the name in passing a few times, but didn't know much about Hollywood's first (silent-era) cowboy movie star. He was killed while driving his 1937 Cord 812 down this stretch of road in 1940. Apparently, the impact of the crash didn't do him in, but the shiny aluminum suitcase in the back did, as it flew to the front of the car and crushed his skull. The plaque at the memorial doesn't make mention of this, which may be just as well. "Whose spirit left his body on this spot" sure sounds better than the suitcase-in-the-head bit.
Memorial to Hollywood's cowboy movie star Tom Mix, who died on this spot in a car crash in 1940.
The city of Florence is built around a state prison. There's also a downtown historic district and a museum. All I know is that the wind storm I rode through in Florence consumed the rubber pad on my luggage rack (An ill-advised reinstallation of the piece was more to blame than the wind–it turns out that more than two strips of servo tape are needed to hold it in place). I also watched the largest tumbleweed I'd ever seen blow across Highway 79, not 10 feet in front of me and my padless DL650.
I was also reminded how riding a motorcycle, somehow, makes you more approachable. I stopped at a McDonald's in Florence to regain my equilibrium and have a cup of coffee. "Tough day to be on the bike" or "don't blow away" were typical comments I heard from complete strangers. I'm pretty sure I would have gone unnoticed if I had pulled up in my Accord.
A strong wind made sections of the Apache Trail more interesting than we bargained for.
I rolled into Apache Junction right at 10 a.m., and found my dad next to his new pride and joy. Well, at least until 4:30 p.m. when Top Spoke wanted the bike back. I purchased a bottle of water and a bag of Peanut M&M's from the gas station and we rolled out of town, heading for the real adventure. We hopped on Highway 88 and headed for Tortilla Flat. Highway 88 is a paradox of sorts for the motorcyclist; it features some of the best tight, increasing/decreasing radius turns you will ever find anywhere, but it's also a real tourist trap. And you will have to deal with overly cautious Mom-and-Pop types who don't want to plunge the Aerostar over the side. It's a two-lane road, double-yellow all the way. But if you can muster up some patience, it pays dividends in spades.
The long, lonely road home—avoid the soft shoulders until you get there.
This stretch of Highway 88 is also known as the Apache Trail. It was originally used by stagecoaches traversing the Superstition Mountains more than a century ago. Tortilla Flat is the last remaining stagecoach stop along the trail, and boasts a population of six. You can stop and have a meal, and shop for trinkets and trash, but the town itself is mostly forgettable. About five miles past the town, Highway 88 turns into a dirt road. We're talking about a maintained, unpaved road here. If you're looking for a single-track Louis and Clark ride, look elsewhere. But for the two of us, it was pretty adventurous (and we were on adventure bikes). For much of the ride, you're cruising right next to Apache Lake. Arizona is known for a lot of things, but boating and fishing aren't among them. We saw both activities, along with saguaro cactus lining the shores of the water and a few one-lane bridges, just to keep you honest. The dirt road is very smooth in places; sometimes you forget it's not paved.
100-year-old Roosevelt Dam, named after Theodore, our 26th president, appears out of nowhere after 45 minutes of switchbacks.
After about 45 minutes of switchbacks and elevation changes, the Roosevelt Dam appears out of nowhere. It's quite impressive, even for a 100-year-old structure. The road segues into asphalt, and at the top of a steep climb you're met by another large body of water, Roosevelt Lake. The dam and the lake were named after our 26th President. We stopped for a bite and a tank of gas at an innocuous-looking greasy spoon. An older fella took our money after filling up; I can't remember the last time I paid for gas after topping off the tank. The gas pumps even have real numbers on them, the kind that spin around back to zero when you pull the handle.
MAP BY BILL TIPTON/COMPARTMAPS.COM
We backtracked to the dam and continued to Globe. Once we left the safety of the canyon, we were again subjected to a modest headwind. I did the best I could, hiding behind the windscreen on my Suzuki, hoping nothing else would fall off. For a minute or two I felt bad for my dad on his Buell–the Ulysses has just a tiny plastic bikini fairing. I'm sure he was getting a healthy dose of bad Chi from the gale-force wind. We stopped along the way and switched bikes for awhile, but apparently he had so much fun on the Buell he couldn't wait to get back on the thing. So much for pity.
We parted ways in Miami as the witching hour was approaching, and he didn't want his bike to turn into a pumpkin if he didn't return it on time. I rejoined Highway 77 and began the climb up Pinal Pass. As I crested the 4,983 feet, I realized how much I relished this day trip. Spending the day on my DL, riding unpaved roads, enjoying my dad's company over a really average grilled ham-and-cheese sandwich and taking in some of Arizona's lesser-known landmarks. I even had enough time to order Suzuki part number 46321-06G00 when I got home.
Arizona Motorcycle Rides
Great Roads West
A West Texas Motorcycle Ride Through Hill Country, Big Bend, and Oil Wells
Mike Henry - January 23, 2023


FREE CONCERT! All four Charlotte-area locations will perform as one ensemble!
The Project Harmony Concert will be followed by the Charlotte Symphony Orchestra concert.
To purchase separate tickets for the CSO concert, please visit www.charlottesymphony.org.
Project Harmony is social transformation through music.
Project Harmony is a music and leadership program aimed at expanding the vision of what's possible in the lives of area youth, inspired by the renowned El Sistema youth and community development program of Venezuela.


Chloe CL2208 Metal Sunglasses. Goldtone metal frames with gradient lenses. Transparent blue plastic arms with goldtone tips and engraved Chloe name plate on exterior sides.
Chloe® is a registered trademark of Chloe. More Than You Can Imagine, Inc. is not affiliated with Chloe.


text:
Q: boto3: StreamingBody [Errno 104] Connection reset by peer I have a StreamingBody from s3 that needs to be opened for an extended period of time (due to high processing time and the files being very large). This occasionally causes the error [Errno 104] Connection reset by peer.
I am not sure what is causing it but here is the an excerpt:
import boto3

def csv_stream(s3, key):
    obj = s3.get_object(Bucket='bucket', Key=key)
    body = obj['Body']
    text = codecs.getreader('utf-8')(body)
    return csv.reader(text)

s3 = boto3.client('s3')

for key in get_keys():
    for row in csv_stream(s3, key):
        do_long_transformations(row)

This job usually takes at least half a day to complete.
What would be the fix here, since restarting the job is always very expensive.

A: Adding a retry config seems to solve it
s3 = boto3.client('s3', config=Config(retries={'max_attempts': 3}))

redpajama_set_name: RedPajamaStackExchange


text:
Independent integrates article comments with Twitter and Facebook
The Independent has installed a new commenting system on its website in the shape of Disqus – the same as we use on Journalism.co.uk no less.
The system allows users to login to leave a comment using a Disqus profile, but also, and more importantly, with their Twitter username and password, Facebook login or OpenId identification.
With the Twitter and Facebook logins there's also the option to share your article comment via these sites.
Jack Riley, digital media editor at the Independent, explains in a blog post that the new system has been trialled on the site's sport section for the past week and has improved the level of "constructive debate".
We're encouraging people to use credentials linked to their personal profiles not just because openness and accountability are great, fundamental things which underpin good journalism as well as good commenting (and why should the two be different?), but also because by introducing accountability into the equation, we're hoping the tone and standard of the comments will go up (…) It's about first of all letting people authenticate their commenting using systems with which they're already familiar (in Facebook's case, that's 400 million people worldwide and counting), and secondly, it's about restoring your trust in our comments section, so that some of the really great submissions we get on there rise to the top, the bad sink to the bottom, and the ugly – the spam and abuse that are an inevitable adjunct of any commenting system – don't appear at all.
This entry was posted in Online Journalism and tagged Disqus, Facebook, Independent, Jack Riley, Twitter on May 25, 2010 by Laura Oliver.
← TechCrunch: Publish2 takes on Associated Press with new Online News Exchange Comment: Reaction to the new Times and Sunday Times websites →
3 thoughts on "Independent integrates article comments with Twitter and Facebook"
Pingback: Twitter is most-used social network to login to news sites | Journalism.co.uk Editors' Blog
newsjunkie aka sciencebod April 11, 2011 at 7:32 pm
First we lost Disqus comments a week or so ago on most of the older postings, which I was informed was the result of a tachnical glitch – but they were never restored.
Now, today, there is no comment facility on ANY of the current articles in the Independent, and again, all previous posts have been stripped of comments.
It's as if Disqus had never existed, which is, to put it mildly, bizarre – and hardly calculated to create or maintain a feel-good factor between the Independent and its online readership.
How about a word of explanation on the Home Page – or do I have to email the Editor – again?
newsjunkie aka sciencebod
Pingback: Ten pros and cons for Facebook comments | Small Business Commerce Association Community


text:
Home Tags Posts tagged with "amount of money"
amount of money
Dolly Parton is set to make a fortune after Whitney Houston's death
Dolly Parton, who originally wrote and recorded the biggest hit I Will Always Love You, is set to earn millions of dollars since the Whitney Houston's death.
Having already made a large amount of money from Whitney Houston's smash hit version of I Will Always Love You, country singer Dolly Parton is set to earn millions more dollars since the death of the legendary singer last Saturday.
Originally co-written and recorded in 1973, the song was a modest success for Dolly Parton. The song was then revived by Whitney Houston in 1992 when it was included on the soundtrack of her romantic thriller The Bodyguard, in which she starred alongside Kevin Costner.
Dolly Parton, who originally wrote and recorded the biggest hit I Will Always Love You, is set to earn millions of dollars since the Whitney Houston's death
I Will Always Love You soared straight to the top of the charts where it spent 10 weeks at number one – making Whitney Houston the record-holder for the longest time spent at the top spot by a solo female artist in Britain.
Dolly Parton has previously commented on the large amount of money she has made in royalties due to Whitney Houston's version – joking in an interview with CNN last month that it had allowed her to buy "a lot of cheap wigs".
According to figures from the Official Chart Company, I Will Always Love You is due to enter the charts at number 10 this Sunday. Its popularity has also been fuelled by actress Jennifer Hudson's tribute performance of the track at the Grammy Awards on Sunday.
Whitney Houston's other hits One Moment In Time, Saving All My Love For You, I Wanna Dance With Somebody, My Love Is Your Love, How Will I Know and I Have Nothing are also expected to appear in the Top 40, with a total of 27 singles by Houston appearing in the Top 200.
On Whitney Houston's death, Dolly Parton commented: "Mine is only one of the millions of hearts broken over the death of Whitney Houston. I will always be grateful and in awe of the wonderful performance she did on my song and I can truly say from the bottom of my heart, <<Whitney, I will always love you. You will be missed.>>"
[youtube 6j4U66RYzQo]
[youtube 3JWTaaS7LdU]
Somalia famine killed 260,000 people from 2010 to 2012
A new report by the UN food agency and the US-funded Famine Early Warning Systems Network has found that nearly …Read More »
Pope Francis to visit St Francis of Assisi's tomb
Pope Francis has arrived in the town of Assisi to pray at the shrine of the 13th-Century saint whose name …Read More »
Beirut explosion: At least 37 people injured after a car bomb explodes in Beir el-Abed area
Dozens of people have been injured after a car bomb exploded in a stronghold of Lebanon's Shia militant group Hezbollah …Read More »
Russian Plane Crash: All 92 on Board Feared Dead
Russia's defense ministry has announced that all 92 people on board a military aircraft are feared dead after it crashed …Read More »
Edward Snowden applies for political asylum in Russia
Fugitive whistleblower Edward Snowden has applied to Russia for political asylum, Russian officials say. Foreign ministry consul Kim Shevchenko said …Read More »


When it comes to Australian luxury vacations, sporting events might not immediately spring to mind. However, sports have a different flavor in many parts of the world, and in Australia there is a culture of luxury that has come up around several of the country's most-beloved sports — especially the unusual sport of cricket. Indeed, as Christopher Hitchens and others have noted, in many ways it isn't necessarily a sport at all so much as it is a ritual of sportsmanship itself, played with a spirit of ceremony as much as competition.
During the Summer of Cricket, many games and test matches will be played for the enjoyment of Australia's citizens and visitors, creating an atmosphere of joviality and high-spirited entertainment. Matches unfold gradually, allowing for people to socialize, spend time in good company and enjoy highly-comfortable arrangements.
Take in the game at a high-class restaurant over a high-definition television while enjoying a perfectly-cooked Australian steak paired with just the right wine from the Yarra Valley, and you will see what people mean about the game being luxurious. Alternatively, strawberries and champagne are available to guests at select events, a delightful contrast to the popcorn and soda approach to many sporting events.
Then there are epic events where rivalries are played out in this civilized, comfortable contest. See the Ashes, a decades-long competition between Australia and England for a trophy that once symbolized "the ashes of English cricket." Or see the battle between the India and West Pakistan teams versus their Aussie counterparts. These international gatherings allow for guests to mingle with other visitors, often from around the entire world as people travel thousands of miles to see their favorites play live. With the comfort of accommodations and the joy of the world's most unique sport, the Summer of Cricket is sure to make a unique vacation experience.


Both the simplicity and stunning flavours of this soup will be sure to surprise you. Consisting of just a handful of all natural ingredients, the roasted carrot is the hero of the dish. It's caramel sweetness if perfectly complemented by the precise addition of just the right amount of potato, onion, garlic and ginger. Full of flavour and extra low in calories.
Carrot, potato, GF vegetable stock, onion, ginger.