text:
\section{Introduction}

Text, image, and audio search engines are ubiquitous in our everyday life and help us navigate a world of information orders of magnitude larger than any one human could read in their lifetime. Search engines are growing increasingly complicated and can utilize billion-parameter models \cite{nayak_2019}. Despite search's foundational importance, there is a lack of principled techniques to understand the rationale behind a search engine's matches and responses. At its core, these systems face the same challenges that classifiers and regressors face: richer model spaces can improve performance but yield un-interpretable models. For high-risk domains like medicine, incorrect search results could result in serious consequences. In other domains, bias in a search engine can disproportionately hide certain voices and have far-reaching systemic consequences.

\begin{figure}
    \centering
    \vspace{-.0in}
    \includegraphics[width=\linewidth]{figures/architectures.jpg}
    \vspace{-.3in}
    \caption{Architectures for search engine interpretability. Top: Standard classifier interpretation that highlights pixels that contribute to the prediction. Middle: Our marginal search interpretation architecture modifies the classifier interpretation architecture to highlight pixels that contribute to the similarity between images. Bottom: ``Joint'' search interpretation extracts a full pairwise correspondence between objects in the query and retrieved image, all while treating the search engine as a black box.}
    \label{fig:shap_architecture}
    \vspace{-.2in}
\end{figure}

\begin{figure*}
    \centering
    \includegraphics[width=\linewidth]{figures/marginal_comparison.png}
    \vspace{-.2in}
    \caption{Comparison of marginal search interpretation methods. All methods identify pixels that most contribute to the image's similarity. One exception is Integrated Gradients which struggles, in part because well trained classifiers are invariant to minor pixel changes and which results in uninformative gradients.}
    \label{fig:comparison}
\end{figure*}

To help solve this challenge, we present a unified framework for extending model interpretability to search engines, recommendation systems, and retrieval systems. With these tools, practitioners can gain insight into the rationales behind their algorithm's matches \textit{without} knowledge of an algorithm's internal construction. We explore this topic through the lens of visual search, but note that these techniques could also apply to text, tabular, or audio systems. This work identifies two distinct ways to interpret search engines: ``Marginal Interpretations'' highlight the most important pixels in the match and ``Joint Interpretations'' extract a full correspondence between query and retrieved image pixels. To extract marginal interpretations we introduce a generic transformation, as shown in Figure \ref{fig:shap_architecture}, to transform search engines and retrieval systems into ``classifiers'' that can be interpreted with existing classifier interpretation methods. To extract joint interpretations, we present new bi-linear generalizations of Class Activation Maps \cite{zhou2015cnnlocalization}, LIME \cite{lime}, and SHAP \cite{shap}. These joint interpretations are new to the literature and give practitioners tools to understand search engines with much greater nuance and detail. Formally, we axiomatically characterize joint interpretations and show that projecting Harsanyi Dividends \cite{harsanyi1963simplified} provides a unique solution to this characterization. This not only extends a correspondence between model explainability and cooperative game theory, but also yields better empirical results on a search engine evaluated on the PASCAL VOC dataset \cite{Everingham10}. In summary:

\begin{itemize}
    \itemsep0em
    \item We introduce and evaluate new, \textit{model-agnostic}, methods for interpreting search engines and recommendation systems.
    \item We extend existing approaches to extract full pairwise correspondences between item characteristics from similarity judgments.
    \item We relate search engine interpretability to the theory of Harsanyi Dividends, and show that this provides a simple way to calculate optimal credit-assignment schemes for more complex settings like search.
\end{itemize}

\section{Background}

This work focuses on search, retrieval, and recommendation architectures. Often, these systems use similarity between objects or learned features \cite{bengio2013representation} to rank, retrieve, or suggest content \cite{bing_2017, su2009survey}. More formally, we refer to systems that leverage a distance, relevance, or similarity functions of the form: $d: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$ to quantify the relationship between items from sets $\mathcal{X}$ and $\mathcal{Y}$. In search and retrieval, $\mathcal{X}$ represents the space of search queries and $\mathcal{Y}$ represents the space of results, the function $d$ assigns a relevance to each query result pair. Finding the most relevant results for a query $x \in \mathcal{X}$ takes the form:

$$\argmax_{y \in \mathcal{Y}} d(x,y)$$

Specializing this notion yields a variety of different kinds of ML systems. If $\mathcal{X} = \mathcal{Y} = \Range(\mathcal{N}(\cdot))$ where $\mathcal{N}$ is an image featurization network such as ResNet50 \cite{he2016deep}, the formalism yields a visual search engine or ``reverse image search''. Though this work focuses on visual search, we note that if $\mathcal{X}$ is the space of character sequences and $\mathcal{Y}$ is the space of webpages, this represents web search. If $\mathcal{X}$ are users and $\mathcal{Y}$ are the space of items, such as songs or news articles, the formalism represents a recommendation problem. In this work we aim to extract meaningful ``interpretations'' or ``explanations'' of the function $d$.

\subsection{Model Interpretability}
The Bias-Variance trade-off \cite{kohavi1996bias} affects all machine learning systems and governs the relationship between a model's expressiveness and generalization ability. In data-rich scenarios, generalization error is dominated by a model's bias and increasing the size of the model class can improve performance. However, increasing model complexity can severely degrade its ``interpretability''. Larger model classes have more parameters to interpret and these parameters can lose their connection to physically meaningful quantities. This is a challenge not just for classification and regression systems, but for search and recommendation architectures as well. For example, the best performing competitors on the Netflix prize dataset \cite{bennett2007netflix} involve significantly more complex models than earlier baselines \cite{rendle2019difficulty}. Famously, the Netflix grand prize winner, the ``BellKor'' algorithm \cite{koren2009bellkor}, boosts and ensembles several different underling methods making it difficult to interpret the model by inspecting its parameters.

To tackle these challenges, some works introduce model classes that are naturally interpretable \cite{nori2019interpretml,hastie1990generalized}. Alternatively, to other works have proposed \textit{model-agnostic} methods to ``explain'' the predictions of classifiers and regressors. Many of these approaches try to model and explain the ``local'' structure around a specific prediction. For example, the black-box interpretability method LIME samples the model's behavior around a neighborhood of a prediction and distills this behavior with a simpler interpretable model like a distance-weighted, regularized, linear model. Other approaches like DeepLift and Layer-wise relevance propagation project signal from a deep network prediction backwards through the network structure. Class Activation Maps (CAMs) project the predicted class of a deep convolutional network onto the penultimate features to create a low resolution heatmap of class-specific network attention. This yields robust interpretations with only a single network evaluation, and has applications in weakly supervised semantic and instance segmentation \cite{ahn2019weakly,wang2020self}. Finally, Additive Shapley Explanations (SHAP), uses the notion of optimal credit assignment through Shapley Values \cite{shapley1951notes} to weight each feature's importance in a prediction. Importantly, \cite{shap} showed that LIME, DeepLift \cite{shrikumar2017learning}, Layer-Wise Relevance Propagation \cite{bach2015pixel}, and Integrated Gradients \cite{sundararajan2017axiomatic} are all implicitly approximating the Shapley Value and thus provided a unified understanding of these methods. We extend this connection in our work.

\begin{figure}[t]
    \centering
    \includegraphics[width=\linewidth]{figures/mistakes.jpg}
    \vspace{-.2in}
    \caption{A pair of incorrectly matched images and their corresponding marginal interpretations from Kernel SHAP which highlight important pixels in red. This visualization shows that the search engine noticed similarity between the keg and the cup, due to the geometry of these objects.}
    \label{fig:diagnose_mistakes}
\end{figure}

\begin{figure*}[t]
    \centering
    \includegraphics[width=.8\linewidth]{figures/sam_projection.png}
    \vspace{-.2in}
    \caption{Visualization of how regions of two similar images can be put in correspondence through the joint search interpretability method SAM. This correspondence also allows us to transfer labels or attention between similar images in a principled way.}
    \label{fig:attention_projection}
\end{figure*}

\section{Marginal Search Interpretability}

Though there is a considerable amount of literature on classifier interpretability, black-box search engine interpretability has only recently been investigated. We build on the rich classifier interpretability literature to present a general transformation that allows one to extend model agnostic explanations to a broader class of systems such as search engines, recommendation systems, and content-based retrieval algorithms. This approach extracts the features, words, or superpixels of the data that contribute the most to the overall similarity judgement. More formally, given a similarity function $d: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$ and elements $x \in \mathcal{X}$ and $y \in \mathcal{Y}$ we form the following pair of functions:

\vspace{-13pt}
\begin{align}
    f_y(x): \mathcal{X} \to \mathbb{R} &\vcentcolon= f_{link}(d(x,y)) \\
    f_x(y): \mathcal{Y} \to \mathbb{R} &\vcentcolon= f_{link}(d(x,y))
\end{align}
\vspace{-13pt}

Where $f_{link}$ is a function to transform the output of $d$ so that explanations have more natural and additive units resembling log-odds for classification \cite{shap,nelder1972generalized}. For the commonly used cosine similarity between features shown in Figure \ref{fig:shap_architecture}, we use $\operatorname{atanh}: [-1, 1] \to \mathbb{R}$. By using these functions with existing classifier interpretability techniques like LIME, SHAP, DeepLift, and Layer-wise relevance propagation we can compute the most important features, superpixels, or words in both the query, $x$ and retrieved data, $y$, respectively. We refer to this approach as ``marginal'' search interpretability as it finds important substructure within $x$ and $y$ but does not generate a ``correspondence'' between the structures of $x$ and $y$. In Section \ref{sec:transport} we discuss this connection in greater detail.

Figure \ref{fig:shap_architecture} middle row demonstrates this approach applied to a simple deep visual search architecture where a deep network featurizes both query and retrieved images before comparing their similarity. Transforming the search query via partial function application and using different link function ($f_{link} = \operatorname{atanh}$) enables direct use of existing model-interpretability approaches. We compare these approaches visually in Figure \ref{fig:comparison}.

In addition to extending black-box interpretation methods to search we also extend the white-box convolution network interpretation approach, CAM, to search. We refer to this approach as ``Search Activation Maps (SAM)''. CAM and SAM are both apply to convolutional networks with a penultimate global average pooling layer. Despite this limitation, CAM is widely used because it only requires a single forward pass of the network and is fully differentiable. Differentiability allows CAM regularization and manipulation in more complex vision architectures \cite{wang2020self}. To generalize CAM to search and recommendation architectures, let $f_{hwc}$ represent the layer of activations of the query image before the global average pooling layer with spatial height $h$, width $w$, and feature size $c$. Let the function $\norm_c$ represent normalization with respect to the channel dimension $c$. Let $g_c$ represent the average pooled activations of the retrieved image. Contracting $f$ and $g$ along the ``feature'' dimension $c$ yields an attention map $a_{hw}$ over the query image. In Einstein notation \cite{einstein1923foundation}:

\vspace{-15pt}
\begin{equation}
    a_{hw} \vcentcolon= \norm_c({f_{hwc}}) \norm_c({g^c})
\end{equation}
\vspace{-15pt}

Like CAM, SAM can be overlayed on query image pixels to visualize the regions that most contribute to the similarity. Additionally, one can switch the query and target images in the above formulation to visualize attention over the retrieved image pixels.

Visualizing why pairs of images are considered similar can help model users and builders understand their retrieval systems. For example, Figure \ref{fig:diagnose_mistakes} demonstrates how this approach can identify why a particular image search system erroneously matched a pair of irrelevant images. This is analogous to how classifier interpretation approaches allow model designers to diagnose errors \cite{lime}.

\section{Joint Search Interpretability}

Visualizing the pixels that most ``explain'' the similarity judgement provides a simple way to inspect where a black-box search engine is ``attending to''. However, this visualization is only part of the story. Image pairs can be similar for many different reasons, and a good explanation should clearly delineate these independent reasons. For example, consider the pair of images in the bottom left of Figure \ref{fig:shap_architecture}. These images show two similar scenes of people playing with dogs with several similar objects, but in different arrangements. We seek not just a heatmap highlighting similar aspects, but a data-structure which tells us about how parts of the query image \textit{correspond} to parts of a target image and vice-versa. In this section we will present generalizations of CAM, LIME, and SHAP to this richer interpretation setting.

\subsection{Joint Search Activation Maps (SAM)}

We begin with generalizing CAM to the joint setting. By modifying the tensor contraction used in marginal SAM, we can compute a tensor which represents the affinities between regions of the query and retrieved images.  More formally, let $f_{hwc}$, represent the layer of activations of the query image before the global average pooling layer of a trained convolutional network. This tensor has spatial height $h$, width $w$, and feature size $c$. Let $g^{ijc}$ represent the corresponding tensor of activations for the retrieved images with spatial dimensions $i$, and $j$. Contracting these tensors across $c$ yields:

\vspace{-13pt}
\begin{equation}
    \label{eqn:jsam}
    a_{hw}^{ij} \vcentcolon= \norm_c(h_{hwc})\norm_c(g^{ijc})
\end{equation}
\vspace{-13pt}

Which is a fourth rank tensor whose values are cosine distances between each combination of features in the query and retrieved image. This tensor not only captures the fine-grained correspondence between query and retrieved images, but also has an operational interpretation as a transport mapping which can project ``attention'' from the query to the retrieved image and vice-versa. We elaborate on this connection in Section \ref{sec:transport}.

\subsection{Joint Search LIME}

Next, we generalize LIME to provide joint search interpretations. In the classifier formulation of LIME, one fits a locally-weighted and regularized linear approximation to the neighborhood of a particular model's prediction. For image classifiers, one forms thus ``neighborhood'' by decomposing the input image into superpixels \cite{achanta2010slic} and perturbing the image by replacing random superpixels with a background color. This process is called ``toggling'' the superpixels, and the boolean states of these superpixels are used to form the features of a linear regression dataset. The ``targets'' are the log probabilities given to the perturbed images by the image classifier under examination. Fitting a locally weighted and regularized linear model to this dataset captures the impact of each superpixel on the prediction. In particular, the fit model is of the form:

\vspace{-10pt}
\begin{equation}
    y \sim a_0 + \sum_q a_{q} s_q
\end{equation}
\vspace{-13pt}

\begin{figure}
    \centering
    \includegraphics[height=130px]{figures/marginal_eval.jpg}
        \vspace{-.1in}
    \caption{Marginal interpretation evaluation strategy. A good method should highlight pixels in the query image (top left and middle) that, when censored (top right), have the largest possible impact on the similarity.}
    \label{fig:marginal-eval}
\end{figure}

Where $s_q=1$ when a superpixel $q$ is toggled ``on'' and the learned weights $a_q$ represent the impact of each superpixel. We point readers to the original work \cite{lime} for a more detailed description of the approach. To generalize LIME to the domain of joint search interpretability we instead consider locally approximating the search engine with a regularized \textit{bi-linear} approximation to capture cross image interactions. More specifically, we decompose both query and retrieved images into superpixels and form a dataset by toggling superpixels in both images simultaneously. We then map these ``perturbed'' pairs through similarity function $d$ and the link function $f_{link}$. Finally, we fit a bi-linear model of the form:

\vspace{-10pt}
\begin{equation}
    y \sim a_0 + \sum_q a_{q} s_q + \sum_r a_r s_r + \sum_{q,r} a_{qr} s_q s_r
\end{equation}
\vspace{-13pt}

Where $s_q$ are the indicator variables that a superpixel, $q$, in the \textbf{q}uery image is toggled on, and likewise $s_r$ indicates the presence of superpixel $r$ in the \textbf{r}etrieved image. The model parameters: $a_0$, $a_q$, $a_r$, and $a_{qr}$ are the learned constant, query-linear, retrieved-linear, and bi-linear terms, respectively. As in the original implementation of LIME, one can include a local weighting when fitting the bi-linear model to increase the influence of less-perturbed pairs of images and a L1 penalty to increase the sparsity of solutions.

\begin{figure}
    \centering
    \includegraphics[height=130px]{figures/joint_eval.jpg}
    \vspace{-.1in}
    \caption{Joint interpretation evaluation strategy. A good method should project query image attention (top left and middle) to corresponding structure in the retrieved image (bottom left and middle). When censoring all but this shared structure (right column) the search engine should view these images as similar.}
    \label{fig:joint-eval}
\end{figure}

\subsection{Joint Search SHAP}

Additive Shapley Explanations supply a principled and axiomatic framework for classifier interpretation and unify several existing approaches. This work looks to extend this framework to the richer functional forms and interpretations needed for search and recommendation systems. SHAP explains predictions by weighting features according to how much ``credit'' each receives for the prediction in an optimal credit assignment framework. More formally, a ``coalition game'' is a set $N$ of $n$ players and a function $v: 2^N \to \mathbb{R}$ where $v(\emptyset) = 0$. In cooperative game theory, this function $v$ represents the expected payoff for a given cooperating coalition of players. Shapley Value Theory shows that the unique, fair credit assignment to each player, $\phi_v(i \in N)$, can be calculated as:

 \vspace{-15pt}
 \begin{equation}
    \label{eqn:shap}
     \phi_v(i) \vcentcolon= \sum_{T \subseteq N \setminus \{i\}} \frac{|T|! (n-|T|-1)!}{n!}(v(T \cup \{i\}) - v(T))

 \end{equation}
 \vspace{-15pt}

This equation captures the average increase in value that a player $i$ brings to a coalition $T$ by weighting each increase, $v(T \cup \{i\}) - v(T)$, by the number of ways this event could have happened during the formation of the ``grand coalition'' $N$. We note that this assignment, $\phi_v$, is the only one that satisfies four reasonable properties: Symmetry under player re-labeling, no credit assignment to ``dummy'' players, linearity, and ``efficiency'' which states that Shapley values should sum to $v(N)$ \cite{young1985monotonic}.
To connect this work to model interpretability, one can identify the `features'' used in model as the ``players'' and interpret the characteristic function, $v(T)$, as the expected prediction of the model with features corresponding to $N \setminus T$ replaced by values from a ``background'' distribution. In our experiments we use a blank image.

To guide our generalization of SHAP we note that both SAM and LIME generalize to joint interpretation methods by replacing linear forms with multi-linear forms to capture interactions between the query and retrieved images. Following this pattern, we seek higher order generalizations of the Shapley Value. In particular \cite{harsanyi1963simplified} show that the Shapley Values arise from distributing ``Harsanyi dividends'' associated with every coalition of players, evenly across members of the coalition. More specifically, the Harsanyi Dividend for a coalition $S$ is defined as:

\vspace{-13pt}
\begin{equation}
      d_v(S) \vcentcolon=
  \begin{cases}
    v(S) & \text{if $|S|=1$} \\
    v(S) - \sum_{T \subsetneq S} d_v(T) & \text{if $|S|>1$} \\
    \end{cases}
\end{equation}
\vspace{-13pt}

These dividends, also known as Mob\"ius coefficients \cite{grabisch2016set}, provide a detailed view of the function's behavior at every coalition. In this work we generalize the notion of sharing these dividends between individuals to sharing these dividends between sub-coalitions. This insight provides natural motivation for a recent Generalization of Shapley Values called the Shapley-Taylor Indices \cite{sundararajan2020shapley}. More specifically, \cite{sundararajan2020shapley} define a generalization of the Shapley Values to coalitions of a size $k$ using the discrete derivative operator, which leads to their namesake connection with Taylor series expansions. However, we show that in the language of Harsanyi Dividends the connection between the Shapley Values and the Shapley-Taylor index is much more apparent. Through the lens of sharing dividends, we arrive at a new formula for the Shapley-Taylor values for coalitions $|S| = k$:

 \vspace{-10pt}
 \begin{equation}
    \phi^k_v(S) = \sum_{T:S \subset T} \frac{d_v(T)}{\binom{|T|}{|S|}}
 \end{equation}
 \vspace{-10pt}

 In this light, one can see that Shapley-Taylor values simply divide up the high order Harsanyi Dividends evenly among all lower-order terms. This reproduces the Shapley Values when $k=1$. When $k=2$ it assigns credit to each individual $|S|=1$ and pair of individuals $|S|=2$. This insight not only allows for deriving the Shapley Values and the Shapley-Taylor indices, but also allows for the creation of new optimal credit assignment weights for more complex problems. In our motivating example of interpreting visual search engines, we keep terms associated with each superpixel, and cross-image pairs of superpixels (first superpixel is in the query image, and the second superpixel is in the retrieved image). We divide all other dividends and distribute them their smaller coalitions. This ``cross-linear'' optimal credit assignment has the same binary terms as the Shapley-Taylor indices, but different unary terms due to the splitting of intra-image terms among the individual superpixels. Put another way, Harsanyi dividend sharing allows one to create ``jagged'' generalizations of the Shapley-Taylor indices that can yield optimal credit assignments on more complex collections of coalitions.

Though the Harsanyi Dividends and Shapley-Taylor indices provide a robust way to allocate credit, they are difficult to compute. The authors of the Shapley-Taylor indices provide a sampling-based approximation, but this approach requires estimating each interaction term with a separate sampling procedure, which scales poorly as dimensionality increases. To make this approach more tractable for large scale datasets we draw a parallel the unification of LIME with Shapley Values through a specific weighting kernel. In particular we introduce a kernel for second order Shapley-Taylor indices:

 \vspace{-13pt}
\begin{equation}
    k(N, S) = \frac{|N| -1}{\binom{|N|}{|S|} \binom{|S|}{2} (|N|-|S|)}
\end{equation}
 \vspace{-13pt}

Using this kernel, one can instead sample random coalitions, evaluate $v$, and aggregate the information into a bilinear model. This allows one to approximate all Shapley-Taylor indicies of $k=2$ with a single sampling procedure, and converges much faster for high dimensional problems and we report convergence rates in Section \ref{sec:kernel-convergence} of the Supplement. This also allows us to view joint search LIME as a Shapley-Taylor index approximator much like regular LIME is to the Shapley Values.

\subsection{Connection to Attention Transport}
\label{sec:transport}

Beyond measuring of the interaction strength between regions in the query and retrieved image, joint explanations also have an operational interpretation as a method of attention transportation between images. For example in Equation \ref{eqn:jsam} which characterizes Joint SAM the co-variant and contra-variant positions of the spatial indices indicate that the tensor, $a^{ij}_{hw}$, also encodes a mapping. More specifically, let $q^{hw}$ denote an attention map over the query image, we form a corresponding attention map on the retrieved image using the tensor contraction: $r^{ij} \vcentcolon= a^{ij}_{hw} q^{hw}$. Figure \ref{fig:attention_projection} demonstrates how projecting different localized attention maps can visualize detailed information about how specific regions of the query image correspond to regions of the target image.

By normalizing the joint interpretation tensor, the connection to transport theory becomes even more explicit. More specifically, we can view the normalized attention weights as a joint distribution over pairs of regions. The marginals of this distribution are precisely the marginal search interpretations. In summary, because a joint distribution encodes a transport plan between its marginals \cite{dobrushin1970prescribing}, a joint interpretation acts as a transport plan between the marginal interpretations.

For LIME and Kernel SHAP, these transport strategies occur in the superpixel space rather than the image region space. In particular, one can use the interaction terms as a linear map to transfer attention from query to retrieved image superpixels. One can then project the weights onto the pixels by coloring each superpixel with its attention.

\section{Experimental Evaluation}

Evaluating the faithfulness of an interpretability method requires careful experimental design. Interpretations that ``look good'' to a human might not be faithful to the underlying model which could model the task poorly. We aim to evaluate how well our proposed joint and marginal interpretation approaches capture the structure of the underlying search engine independent of how the model performs semantically. To this end we expand the ``interpret-and-censor'' evaluation strategy of \cite{shap}. This approach first interprets a prediction, then censors the most important features, and finally re-evaluates the classifier. If the interpretation method works well, this should yield a large change in the prediction. The most important part of this method is that it directly measures faithfulness to the underlying classifier and does not conflate this with the performance of the classifier on the task.

\begin{table*}[t]
\caption{Comparison of speed and performance of marginal and joint search interpretation methods across three different search engines. $\Delta d$ (higher is better) represents the change in similarity when censoring the top 20\% of pixels in the marginal interpretation as shown in Figure \ref{fig:marginal-eval}. $d$ (lower is better) is the similarity between attention-censored images as shown in  Figure \ref{fig:joint-eval}. mIoU (higher is better) represents the performance on a binary semantic segmentation label propagation task as described in Section \ref{sec:eval-label-prop}.}
\label{tab:results}
\vspace{-.15in}
\begin{center}
\begin{small}
\begin{sc}

\begin{tabular}{cc|cc|ccc}
\toprule
                             &                        & \multicolumn{2}{c|}{Marginal}                         & \multicolumn{3}{c}{Joint}                                               \\
Search Backbone              & Method                 & Time (s)                  & $\Delta d$                & Time (s)                  & $d$                         & mIoU          \\ \midrule
\multirow{6}{*}{ResNet50}    & SBSM                   & 19.5 $\pm$ 0.0            & 0.1 $\pm$ 0.003           & 87.4 $\pm$ 0.01           & 0.32 $\pm$ 0.005            & 0.52          \\
                             & SAM                    & \textbf{0.09 $\pm$ 0.001} & 0.07 $\pm$ 0.003          & \textbf{0.03 $\pm$ 0.0}   & 0.27 $\pm$ 0.003            & \textbf{0.65} \\
                             & LIME                   & 32.6 $\pm$ 0.02           & 0.11 $\pm$ 0.004          & 120.9 $\pm$ 0.06          & 0.26 $\pm$ 0.004            & 0.62          \\
                             & Integrated   Gradients & 3.4 $\pm$ 0.0             & 0.1 $\pm$ 0.004           & -                         & -                           & -             \\
                             & Kernel   SHAP          & 44.4 $\pm$ 0.1            & 0.11 $\pm$ 0.004          & 129.4 $\pm$ 0.87          & \textbf{0.25 $\pm$ 0.005}   & \textbf{0.65} \\
                             & Partition   SHAP       & 45.0 $\pm$ 0.01           & \textbf{0.15 $\pm$ 0.004} & -                         & -                           & -             \\ \midrule
\multirow{6}{*}{VGG11}       & SBSM                   & 16.3 $\pm$ 0.01           & 0.12 $\pm$ 0.005          & 68.7 $\pm$ 0.02           & 0.41 $\pm$ 0.007            & 0.53          \\
                             & SAM                    & \textbf{0.07 $\pm$ 0.003} & 0.1 $\pm$ 0.005           & \textbf{0.05 $\pm$ 0.001} & \textbf{ 0.31 $\pm$ 0.003}  & \textbf{0.72} \\
                             & LIME                   & 29.8 $\pm$ 0.02           & 0.12 $\pm$ 0.005          & 109.0 $\pm$ 0.04          & 0.32 $\pm$ 0.005            & 0.63          \\
                             & Integrated   Gradients & 3.1 $\pm$ 0.01            & 0.08 $\pm$ 0.005          & -                         & -                           & -             \\
                             & Kernel   SHAP          & 41.4 $\pm$ 0.07           & 0.12 $\pm$ 0.005          & 115.5 $\pm$ 0.71          & \textbf{ 0.31 $\pm$ 0.006 } & 0.67         \\
                             & Partition   SHAP       & 41.5 $\pm$ 0.03           & \textbf{0.16 $\pm$ 0.006} & -                         & -                           & -             \\ \midrule
\multirow{6}{*}{DenseNet121} & SBSM                   & 21.4 $\pm$ 0.01           & 0.16 $\pm$ 0.006          & 88.5 $\pm$ 0.01           & 0.6 $\pm$ 0.008             & 0.54          \\
                             & SAM                    & \textbf{0.17 $\pm$ 0.004} & 0.14 $\pm$ 0.006          & \textbf{0.12 $\pm$ 0.001} & 0.52 $\pm$ 0.005            & 0.63          \\
                             & LIME                   & 34.4 $\pm$ 0.02           & 0.18 $\pm$ 0.006          & 130.5 $\pm$ 0.06          & 0.5 $\pm$ 0.006            & 0.62          \\
                             & Integrated   Gradients & 3.6 $\pm$ 0.0             & 0.14 $\pm$ 0.007          & -                         & -                           & -             \\
                             & Kernel   SHAP          & 46.2 $\pm$ 0.06           & 0.19 $\pm$ 0.007          & 130.5 $\pm$ 1.03          & \textbf{0.49 $\pm$ 0.008}   & \textbf{0.65} \\
                             & Partition   SHAP       & 47.6 $\pm$ 0.02           & \textbf{0.24 $\pm$ 0.007} & -                         & -                           & -             \\ \bottomrule
\end{tabular}

\end{sc}
\end{small}
\end{center}
\vskip -0.2in
\end{table*}

\subsection{Marginal Search Interpretation}

We first generalize the interpret-and-censor evaluation to marginal search interpretation methods. Intuitively speaking, a good marginal search interpretation should highlight the ``most important'' pixels that contribute to the similarity of the two images. Censoring these important pixels by filling them with a mean-color background should dramatically reduce the similarity between these two images. The magnitude of the change in the similarity, $\Delta d$, between uncensored and censored image pairs is a direct measurement of the interpretability method's quality and is a direct analogue of the interpret-and-censor strategy of \cite{shap}. Figure \ref{fig:marginal-eval} shows a diagram and example of our proposed strategy. We use each marginal interpretation strategy to extract an attention map over the query image pixels. Finally, we censor the top 20\% of important pixels in the query and re-evaluate the engine's similarity computation and record the change in distance, $\Delta d$. The marginal columns of Table \ref{tab:results} show the results of this experiment, and compares computation times across marginal methods, we find that SAM is the k, fastest method, but that PartitionSHAP provides the best quality interpretations and significantly outperforms the current best visual search approach SBSM \cite{dong2019explainability}. The dominance of Partition SHAP suggests that the formal basis for SHAP also yields higher-fidelity interpretation methods. We describe our models, data, and implementation details in Section \ref{sec:eval-details}

\subsection{Joint Search Interpretation}
\label{sec:eval-label-prop}

We also extend the interpret-and-censor evaluation strategy to joint interpretations by evaluating an interpretation's ability to project attention maps. We generate different query attention maps using Pascal VOC's ground truth semantic segmentation labels. Intuitively speaking, a good joint interpretation should find the region of the retrieved image that most closely matches a selected region of the query image. Censoring all but this region, as shown in Figure \ref{fig:joint-eval}, should yield similar images according to the search engine. More specifically, first we censor the query image to a randomly chosen semantic segmentation class. Second, we use the chosen class' segmentation mask as a query attention map attention and project this attention to the retrieved image using the joint interpretation method. Third we then censor all but the top $p$ pixels of the retrieved image where, $p$ is the size of corresponding semantic segmentation mask in the retrieved image. Finally, we measure the similarity between the censored images. A faithful interpretation approach should yield a similar (small $d$) image pair after censoring.

In addition to measuring faithfulness of the representation method we also measure how well the approach performs semantic segmentation label propagation. To measure this, we treat the projected attention as a binary semantic segmentation prediction and compare with the known ground truth segmentation labels. The mean intersection over union (mIoU) of this segmentation prediction provides a second measurement of interpretation quality more focused on a particular downstream task like label propagation. However, we caution that this is not a direct measurement of interpretation faithfulness, but we feel this provides a complementary view that can help creation of better label propagation systems. The joint columns of Table \ref{tab:results} show the results of these experiments over several interpretation methods that we have generalized to the joint domain. We find that SAM is by far the fastest but Kernel SHAP significantly outperforms all methods in terms of faithfulness to the underlying search engine. We describe our models, data, and implementation details in Section \ref{sec:eval-details}

We note that because this is the first work to identify these joint interpretations, there are no existing approaches in the literature to compare against in the joint domain. To compare against a reasonable literature-derived benchmark, we have additionally generalized the SBSM method to the joint setting and detail this construction in the Supplement.

\subsection{Implementation Details}
\label{sec:eval-details}

Our evaluation experiments use visual search engine built from a pre-trained ``backbone'' network that featurizes images and compares their similarity using cosine distance. We report results for engines using ResNet50 \cite{he2016deep}, VGG11 \cite{simonyan2014very}, and DenseNet121 \cite{huang2017densely},  as backbone networks. We use 200 random query images from the Pascal VOC validation set and their closest pairs in the VOC training set. Additionally, we restrict retrieved images so that the query and retrieved images both contain the same class objects and have objects that take up more than 5\% of the image \cite{hamilton2020conditional}. We include these image pairs and evaluation code in the supplement. Experiments use PyTorch \cite{pytorch} v1.7 pre-trained models, on an Ubuntu 16.04 Azure NV24 Virtual Machine with Python 3.6.

\section{Related Work}

The field of black-box visual search engine explainability is relatively new despite the ubiquity of search engines. To our knowledge \cite{dong2019explainability} is the first to present a generic visual search engine explanations. We also note the works of \cite{singh2019exs} and \cite{fernando2019study} who marginally interpret a text search engine using LIME and DeepSHAP. However, these works do not apply their methods to images and do not lift these approaches to the joint domain. The optical-flow architecture FlowNet \cite{fischer2015flownet} uses the correlation between convolutional feature maps to estimate the correspondence between two adjacent video frames as part of a larger optical flow architecture. Though similar to SAM, this approach was not interpreted in the context of search engine explanations. \cite{hou2019cross} also embeds a component similar to SAM within a deep classifier to improve few-shot performance. Other recent works have used learn co-attention within transformer architectures to help pool and share information across multiple domain types \cite{wei2020multi}.

In the field of cooperative game theory, several seminal works provide a variety of ways to extend Shapley Values to more complex interactions. \cite{harsanyi1963simplified} generalized the Shapely value by introducing a ``dividend'' that, when split and distributed among players, yields the Shapley Values. \cite{owen1972multilinear} introduce another way to extend Shapley values using a multi-linear extension of the game's characteristic function. \cite{sundararajan2020shapley} introduce the Shapley-Taylor index which they show is equivalent to the Lagrangian remainder of Owen's multi-linear extension. This work does not relate the Shapley-Taylor indices to Harsanyi dividends or explore interactions in the context search engine interpretability.

\section{Conclusion}

In this work we have proposed and evaluated new approaches to generate model-agnostic explanations for image similarity, search, and retrieval architectures. We characterize search engine interpretability methods as either ``marginal'' or ``joint'' interpretation methods depending on what kind of information they explain. We extended Class Activation Maps (CAMs), Additive Shapley Explanations (SHAP), and Locally Interpretable Model-Agnostic Explanations (LIME) to image retrieval and search and showed it was possible to extract much richer ``joint'' interpretations using extensions of these methods. These approaches enable black and grey-box model introspection and can help diagnose errors and understand the rationale behind a model's similarity judgements. We also provide a new framework for deriving optimal credit assignments from Harsanyi Dividends. We show that this framework generalizes the Shapley-Taylor indices, and provide a Kernel for faster estimation of Shapley-Taylor indices. Finally, we show this game-theoretic formalism yields more faithful model interpretation strategies for several visual search engines.

redpajama_set_name: RedPajamaArXiv


text:
Mendocino County Today: Tuesday, March 25, 2014
By AVA News Service on March 25, 2014
Memorial To Be Broadcast
Coq au Vin Update
No Right To Sleep
Three Inches Forecast
The Governor's Dreams
KZYX's Snowden Moment
Poem For Jim
Historians On Trial
UPDATE: MEMORIAL SERVICE TO BE BROADCAST LIVE VIA LOCAL CABLE CHANNEL AND INTERNET STREAM
For those unable to attend Wednesday's Memorial Service for Deputy Ricky Del Fiorentino, the Mendocino County Sheriff's Office, working with GSA/Information Services, and Mendocino Access TV has arranged for a live broadcast of the service. (The service will also be recorded for delayed playback.) Interested parties are encouraged to access Local Cable Channel 3 or 65 (Ukiah Valley) as the primary source, and the internet-feed at: http://mendocinoaccess.org as a secondary option due to bandwidth limitations. Further information is available in the enclosed Press Release [see below] and on the MCSO's Facebook site (Mendocino Sheriff).
THE LOGISTICS of a mass memorial service for slain deputy Ricky Del Fiorentino are complicated by the absence of places to hold them in Fort Bragg. There's the small-ish Cotton Auditorium, the Fort Bragg High School gym and, outside in the weather, the high school's little jewel of a football stadium; those are the three venues that can accommodate a large crowd. The late Vern Piver's service was standing room only in the gym. Piver, like the late deputy, was enormously popular, not only in Fort Bragg, but throughout the County for both his ebullient personality and his unflagging commitment to youth sports. Fort Bragg City councilman, Jere Melo, also a murder victim at the hands of a maniac, was honored with an outdoor memorial in the football arena. Del Fiorentino, also a man with a Piver-Melo quality of community respect, will draw a huge number of people. So, with rain expected for the middle of the week with the memorial scheduled for Wednesday morning, and school in session? Got to be Cotton Auditorium, which it is on Wednesday, (today, 26 March) at 10am.
ON TUESDAY NIGHT, the 25th, a Candlelight Vigil will be held in front of the Fort Bragg Justice Center located at 700 South Franklin Street in Fort Bragg, California.
BOTH EVENTS are expected to draw unprecedented numbers of mourners lamenting the loss of a very fine man. Deputy Del Fiorentino is survived by his wife, children (ages 21, 19, 18, 6), step-son (age 29), grandson (age 5 months), step-grandchildren (ages 6, 3), parents and siblings. Anyone wishing to make monetary donations to the Ricky Del Fiorentino Memorial Fund can do so by contacting or visiting any of the Savings Bank of Mendocino County locations or by mail to PO Box 3600 Ukiah, Ca 95482 (707-462-6613). To donate online via PayPal to Deputy Del Fiorentino's memorial fund go to the Mendocino County Sheriff's facebook page at:
https://www.facebook.com/MendocinoSheriff
DEPUTY RICKY DEL FIORENTINO MEMORIAL SERVICE TO BE TELEVISED ON TV & INTERNET
Cotton Auditorium (500 North Harold Street, Fort Bragg). Wednesday, March 26, 2014, 10am.
In preparation for the memorial service for fallen Deputy Ricky Del Fiorentino on Wednesday March 26, 2014 at 10:00 am the Mendocino County Sheriff's Office expects attendance to exceed the seating capacity of the Cotton Auditorium. Overflow seating has been established adjacent to the Cotton Auditorium at the Fort Bragg Middle School Gymnasium and a large enclosed outdoor tent beside the Gymnasium. It is expected that there will be a live steaming video feed transmitted from inside the Cotton Auditorium to the Gymnasium and tent for viewing purposes. The live steaming video feed will be broadcasted on cable channels 3 and 65 in the Ukiah Valley area and on the internet at the www.mendocinoaccess.org website. Those who are unable to attend the memorial service are encouraged to watch the cable broadcast if all possible as opposed to the internet broadcast to prevent any disruption in the internet broadcast due to over saturation. (Sheriff's Press Release)
MRS. RICHARD, of the excellent Coq au Vin restaurant at Floodgate, Navarro, visited us Monday morning. Gracious but unhappy, she explained the extended period of medical misfortune which has plunged her culinarily talented family into a morass of insurmountable debt. The Richards, prior to the onset of their medical emergencies, enjoyed a long run of celebrated success in the East Bay. Things went awry in Monterey County. Without re-hashing the entire sad saga, all of it exhaustively recorded by ghoulish Monterey County newspapers, I believe Mrs. Richard and I regret printing the links to the Monterey stories because none of those stories addresses the crucial point: Crooks tend not to work hard. The point of being a crook is to avoid sustained labor of any kind. The Richards have worked long and hard in a business that runs on long, hard hours. I hope Valley residents will see it my way and continue to enjoy the truly fine food at Coq au Vin. The Richards try hard and they deserve a break.
THE QUESTION OF THE DAY: Does Mendocino County's Right To Farm ordinance trump the State of California's Noise Pollution standards? Many residents of the normally quiet and bucolic Anderson Valley are fuming at the early morning noise coming from giant propellers, fueled by propane, that the vineyards are using for frost protection. These things are loud, very loud, sleep-disrupting loud, and their din is inescapable, echoing off the hills after it jolts the Valley floor awake, keeping everyone up and cursing the day the industrial booze industry took over our 26 miles of shattered tranquility.
THE AURAL ASSAULTS began Tuesday morning the 18th a little after 1am. More than a week later they continue, blasting off between midnight and roaring on until an hour or so after daylight.
WE CALLED Chuck Morse, Mendocino County's Ag Commissioner, asking for a clarification. Other people are calling the Sheriff's Office. Yesterday we were informed by an irate citizen that the Sheriff will dispatch the duty officer from Ukiah to measure decibel levels. We haven't heard of an officer actually appearing to take decibel levels, and even if they're found to exceed state guidelines of 60 and above, what can be done about it? The wine industry rules Mendocino County, as most of us know, and it would be a rare official, elected or appointed, who would dare take them on. Everyone seems hazy on the law as it applies to the wind machines.
WE CONTACTED 5th District supervisor Hamburg: Can we get a clarification on the rules? Does the right to farm ordinance trump the noise pollution laws? Several thousand people are awakened every early morning over here for the last week when these fans go on from midnight to past daybreak."
HAMBURG passed us to Morse, who wrote: "Morning Dan, It is a function, this year much more so than in 'normal' years, of the drought and I believe the practice is covered under the right to farm ordinance. A year or two ago I explored the possibility of quieter props on the wind machines, but that didn't go too far, as it tends to minimize the goal in the first place and they were not really available (as they are counter-productive to the goal of moving as much air as possible). I hope we get some cloud cover this week to warm the air mass and keep the wind machines off as much as possible."
BOTTOM LINE is the bottom line of the wineries. We sacrifice our sleep for their vines.
I WONDER if local realtors, henceforth, will be forced to attach sale covenants. "The buyer must agree to forgo sleep from the middle of March until the middle of April of every year…"
THE FAUX APOLOGY from Roederer reprinted in this yesterday's Mendoino County Today is especially galling. It's signed with an illegible scrawl to conceal the identity of the French national who wrote it. The wine people, esp. the furriners, don't like negative attention. They prefer Jon Bonné and sunstruck beautiful people holding golden goblets to blue skies.
THE FRENCH have invested here because they know they can do whatever they want — from labor practices to water draws to chemical applications. American vintners assume their rights of destruction come with citizenship. The French wine industry is heavily regulated. They couldn't do in France what they do here. The monarchical assumption that the industry provides jobs is, strictly speaking, true, but those jobs mostly go to non-citizens whose underpaid labor makes the wine industry possible.
ROEDERER VINEYARD workers struck at harvest time a decade ago because the wealthiest family-owned winery in the world tried to screw them out of a few cents on the dollar for bringing in the grapes. Roederer demonstrated for all time its commitment to worker welfare. The winery's first move was to bus-in replacement workers from somewhere over on I-5. When those workers arrived in Boonville they refused to cross the picket line established by Anderson Valley workers. A hurry-up election was held right in the Boonville vineyard, with sleek French execs and attorneys flown in from Paris for the occasion looking on in their five thousand dollar suits. The workers voted heavily for representation by the United Farm Workers. Roederer quickly hired the infamous union-busting law firm of Littler Mendlesohn, and in a week L&M convened seminars around the County on the theme How To Keep The Unions Out. Wine people all over the Northcoast worried that their unhoused, uninsured, underpaid labor was in revolt; they flocked to these shameful sessions, and here in The Valley, Roederer placed a snitch in their single worker's housing in Navarro whose task it was to identify those workers who were pro-union, and those workers never worked another day in the Anderson Valley.
RASTAFEST, and other local events dependent on amplified sound, are compelled to agree to strict hours. Vineyards don't. Vineyards, of course, are most vulnerable to destructive frosts during the midnight-to-early-morning hours. We all get that, and some of us even sympathize.
AS AN EARLY RISER — 4am to 5am — I always check the temperature posted outside the castle gate. Lately at 4:30, it reads right around 40 degrees, but at 6am, the crack of dawn, it's been at or about 35 degrees. Vines are harmed when it's cold enough to frost, and there indeed have been light frosts over the last week, although one would think the din from the propane blasters, by itself, would keep the vines on the awake side of death. One has to wonder, though, what did pre-industrial vineyards do?
WE UNDERSTAND that this is a drought year, and we understand that the grape growers are short on our water, the usual agent deployed for frost protection. We even agree that given the grim choice of our water vs. the oppressively intrusive noise from the big propellers, we'd rather endure a few hours of noise than see our streams sucked dry.
THINGS COULD BE DONE to reduce the nuisance level of these loud machines, especially near residences, but there doesn't appear to be much organizing to put pressure on County government even for the modest things mentioned in yesterday's post by Mark Scaramella. The dominant wine mob would object to even an attempt to call a hearing on the subject, believing it's their god-given right to keep everybody else awake.
A WEEK of light rain beginning Tuesday night is predicted to bring about three inches, total, of the precious stuff.
GOVERNOR BROWN: "He's never seen 'Chinatown,' but he's trying to deal with the drought by fixing the state's unsustainable water transport system, which his dad helped put in place and he himself tried to fix 30 years ago. And he's still fighting for his dream of a high-speed train from Sacramento to San Diego, a project bogged down in lawsuits. He takes a white model of the train from the window and lovingly places it in the middle of a big picnic table, noting that he has liked trains since he was a kid." (—Maureen Dowd)
GORDY WANTS TO KNOW
As you are an AVA reader and moreover a believer, Bruce, let me caution you about an incomplete judgment by your KZYX Board reporter in re: the internal Board memo finessed mysteriously to you, and given front-page publication. It was, the reporter writes, "an Edward Snowden moment." Not quite. Snowden acknowledged his deed.
Perhaps the usual suspect was surprised to see his FYI in print. But newspapers should have no friends, eh?
Yours, Gordy Black, Mendocino
ED REPLY: Right you are, Gordo, and looking around the room for a friendly face I see only a field of KZYX programmers, one of them sporting a greasy cravat, who stare back in the manner of dumbly quizzical ruminants, and I… Well, not to go on here because even you probably get the point, which is, if you don't, who cares?
FOR JIM ARMSTRONG of Potter Valley, a great American poem...
Autumn Begins in Martins Ferry, Ohio
In the Shreve High football stadium,
I think of Polacks nursing long beers in Tiltonsville,
And gray faces of Negroes in the blast furnace at Benwood,
And the ruptured night watchman of Wheeling Steel,
Dreaming of heroes.
All the proud fathers are ashamed to go home,
Their women cluck like starved pullets,
Dying for love.
Their sons grow suicidally beautiful
At the beginning of October,
And gallop terribly against each other's bodies.
HISTORIANS & JOURNALISTS ON TRIAL
by Tom Cahill
A Review of "Hidden History: The Secret Origins of the First World War" by Gerry Docherty and Jim Macgregor, 2013
Cecil Rhodes was a 19th Century diamond and gold mining magnate in South Africa who, according to his will, was determined to create a secret organisation of English ruling-class elites who would eventually control the world and . . . "render war impossible and promote the best interests of humanity." To this day, scholarships in his name are still being awarded. A young Bill Clinton received one.
Rhodes died young, at 48--fortunately--because by then he was a full-blown sociopath and a mega control freak who had already caused the deaths and misery of a vast population of blacks and whites in Africa in his successful attempt to monopolize that continent's gold and diamond mines. And the secret group he started in London at the turn of the 19th/20th Century included other famous sneak-thieves and royal cheats of the period such as Nathaniel Rothschild and King Edward VII. These were the chief architects of the very first global war--a war "to end all wars" and to "save democracy" as the media hype of the time proclaimed to the undereducated and the gullible.
Thus it was an old boys' club of Brits organized by Rhodes and NOT a German cabal that instigated the "Great War" of 1914-18 as popular history has propagandized us for the past hundred years. This is the theory with lottsa facts and sources NOT of German historians but of two Scots blowing the whistle on English elites. In their most timely contribution to truth and reconciliation titled "Hidden History: The Secret Origins of the First World War," Gerry Docherty and Jim Macgregor have issued a challenge to historians and journalists to stop whoring themselves to the financiers and politicians as well as the industrialists and military brass-hats and their camp-followers who have forever profited from the misfortune of others.
As for the assassination of Archduke Ferdinand and his wife in Sarajevo in June 1914 that allegedly lit the fuse of WW I, the fingerprints of the Rhodes mob are all over the crime that was pinned solely on a group of idealistic young Bosnians.
In their introduction, the authors name a very few brave Anglo/American historians and journalists who dared question the authority of this royal crime syndicate as to the causes of WW I. This is of the utmost importance because historians and journalists should be objective but, alas, much, much too often they are NOT, thus aiding and abetting horrendous crimes against humanity.
Take for instance "The New York Times"--the so-called "free world's" so-called "newspaper of record"--that for the past half century has rarely missed an opportunity to denigrate as "conspiracy nuts" the overwhelming majority of Americans (as high as eighty percent) who disagree with the Warren Commission that Lee Harvey Oswald (a poor shot in the Marine Corps) acted alone with a vintage, inaccurate, cheap, piece-of-trash carbine (NOT a rifle) in assassinating Pres. John Kennedy. So it will be interesting to see if "The New York Times" even deigns to review "Hidden History" about the current mother-of-all-conspiracy theories, ie. who really started WW I and why.
A popular bumper-sticker and poster in the US during the War on Vietnam was "WAR IS GOOD BUSINESS: Invest Your Sons." The poster was illustrated with a photo of the "Pieta," the famous statue by Michelangelo depicting the mother of Christ holding in her arms the tortured corpse of her only son.
Because of the massive wealth stolen by the "robber barons" such as J.D. Rockefeller, J.P. Morgan, Leland Stanford, Cornelius Vanderbilt and many others at the turn of the 19th/20th Centuries, Mark Twain called this period "the Gilded Age." But it wasn't just an American phenomenon. Europe and especially Great Britain was also gripped by a plague of uncontrolled, unconscionable greed at this time.
The difference between the British gang that Docherty and Macgregor call the "Secret Elite" and the American white collar gangstas was that the Brits were (1) better organized, (2) more international in scope and (3) had a broader objective, being that of white, Anglo/Saxon, global domination in which they preceded by decades Adolph Hitler's Aryan super-race.
During the Boer War of 1899-1902 the British also predated the Nazis with concentration camps in which about 20,000 white children died. More than 6,000 white women and untold numbers of black people also died in what the Secret Elite euphemistically called "camps of refuge." And most responsible for this barbarism was Alfred Milner who took over leadership of the Secret Elite when Rhodes died and was knighted for his crimes by King Edward VII.
Most governments in history have a "gangster nature," says Michael Parenti. He's an American political scientist, historian, lecturer and author of many books including "Democracy For The Few" and "Dirty Secrets."
Not long after the Boer War, the Secret Elite went into high gear, conspiring to neutralize Germany both commercially and militarily. By 1911, Winston Churchill was a card-carrying member of this clique and used his post of First Lord of the Admiralty to prepare the Royal Navy for battle against Germany. When war finally overcame the world in 1914, it was an unholy horror that even the author Stephen King could not possibly overdramatize. Fighting became static, the trenches were open cesspools, and men died from diseases as well as from the new wonder- weapons such as machine guns, flame-throwers, tanks, airplanes and-- gasp--poison gas.
The Battle of the Somme in France began 1 July 1916, literally with a bang--a very big one--from tons of explosives placed by miners under a German trench that left a crater three hundred feet across and ninety feet deep. In just that one day, in just that one battle, the British Army alone lost more than 20,000 killed and 40,000 wounded and missing. Most of the casualties fell in the first two hours, cut down in a crossfire by well-placed German machine gunners. There was virtually no gain, and that infamous day's losses were never matched before or since by any army in the world.
Can you imagine dealing with 20,000 corpses as the medical aid stations were overwhelmed with tens of thousands of wounded? Some were blown to small, unidentifiable pieces and/or buried by explosions of artillery shells, never to be seen again. Bones of men killed that day are still being unearthed by farmers along with unexploded munitions and other debris of combat. And can you imagine the number of widows created that single day in 1916? It had become fashionable for young women to marry their sweethearts just before they left for France--a blessing for a safe return.
But I'm getting ahead of myself. "Hidden History" is about the run-up to WW I: who masterminded the plans to provoke Germany; who financed the filthy politics, deceit and hypocrisy; what newspapers beside "The New York Times" beat-the-drum for the bloodletting; what British General was a member of the Secret Elite and most responsible for the carnage at the Somme.
"War against a foreign country only happens when the moneyed class think they are going to profit from it," said George Orwell.
Rhodes' Secret Elite may have even caused WW II as well as the "Great" War in that Germany's punishment at Versailles in 1919 set the stage for a fanatical dictator like Adolph Hitler.
Docherty and Macgregor name names, dates, and places in incredible detail, enough to hang as traitors members of the Secret Elite. One family name repeated over and over and over is "Rothschild"--perhaps the arch-villains of all times. And the author's end their scholarly contribution to justice with one hundred pages of notes, references, and index.
"Hidden History" could be, and should be, a template for Historians Without Borders to chronicle the causes of other wars. Just "follow the money trail" and name names especially of dynasties such as the Rothschilds involved in more than one mass murder.
But beware! This book could affect you as Peter Weiss has his key character lament in "The Persecution and Assassination of Jean-Paul Marat as Performed by the Inmates of the Asylum of Charenton Under the Direction of the Maquis of Sade."
Day and night were not enough for me. When I investigated a wrong, It grew branches And every branch grew twigs. Wherever I turned, I found corruption.
"Hidden History" proves once again--as if further evidence is necessary--that conspiracy is synonymous with politics. The book is not for the faint-of-heart, but is an invaluable contribution for the psycho/spiritual healing necessary for the second great Renaissance now spreading globally.
Jim Armstrong March 25, 2014
A great short American poem.

redpajama_set_name: RedPajamaCommonCrawl


Me is a WOMAN, that's right!
My positive though for today is, "I am glad to have been born a WOMAN!" Let's hear it for the girls!
Atomic weight: 'Don't even go there'.
Physical properties: Generally round in form. Boils at nothing and may freeze at any time. Melts whenever treated properly. Very bitter if not used well.
slightly green when placed next to a better specimen.
Sarcastic Redneck Bumper Stickers……..and more!
Do you want me to accept you as you are, or do you want me to lie to myself and try to like you?
Don't let your mind wander; it's far too small to be let out on its own.
Don't thank me for insulting you; it was a pleasure.
Don't you realize that there are enough people to hate in the world already without you putting in so much effort to give us another?
Thanks to Pam for this one!
Wow, I wonder if this one really exists? Nasty!
for thinking outside the box….
They're there when you need a shoulder to lean on.
understanding what you're all about….
to discover your true colors….
ollow your heart wherever it takes you….
Don't be afraid to love them back….
Have a Grrreat day and week !
Thanks to Kristi, my friend since 8th grade!


text:
CFE Tax: A Guide to Understanding the Basics of Canadian Income Taxation, 7th Edition, Softbound book
Author: Jason Fleming
CFE Tax: A Guide to Understanding the Basics of Canadian Income Taxation, 7th Edition is an easy-to-read text on federal income taxation in Canada. The guide focuses on giving students a technical but basic understanding of income taxation and is designed to introduce readers to the basics of income taxation using plain language. Introductory tax texts are normally very lengthy due to the complexity of the Canadian income tax system, but the author of this work has endeavoured to eliminate the highly technical aspects of Canadian taxation and has provided the basic principles in a simple and readable book.
The new edition has been updated to reflect tax changes up to March 31, 2020
The new edition includes added material on GST, CCA changes (including accelerated investment incentive), new tax credits, proposed stock option changes, and some COVID-19 tax changes
Short and easy-to-read textbook on Canadian income tax appropriate for CFE writers (and university and college students), who can use this book in their studies
Includes many examples to help explain complex tax topics to readers
Est Cost:New editions supplied every two years on standing order subscription


Pakistan: Lucky Cement's profit has fallen as its cost of sales including coal, other fuels and packing materials have risen. Its standalone profit after tax fell by 10.9% year-on-year to US$98.3m in the financial year that ended on 30 June 2018 from US$110m in the same period in 2017.
Increasing the Use of Alternative Fuels at Cement Plants: International Best Practice vii This report was produced in cooperation between IFC, SNIC (Sindicato Nacional da Indústria do Cimento), and ABCP (Associação Brasileira de Cimento Portland).
Cement plant use coal as fuel . Introduction: Rotary kiln cement plant is the main equipment of calcining the cement clinker. The characteristics of a rotary kiln . are with simple structure, convenient and reliable control of the producing process,with little wearing parts . and high running rate.
Electricity Production – Coal is mainly used as Fuel to generate Electricity through ... and suppliers in UAE provides all type of cement plants. A critical review on energy use and savings in the cement industries 2045. 1.2.
Alternative Fuel Use in Cement Manufacturing Implications, opportunities and barriers in Ontario ... 3.1.1 How is the use of alternative fuels in cement production different from waste ... fuels for coal and petcoke therefore provides the best opportunity for emissions reductions.
Agricultural waste for cement production optimization an in-depth analysis on effect of using conventional fuel (mineral coal, pet-coke, Heavy oil and Natural gas) and Agricultural waste (sugar waste and ground nut shell). This mixture is intended for use in a rotary kiln of clinker production, dry process.
The most significant uses are in electricity generation, steel production, cement manufacturing and as a liquid fuel. Request Quotation. The U.S. cement industry uses energy equivalent to about ... According to the Department of Energy, U.S. cement production accounts for 0.33 percent of ... Request Quotation. coal use in cement industry.
COMPARATIVE ANALYSIS OF COAL, FUEL OIL AND NATURAL GAS FOR CEMENT PRODUCTION. ABSTRACT. This study is a comparative analysis of the use of coal, fuel oil and natural gas for cement production noting their physical and chemical properties, cost and availability, impact on the environment and human health.
the type of energy used to fuel the kilns where the cement clinker is produced. Cement manufacturing requires very large amounts of energy and cement manufacturers have used a variety of energy inputs.
However, more recently, other regions have increased their use of coal for cement production due to supply and cost volatility for other fossil fuels. India. India, where coal is the dominant fuel used for cement production, has suffered from inconsistent coal supplies for the past several years.


text:
1 Comment / By Gary Jenkins / January 21, 2019 April 5, 2022 / Gangland Wire Crime Stories
I am adding a new method of supporting the podcast. If you have the venmo app, how about donating an average of fifty cents an episode or just hit me up for a buck every time you think of it. In this second installment, I will talk about the activities of the Hole in the Wall gang as they stole, drank, gambled and ate pizza in Las Vegas. Remember during this time the Chicago outfit and 3 other Midwestern families had manipulated the Teamsters Union pension fund to loan money to a man named Allen Glick. He purchased the Stardust, the Hacienda, the Fremont and the Marina. The Stardust would be the flagship. Frank Balistrieri, boss of the Milwaukee family, had ordered Glick to promote a current stardust employee named Frank Rosenthal to a position of authority in his corporation. In 1971, Chicago Outfit boss Joseph "Joey Doves" Aiuppa ordered Tony Spilotro to move to Las Vegas in support of their interests. Aiuppa is replacing an aging Chicago Outfit member named Marshal Caifano who has been Chicago's man in Vegas. Spilotro opens a gift shop in Circus Circus under his often-used alias, Anthony Stuart. The Circus Circus owner, Jay Sarno, had received a 43-million-dollar Teamster's loan and could not question this new gift shop operator. Tony Spilotro will eventually be found out by the Nevada Gaming Control board and placed in the Black Book of banned persons. As a item of interest, Spilotro invested $70,000.00 in this Gift shop and will sell it for over $700,000.00 dollars. Law enforcement is starting to watch Spilotro and he must use other people to pass messages back and forth between himself and Lefty Rosenthal. They need to avoid any overt contact because the FBI has forged a working relationship with Las Vegas Metro Police Intelligence and Nevada Gaming Control and they are all watching closely by the later 1970s. Frank Cullotta was not known to Vegas authorities and Spilotro brings him out to Las Vegas to assist.
During this time, Spilotro needs to make money and he knows how to do big time scores, like jewelry stores, banks, fur stores and robbing drug dealers. He opens a jewelry store called The Gold Rush limited to use as his headquarters and a front to fence stolen good. This store was just off the strip at 228 West Sahara and is a vacant lot now. By this time in his life Tony Spilotro is the only real boss of organized crime in Las Vegas. As the boss, tony does not go on scores. Aiuppa has assigned Joey the Clown or Joseph Lombardo to be Spilotor's Chicago contact. Spilotro has recruited Frank Cullotta who will recruit other Chicago burglars to join his crew. Between Spilotro and Cullotta, they put together guys who law enforcement will name The Hole in the Wall Gang.
One of the men Frank Cullotta recruited was a career criminal named Ernie Davino. Unlike the others, Ernie Davino was not from Chicago, he was a New Jersey mobster who had migrated to Las Vegas on his own. He was the son of Ernest "Tubby" Davino, a leg breaker and enforcer for New York mobster Albert Anastasia in the 1950s. Ernie Davino would later claim that he and a man named Leo Guardino started the Hole in the Wall Gang and eventually fell in with Frank Cullotta and Tony Spilotro. I think Ernie Davino and Frank Cullotta both wanted to be the alpha male of the gang but Frank's connection to Tony Spilotro gave him the upper hand. No matter who started this gang, it is undisputed that by the end, this was Frank Cullotta and Tony Spilotro's gang.
Another burglar Frank recruited was Wayne Matecki. He was a good candidate because he never moved out to Las Vegas and was unknown to local cops plus he didn't have a criminal record. Anytime Frank lined up a score, Mateki would fly into town and leave immediately after from an airport in California or Arizona. Frank tested Mateki early on by having him hide inside a Las Vegas store after closing. Frank directed him to hide inside when the store closed. Then Mateki was to gather up the fur coats and other expensive items and pile them close to the front door. Frank would drive up to the front and Mateki broke out and they drove off in a few minutes after the alarm sounded. Once Frank saw that Mateki had the heart to do a job, he was an integral part of the team.
Ernie Davino's friend, Leo Guardino was a Chicago burglar and at this point in his life I learned he was tired and trying to go legit. After a lifetime in and out of jail, he was struggling with earning a legitimate living. He was still making scores with Ernie Davino. Frank Cullotta would later describe Leo Guardino as a Chicago career criminal who was trying to go legit when he joined the Hole in the Wall gang. Probably Guardino thought Cullotta and Spilotro's scores would be big and would become his last big scores, so he could retire from the life.
Spilotro recruited the disgraced former Las Vegas Metro cop named Joe Blasko into the gang. He was valuable in counter surveillance and monitoring police scanners during jobs. Frank did ot like the idea of working with this former cop, but Tony Spiloto told Cullotta he must use Blasko on scores, so he could make some money. Davino said he did not mind working with this former cop and found him helpful on jobs.
One of the most unusual members of the hole in the wall gang was a guy named Larry Newman. He was about 6' – 4"and 250 lbs. When Frank first met him, he was serving time for a triple murder. In 1956 Larry Newman became angry at a Chicago bartender. He left and returned with a shot gun and killed the bartender, a waitress and a newsboy who stumbled into the tavern. He had served 11 years of a life sentence and he met Frank Cullotta during this prison sentence. Frank remembers they both worked n the kitchen. He knew Newman was a very dangerous man but he also knew this was the kind of guy wo was capable of anything he was asked to do, up to and including murder. Larry Newman was not the usual Outfit associate. He did not grow up with other gang members nor did he have any relatives that were known to the other gang members. He did have Frank's trust because Frank had spent several years in a prison with Newman. Larry Newman was a child of a wealthy Chicago family and had enough monthly income from a trust fund that allowed him to not have to work. He elected to become a criminal.
During these years when the Hole in the Wall Gang was doing high end commercial burglaries, Larry Newman continued his murderous ways. In an eerily similar crime, Newman became angry with another bartender. A Henry County Illinois man named Ron Scharff ran a joint in suburban Chicago and he once ejected Newman's former girlfriend. She called Larry and mentioned this incident. He became enraged and talked about going back and killing this guy. Frank Cullotta learned of this threat and told Newman to forget about it. But Newman being "Larry Newman" flew back to Chicago and murdered Ron Scharff and his waitress anyway. Exactly like the previous killing without the newsboy walked in on the murder. Ron Scharff's son, Paul Scharff wrote a book, Murder in McHenry on this incident.
To demonstrate just how dangerous Newman was, Frank set up a score on Chicago jewelry store owner named Bob Brown. He put Wayne Mateki and Larry Newman together and gave them the description and address of this Bob Brown's store. Frank thought there would be as much as $100,000 in jewelry in the store. Frank also warned them that the guy was connected to Allen Dorfman, the Outfit's man on the inside of the Teamsters Union and they had to keep quiet about the job. He warned them that above all else, so not tell Spilotro because Dorfman was very close to Spilotro's Chicago boss, Joey the Clown Lombardo. When Newman and Mateki returned Newman told Frank that he had killed the jeweler. Mateki related to Cullotta that they ambushed the man inside his store after closing. Before they left, Newman grabbed a machete off the wall and hacked him to death. New claimed he wanted to avoid any chance the guy could ever identify them because it might be a death sentence. Then he told Frank he hacked him up to make it look like a crazy man had done this killing. Now there is the pot calling the kettle black.
For the Bertha's job Frank's main crew were three men he knew from Chicago, Larry Newman, Wayne Mateki and Leo Guardino, the ex-cop Joe Blasko, and Ernie Davino from New Jersey. Tony Spilotro liked Blasko because he had fed him information when he was still a cop and Leo Guradino vouched for Ernie Davino. This little gang had all the bases covered, Leo Guardino and Ernie Davino specialized in burglaries. Larry Neumann and Wayne Matecki were capable of anything, including armed robbery and murder. Joe Blasko could help in counter surveillance and use his cop connections to get information like home address or license plate registrations. Tony will ask Frank to take on one more man to add some electronic expertise to their kit of tools. Sal Romano will complete the Hole in the Wall gang for the Bertha's Gifts and Furnishings score. Frank Cullotta will later claim that Ernie Davino invited Sal Romano to go on the Bertha's job because he is an expert in locks and alarms. Since Sal was a long time Chicago Outfit burglar, it is more likely that either Tony or Frank invited him to join.
Sal Romano was a gentleman burglar. On one occasion Romano was doing a burglary set up by Outfit member Paul Shiro. This job was down in Phoenix and after he made entry, a small dog started barking and raising a ruckus. Romano backed out and took his crew with him. When he returned to face Shiro, he was asked why he didn't just kill the dog and Romano quietly said, "I don't do dogs."
During these years Frank opened a pizza place and claims he was the first person to introduce Chicago deep dish style pizza in Las Vegas. Upper Crust Pizza and the adjoining My Place lounge were located at 4110 S. Maryland Parkway. This is now the Pioneer Plaza, a small strip mall. One funny story is that Cullotta found a camera and microphone the FBI had hidden in the ceiling of this place. He gave it to Oscar Goodman who made the feds produce a letter certifying they had court authorization to install this device. Most mob fans have seen the famous picture of Frank and many of his gang sitting at a sidewalk table in front of his joint. They look like Tony Soprano and his crew out in front of the meat market.
A tragic story started at this sidewalk table at the Upper Crust. This might be titled, "The night the cops killed Frankie Blue." Frank Bluestein or Frankie Blue was a young guy whose father, Charles Bluestein was an Outfit associate involved in the Culinary Workers Union in Vegas. Frankie Blue had moved to Las Vegas and was working as a maître d' at the Hacienda Hotel & Casino. Remember, this was an Outfit controlled casino that was part of the 4 casinos that Lefty Rosenthal managed for Allen Glick.
The newly formed FBI surveillance team joined with Las Vegas Metro Intelligence to conduct moving surveillance as well as audio surveillance on the Spilotro crew. On the evening of June 9, 1980, two of Kent Clifford's Intelligence Unit members, Detective David Groover and Sergeant Gene Smith were conducting another routine surveillance of the Tony Spilotro gang. On that night as they were camped outside the Upper Crust pizza parlor and the adjoining My Place bar, located at Flamingo Road and Maryland Parkway. One of the officers, Detective David Groover would later remember, "We put in a lot of long tedious hours watching those guys. But in that kind of work things could change very quickly, and that night they did."
Frank remembers that former Chicago resident Frankie Bluestein came into the Upper Crust driving a late model Lincoln with Illinois tags. He knew Frankie Blue and that he was about ½ a wise guy but not a real criminal. Frank, Tony and some others were sitting in front and the guy went inside and ordered a pizza. He came back out to chat with them while he waited. Frank warned him that he would attract the attention of the Las Vegas Metro Intelligence driving a Lincoln with Illinois tags. The guy laughed and said that might be good because somebody was following him, and he thought they were trying to set him up for a robbery. The guy also volunteered that he wasn't afraid because he had a gun. The guy got his pizza and left.
Las Vegas Metro Intelligence Sergeant Gene Smith remembers he and his partner Detective David Groover saw an unknown guy talking with Tony Spilotro and Frank Cullotta and other members of the gang. They watched as he drove away in a 1979 Lincoln town Car with Illinois tags. They wanted to identify him, so they followed a short distance when the guy started driving at a high rate of speed and swerving in and out of traffic. The officers threw a Kojak light on the roof of their car and pulled up behind the Lincoln. Suddenly, the Lincoln pulled over and the driver jumped out. Both officers exited their car drawing their weapons and Sergeant Smith saw a gun in the driver's hand and fired immediately.
The reaction of Spilotro was tremendous. Tony Spliotro asked Frank Cullotta to hire a couple of black guys to come to town and kill some cops. He wanted this to look like a race problem in order to extract revenge against Las Vegas Metro Intelligence but not bring heat down on the Outfit.
Intelligence Unit Commander Kent Clifford learned that Spilotro had put a hit out on his two cops. During this time, somebody fired a shot gun into the side of one of the cop's homes and another person fired a shot gun into Spilotro's home. Commander Kent Clifford took drastic action to stop the escalation of hostilities. He flew to Chicago, went to the homes of several Outfit bosses and found only the maids or a wife. He went to Teamster consultant and Outfit associate Allen Dorfman's office and told him the addresses he had visited. He then told Dorfman that if anything happened to his officers, he would return with several men and kill everybody he found around those houses. Commander Clifford claims he received a phone call later that afternoon advising him that his men were safe in the future.
I would imagine that Aiuppa and the rest of the bosses were wondering what the hell Tony Spilotro was doing out in Las Vegas. Drawing attention like that can endanger the skim. As far as Spilotro's becoming a persona non-grata with the Outfit soon after this, some folks like to believe that because of a supposed affair with Lefty's wife, Jeri, but this is not the kind of stuff that will get you killed. Bringing unnecessary heat, informing and stealing money from the bosses are what gets the mob guy killed.
In the third and final installment I will describe the night of the robbery. The Hole in the Wall Gang has been doing good scores. They feel safe at their joint, the Upper Crust pizza and My Place lounge. They are getting treated like rock stars and get comped at the Stardust. They have connections all over Vegas. They are the only mob in Vegas. What can go wrong?
Don't forget to listen to Aaron on the Big Dumb Fun Show.
To go to the our store click here
To subscribe on iTunes click here, give me a review and I will send you a link to see the film for free.
1 thought on "Hole in the Wall Gang Part 2"
Mike bailey
I was there that night they got busted I used to run small errand for Frank cullotta over the Golden Steer and the Leaning Tower Frank used to own a place called Coney Island arcade and Boston City Pizza was right next to it he had hands in that also later in life I went to prison I unfortunately had the opportunity to be cell mates Ernie d'avino dick sucking Punk I know a whole lot more it's a lot of the story that's missing


Jordyn Wieland reached first on an error by 1b.
Becca Oldham walked; Jordyn Wieland stole second, advanced to third.
Kate Jolliff Rosado doubled, RBI; Becca Oldham advanced to third; Jordyn Wieland scored, unearned.
Goett singled, RBI; Hughes scored.
Frerichs singled, RBI; Goett scored.
Emi Hug singled, RBI; Amanda Atteberry scored.
Kate Jolliff Rosado doubled, RBI; Mackenzie Jackson scored.
Burnell homered, 2 RBI; Hughes scored.
Mata pinch hit for Daniels.
Whelen pinch ran for Mata.
Wood to 3b for Howard.
Kendall Farr hit by pitch.
Becca Oldham reached first on an error by 3b; Josie Koontz advanced to third.
Kate Jolliff Rosado flied out to cf, RBI; Josie Koontz scored, unearned.
Kendall Farr Beth Mersinger placed on second.
Kendall Farr singled, RBI; Beth Mersinger scored, unearned.
Elizabeth Hawkins singled; Amanda Atteberry advanced to third.
Emi Hug reached on a fielder's choice, RBI; Elizabeth Hawkins out at second rf to ss; Amanda Atteberry scored.
Maegan Stone to p for Kendall Farr.
Fransen Frerichs placed on second.
Fransen singled, RBI; Frerichs scored, unearned.
Daniels doubled; Fransen advanced to third.


Join my Facebook class on the 10th of October to learn about new essential oils and products launched by doTERRA. The doTERRA convention is always the most exciting time of the year because our company announces new products, research, events, initiatives, and a ton of additional updates. My goal with this experience is to share all of this excitement with you just in case you were unable to make it to convention this year!
Join my Facebook class on the 17th of October to learn about how to age gracefully with essential oils.
During the class I will share a wealth of information on the plentiful benefits essential oils offer, their multitude of uses, various incredible skin care routines, immune-boosting blends, and why dōTERRA® has such a great reputation.


Edmund Bradden provides an introduction to Chris Knight's theory of human origins. Reposted from the 'Radical Anthropology Group' website.
At various times, homosexuality has been considered the noblest of loves, a horrible sin, a psychological condition or grounds for torture and execution. David F. Greenberg argues that homosexuality is only deviant because society has constructed, or defined, it as deviant. The book examines a vast terrain of examples and details in the history of homosexuality.


"Do you want to screw or is returning what I think might be a show of interest going to turn into a sexual harassment lawsuit?"
In the alternate timeline where Sombra was not stopped, a human finds himself suddenly in Sombra's body... just after the dread tyrant has conquered Equestria.
To save Ponyville, Flurry Heart may have to destroy it.
Princess Celestia summons Princess Twilight Sparkle for tea.
Two friends celebrate opening day of Fluttershy's new business.
In Equestria, there are many roleplaying games. Some of them are even fun.
Princess Celestia struggles to be the princess that Equestria believes her to be.
- to help: disable notifications, only check at designated times, etc..
- dont use a screen to satisfy your hobbies, just go do them without one.
"There are no selfless actions because you derive a pleasurable sense of righteousness from doing good things, so you're really doing it for your own benefit instead of theirs."
Thanks so much for the fav on Herding Instincts and remember that comments are greatly appreciated!