text:
\section{COLD: computing power cost-aware online and lightweight deep pre-ranking system}
In this section, we will introduce our newly designed pre-ranking system COLD in detail. The core idea behind COLD is to take into consideration of both model design and system design. Computing power cost in COLD is also a variable that can be optimized jointly with model performance. In other words, COLD is a flexible pre-ranking system and the trade-off between model performance and computing power cost is controllable.

\begin{comment}
\begin{figure}[h]
    \begin{center}
      \includegraphics[width=0.4\textwidth] {image/constrain.pdf} \\
      \caption{The feature group selection instruction. The model version which is under the online RT and QPS constrains with the highest GAUC is selected as the final COLD model. }
      \label{cold:constrain}
    \end{center}
  \end{figure}
\end{comment}

\begin{figure*}[pt]
    \begin{center}
\centering
      \includegraphics[width=0.8\textwidth] {./image/row-column-fc.pdf}
      \caption{Row V.S. Column based computations. Column based method is cache-friendly and enables further acceleration.}
          \label{fc}
    \end{center}
  \end{figure*}

\subsection{Deep Pre-Ranking Model in COLD}

Unlike the vector-product based DNN model, which reduces computing power cost by restricting model architecture and thus causes loss of model performance, COLD allows applying arbitrary complex architecture of deep models to ensure the best model performance. In other words, SOTA deep ranking models can be used in COLD. For example, in our real system, we take \textbf{GwEN} (group-wise embedding network, referred as baseModel in \cite{zhou2018deep}) as our initial model architecture, which is an early version of the online model in our ranking system. Figure \ref{cmp} illustrates GwEN, which is a fully connected layer with the concatenation of feature group-wise embedding as inputs.  Note that cross features are also included in GwEN network.

Of course, the computing power cost of online inference by applying deep rank models with complex architecture directly is unacceptable, with a larger size of the candidate set to be ranked in the pre-ranking system. To tackle the critical challenge, we apply two ways of optimization strategy: one way is to design a flexible network architecture that can make a trade-off between model performance and computing power cost, the other way is to explicitly reduce computing power cost by applying engineered optimization tricks for inference acceleration.

\subsection{Design of Flexible Network Architecture}
Generally speaking, we need to introduce suitable designs of network architecture to get a lightweight version of the deep model from the full version of the initial GwEN model. Techniques such as network pruning, feature selection, and neural architecture search, etc. can be applied in this task. In our real practice, we choose the feature selection approach which is convenient for a controllable trade-off between model performance and computing power cost. Other techniques are also applicable, which we leave readers for further trying
\cite{Huang:2019h,Liu:2020t}.

Specifically, we apply the SE (Squeeze-and-Excitation) block \cite{hu2018squeeze} for feature selection. SE block is firstly used in CV (Computer Vision) to explicitly model the inner-dependencies between channels. Here we use SE block to get the importance weights of group-wise features and select the most suitable ones in COLD, by measuring both model performance and computing power cost.

\subsubsection*{\textbf{Importance weight calculation}}
 Let $e_i$ denote the embedding of $i_{th}$ input feature group. The total number of the feature groups is $M$.
The SE Block squeezes the input $e_i$ into a scalar value of weight $s_i$, which is calculated as:
\begin{equation}
s = \sigma(\bm{W}[e_1,...,e_m]+b),
\end{equation}
Where $\mathbf{s} \in \mathbb{R}^{M}$ is a vector,  $\mathbf{W} \in \mathbb{R}^{k \times M}, b \in \mathbb{R}^{M}$. $\mathbf{W}$ and $b$ are learn-able parameters.
Then the new weighted embedding $v_i$ is calculated by field-wise multiplication between the embedding $e_i$ and the importance weight $s_i$.

\subsubsection*{\textbf{Feature group selection}}
The weight vector $\textbf{s}$ represents the importance of each feature group. We use the weight to rank all feature groups and select K groups of features with top weights. Then an offline test is conducted to evaluate the model performance and system performance of the candidate lightweight version of the model with selected  K groups of features. Metrics include  GAUC\cite{zhou2018deep}, QPS (Queries Per Seconds, which measures the throughput of the model), and RT (return time, which measures the latency of model). With several heuristic tries of number K, i.e., groups of features, we finally choose the version with the best GAUC  under a given constraint of system performance as our final model.
In this way, the trade-off between model performance and computing power costs can be conducted in a flexible way.

\subsection{Engineered Optimization Tricks}
Apart from reducing the computing power cost by flexible network architecture design, which is hard to avoid the hurt of model performance to some degree, we also apply various optimization tricks from an engineering view,  to further bring space for COLD to apply more complex deep models to reach better performance.

Here we introduce the hands-on experience in the case of our display advertising system in Alibaba. Situations may vary from system to system. Readers can make choices according to the actual situation. In our display advertising system, the online inference engine of the pre-ranking module mainly contains two parts: feature computation and dense network computation. In feature computation, the engine pulls user and ad features from the indexing system and then computes cross-features. In dense network computation, the engine first turns features into embedding vectors and concatenate them as the input of the network.

\subsubsection*{\textbf{Parallelism at All Level}}
To achieve low latency and high throughput inference with low computing power cost, leveraging parallel computing is important. Our system, therefore, applies parallelism whenever is possible. Fortunately, the pre-rank score of different ads is independent of each other. This means they can be computed in parallel with the cost that there may be some duplicated computation related to user features.

At a high level, one front-end user query will be split into several inference queries. Each query handles parts of the ads and the results will be merged after all the queries return. Therefore there are trade-offs when deciding how many queries to split. More queries mean few ads for each query and therefore lower latency for an individual query. But too many queries can also lead to huge duplicated computation and system overhead. Also, as the queries are implemented using RPC (Remote Procedure Call) in our system, more queries mean more network traffic and could have a higher chance of delay or failure.

When handling each query, multi-thread processing is used for feature computation. Again, each thread handles parts of ads to reduce the latency. Finally, when executing dense network inference, we use GPU to accelerate the computation.

\begin{comment}
\begin{figure*}[h]
\centering
\begin{minipage}[h]{0.4\textwidth}
\centering
      \includegraphics[width=0.9\textwidth] {image/row.png}
      \caption{\label{fig:row}Row based Feature Computation. }
\end{minipage}

\begin{minipage}[h]{0.4\textwidth}
\centering
      \includegraphics[width=0.9\textwidth] {image/column.png}
      \caption{\label{fig:column}Column based Feature Computation. }
\end{minipage}
\end{figure*}
\end{comment}

\subsubsection*{\textbf{Column based Computation}}
Traditionally, feature computation is done in a row based manner: ads are being processed one by one. However, such a row based method is not cache-friendly. Instead, we use a column-based method to put computations of one feature column together. Figure \ref{fc} illustrates the two kinds of computation modes. By doing so, we can use techniques like SIMD (Single Instruction Multiple Data) to accelerate feature computation.

\subsubsection*{\textbf{Low precision GPU calculation}}
For COLD model, most of the computation is the dense matrix multiplication, which leaves optimization space. In NVIDIA's Turing Architecture, the T4 GPU provides extreme performance for Float16 and Int8 matrix multiplication which perfectly fits our case. The theoretical peak FLOPS for Float16 can be 8 times higher than Float32. However, the Float16 loses some precision. In practice, we found that for some scenario, as we use sum-pooling for some feature groups, the input of the dense network could be a very large number and exceed Float16 representation. To solve this, one solution is to use normalization layers like the batch-norm layer. However, the BN layer itself contains moving-variance parameters whose scale could be even larger. This means the computation graph needs to be mix-precision \cite{Micikevicius:2017vd} that fully-connected layers use Float16 and batch-norm layers use Float32. Another approach is to use a parameter-free normalization layer. For example, the logarithmic function can easily transform large numbers into a reasonable range. However, log() function can not handle negative values and can result in a huge number when input is near zero. Therefore, we design a piece-wised smooth function called linear-log operator to handle that unwanted behavior, as shown in Eq. \ref{eq:log}

\begin{equation}
    linear\_log(x) = \left\{
        \begin{array}{rcl}
        &-log(-x)-1   & {x< -1} \\
        &x   &  {-1\leq x\leq 1}.\\
        &log(x)+1   & {x>1}
        \end{array}
    \right.
\label{eq:log}
\end{equation}

The graphics of the linear\_log() function can be seen in Figure \ref{fig:log}. It transforms Float32 numbers into a reasonable range. So if we put a linear\_log operator in the first layer, it guarantees the input of the network would be small. Also, the linear\_log() function is $C^1$ continuous, so that it won't make network training harder. In practice, we found that after adding this layer, the network can still reach the same accuracy comparing to the original COLD model.

 \begin{figure}
     \begin{center}
       \includegraphics[width=0.45\textwidth] {image/log.pdf} \\
       \caption{\label{fig:log}The linear\_log function}
     \end{center}
\end{figure}

After using Float16 for inference, we found that the running time of CUDA kernel drops dramatically, and the kernel launching time becomes the bottleneck. To boost actual Querys Per Seconds (QPS), we further use MPS (Multi-Process Service) to reduce overhead when launching the kernels. Combining Float16 and MPS, the engine throughput is twice as before.

\begin{figure}[th]
    \begin{center}
      \includegraphics[width=0.45\textwidth] {image/COLD-infraustructure.pdf} \\
      \caption{Infrastructure of fully online infrastructure of COLD pre-ranking system.}
      \label{intro:infra-cold}
    \end{center}
  \end{figure}

\subsection{Fully Online Infrastructure of COLD Pre-Ranking System}
Benefiting from the unrestricted model architecture, COLD can be implemented under a fully online infrastructure: both training and serving are executed in an online manner, as illustrated in Figure \ref{intro:infra-cold}. From the industry view, it's the best system practice currently that the ranking system owns. This infrastructure benefits COLD in two folds:
\begin{itemize}
    \item Online learning of COLD model brings its excellent ability to handle the challenge of data distribution shift. To our experience, as shown in the next experimental section, the improvement of model performance that COLD model over vector-product based DNN model is more significant when the data changes dramatically (e.g., Double 11 event in China). Besides, COLD model is more friendly to the new ads with online learning.
    \item The fully online pre-ranking system of COLD provides us with a flexible infrastructure that supports efficient new model developing and online A/B testing. Remember that for vector-product based DNN model, vectors of the user and ad side need to be pre-calculated offline and load to inference engine by index. Thus it involves development over several systems to conduct A/B testing of two versions of vector-product based DNN model. To our experience, the typical time cost to get a solid A/B testing result is several days, which in turn is several hours for COLD. Besides, fully online serving also helps COLD to avoid delayed switch that the vector-product based DNN model suffers.
\end{itemize}

\section{Conclusion}

In this paper, we introduce our new generation of pre-ranking system COLD in detail . It is designed from a brand-new perspective. Instead of saving computing power with hard restriction of model architecture which causes loss of model performance, COLD takes into consideration both model design and system design. Computing power cost in COLD is also a variable that can be optimized jointly with model performance. With the co-design of the model architecture and computing power cost, COLD turns to be a flexible pre-ranking system that the trade-off between model performance and computing power cost is controllable. This new pre-ranking system enables a better pursuit of model performance. Experiments show COLD model achieves more than 6\%\ RPM lift over vector-product based DNN model, our latest online version of the pre-ranking model, which is significant to the business. Besides, COLD can be implemented with a fully online infrastructure for both training and serving, achieving the best system practice that the current ranking system owns. Since 2019, COLD has been deployed in the display advertising system in Alibaba and serves the main traffic of almost all products, contributing a significant business revenue growth.

\section{Experiments}

We conducted experiments on COLD before it was deployed into our display advertising system. Specifically, we need to evaluate the offline performance, comparing COLD model with the previous ad pre-ranking ctr model and the current ranking ctr model. Then we need to conduct the online A/B test to evaluate the online performance.
\subsection{Competitors and Experiment Setup}
We compare COLD with the following baselines:
\begin{itemize}
        \item {\textbf{Vector-Product based DNN}} is the previous pre-ranking model in our display advertising system. The scores is calculated with inner products form between user embedding and ad embedding.

        \item {\textbf{DIEN}} \cite{zhou2019dien}  is our previous version of ranking model. It integrates GRU with attention for capturing the evolution trend of user interests.
\end{itemize}

\textbf{Experiment Settings.}
COLD is trained  with more than 90 billion samples. Layers of FCN (Fully Connected Network) are set by $1024 \times 512 \times 256 \times 128 \times 64 \times 2$. The number of embedding dimension is set to be 16. we use Adam solver to update the model parameters. The GAUC \cite{zhou2018deep} is used as the metric to evaluate the offline performance of the models. The top k recall is also used to indicate the alignment degree between the pre-ranking model and the ranking model.
\begin{equation}
  recall = \frac{|\{\text{top k ad canidates}\} \cap \{ \text{top m ad canidates}\}|}{|\{\text{top m ad canidates}\}\}|}.
\end{equation}
The top k ad candidates are ranked by the pre-ranking model and the top m ad candidates are ranked by ranking model. The ranking metric is cost per mile (CPM=CTR $\times$ Bid).

\subsection{Offline and Online Evaluation}

We conduct offline evaluation on the feed scenes in Taobao. As shown in Table \ref{exp:offline},  COLD  outperforms the vector-based model both in GAUC and recall. It also maintains a comparable result with DIEN.

The online A/B testing is also performed. As shown in Table \ref{exp:online}, the CTR of COLD increases 6.1\%\ and 9.1\%\ compared to the vector-product based model, which means that COLD can provide more accurate ads for users. The RPM also increases 6.5\%\ and 10.8\%\ which indicates the COLD method can bring more revenue for the Taobao display advertising platform. The results of COLD in double 11 event is better than normal, which proves that COLD can adapts to the newest data distribution when the data changes dramatically.

\begin{table}[t]
  \caption{Offline Evaluation Results}
\label{exp:offline}
  \begin{tabular}{lcl}
    \toprule
     Method & GAUC & Recall\\
    \midrule
 Vector-based Model & 0.6107 & 88\% \\
 COLD & 0.6349 & 96\%\\
DIEN & 0.6476 & 100\%\\
  \bottomrule
\end{tabular}
\end{table}

\begin{table}
  \caption{Online Evaluation Results.}
\label{exp:online}
  \begin{tabular}{lcl}
    \toprule
 Method & CTR & RPM \\
    \midrule
 Normal Days & +6.1\%\ & +6.5\%\ \\
 Double 11 Event& +9.1\%\ & +10.8\%\ \\
  \bottomrule
\end{tabular}
\end{table}

\subsection{Ablation Study}

We conduct experiments to evaluate the performance and computing power cost between different models and feature groups. The feature group candidates are selected by ranking results based on the importance weight of SE block.
We also conduct experiments to demonstrate the benefits of different optimization techniques.

We first compare the system performance for different models, especially the vector-product based model, complicated ranking model, and COLD. Vector-product based model runs on a CPU machine with 2 Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz (96 cores) and 512GB RAM. Ranking model and COLD run on a GPU machine with NVIDIA T4 equipped. As shown in Table \ref{tab:qps}, the vector-based model has the lowest latency and highest throughput, while the ranking model is the opposite. Our COLD finds a nice balance between the two.

\begin{table}
  \caption{The System Performance for Different Models}
\label{tab:qps}
  \begin{tabular}{lccc}
    \toprule
 Method & QPS & Latency & GAUC  \\
    \midrule
 Vector-based Model  & 100000 & 1ms & 0.6107 \\
 COLD with Less Features & 13000 & 6ms & 0.6250 \\
 COLD & 12000 & 7ms & 0.6349\\
 COLD with More Features & 10000 & 9ms & 0.6320 \\
DIEN & 1100 & 15ms & 0.6476\\
  \bottomrule
\end{tabular}
\end{table}

We then compare the row-based feature computation with the column-based method. We run our engine on a machine with 2 Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz (96 cores) and 512GB RAM. For each query, we set the batch size as 300. Table \ref{tab:rt_column} shows the running time for different stages of feature computation. As we can see, the column-based method accelerates the ad features computation by a factor of six. Table \ref{tab:qps_column} shows
the throughput of the two methods. Column based method has only one-third of CPU load under the same QPS or obtains 3X throughput with similar CPU load.

\begin{table}
  \caption{The Run Time Performance}
\label{tab:rt_column}
  \begin{tabular}{cccc}
    \toprule
 Method & \makecell*[c]{User/Scene \\Features} & Ad Features & Post-Processing \\
    \midrule
 Row based & 0.49ms & 4.90ms & 0.75ms \\
 Column based & 0.40ms & 0.83ms & 0.49ms\\
  \bottomrule
\end{tabular}
\end{table}

\begin{table}
  \caption{The QPS and CPU load Performance}
\label{tab:qps_column}
  \begin{tabular}{ccc}
    \toprule
 Method & QPS & CPU load \\
    \midrule
 Row based & 3000 & 29\% \\
 Column based & 3000 & 10\%\\
 Column based & 9000 & 33\%\\
  \bottomrule
\end{tabular}
\end{table}

Finally, we compare the performance using different CUDA features. The experiments run on a GPU machine with NVIDIA T4. When running the experiments, we gradually increase the QPS from the client-side, until the more than 1\% of the server responding time begins to exceed the latency limit. We then record the current QPS as usable QPS. As shown in Table \ref{tab:qps_cuda}, the float32 version has the lowest usable QPS. Using float16 alone can decrease GPU load, but could not improve the usable QPS. Which means matrix computation is no longer the bottleneck. Combining float16 and CUDA MPS, we can double the usable QPS comparing to float32, and can fully utilize the GPU without exceeding the latency limit.

\begin{table}
  \caption{The System Performance for Different CUDA setting}
\label{tab:qps_cuda}
  \begin{tabular}{ccc}
    \toprule
 method & QPS & GPU load \\
    \midrule
 Float32 & 4150 & 90\% \\
 Float16 & 4500 & 67\%\\
Float16+MPS & 9400 & 100\%\\
  \bottomrule
\end{tabular}
\end{table}
\section{Experiments}

We conduct careful comparisons to evaluate the performance of the proposed pre-ranking system COLD. As an industrial system, comparisons are conducted on both model performance and system performance. To the best of our knowledge, there are merely public datasets or pre-ranking systems for this task. The following experiments are executed in the online display advertising system in Alibaba.

\subsection{Experimental Settings}
The strongest baseline of COLD model is the SOTA vector-product based DNN model, which is the latest version of the online pre-ranking model in our display advertising system.

Both COLD model and vector-product based DNN model are trained with more than 90 billion samples, which are collected from logs of the real system. Note that the vector-product based DNN model shares the same user and ad features with COLD model. The vector-product based DNN model could not introduce any user-ad cross features, while COLD model uses user-ad cross features. For a fair comparison, we also evaluate the performance of COLD model with different groups of cross features.
For COLD model, the feature embedding vectors are concatenated together and then fed into a fully connected network~(FCN). The structure of this FCN is $D_{in} \times 1024 \times 512 \times 256 \times 128 \times 64 \times 2$, where $D_{in}$ means the dimension of the concatenated embedding vectors of selected features. For the vector-product based model, the FC layers are set by $200 \times 200 \times 10$. The dimension of input feature embedding is set to be 16 for both two kinds of models. We use Adam solver to update the model parameters. GAUC \cite{zhou2018deep} is used as the metric to evaluate the offline performance of the models. Besides, we introduce a new metric of top-k recall, to measure the alignment degree between the pre-ranking model and subsequent ranking model. The top-k recall rate is defined as:
\begin{equation}
  recall = \frac{|\{\text{top k ad candidates}\} \cap \{ \text{top m ad candidates}\}|}{|\{\text{top m ad candidates}\}\}|},
\end{equation}
where the $\text{top k candidates}$ and the $\text{top m candidates}$ are generated from the same candidate set, which is the input of the pre-ranking module. The top k ad candidates are ranked by the pre-ranking model and the top m ad candidates are ranked by ranking model. The ranking metric is eCPM (expected Cost Per Mille, eCPM = pCTR * bid).
In our experiments, the ranking model uses DIEN \cite{zhou2019dien}, a previous version of the online ranking system.

For evaluation of system performance, we use metrics including QPS (Queries Per Seconds, which measures the throughput of the model), RT (return time, which measures the latency of model). These metrics reflect the computing power cost of the model under the same size of the candidate set to be pre-ranked. Roughly speaking, larger QPS under lower RT means lower computing power cost for a given model.

\subsection{Evaluation on Model Performance}
Table \ref{exp:offline} shows the offline evaluation results of different models.
We can see that COLD maintain a comparable GAUC with our previous version ranking model DIEN, and achieves significant improvement both in GAUC and Recall compared with the vector-product based model.

\begin{table}[t]
  \caption{Offline evaluation results}
\label{exp:offline}
  \begin{tabular}{lcl}
    \toprule
     Method & GAUC & Recall\\
    \midrule
        Vector-Product based DNN Model & 0.6232 & 88\% \\
        COLD  & 0.6391 & 96\%\\
        DIEN  & 0.6511 & 100\%\\
  \bottomrule
\end{tabular}
\end{table}

We also conduct careful online A/B testing. Table \ref{exp:online} shows the lift of COLD model over the vector-product based DNN model. In normal days, COLD model achieves 6.1\%\ CTR and 6.5\%\ RPM (Revenue Per Mille) improvement, which is significant to our business. Moreover, the improvement turns to be 9.1\%\ CTR and 10.8\%\ RPM during the double 11 event. It proves the value of the fully online infrastructure of COLD which can enable the model to be adapted to the newest data distribution when the data changes dramatically.

\begin{table}
  \caption{Results of online A/B testing by comparing COLD model with vector-product based DNN model.}
\label{exp:online}
  \begin{tabular}{lcl}
    \toprule
     Time & CTR lift & RPM lift \\
    \midrule
     Normal Days & +6.1\%\ & +6.5\%\ \\
     Double 11 Event& +9.1\%\ & +10.8\%\ \\
  \bottomrule
\end{tabular}
\end{table}

\subsection{Evaluation on System Performance}

\begin{table}
  \caption{Comparison of system performance of pre-ranking system that servers with different models}
\label{tab:sys}
  \begin{tabular}{lcc}
    \toprule
       Model & QPS & RT  \\
    \midrule
       Vector-Product based DNN Model  & 60000+ & 2ms \\
       COLD  & 6700 & 9.3ms  \\
       DIEN & 629 & 16.9ms \\
  \bottomrule
\end{tabular}
\end{table}

We evaluate the QPS and RT of the pre-ranking system that serves with different models. Table \ref{tab:sys} gives the results. Vector-product based model runs on a CPU machine with 2 Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz (96 cores) and 512GB RAM. COLD models and DIEN run on a GPU machine with NVIDIA T4 equipped. In this time, the vector-product based DNN model achieves the best system performance, which is as expected. The computing power of DIEN costs the most. COLD achieves the balance.

\subsection{Ablation Study of COLD}
To further understand the performance of COLD, we conduct experiments on both model design view and engineered optimization techniques view.
For the latter one, as it is hard to decouple all the optimization techniques from the integrated system and compare each of them, here we conduct an evaluation on the most significant factor of low precision computation of GPU.

\begin{table}
\begin{threeparttable}
  \caption{Trade-off performance of pre-ranking system with different versions of COLD model}
\label{tab:tradeoff}
  \begin{tabular}{lccc}
    \toprule
       Model & QPS & RT & GAUC \\
    \midrule
       COLD (No Cross Features)& 6860 & 8.6ms & 0.6281 \\

       \textbf{COLD} \tnote{*} & 6700 & 9.3ms & 0.6391 \\
       COLD (All Features)& 2570 & 10.6ms & 0.6467\\
  \bottomrule
\end{tabular}
\begin{tablenotes}
\item[*] This is the balanced version of COLD model that we use as product version. It users partial of cross-features.
\end{tablenotes}
\end{threeparttable}
\end{table}

\textbf{Trade-off performance of the pre-ranking system with different versions of COLD model}.
In the model design stage, we use the SE block to get the feature important weights and select different groups of features as the candidate version of models. Then we conduct the offline experiments to evaluate the models with QPS, RT, and GAUC.  Table \ref{tab:tradeoff} shows results. Obviously, computing power cost of COLD model varies with different features. This fits our design of flexible network architecture. COLD model with more cross features achieves better performance, which also increases the burdens for online serving correspondingly. In this way, we can make a trade-off of model performance and computing power cost. In our real system, we choose the balanced version manually by experience.

\begin{comment}
\textbf{Comparison between row-based and column-based feature computation}.  We run our inference engine on a machine with 2 Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz (96 cores) and 512GB RAM. For each query, we set the batch size as 300. Table \ref{tab:rt_column} shows the running time for different stages of feature computation. As we can see, the column-based method accelerates the ad features computation by a factor of six.
\begin{table}
  \caption{The Runtime Performance}
\label{tab:rt_column}
  \begin{tabular}{cccc}
    \toprule
       Method & \makecell*[c]{User/Scene \\Features} & Ad Features & Post-Processing \\
    \midrule
       Row-based & 0.49ms & 4.90ms & 0.75ms \\
       Column-based & 0.40ms & 0.83ms & 0.49ms\\
  \bottomrule
\end{tabular}
\end{table}
\end{comment}

\textbf{Comparison of computing with different GPU precisions.} The experiments run on a GPU machine with NVIDIA T4. When running the experiments, we gradually increase the QPS from the client-side, until the more than 1\% of the server responding time begins to exceed the latency limit. We then record the current QPS as usable QPS. As shown in Table \ref{tab:qps_cuda}, the Float32 version has the lowest usable QPS. Using Float16 alone can improve about 21\% of the usable QPS. Combining Float16 and CUDA MPS, we can double the usable QPS comparing to Float32, and can fully utilize the GPU without exceeding the latency limit.

\begin{table}
  \caption{Comparison of system performance by computing with different GPU precisions}
\label{tab:qps_cuda}
  \begin{tabular}{cc}
    \toprule
      Precision & QPS  \\
    \midrule
      COLD (Float32) & 2800  \\
      COLD (Float16) & 3400 \\
      COLD (Float16+MPS) & 6700 \\
  \bottomrule
\end{tabular}
\end{table}

\section{Introduction}

\begin{figure*}[pt]
    \begin{center}
\centering
      \includegraphics[width=0.9\textwidth] {./image/4g.pdf}
      \caption{The development history of the pre-ranking system from model view. $x_u,x_a,x_{ua}$ are the raw features of user, ad and cross. $e_u,e_a,e_{ua}$ are the embeddings of user, ad, and cross features. The first generation is the non-personalized ad-wise statistical model.  LR (Logistic Regression) model is the second generation which is a lightweight version of the large scale ranking model in the age of shallow machine learning \cite{cascaderanking}. It can be deployed in online learning and serving manner. Vector-product based deep learning architecture is the third generation and current state-of-the-art pre-ranking model. It significantly boosts the model performance over the previous generation. COLD is our proposed new generation of the pre-ranking model. }
          \label{cmp}
    \end{center}
  \end{figure*}

Users have been struggling with information overload due to the rapid growth of internet services these years. Search engine, recommender systems, and online advertising have become foundational information retrieval applications which server billions of users every day. Most of these systems \cite{youtube:recommend,cascaderanking,zhou2018deep,Wu:2018uo,zhou2019dien,Chen:2019tc,pi2019deep,Fan:2019ge} follow a multi-stage cascade architecture, that is, candidates are extracted by sequential modules such as matching, pre-ranking, ranking, and reranking, etc. Figure \ref{intro:cascade} gives an brief illustration.

There have been numerous papers discussing how to build an effective and efficient ranking system \cite{youtube:recommend,gai2017learning,Grbovic:2018bw,zhou2018deep,zhou2019dien,Wu:2018uo,pi2019deep,Chen:2019tc,Fan:2019ge,Lyu:2020ue}. However, very few works pay attention to the pre-ranking system \cite{cascaderanking,Wu:2018uo}. For simplicity, in the rest of the paper, we limit ourselves to discuss the design of the pre-ranking system in the display advertising system. Techniques discussed here can be easily applied in recommender systems, search engines, etc.

For a long time, it is believed that pre-ranking is just a simplified version of the ranking system, considering the computing power cost challenge of online serving with a larger size of the candidate set to be ranked. Take the display advertising system in Alibaba as an example. Traditionally, the size of the candidate set to be scored for the pre-ranking system scales up to tens of thousands, which turns to be hundreds in the subsequent ranking system. On the other side, both ranking and pre-ranking systems have strict latency limit, e.g., $10 \sim 20$ milliseconds. In this situation, the pre-ranking system is often designed as a lightweight ranking system by simplifying the ranking model to handle the explosion of computing power for online inference.

\subsection{Brief Introduction of the Development History of the Pre-Ranking System}
Looking back at the development history of the pre-ranking system in industry, we can simply categorize it into four generations from the model view, as shown in figure \ref{cmp}.
The first generation is the non-personalized ad-wise statistical score. It calculates the pre-rank score by averaging out the recent CTR (Click-Through Rate) of each ad. The score can be updated with high frequency.
LR (Logistic Regression) model is the second generation which is a lightweight version of the large scale ranking model in the age of shallow machine learning. It can be deployed in the online learning and serving manner \cite{McMahan:2013cq}.
Vector-product based deep learning model \cite{youtube:recommend} is the third generation and current state-of-the-art pre-ranking model. In this method, user-wise and ad-wise embedding vectors are pre-calculated separately in an offline manner with no user-ad cross features, then the inner product of the two vectors is calculated online to obtain the pre-ranking score. Although vector-product based DNN has significantly boosted the model performance of the first two generations, it still suffers from two challenges which leave space for further improvement: (i) \textbf{Model expression ability}. As shown in \cite{Zhu:2018gi}, the expression ability of the model is limited by restricting deep models with vector-product form; (ii) \textbf{Model update frequency}.
The embedding vectors of the vector-product based DNN need to be pre-calculated offline and then loaded into the memory of the server for online calculation.
It means the vector-product based DNN model can only be updated in a low-frequency manner, which make it hard to adapt to the newest data distribution shift, especially when the data changes dramatically (e.g., Double 11 event in China).

To summarize, all the above mentioned three generations of the pre-ranking system follow the same paradigm: computing power is treated as a constant constraint, under which the pre-ranking model corresponding with the training and serving system is developed. That is, the design of the model and the optimization of the computing power is decoupled, which usually leads to a simplification of the model to fit the requirement of computing power. This results in suboptimal performance.

\subsection{COLD: New Generation of Pre-Ranking System}
In this paper, we rethink the challenge of the pre-ranking system from an algorithm-system co-design view. Instead of saving computing power with restriction of model architecture which limits the performance, here we design a new pre-ranking system by jointly optimizing both the pre-rank model and the computing power it costs. We name it COLD (\textbf{C}omputing power cost-aware \textbf{O}nline and \textbf{L}ightweight \textbf{D}eep pre-ranking system), as illustrated in  Figure \ref{cmp}.
We treat COLD as the fourth generation of the pre-ranking system. COLD takes into consideration of both model design and system design. Computing power cost in COLD is also a variable that can be optimized jointly with model performance. In other words, COLD is a flexible pre-ranking system that the trade-off between model performance and computing power cost is controllable.

Key features of COLD are summarized as follows:
\begin{itemize}
    \item Arbitrary deep model with cross features can be applied in COLD under a constraint of controllable computing power cost. In our real system, COLD model is a seven-layered fully connected deep neural network with SE (  Squeeze-and-Excitation) block \cite{hu2018squeeze}. SE block benefits us to conduct feature group selection to get a lightweight version from a complex ranking model. This selection is executed by taking into consideration of both model performance and computing power cost. That is, computing power cost for COLD model is controllable.
    \item Computing power cost is explicitly reduced by applying optimization tricks such as parallel computation and semi-precision calculation for inference acceleration. This further brings space for COLD to apply more complex deep models to reach better performance.
    \item COLD model works in an online learning and severing manner, bringing system excellent ability to handle the challenge of data distribution shift.  The fully online pre-ranking system of COLD provides us with a flexible infrastructure that supports new model developing and fast online A/B testing, which is also the best system practice currently that ranking system owns.
\end{itemize}

\begin{figure}[t]
    \begin{center}
      \includegraphics[width=0.4\textwidth] {image/trend.pdf} \\
      \caption{Model expression ability v.s. update frequency of the four generations of ranking system.}
      \label{intro:trend}
    \end{center}
  \end{figure}

Figure \ref{intro:trend} gives a comparison of all the four generations of ranking system w.r.t. model expression ability and update frequency. COLD achieves the best tradeoff. Since 2019, COLD has been deployed in almost all products involving the pre-ranking module in the display advertising system in Alibaba, serving hundreds of millions of users with high concurrent requests every day. Compared with vector-product based DNN, our latest online version of the pre-ranking model, COLD brings us more than 6\% RPM improvement, which is a significant improvement for the business.

The rest of the paper is organized as follows: section 2 will give an overview of an industrial pre-ranking system, then section 3 will introduce the details of COLD, including issues of model design, optimization of computing power cost and the whole infrastructure, section 4 and 5 will give experimental comparison and conclusion.

\section{COLD Model}

To address the problems of previous pre-ranking methods and make the co-design between model structure and computing power it costs, we propose to use FC (Fully Connected) layers and cross features in COLD, as shown in Figure \ref{cmp}. More complexly model makes it more difficult to satisfy the computing power cost constraint.
Inspired by the SE (Squeeze-and-Excitation) block \cite{hu2018squeeze} which is firstly used in CV (Computer Vision) to explicitly model the inner-dependencies between channels, we use SE block to get the importance weights and select the most suitable features in COLD. Specifically,
the embedding of $i_th$ input feature group is represented as $e_i$.
The SE Block squeezes the input $e_i$ into a one-dimensional weight $s_i$, which is calculated as:
\begin{equation}
s_i = \sigma(\bm{W}e_i+\bm{b}),
\end{equation}
Where $\mathbf{W} \in \mathbb{R}^{k \times 1}, \mathbf{b} \in \mathbb{R}^{1}$.
The weight $s$ can represent the importance of each feature group. We use the weight to rank the feature group and  select several feature groups with different top K weights as the candidate feature groups and conduct the offline test to evaluate the performance and computing power cost based on GAUC\cite{DIN}, Querys Per Seconds (QPS) and RT. Finally we choose the feature groups with the best GAUC  under the system computing power and latency constrain as our final model. In this way, the design of the model and the optimization of the computing power is not separated anymore. The trade-off between them can be conducted in a more flexible way.

The parameters of the COLD model are updated with the online learning method, which makes it more friendly to the new ads and can adapt to the newest data distribution. It is worth mentioning that the COLD can support UIC\cite{pi2019practice} to model the more complex user behaviors without increasing too much computing resources and latency.

\section{Overview of pre-ranking system}

As illustrated in figure \ref{intro:cascade}, pre-ranking can be viewed as a connecting link between matching and ranking modules. It receives the result of matching and performs a rough selection to reduce the size of the candidate set for the following ranking module. Take the display advertising system in Alibaba as an example, the size M of the candidate set that is fed into the pre-ranking system often reaches ten thousand. Then the pre-ranking model selects top N candidates by certain metrics, e.g. eCPM  (expected Cost Per Mille) for advertising system \footnote{eCPM = pCTR * bid for CPC (Cost Per Click) bidding based advertising system}. The magnitude of N is usually several hundred. These winning N candidates are further ranked by a complex ranking model to get the final results to be displayed to users.

Generally speaking, pre-ranking shares similar functionality of ranking. The biggest difference lies in the scale of the problem. Obviously, the size of candidates to be ranked is 10x or larger for the pre-ranking system than the ranking system. Directly apply ranking models in the pre-ranking system seems impossible, which will face the great challenge of computing power cost. How to balance the model performance and the computing power it costs is the key consideration for designing the pre-ranking system.

\subsection{Vector-Product based DNN Model}
Driven by the success of deep learning, vector-product based DNN model \cite{youtube:recommend} has been widely used in pre-ranking systems and achieves state-of-the-art performance. As shown in Figure \ref{cmp},  architecture of vector-product based DNN model consists of two parallel sub neural networks. User features are fed to the left sub network and ad features to the right.  For each sub network, features are fed into the embedding layer first and then concatenated together, followed by FC (Fully Connected) layers. In this way, we obtain two fix-size vectors $\bm{v_u}$ and $\bm{v_a}$ which represents the user and ad information respectively. Finally, the pre-ranking score $p$ is calculated as follows:

\begin{equation}
 p = \sigma(\bm{v_u}^T \bm{v_a}), ~~~ where ~~~ \sigma(x)=\frac{1}{1+e^{-x}}.
\end{equation}
Training of the vector-product based DNN model follows the same way as the traditional ranking model. In order to focus on the key part of the pre-ranking model, we omit the details of training. Readers can refer to previous work such as
\cite{zhou2018deep,zhou2019dien} for detailed introduction.

\begin{figure}[th]
    \begin{center}
      \includegraphics[width=0.45\textwidth] {image/vector-product-dnn-infraustructure.pdf} \\
      \caption{Infrastructure of pre-ranking system with vector-product based DNN model.}
      \label{intro:infra-vector}
    \end{center}
  \end{figure}

\subsection{Shortcomings of the Pre-Ranking System with Vector-Product based DNN Model}
The vector-product based DNN model is efficient in latency and computing resources. Vectors of $\bm{v_u}$ and $\bm{v_a}$ can be pre-calculated separately in an offline manner and score $p$ can be calculated online. This makes it friendly enough for tackling the challenge of computing power costs. Figure \ref{intro:infra-vector} illustrates the classic implementation of infrastructure.  Compared with previous generations of the pre-ranking model, the vector-product based DNN model achieves significant performance improvement.

However, the vector-product based DNN model pays too much attention to reduce the computing power cost, by restricting the model to be in the vector-product form, this results in suboptimal performance. We summarize the shortcomings as follows:
\begin{itemize}
    \item The model expression ability is limited by the vector-product form, and can not utilize the user-ad cross features. Previous work \cite{Zhu:2018gi} has shown the obvious superiority of incorporating complex deep models over vector-product form networks.
    \item The user and ad vectors of $\bm{v_u}$ and $\bm{v_a}$  need to be pre-calculated offline by enumerating all users and ads, to reduce the computing resources and optimize the latency.  For businesses with hundreds of millions of users and tens of millions of ads, the pre-calculation usually costs several hours, making it hard to be adapted to the data distribution shift. This will bring great hurt of model performance when the data changes dramatically (e.g., Double 11 event in China).
    \item The model update frequency is also affected by the system implementation. For the vector-product based DNN model, the daily switch between versions of user/ad vectors indexes needs to be executed at the same time, which is hard to be met with two indexes stored in different online systems. To our experience, the delayed switch also hurts the model performance.
\end{itemize}

These shortcomings of the pre-ranking system with vector-product based DNN model originates from the excessive pursuit of computing power reduction and are hard to be tackled completely. In the following sections, we will introduce our new solution, which breaks the classic designing methodology for the pre-ranking system.

\section{The Optimization for Inference }

\begin{figure*}[h]
\centering
\begin{minipage}[h]{0.4\textwidth}
\centering
      \includegraphics[width=0.9\textwidth] {image/row.png}
      \caption{\label{fig:row}Row based Feature Computation. }
\end{minipage}
\begin{minipage}[h]{0.4\textwidth}
\centering
      \includegraphics[width=0.9\textwidth] {image/column.png}
      \caption{\label{fig:column}Column based Feature Computation. }
\end{minipage}
\end{figure*}
After the design of our COLD model, it is still a challenging task to performing the inference on tens of thousands of Ads online. The computation power cost  and latency becomes  larger than the vector-based method. So we conduct several optimization on the inference pipeline.

The real-time predicting service mainly contains two parts: feature computation and dense network computation. In feature computation, the engine pulls user and ad features from the indexing system and computes cross-features. In dense network computation, the engine will first turn features into embedding vectors and concatenate them as the input of the network. Then our optimized inference engine will perform the network forwarding pass and finally return the predicted CTR score. As latency is critical to our ad serving system, the budget for the running time of pre-ranking is very limited. The latency for computing the pre-ranking score can not be longer than 20 ms. In practice, this means we need to keep the averaging run time lower than 7ms so that the long tail queries would not affect the performance of our system.

\subsection{Parallelism at All Level}

In order to achieve low latency inference, leveraging parallel computing is important. Our system, therefore, applies parallelism whenever is possible. Fortunately, the CTR score of the ads is independent of each other. This means they can be computed in parallel with the cost that there may be some duplicated computation related to user features.

At a high level, one front-end user query will be split into several CTR score prediction queries. Each query handles parts of the ads and the results will be merged after all the queries return. Therefore there are trade-offs when deciding how many queries to split. More queries mean few ads for each query and therefore lower latency for an individual query. But too many queries can also lead to huge duplicated computation and system overhead. Also, as the queries are implemented using RPC (Remote Procedure Call) in our system, more queries mean more network traffic and could have a higher chance of delay or failure. In practice, we find that having 300\~ 400 ads for each query is most efficient for our cluster.

When handling each query, multi-thread processing is also used for feature computation. Again, each thread handles parts of ads to reduce the latency. And finally, when performance dense network inference, we use GPU to accelerate the computation as the most time-consuming operator of our COLD is dense matrix multiplication.

We have been using those parallel computing techniques in our ranking system for a long time. However, they are not enough for the latency requirement of the pre-ranking model. Some specialized optimization is required.

\subsection{Column based Computation}
Traditionally, feature computation is done in a row based manner, as shown in Fig \ref{fig:row}. The ads are being processed one by one. However, such a row based method is not cache-friendly. Instead, we use a column-based method (Fig \ref{fig:column}) to put computations of one feature column together. By doing so, we are able to use techniques like SIMD (Single Instruction Multiple Data) to accelerate feature computation.

\subsection{Utilizing CUDA Features}
Even though GPU is good at accelerating deep neural networks, it is notoriously hard to fully utilize its computation power. The default implementation in open-source training frameworks like Tensorflow could result in high latency and low throughput for online inference. Such low performance is usually due to the poor arrangement of computation graph which contains unnecessary kernel launch and memory copy. Therefore, we have implemented our own inference engine which rewrites the computation graph to obtain the best performance.

For our ranking models which contain lots of complicated units like attention and GRU, techniques like kernel fusion are already enough since the bottleneck is data movement. But for COLD, the network involves almost only dense matrix multiplication, which needs extreme optimization.

In NVIDIA's Turing Architecture, the T4 GPU provides extreme performance for Float16 and Int8 matrix multiplication which perfectly fits our case. The theoretical peak FLOPS for float16 can be 8 times higher than float32. However, the float16 loses some precision. In practice, we found that for some scenario, as we use sum-pooling for some feature groups, the input of the dense network could be a very large number and exceed float16 representation. To solve this, one solution is to use normalization layers like the batch-norm layer. However, the BN layer itself contains moving-variance parameters whose scale could be even larger. This means the computation graph needs to be mix-precision that fully-connected layers use float16 and batch-norm layers use float32. Another approach is to use a parameter-free normalization layer. For example, the logarithmic function can easily transform large numbers into a reasonable range. However, log() function can not handle negative values and can result in a huge number when input is near zero. We, therefore, designed a piece-wised smooth function called linear-log operator to handle that unwanted behavior, as shown in Eq. \ref{eq:log}

\begin{equation}
    linear\_log(x) = \left\{
        \begin{array}{rcl}
        &-log(-x)-1   & {x< -1} \\
        &x   &  {-1\leq x\leq 1}.\\
        &log(x)+1   & {x>1}
        \end{array}
    \right.
\label{eq:log}
\end{equation}

The graphics of the linear\_log() function can be seen in Fig. \ref{fig:log}. It transforms float32 numbers into a reasonable range. So if we put a linear\_log operator in the first layer, it guarantees the input of the network would be small. Also, the linear\_log() function is $C^1$ continuous, so that it won't make network training harder. In practice, we found that after adding this layer, the network can still reach the same accuracy comparing to the original COLD.

 \begin{figure}
     \begin{center}
       \includegraphics[width=0.2\textwidth] {image/log.pdf} \\
       \caption{\label{fig:log}The linear\_log function}
     \end{center}
\end{figure}

After using float16 for inference, we found that the running time of CUDA kernel drops dramatically, and the kernel launching time becomes the bottleneck. In order to boost actual Querys Per Seconds (QPS), we further use MPS (Multi-Process Service) to reduce overhead when launching the kernels. Combining float16 and MPS, the engine throughput is twice as before.

redpajama_set_name: RedPajamaArXiv


text:
Title: Contracted
Writer/ Director: Eric England
Cast: Najarra Townsend, Caroline Williams, Alice Macdonald, Katie Stegeman, Matt Mercer, Charley Koontz
Samantha (the gorgeous Najarra Townsend) is at a party sans her pretty girlfriend Nikki (Katie Stegeman). Samantha calls her up, but she doesn't pick up the phone. Her friend Alice (Alice Macdonald) gives her a couple of shots to loosen her up. She also calls out Alice's creeper, Riley (Matt Mercer), who also happens to be this party. But, the actual danger comes not from him, but from a dude named BJ (Simon Barrett) who hands her another drink. Half conscious, Sam ends up in his car where he rapes her.
The next day she wakes up with blood running down her crotch and sharp pains. Soon things go from worse to totally fucking fucked up, as she begins to fall apart, literally and figuratively. She has "contracted" what is clearly the single worst STD in the history of mankind. To make things harder on her, her mom (Caroline Williams) thinks that she is back on drugs and doesn't approve of her lesbian lifestyle, her actual girlfriend is increasingly distant, her creeper keeps showing, and her work just plain sucks. The bad things continue to pile up and what happens as the movie progress will churn your stomach and make sure you NEVER forget your condom!
Make no bones about it, this is the one of the absolute best body horror movies to come down the pike, in a very long time, as well as being one of the most disgusting ones, too. Writer/ director Eric England has crafted an excellent movie that moves at a quick clip and keeps you interested up until the crushing ending. From start to finish this grim horror all the way and all the better for it. Working both as an allegory for date rape and warning of the dangers of STDs, this one should be played at a sex ed class!
The acting is truly excellent. Najara Townsend is amazing in the lead. We feel for her, throughout the movie. She is a likeable if flawed person going through an increasingly horrible situation. After seeing this movie, I am definitely a huge fan of this talented, young actress. Also, excellent is Caroline (The Texas Chainsaw Massacre Part 2) Williams who gives what maybe the best performance of her career, thus far. Her scenes with Townsend add an emotional weight to the story that helps to add to Samantha's whole ordeal.
There is some gore, but most of the shocks come from the actual disease and how it affects Samantha's body. What happens to her is increasingly barf inducing. Even, I seasoned horror viewer that I am, was in awe of what happens to her body. I kept screaming, "holy shit!" and "That's fucked up!". No spoilers here, since it's best to not how far this goes, so all I will say is "maggots". That's all you get from me, this time! The impressive and disgusting make-up FX by Mayera Abeita are gruesomely realistic. Plus, she has the very difficult task of making the gorgeous Townsend look more and more fucked up, something she achieves with disgusting perfection.
Contracted is up there with some of Cronenberg's work, as far as quality body horror goes. It might also be the best disease horror movie ever. Off the top of my head, I can't think of a better one. Utterly disgusting, smart, intense, fast paced, and with excellent acting, I give this medical nightmarish chunkblower my highest possible recommendation.
**** out of ****
body horror, Caroline Williams, Contracted, Eric England, Gore, horror films, lesbians, movie reviews, Najarra Townsend Edit
BATMAN V SUPERMAN XXX: AN AXEL BRAUN PARODY (DVD Review)
All pics come courtesy of Wicked Pictures
Title: Batman V Superman XXX: An Axel Braun Parody
Produced and directed by: Axel Braun
Cast: Ryan Driller, Giovanni Francesco, Aiden Ashley, Britney Amber, Carter Cruise, Kleio Valentien, Brendon Miller, Derrick Pierce, and Alison Tyler with a special appearance by Celeste Star.
Studio: Wicked Pictures/Wicked Comix
Min: 133:45 minutes
Date of Production: 3/10/2015
Lex Luther (Derrick Peirce) is about to get possibly impeached from the presidency of the United States for his supposed connection to an attack on Metropolis by Bizaro. This is mentioned on the TV by Lois Lane (the always beautiful Celeste Star in a non-sex role). From a distant planet Superman (Ryan Driller) watches the report alongside the busty and sexy Maxima (Britney Amber). Back on Earth, Batman (Giovanni Francesco) is looking for the Joker (Brendon Miller) who, as Harley Quinn (Kleio Valentein) informs him, has a nefarious plot to hurt Lois.
Perhaps the most single anticipated porno of the year this is the movie that horny DC comic-book fans want and need. I, for one, have been excited about this for the longest of time, much more so than the actual mainstream studio film that is to be released next year. And, I am pleased to say that this one delivers both in it's comic-book/ cosplay content, as well as being prime jerk-off material. In short, it is exactly what you want in a movie like this.
There are lots of winks and nods to DC movies and comics, that the fans will love. Braun is himself a huge comic-book fan, which is exactly why we geeks get excited about and love his films. Characters get named, scenes and stories from various source materials get used. I won't mention them cause part of the fun is in discovering these jewels on your own. The suits all look really cool and close to the source material. Catwoman (the gorgeous Aiden Ashley) in particular blows away (pun unintended) the Anne Hatheway take on the character from the overrated The Dark Knight Rises. In fact, I will go so far as saying that Aiden is hotter and does a better job than Hatheway in capturing the feel and personality of the character. Sue me if this comment pisses you off, but I stand by it.
In fact everyone is so well cast as their respected characters. They really capture the essence, look, and feel of these classic DC characters. Aside from Aiden, particular props need to go to, of course, Giovanni Francesco and Ryan Driller in the title roles, Derrick Peirce who is excellent as Luthor, and Kleio Valentien who pure perfection as Harley Quinn. She seems to be having a blast playing her. I was so excited and happy to see that Celeste Star makes a special appearance as Lois Lane in the trailer. Though she doesn't have sex or gets nude she looks stunning as ever, and what happens to her in the movie is shocking (even if the trailer spoils it). It is safe to say that this will NOT occur in the mainstream movie. But, perhaps my favorite acting came from Brendon Miller as a Heath Ledger inspired Joker. He is just awesomely evil here!
But of course, in the end it's the sex that's going to be what truly sells this one. And, I am pleased to say that this movie is a scorcher and will have you reaching for the Kleenex with superhuman speed. The first scene has Supes getting a BJ from Maxima. I've always thought the character of Maxima, a warrior alien woman who wants Superman, to be a hottie. But, it's doubtful you will ever see her in a DC movie, much less in a comic-book appropriate costume like this, or played by someone as sexy as the gorgeous and hot Ms. Britney Amber. She is quite apt at giving killer head making for a solid opening scene.
Next up Harley Quinn gets boned by Batman. You see the only way she'll tell the Dark Knight what the Joker's plan is if he impales her with his dick. As, I said before Kleio is just perfect in this. She channels the beloved character perfectly and just looks hot as fuck as she sucks Batdick; gets penetrated, and takes a load to her chest. I could totally see Kleio getting a best supporting role nomination for this one.
Up next we get Peirce's Lex Luthor piercing Supergirl's snatch in an absolutely fantastic scene. I had a thing for Supergirl as a little kid, so seeing her being bought to life by way of one of the hottest performers today in the gorgeous Carter Cruise was perfect for me. I was turned on the moment Carter walks onscreen. She gives her it all, and takes all of his schlong. It's an energetic fuck and the cumshot is fan-fucking-tastic. If you needed further proof of why Carter rules, here it fucking it is!
The next scene, though, is the movie's highlight and hottest one. Catwoman and Harley Quinn start off with some hot lesbian action, and soon the Joker joins in. As I already said Aiden and Kleio are just so good and natural in their roles. I love that Aiden stays in character throughout the whole scene with lots of cat like licking. The Joker is clearly a lucky psycho plowing both these babes, who happily take his penis. They share his load in a scorcher of a cumshot. This one alone is worth the price of admittance and could easily become a favorite scene of any horny comic-book reader to revisit. I know I will!
The climax of the movie will have you climaxing in joy as Batman and Superman screw Wonder Woman (Alison Tyler). Alison was clearly made by the heavens (or the Greek gods) to play the most famous superheronie on God's green earth. Her voluptuous body and 6 foot frame look AMAZING in the very kick ass and sexy costume making her the very definition of an Amazon goddess. The internet has been less than kind to the casting of Gil Gadot as Wonder Woman in the aforementioned mainstream movie, but no one in their right mind will bitch about Alison. Her boobs are wondrous enough that I could spend a lifetime talking about my love for them.
She gets rotisserie style fucked by the Man of Steel and the Dark Knight. She also sucks them both off. They end up blowing their heroic load on her massive and glorious chest. This is turly DC's Trinity in all of it's smuttiest and sexiest best!
Shot in Hi-Def 1080p (you will have to see it online to see it in full hi-def) the DVD (which is only standard definition, of course) looks gorgeous with deep blacks and rich colors. The stereo surround sound is also quite good with clear dialogue and a driving soundtrack. The extras are mostly on the second disc. They include a fun behind the scenes feature, a non-sex version of the movie, 3 bonus scenes from other films, and trailers, including it's own trailer, though that one is on the first disc.
An entertaining, fun, sexy porn parody Batman V Superman: An Axel Braun Parody is a must have for fans who love porn and superheroes. Axel again delivers the goods for this type of smut like only he can. With every single scene being worth watching and wanking off to, solid performances by everyone involved, and a plethora of comic-book coolness, this one needs to be at the top of your must see list.
4 out of 4 Woodies
You can watch BATMAN V SUPERMAN XXX: AN AXEL BRAUN PARODY here: http://www.adultdvdempire.com/1744060/batman-v-superman-xxx-an-axel-braun-parody-porn-videos.html?partner_id=42272314
adult, Aiden Ashley, Alison Tyler, Axel Braun, Batman, Batman V Superman XXX, Britney Amber, Carter Cruise, Celeste Star, Cosplay, DC comics, Kleio, Nudity, porn parodies, superheroes, Superman Edit
A Return to Jurassic Park
I will finally be seeing Jurassic World tomorrow. The fourth installment in the franchise is already a HUGE hit at the box office and according to the reviews is actually a worthy follow-up to the Spielberg classic, unlike the often maligned second and third installments. Does the first film hold up? Are the sequels really THAT bad?
I should mention that I clearly remember seeing the first film in the movies, twice actually! I was a huge fan of it (having grown up loving dinosaurs, as a child), so much so that I read the book (which is one of my favorite novels, ever), own a mint in box Raptor, and owned it on VHS. I saw it a bunch of more times and while I always liked it, I never enjoyed it as much as that first viewing. I never bought it on DVD, so now watching the blu-ray (which I only bought recently), what would my thoughts be of this blockbuster film? Let's see!
ALL REVIEWS ARE OUT OF FOUR STARS
Jurassic Park- A rich, old dude named John Hammond (Richard Attenborough) opens up a park where genetically created dinos live. He calls upon some experts to give a look and has two kids from his family come over to the island, too. Needless to say things go downhill, as the dinos are set free and begin to attack.
Jurassic Park was a massive hit when it came out. While, some have called it a perfect film, I would disagree.
On the upside, Spielberg directs the movie with a sense of awe and wonder that gives it's amazing and a lot of heart. The CGI is flawlessly mixed with the impressive practical FX of Stan Winston. The dino attacks are nail-biting and rousing. The first T-Rex attack is absolutely smashing, in particular, on blu-ray. The raptors hunting the kids is tense and the climax rocks hard.
Good acting from the cast, including both Sam Niel and Jeff Goldbulm and a likeable and hot heronie in Laura Dern all add to the plus.
The movie is hampered by annoying kids and, having read the book, a desire from me, for it to be a gorefest. The book is better for sure, but this is a fun, little sci-fi classic, and one that actually has aged quite well.
The Lost World: Jurassic Park- Based on the Michael Crichton book of the same name, this sequel, tell us that dinos were bred on a different island where they are flourshing. Hammond sends Ian Malcolm (Jeff Goldbulm) to go there and join his hot girlfriend (Julianne Moore). His daughter stows away and joins them, among others, and they see that there are people there intent on bringing dinos to San Diego. Much screaming and dying ensues.
When, I first saw this I didn't like it much at all, aside from the upped gore quota. I am glad to say the movie aged well, though, and is better than I recall it being. But is still highly flawed and really not that great.
First off, it's overlong and highly uneven. The length hurts the pacing, as does the annoying kid (what is with kids in the franchise?!). Her scene where she uses her gymnastics to drop-kick a raptor is THE worst scene in the whole series.
That said the acting is good. Goldbulm is, well, gold. Moore is a beautiful and talented woman but kinda wasted here. The FX (both practical and CGI) are as flawless as in the first film. But, it's the vicious dinosaur scenes that makes this one worth seeing. While, still staying within the (old)parameters of the PG-13 rating, there is some solid carnage including a spectacular moment where two T-Rexes rip a dude in half.
Speilberg has some solid and tense moments, that combined with the aforementioned gore make the movie slide towards a more horrific feel than the first film.
Never is this more evident than the fun climax where a T-Rex rampages through San Diego. Fans of giant monster horror movies will be happy. I know I was.
**1/2
Jurassic Park III- My favorite character from the first film Alan Grant (Sam Niel) gets tricked by annoying couple to go to the second island and look for their stupid, fucking kid.
This is interesting. I remember really liking this one and thinking it was fun, when I saw it in the movies. Watching it now, I think what the fuck was I thinking?!
This movie does NOT hold up, at all. The CGI are no where near as good as the first two. Why? My guess this shit was rushed. They look pretty bad in some shots. The couple are AWFUL! Tea Leoni as the wife is a pretty lady, for sure, but God is she truly a pain in my fucking ass here! William Macy is wasted as her stupid husband. I HATED their scenes together. Fuck, I hated them both, even when they were separate from each other. I kept wishing that they would die, but... well let's just say the body count is lower, this time around and violence heavily toned down. There are lapses in logic through out and total stupidity abounds.
The dinosaurs scenes can be fun, though. The new dinosaur, the massive Spinosaurus is cool. His fight with the T-Rex is pretty awesome but, too short. The best scene is the pteradon attack. They're one of my fav dinos, but even this hurt by the lame-brained directing and script.
It's not the worst film ever made, but it is the very definition of middle of the road and being utterly forgettable, especially when the weak-ass climax hits followed by the lame and highly crappy ending.
So what are your thoughts on the series? Please, no spoilers on the 4th film, since I, as I stated before, I will be seeing it tomorrow.
Gore, horror, Jurassic park, Jurassic Park III, movie reviews, Raptors, science fiction, Spinosaurus, Steven Spielberg, T-Rex, The Lost World: Jurassic Park, Tyrannosaurs Rex, Velociraptors Edit
MILE HIGH MEDIA KICKS UP THE ALL-GIRL ACTION IN 'LESBIAN BABYSITTERS 13'
All pics courtesy of Mile High Media and Sweetheart Video:
You know if I needed babysitters, I would hope they are as hot as the ones in this film. Clearly another scorcher from Mile High Media/ Sweetheart Video and Dana Vespoli, the cover alone is enough to sell me on it. Of course all of the following seals the deal!
From the Mile High Media press release:
Naïve Babysitter Claire Robbins Is in Over Her Head in Sweetheart Video's Newest Release
MONTREAL, Quebec (June 17, 2015) – Mile High Media takes the all-girl action up a notch in "Lesbian Babysitters 13," now available on DVD. In the erotic flick, naïve babysitter Claire Robbins is in over her head, as she starts her new job taking care of a dysfunctional lesbian couple's child. "Lesbian Babysitters 13" is Sweetheart Video at its best.
In addition to Claire Robbins, "Lesbian Babysitters 13" features Kimberly Kane, Kleio Valentiien, Dana Vespoli, and Nina Elle. Vespoli also directs the film.
In "Lesbian Babysitters 13," kindhearted Claire thinks she's found the perfect job caring for the baby of an uber hip lesbian couple. However, she soon discovers tattoo shop owner Jessica (Elle) is struggling to keep her relationship from falling apart. Her lover Jill (Vespoli), a professor, is becoming more and more withdrawn. Claire soon finds herself struggling to help Jessica while becoming entangled in the couple's bizarre world.
Kimberly Kane plays Professor Smyte, a colleague of Jill's from the university, who takes a special interest in young Claire. Meanwhile, Kleio Valentien plays a porn star with nothing to lose. This must-see all-girl movie is the latest installment in Sweetheart's celebrated series and has all the passion and intrigue that make this title so popular with fans of lesbian romance.
Mile High Media Vice President Jon Blitt states, "Dana tells a fun and sexy story in the 13th volume of 'Lesbian Babysitters.' Combine it with a great cast and dynamic sex scenes, and we have another winner from Sweetheart Video!"
Sweetheart Video's goal is to create original, innovative girl / girl erotica for the discerning viewer. Sweetheart films are storyline-driven, carefully cast, and always containing real sexual encounters between women who truly want to be together.
Mile High Media hosts membership sites for its acclaimed studios. SweetheartVideo.com provides consumers with all the passionate girl / girl scenes Sweetheart Video is known for, direct on their computer.
About Mile High Media:
Mile High Media has been a pillar of the adult industry for over 20 years. The Canadian managed company began as a Canadian licensee for American adult movies, ensuring it distributed only the highest quality content. Understanding the diverse and ever changing needs of its consumer base, Mile High determined the best way to provide a wide array of superior entertainment was to produce its own. In 2003, Mile High launched Doghouse Digital, the first of its acclaimed lines catering to specific niches. Growing market trends and demand led Mile High to create Sweetheart Video (lesbian line), Sweet Sinner (couple line), Reality Junkies (reality gonzo line), Hardcore Parody (parody line) and Icon Male (Gay romance line). With over 2000 titles, Mile High's studios yield some of the most viewed adult movies in the world, available on all platforms, including broadcast, IPTV, VOD and mobile applications. Mile High is committed to evolve alongside consumers tastes, reflecting new developments in technology and entertainment. This philosophy led AEBN.net to name Mile High the 2012 Studio of the Year. For more information, visit www.MileHighOnline.com, www.Facebook.com/MileHighMedia, or www.Twitter.com/MileHighMovies.
Goddamn!! See what I mean?! It's a must have for sure! Lesbian Babysitters 13 hit DVD yesterday, on June 17, 2015. You can see it on the link mentioned about or here: http://www.adultdvdempire.com/1744567/lesbian-babysitters-13-porn-videos.html?qs=Lesbian%20Babysitters%2013&partner_id=42272314
adult, all girl, Ass, Babes, Claire Robbins, Dana Vespoli, hardcore porn, Kimberly Kane, Kleio, Lesbian Babysitters 13, lesbians, Mile High Media, Nudity, porn stars, Sweetheart Video, Tattoos Edit
BATMAN V SUPERMAN XXX: AN AXEL BRAUN PARODY (DVD R...
MILE HIGH MEDIA KICKS UP THE ALL-GIRL ACTION IN 'L...
In Remembrance of the Last Classic Horror Legend: ...
Insidious: Chapter 3
Special Edition NYC 2015 (Sun)
MILE HIGH MEDIA GIVES FANS 8 VERY BAD GIRLS IN PRI...
Hot Asian of the Month: Lana Condor
Veteran Director Stormy Daniels' "Waiting On Love"...
Wicked Comix Releases Official Trailer For 'Batman...

redpajama_set_name: RedPajamaCommonCrawl


text:
Panni est une commune italienne de la province de Foggia dans la région des Pouilles.

Administration

Hameaux

Communes limitrophes
Accadia, Bovino, Montaguto, Monteleone di Puglia, Orsara di Puglia,  Savignano Irpino

Notes et références

Commune dans la province de Foggia

redpajama_set_name: RedPajamaWikipedia


Last month, I wrote about fear. Fear of embarrassing oneself is often a hitch factor in creativity. "What if this idea is no good?" "What if people laugh at this idea?" are pretty common ones, and that sort of fear can be paralyzing.
But it's also exceedingly common. It takes a long time to overcome that fear of failure, that fear of not being good enough. Many people, and I'll add that many successful people retain that crippling fear that what they make is not fit for public consumption, and that makes them hesitate.
Which is not to say that creation is easy, or that everything that we make is fit for public consumption. Nickolaus Pacione has taught me that, if nothing else. But I'll never get anywhere if I don't start. I'll never develop good ideas if I don't work through the bad ones, and have enough of each to recognize a bad one when it presents itself. More importantly, if I hadn't cycled through a bunch of ideas, I'll never get comfortable with the concept that ideas are easy to come by. Ideas are cheap. Good ideas are plentiful. Being willing to discard mediocre ideas means that I spend less time wrangling them. Because experience has shown me that if I wait five minutes, a better idea will come up.
Is also seems that, after five years, I'm going to hit 20,000 views of this blog. I'm aware that the majority are Google search hits, and that I have yet to break a thousand hits in a single month. But thank you to everyone who has read this strange personal/writing blog.
For a good time? Go to an author's reading. The Northshire Bookstore recently had an author signing for Nancy Marie Brown who has just released Song of the Vikings a book about the very interesting historian/skald of Iceland, Snorri Sturluson.
I learned a lot, just in an hour. I knew, for example, that my 'escapist' reading eas looked down on when I was in college, that Tolkien had only a grudging respect in academia in the early 90's because he had written part of the Oxford English Dictionary, and because his books had sold so well. How much more frustrating it must have been earlier, when he didn't have the cache of being so popular. He was only an Oxford Dean, back then.
I love my Tolkien. I love my Icelandic saga. While they are not the same, they go hand-in-hand with a certain number of sensibilities. And those have been passed onto a few generations of writers. And I consier that a good thing. But it's always interesting to understand the origins of something I enjoy, as well as it's influences. It's why I'm totally hooked on movie commentaries and extras. What was the process that lead to the creation of something like the Lord of the Rings or Pan's Labyrinth?
Snorri Sturluson was a poet and powerful landowner in 13th Century Iceland. He wrote many of the stories we clasically associate with Norse mythology, and laid the foundations for the early modern view of heroism. But Snorri's story also has unexpected moments of connection to the work of Tolkien. For example, the day of his death. September the 22nd. Which is, for the non-Tolkein fanatics out there, Bilbo and Frodo's shared birthday. Hm.
Nancy Marie Brown is a good speaker, interesting and informative. She really knew how to hook me as a listener in. She was pleasant to speak with and clealy knew her stuff. Who wouldn't want a book or two signed by her?
According to her blog, what prompted her to start writing this book was reading Neil Gaiman's American Gods. Neil Gaiman, if you aren't a long-time reader, is, in my opinion, the second-most dropped name in English. I can't wait to dive into Song of the Vikings.


\r\nStarting depletion tomorrow. the goal is to get under 160lb and 20% bf. We will see where I land.
\r\nThis am was ok as well as I was getting the weights right to feel the "burn\' without failure. Next week\'s depletion workouts will go a lot smoother.
\r\nI am on the Tuesday/Wed depletion cycle . I want to have Sat\'s for the load.
\r\nDay 1 started July 17 , 2018, 6 week cycle.
I\'m interested to follow your cut. I\'m planning to start one myself in the next week or two. Good work getting through the hard part of the week!
\r\nI did animal crackers, Ezekiel flax bread, pretzel nuggets, and whether\'s hard candies. My load was between 700-800 carbs and 50 g fat. Next week I may use bagels and keep my carbs between 600-700. I used protein shakes for my protein because I couldn\'t stuff in more food after eating the carbs.
\r\nI looked good on Sunday before Power Workout. My goal in the next 5 weeks is to nail the workouts, the diet isn\'t hard at all. Feel like I am full of poop though after the carb load. I am taking aloe vera tablets this week to get things moving, although I took care of some business, there is more that needs to go.
\r\nWill see what happens or if last week was effective on this Friday\'s weigh-in, Goal is 164 lbs.
\nI looked good, lean, until today. Feel/look this am bloated but it is PMS time.
Well , I am almost off of this diet break. I am working with a trainer for this next round and maintenance and Istill have to learn how to be off plan. I am ready for the structure of dieting. So next Monday , back in for another 8 weeks.
Loved to see another female doing this, we\'re about the same size too.
\r\nI\'m 5\'8 161.8 23%BF Hips 39" waist is sitting at 25.5" hoping to drop to 18% so I can start a lean bulk.
\r\nI haven\'t started yet but should be ready to do so in about a week.
Thanks into 4th week this week of second round. If you will be coming here regularly..ill start posting stats again.


The new Material Design text field boxes. Text field boxes increase text field identifiability and scannability by using a transparent rectangular fill to enclose the label and input text.
Help and inspire fellow designers by sharing your Sketch resource.


For the first three races, the endurance event will be held in parallel with the cross-country races; the Stewart 45 is of course a stand-alone endurance event and will be our grand-finale where we will crown the first H2H Endurance Champions.
The series will offer two fields, Open Men and Open Women, and series standings will be based on the best three out of four races. For the complete series rules, click here.
What's new for the 2016 season?
We've got some exciting changes in store for the 2016 H2H Series. First of all, a big THANK YOU to AC Daughtry Security Systems for coming on board as the series sponsor.
The racing schedule has been finalized, there is one change from the tentative schedule we published in December: we're excited to welcome March Mayhem to the series, and unfortunately due to scheduling conflicts the Tymor Challenge will not be part of the series this year, but will be back in 2017. For the complete 2016 racing schedule, click here.
Read more: What's new for the 2016 season?
Our annual awards banquet is set for Friday, January 29, 2016, 7 pm at the Hilton Garden Inn in Rockaway, NJ. Pre-registration only so don't miss it! Come and celebrate with the racers on the podium from each class and category, and help kick off the 2016 season.
Check out the final standings below, and stay tuned to this webpage or follow us on facebook for info on the 2015 Awards Ceremony.
Check out the fun new course for 2015, running in a counter-clockwise direction for a chance.


GBSN runs a number of weekly breastfeeding support groups across Gloucestershire, in Cheltenham, Cirencester, Stroud, Dursley, Churchdown and Tewkesbury. These groups are all run by our peer supporters and offer information, encouragement and support to families at any stage of their breastfeeding journey. We also have a qualified Breastfeeding Counsellor on hand at each group, as well as a wealth of books and DVDs for you to borrow free-of-charge.
You will be made to feel very welcome at any of our groups, and you can find their meeting times below. For more information about any of our groups, along with any relevant events or news please visit its individual page.


From the bestselling author of Ageless Body, Timeless Mind.
Essential ideas from the work of Deepak Chopra, M.D. are arranged to create a transcendent experience for the listener, a journey into healing. Along the path, we discover that what we think and feel can actually change our biology. We learn to go beyond self-imposed limitations that create disease, and to seek that place inside ourselves that is at one with the infinite intelligence of the universe, the source of life. By the end of this audio, the listener's consciousness will have been altered by the experience of the journey itself. Such change has the power to transform out lives, to grace us with the gifts of lasting peace and perfect health.
The end of the audio contains techniques for the Mindfulness Mediation, which can access the silent space between your thoughts, and tap into the inner wisdom that will make all you dream s comes true.
In this book I have sought to select essential thoughts from my various books and arrange them so as to give the reader the experience of a journey into healing. I hope this journey will influence your perceptions, the better to free your body's natural tendency to perfect health. To complement this text I have added pages on meditation, one of the best ways to access our inner intelligence, to experience who we really are.


text:
GOLD FRINGED FLAGS...
..."The gold-fringed flag only stands inside military courts that sit in summary court martial proceedings against civilians."
The U.S. Constitution is a "Let's Pretend" document without the Rule of Law.
Last week, Jeffrey Imelt, CEO of General Electric (GE) and a member of Obama's Economic Advisory Council with specific responsibility for job creation, told businesses to stop sitting on their cash and begin hiring people. Imelt is a particularly offensive man whose only real talent is manipulating government programs to benefit his company at taxpayer expense.
In an article I wrote last week for another publication (Link here) explaining why this administration is impotent when it comes to job creation, I said:
"In our crisis economy, there is no "other people's money" available from taxpayers, stockholders, partners or investors. Thus, the union thugs, corporate executives, administrators, and others who don't understand risk management – those with whom you've surrounded yourself – can't solve the problem. They only know how to succeed using other people's money… and the state of the world economy dictates that there is no more 'other people's money.' Well, there is – but you will let the world starve before you make the source known."
There is so much corruption in our business community, it makes me ill.
One example: In 2007, GE, the world's largest industrial company, was one of the biggest beneficiaries of funds set aside to rescue banks. Is GE a bank? No, but what does that matter?
GE has a company called General Electric Capital, one of the world's largest and most diverse financial operations. GE Capital makes loans for commercial real estate, aircraft leasing and issues credit cards for stores like Wal-Mart. If categorized as a "banking company," it would be the seventh largest bank in America. Because GE Capital is not a bank, it did not qualify for TARP funds under the Temporary Liquidity Guarantee Program (TLGP) funds. TLGP is a Federal Deposit Insurance Corporation (FDIC) program designed to guarantee unsecured bank, thrift and holding company debt, thus strengthening confidence in the banking system. GE didn't qualify for the program – but close to $80 billion was funneled through two small financial institutions into GE's corporate coffers.
You see, GE Capital owned an FDIC-insured savings and loan and an industrial loan company in Utah. These two very small financial institutions made up only three percent of GE's total asset base… but gained qualification to the TARP funds from the Obama administration under a loophole. Special privileges had to be manipulated through the system, but it was done quickly and, with Imelt in charge of NBC News, a GE company, publicity was contained.
Anyone who buys a General Electric product is supporting a company that has become one of the most skilled tax avoiders in America. In 2010, GE set aside a mere 7.4 percent of its $5.1 billion U.S. profits to pay taxes. Actually, it paid practically nothing.
How do things like this happen? How do American taxpayers get nailed to the wall over and over and over again? I go back to my opening statement: The U.S. constitution is a "Let's Pretend" document without the Rule of Law.
I'm always 'preaching' about finding the core problem rather than being continually diverted by issues about which we can do nothing. This is a prime example. The core problem appears to be our court system. There are some interesting facts you should know about the American Flag, for instance. What does that have to do with our court system? Read on…
President Dwight David Eisenhower signed Executive Order 10834 on August 21, 1959. It is printed in the Federal Register at 24 F.R. 6865, pursuant to law, and says: "A military flag is a flag that resembles the regular flag of the United States, except that it has a yellow fringe border on three sides."
Every nation in the world accepts The Law of the Flag. Basically, it designates the rights under which a ship owner who sends a vessel into a foreign port functions and provides notice to those who enter into contracts with the ship master that he will use the Law of the Flag he flies to regulate contracts. The Law of the Flag regulates the laws under which contracts entered into will be governed. (See Ruhstrat v. People.)
In other words, Admiralty Law says that when a ship flies a Spanish flag, if you sign a contract with the Captain of that ship, any agreements or contracts the Captain signs will fall under Spanish law unless otherwise specified.
Further, there is a doctrine called "four cornering" a flag. According to usatherepublic.com, "By the doctrine of 'four cornering' the flag establishes the law of the country that it represents. For example, the embassies of foreign countries, in Washington D.C., are 'four cornered' by walls or fencing, creating an 'enclave.' Within the boundaries of the 'enclave' of the foreign embassy, the flag of that foreign country establishes the jurisdiction and law of that foreign country, which will be enforced by the Law of the Flag and international treaty. If you enter an embassy, you will be subject to the laws of that country, just as if you board a ship flying a foreign flag, you will be subject to the laws of that flag, enforceable by the 'master of the ship' (Captain)."
So, when you enter your local courtroom and find a yellow-fringed flag, you may want to seriously question under what laws people who appear in that courtroom are being tried: The U.S. Constitution? Or, the U.S. Military? Or, Maritime Law? Or, the Uniform Commercial Code (corporate law)?
Admiralty law tells us that when you see a flag, you need to understand that in areas where a flag is "four cornered," you are being told which laws dominate that space. We are governed by the laws of the country the flag represents. According to President Eisenhower, gold-fringed flag tells us we are under the rules and regulations of the military force that flies the flag.
FLAG Martial Law; "Pursuant to 4 U.S.C. chapter 1, §§1, 2, & 3; Executive Order 10834, August 21, 1959; 24 F.R.6865; a military flag is a flag that resembles the regular flag of the United States, except that it has a YELLOW FRINGE border on three sides. The president of the United States designates this deviation from the regular flag, by executive order, and in his capacity as Commander-in-Chief.
FLAG Martial law; "Ancient custom sanctions the use of the fringe on regimental colors and standards, but there seems to be no good reason or precedent for its use on other flags." The Adjutant General of the Army, March 28, 1924, (1925); 34 Ops. Atty. Gen. 483, 485.
According to many Internet sites espousing expertise involving Flags, courtrooms that display fringed flags behind a judge is a military courtroom. Such a courtroom tries cases on the basis of military law, not constitutional law, not common law, not civil law, and not statute law. According to Army Regulations, (AR 840-10, Oct. 1, 1979.) "the Flag is trimmed on three sides with Fringe of Gold, 2 1/2 inches wide," and "such flags are flown indoors, ONLY in military courtrooms."
The gold-fringed flag only stands inside military courts that sit in summary court martial proceedings against civilians. These courts are partially governed by local rules, but more especially by "The Manual of Courts Martial", U.S., 1994 Ed., at Art. 99, (c)(1)(b), pg. IV-34, PIN 030567-0000. A judge who sits 'under' a gold or yellow fringe flag becomes the 'captain' or 'master' of that ship or 'enclave' – or courtroom? That judge has the absolute power to make the rules as he goes. The gold or yellow fringe flag warns that you are leaving your constitutional rights outside the four walls in the room where such a flag is "cornered."
Is this why so many judges are appointed and not elected by the people? Federal judges are appointed by the President, the national military commander in chief. State judges are appointed by Governors, the state military commanders. Are judges appointed because the courts are military courts and civilians do not 'elect' military officers?
Under martial law, you are presumed guilty until proven innocent. That would certainly explain many court decisions, wouldn't it?
Article IV, section 3, of the Constitution of the United States of America says "…no new State shall be formed or erected within the Jurisdiction of any other State." So – why have the judges of State and Federal courts been allowed to erect foreign 'enclaves' within our public courthouses under a foreign flag with yellow fringe? The soil on which these courtrooms are built belongs to each of our states. Does this make sense to you? It doesn't to me. It only makes sense if the states are viewed as territories, rather than states.
There are other theories – that Maritime Law gives the legal system the right to treat people as corporate assets, for example – but all of the theories come to the same conclusion: Any courtroom with a fringed flag behind the judge should be highly suspect relative to the law under which people are tried.
Your job on Monday ("should you accept it" – for Mission Impossible fans): Visit your local courtrooms and take a picture of the flag in each. Be sure the pictures document where the flag is located – is it behind the presiding judge? The odds are very high that the flags in your courtrooms are fringed with gold. Is this an indication that the courts in your state function under Maritime Law? Probably. I have for years looked for reason in the decisions coming from American courtrooms – and as strange as it sounds, this makes a lot of sense and explains a lot of crazy decisions that have nothing to do with constitutional law.
This opens doors of lawful activity for Tea Party Groups… for all citizens, actually. Rather than protest – or sit quietly by doing nothing – take pictures of your courtrooms. Visit the links provided regarding flags and what they mean. Raise enough money to place a full-page ad in your local newspaper and publish the photos with a story. Educate the rest of the electorate as to why our courts allow corporate and government fraud and why the law seems to be interpreted differently for individuals than it does for people like Jeffrey Imelt, GE, and the politicians who support scumbags like him.
When you add judges whose Oaths of Office are not properly filed (see my NewsWithViews article) as constitutionally required by the state and the federal government, you will understand why taxpayer rape is being condoned by our legal system.
And here's the second project for you – should you accept it, of course. Find the names of your judges – city, county and state. Call your county clerk and ask where the Oaths of Office for city and county judges can be found. Call your Secretary of State to find where each state judge's filed Oath of Office is located (including the state supreme court). These are public documents. You are entitled to copies of them. When you find a judge whose Oath of Office is not on file, ask for a signed statement to that effect.
Bear in mind that when judges do not have a legitimately FILED Oath of Office, every decision that person has made while on the bench is illegitimate. Most state constitutions say a public official without a FILED (not just taken, but legally filed) Oath of Office is viewed as if the office requiring the Oath has been "vacant." Find the exact wording of your state's constitution (look under Oaths of Office) regarding filed Oaths. Publish in your advertisement about court flags what the Constitution says along with the names of any judges who do not have one on file. Remember, district attorneys and federal judges and prosecutors take Oaths of Office, too.
We citizens can do much to clean up the mess we allowed "them" to make – and the cleaning job must begin locally! Remove "their" base.
1 - Sovereign Family of the World
2 - Gold Fringed Flag
3 - Get that Gold Fringe off my Flag
© 2011 Marilyn M. Barnewall - All Rights Reserved
Marilyn MacGruder Barnewall began her career in 1956 as a journalist with the Wyoming Eagle in Cheyenne. During her 20 years (plus) as a banker and bank consultant, she wrote extensively for The American Banker, Bank Marketing Magazine, Trust Marketing Magazine, was U.S. Consulting Editor for Private Banker International (London/Dublin), and other major banking industry publications. She has written seven non-fiction books about banking and taught private banking at Colorado University for the American Bankers Association. She has authored seven banking books, one dog book, and two works of fiction (about banking, of course). She has served on numerous Boards in her community.
Barnewall is the former editor of The National Peace Officer Magazine and as a journalist has written guest editorials for the Denver Post, Rocky Mountain News and Newsweek, among others. On the Internet, she has written for News With Views, World Net Daily, Canada Free Press, Christian Business Daily, Business Reform, and others. She has been quoted in Time, Forbes, Wall Street Journal and other national and international publications. She can be found in Who's Who in America, Who's Who of American Women, Who's Who in Finance and Business, and Who's Who in the World.
Web site: http://marilynwrites.blogspot.com
E-Mail: marilynmacg@juno.com
FRIENDS, AMERICANS, COUNTRYMEN, LEND ME YOUR EARS
AGENDA 21 EQUALS NATIONAL SUICIDE
DEAD BODIES, DECEPTION AND HONOR
BANKSTERS, PRANKSTERS AND SANDY HELL HOLES
MATURITY AND MOTHERHOOD
ROLLING STONE TO CAUSE ROLLING HEADS?
BALLOT BLUES: SAY "BYE-BYE" OBAMA
COMMIES, CRIMES, AND CHASE
"INSIDE JOB" - CONTRIVED CRISIS
DECENTRALIZING THE CENTRALIZED POWER OF THE FEDERA...