text:
\section{Introduction}
\label{sec:intro}
Large Transformer-based model on large corpora~\cite{devlin2018bert, radford2018improving,radford2019language,brown2020language, lewis2019bart,liu2019roberta,dong2019unified,raffel2019exploring} has displayed tremendous success in natural
language processing (NLP) ever since the pioneer work~\cite{vaswani2017attention}.
Inspired from these breakthroughs, the building block of Transformer,
self-attention has greatly revolutionized computer vision (CV) where convolutional neural network has once been the dominant backbone,
from the integration of convolution and self-attention~\cite{wang2018non,carion2020end,ramachandran2019stand,wang2020axial} to  general vision Transformer backbones~\cite{dosovitskiy2020image,wang2021pyramid,chen2021mobile,liu2021swin,dong2021cswin}.

The NLP success not only relies on the Transformer's computational efficiency and scalability enabling training on models with over one hundred billion parameters~\cite{brown2020language},
but also the self-supervised learning paradigm leveraging vast amount of text on the internet.
The two popular paradigms, GPT based on autoregressive language modeling and BERT based on masked language modeling, are also redesigned in computer vision in order to take advantage of enormous quantities of web images.

Nevertheless, it is challenging to design similar-style objectives in vision,
as images, being a high-dimensional and redundant modality, differ from texts in two aspects.
First, the texts are naturally composed of discrete characters with a pre-existing vocabulary, while the images present continuous values in the color space.
To acquire discrete image tokens,
iGPT~\cite{chen2020generative} quantized the pixel values using k-means,
ViT~\cite{dosovitskiy2020image} divide the image into non-overlapping fixed-size patches,
and BEiT~\cite{bao2021beit} tokenize the image to discrete visual tokens via discrete VAE~\cite{oord2017neural,rolfe2016discrete}.
Second, the discrete tokens in texts contain highly semantic meanings, while the images, being discretized in pixel-level and patch-level, consist of heavy redundant tokens.
The latent codes used in BEiT~\cite{bao2021beit} may be more semantic than pixel-level and patch-level tokens, but it is learned through a low-level vision objective, per-pixel loss, which is proved to be vulnerable to the perceptual difference between images
as the pixel representations of perceptually similar images might be very different.
Hence we ask that
\emph{might there be a way to learn perceptual discrete visual tokens which will be helpful for image pre-training.}

Motivated by this observation and answering the above question, we propose to learn a perceptual codebook (PeCo) for images for the first time,
with the goal of facilitating subsequent pre-training tasks.
To achieve that, we propose a simple yet effective way, which is to enforce perceptual similarity between the original image and the reconstructed image in a vector quantized variational autoencoder (VQ-VAE)~\cite{oord2017neural}.
The perceptual similarity is measured by perceptual distance between deep visual features extracted from a deep neural network architecture.

The perceptual distance is not new and related to the widely used perceptual loss, \ie aligning the ImageNet-trained VGG features, in many
synthesis tasks such as neural style transfer~\cite{gatys2016image,chen2017stylebank}, image super-resolution~\cite{johnson2016perceptual}, conditional image synthesis~\cite{dosovitskiy2016generating,chen2017photographic}. It was generally believed that constraining perceptual loss in those synthesis tasks will improve the image quality, \eg the synthesized image reveals finer details.
Nonetheless, our adoption here is non-trivial, as in this paper, we care about the perceptual knowledge of the learned codewords and its effectiveness in pre-training
rather than the quality of the reconstructed images.

More importantly, we do not use VGG features as VGG is trained on ImageNet with labels.
Instead, we turn to a self-supervised trained network for the deep visual features and we show that even with self-supervised network, the learned codewords exhibit semantic meanings. In this way, the whole learning pipeline can enable unsupervised pre-training.
In the experiments, we train the codebook on ImageNet-1K~\cite{deng2009imagenet}
and adopt visual Transformers with BERT pre-training~\cite{devlin2018bert,bao2021beit} over the learned perceptual discrete visual tokens.
We demonstrate that equipped with such perceptual visual tokens, better performance can be achieved compared with the strong competitor BEiT~\cite{bao2021beit} using DALL-E~\cite{ramesh2021zero} codebook trained over $250M$ images without the perceptual loss.

We fine-tune the pre-trained model on various downstream tasks: image classification, object detection, and semantic segmentation. Experimental results show that our pre-trained model transfers better than  BEiT~\cite{bao2021beit} with only the codebook change. Concretely, we achieve $\textbf{84.5}\%$ Top-1 accuracy on ImageNet-$1$K with ViT-B model, outperforming BEiT~\cite{bao2021beit} by $\textbf{+1.3}$ with the same 800 pre-training epochs. It is also better than the result reported (83.6\%) in the concurrent work MAE\cite{he2021masked}, which is achieved with longer pretraining epochs (1600 epochs). For the object detection and semantic segmentation task, compared to BEiT, our pretrained model improves the performance on COCO~\cite{lin2014microsoftcoco} by $\textbf{+1.3}$ box AP and $\textbf{+1.0}$ mask AP, and $\textbf{+1.0}$ mIoU on  ADE20k~\cite{zhou2017scene} segmentation dataset. For larger models, our ViT-Huge gets $\textbf{88.3\%}$ Top-1 accuracy on ImageNet-1K without any additional data, outperforms the start-of-the-art method MAE~\cite{he2021masked} with a large margin $\textbf{+0.5\%}$

\section{Related Works}
\noindent \textbf{Self-supervised Learning.}
Self-supervised learning has attracted increasing attention over the past few years,
as deep learning networks become more and more data-hungry and it's impossible to label everything in the world.
There are two main categories along this path, contrastive and generative~\cite{liu2021self}.
One emerging field is self-supervised contrastive learning,
training an encoder to the representation measured by contrastive loss~\cite{hadsell2006dimensionality,dosovitskiy2014discriminative} via comparing similar and dissimilar samples. The representative methods include MOCO~\cite{he2020momentum,chen2020improved,chen2021empirical}, SimCLR~\cite{chen2020simple,chen2020big}, BYOL~\cite{grill2020bootstrap}, SwAV~\cite{caron2020unsupervised} and more~\cite{oord2018representation,bachman2019learning}.
However, contrastive-based methods heavily depend on the strong data augmentation and effective negative sampling.
The other recent resurgent field is generative self-supervised learning,
training an encoder and a decoder under the objective of reconstruction loss.
The typical
objectives, autoregressive and denoising autoencoder, aiming at recovering the corrupted or masked input, has yielded the most successful frameworks~\cite{devlin2018bert, radford2018improving,radford2019language,brown2020language, lewis2019bart,liu2019roberta,dong2019unified,raffel2019exploring,joshi2020spanbert} in NLP.
Thanks to the pre-existing vocabulary in language, recovering the missing word can be transformed into
predicting all the possible words with the probability estimation, converting the prediction problem to an easier classification problem.
While in CV, on the other hand, most attempts~\cite{van2016pixel,oord2016conditional, chen2020generative,he2021masked} still resort to regression for generative methods
due to the lack of visual vocabulary,
\eg iGPT~\cite{chen2020generative}.
Recently, BEiT~\cite{bao2021beit} successfully adopts a classifier for prediction by directly adopting a VQ-VAE as the visual tokenizer.
Yet there exists a major difference between the language vocabulary and the visual vocabulary.
That is, the words of language are highly semantic, while the visual words of images are mostly not.
Most recently, a concurrent work MAE~\cite{he2021masked} also observes this information density difference, but still reconstructs pixels, the non-semantic entities.
In this work, we attempt to learn a perceptual visual vocabulary for BERT pre-training of vision transformers, showing superior transfer performance than BEiT~\cite{bao2021beit} and MAE~\cite{he2021masked}.

\noindent \textbf{Discrete Visual Supervision.}
Exploring masked image modeling or image inpainting task for self-supervised pretrained tasks has never been stopped in vision community, especially when BERT~\cite{devlin2018bert} achieves great success in various tasks of NLP. To apply the cross-entropy loss function for vision tasks, iGPT~\cite{chen2020generative} clusters the pixel values to simulate the process of BPE~\cite{sennrich2015neural} process for different words in language. ViT~\cite{dosovitskiy2020image} attempts to directly divide the raw pixel values into multiple groups and assign a discrete label for each group GRB value. Recent work VQ-VAE~\cite{oord2017neural} proposes to adopt encoder and decoder to quantize the visual contents to a learnable codebook with fixed size.

\noindent \textbf{Perceptual Similarity.}
The perceptual similarity, as its name suggests, is to mimic the human perceptual
judgment of image similarity.
Numerous efforts have been proposed to achieve that, such as SSIM~\cite{wang2004image}, MSSIM~\cite{wang2003multiscale}, FSIM ~\cite{zhang2011fsim}, and HDR-VDP~\cite{mantiuk2011hdr}.
It has been shown in~\cite{zhang2018unreasonable} that the internal activations of network trained for classification task surprisingly coincide with human judgment.
Such deep features have been widely used in image generation~\cite{gatys2016image,johnson2016perceptual,chen2017stylebank,wang2018perceptual,dosovitskiy2016generating,bruna2015super,ledig2017photo,esser2021taming} with the goal of synthesizing
indistinguishable realistic images.
The loss is called perceptual loss or VGG loss as the network used is often VGG architecture.
Differently, we aim at learning a more semantic visual vocabulary from VQ-VAE for subsequent image pre-training task.
Moreover, to enable self-supervised learning, we adopt a self-supervised trained network rather than ImageNet-trained networks.
We show that (1) the learned visual words are indeed more semantic and (2) such perceptual codebooks are indeed beneficial for pre-training.

\section{Method}
In natural language, the words are naturally discrete tokens which contain high semantic information. In contrast, vision signals are continuous with redundant low-level information.
While there are various ways to discretize the image in prior works, the semantic level of the resulting visual tokens has been largely ignored.
In this section, we start by briefly describing the discrete representation learning from VQ-VAE,
and then introduce the process of how to learn a perceptual codebook,
followed by BERT pre-training over the learned perceptual visual tokens.

\subsection{Learning Discrete Codebook for Visual Content}
We utilize VQ-VAE~\cite{oord2017neural}
to convert the continuous image content into the form of discrete tokens.
Consider an image $x \in \mathbb{R}^{H \times W \times 3}$, VQ-VAE is able to represent it with discrete visual codewords $\{z_q^1, z_q^2, \cdots, z_q^N\} \in \mathcal{V}^1 \times \mathcal{V}^2 \times \cdots \times \mathcal{V}^N$,
where $z_q^i$ comes from a visual codebook (vocabulary) $\mathcal{V}^i =\{e_k^i \in \mathbb{R}^D\}_{k=1}^{K_i}$ consisting of $K_i$ $D$-dimensional codewords. Usually we have $K_1 = K_2 = \cdots = K_N = K$ for simplicity,
and $N = h \times w$ with $h \times w$ being the spatial resolution of the latent space.

Specifically, to learn such latent codeooks, VQ-VAE contains three major parts: an encoder,
a quantizer and a decoder.
The encoder maps the input image to intermediate latent vectors $z = \text{Enc}(x)$,
 where $z \in \mathbb{R}^{h \times w
 	\times D}$
 The quantizer is in charge of
quantizing each vector at position $(i,j)$ to be codewords coming from the corresponding codebook $\mathcal{V}^{i,j} = \{e_k^{i,j}\}_{k=1}^K \subset
\mathbb{R}^D$ according to nearest neighbor assignment.
That is,
\begin{align}
	k^* & = q(z^{i,j}) = \argmin_{k \in \{1,2,\cdots,K\}} \Vert z^{i,j} - e_k^{i,j} \Vert. \\
	z_q^{i,j} & = r(k^*) =  e^{i,j}_{k^{*}},
\end{align}
where $q$ is the quantization encoder that maps the vector to an index of the codebook,
and $r$ is the quantization decoder that reconstructs the vector from the index.
Based on the quantized codewords $z_{\mathbf{q}}$, the decoder aims to reconstruct the input image $x$. Suppose the reconstruct result is $\hat{x} = \text{Dec}(z_{\mathbf{q}})$. Since the quantizer is non-differentiable, to back-propagate gradient into encoder, the gradient is
approximated like the straight-through estimator~\cite{bengio2013estimating} and just copied from decoder to encoder~\cite{oord2017neural}. The training objective of VQ-VAE is defined as,
\begin{align}
\mathcal{L}_{\text{VQ-VAE}}(\text{Enc}, \text{Dec}, \{\mathcal{V}\}) &= \mathcal{L}_{pixel}
  + \Vert \text{sg}[\text{Enc}(x)] - z_{\mathbf{q}} \Vert_2^2 \nonumber \\ &+ \beta\Vert \text{sg}[z_{\mathbf{q}}] - \text{Enc}(x) \Vert_2^2.
\label{eq:origvqloss}
\end{align}
Here, $\mathcal{L}_{pixel} = \Vert x - \hat{x} \Vert$ is a reconstruction loss, $\text{sg}[\cdot]$ is the stop-gradient operator, $\beta$ is a commitment loss hyper-parameter set to 0.25 in all our experiments.

\begin{figure}[t]\centering
\includegraphics[width=0.995\linewidth]{fig/recons_comparison_v1.pdf}\vspace{-.7em}
\caption{Image reconstruction comparison with different losses. An example contains three images showing input (left), reconstructed image using pixel-wise loss (middle), and
reconstructed image using pixel-wise loss and feature-wise loss (right). We can see that perceptually the right image appears more similar to the input compared with the middle image, although the middle image gets lower pixel-wise loss.
We notice that the right image has some checkboard artifacts which are produced by the ResNet-50 network structure as indicated in ~\cite{odena2016deconvolution}.}
\label{fig:recons_comparison}
\end{figure}

\subsection{Learning a Perceptual Codebook for Visual Content}
In the vanilla VQ-VAE, the codebook is learned by an element-wise pixel loss, \ie $\mathcal{L}_{pixel}$, between the original image and the reconstructed image. However, this per-pixel loss may prevent the network from capturing \emph{perceptual} difference since the loss only accounts for the correctness of individual pixels. Therefore, a small shift and rotation operation on the original image may not cause perceptual change but induce a large $\ell_1$/$\ell_2$ change. Since the codebook is trained with such per-pixel loss, the codewords probably fail to encode perceptual information.

Therefore, we propose a simple strategy by enforcing perceptual similarity between the original image and the reconstructed one beyond the pixel loss.
The perceptual similarity is not based on pixel differences but instead on differences between high-level image features representations extracted from a pre-trained deep neural network.
We hope this feature-wise loss will better capture perceptual difference and offer invariance towards low-level variations.
We show the comparison of using different losses in Figure~\ref{fig:recons_comparison} from the perspective of image reconstruction, suggesting that images with lower pixel-wise loss may not appear perceptually similar.

Formally, let $f_l(x)$ be the normalized activations of the $l$-th layer of a network $F$ when processing the image $x$.
The size of the feature map is $H_l \times W_l \times C_l$ with $H_l$ being the height, $W_l$ being the width and $C_l$ being the channel dimension.
Usually, multi-scale features, more comprehensive and discriminative, from multiple layers at different depth are extracted to calculate the perceptual similarity for better semantic capture. The perceptual metric for the input image $x$ and the reconstructed image $\hat{x}$ can be formulated as,
\begin{align}
\mathcal{L}_{\text{percep}} = \sum_{l \in \mathcal{S}} \Vert f_l(x) - f_l(\hat{x}) \Vert_2^2,
\label{eq:perceptloss}
\end{align}
where $\mathcal{S}$ denotes the number of layers from which the features are extracted.

Therefore, the overall objective function is,
\begin{align}
\mathcal{L}_{\text{VQ-VAE}_{percep}} & = \mathcal{L}_{pixel} + \lambda \mathcal{L}_{percep} \nonumber \\
&+ \Vert \text{sg}[\text{Enc}(x)] - z_{\mathbf{q}} \Vert_2^2 \nonumber \\ &+ \beta\Vert \text{sg}[z_{\mathbf{q}}] - \text{Enc}(x) \Vert_2^2,
\label{eq:recloss}
\end{align}
where $\lambda$ is the hyper-parameter for the loss weight of $\mathcal{L}_{percep}$, we will study different vaules of loss weight $\lambda$ in the experiments.

\subsection{BERT Objective over Perceptual Codebook}
We adopt the BERT objective to perform the \emph{masked image modeling} task over the discrete  visual tokens as in BEiT~\cite{bao2021beit}.
For a given image $x$,
the input tokens are image patches which are non-overlappingly split from the whole image,
and the output tokens are discrete perceptual visual words obtained through learning Eqn~\ref{eq:recloss}.
Let the input be $\{x^1, x^2, \cdots, x^N\}$, and the groundtruth output be
$\{z_q^1, z_q^2, \cdots, z_q^N\} = r(q(Enc(x)))$.
The goal of the masked image modeling is to recover the corresponding visual tokens from the masked input where a portion of input tokens have been masked.

Precisely, let $\mathcal{M}$ be the set of masked index.
Then the masked input $\bar{x}$ is represented as,
\begin{align}
	\left\{
	\begin{array}{cc}
		x^i, & i \notin \mathcal{M}\\
		m, & i \in \mathcal{M}\\
	\end{array}
	\right. , i = 1,2,\cdots,N,
\end{align}
where $m$ is a learnable mask token as same dimension as non-mask tokens.
The masked input tokens are fed into a $L$-layer vision Transformer with the last layer's hidden output being denoted as $\{h^1, h^2, \cdots, h^N\}$.
We aim at recovering the corresponding visual token from the hidden vector at masked positions.
To achieve that with the classification loss,
a $K$-way classifier is appended after the hidden vector $h^i$ to get the probability estimation about all possible discrete tokens in the corresponding codebook $\mathcal{V}^i$.
Suppose the groundtruth discrete visual tokens corresponding to the masked patches are $z_q^t$ with $t \in \mathcal{M}$, the pre-training objective can be formulated as,
\begin{align}

	\mathcal{L}_{\text{pre-training}} =  - \sum_{t \in \mathcal{M}} \text{log} P(z_q^t|\bar{x}),
	\label{eq:perceploss}
\end{align}
where $ P(z_q^t|\bar{x})$ is the estimated target token probability for masked patches of corrupted image $\bar{x}$. After pre-training the model, we apply the model to various downstream tasks including ImageNet-$1K$~\cite{deng2009imagenet} classification, COCO object detection~\cite{lin2014microsoftcoco}, and ADE20k~\cite{zhou2017scene} Segmentation.

\subsection{Pre-training Details}

\noindent \textbf{Vector Quantizer.}
We use standard k-means algorithm for vector quantization.
We set the codebook size $K$ as 8192 for fair comparison.
When the size of the discrete latent space $K$ is large, we observe that only a few of codewords are selected to represent image and get trained.
Many other codewords are wasted.
To overcome this issue, we adopt exponential moving averages (EMA)~\cite{oord2017neural} to update the codebook which is proved to be useful for increasing utilization of codewords in a codebook.

\noindent \textbf{Perceptual Codebook Learning Setup.} We train the perceptual codebook using the train set of ImageNet-1k dataset by default. For the encoder and decoder of VQ-VAE, we choose traditional convolutional based backbone.
The network contains two residual blocks at each resolution, and a downsample and upsample layer for resolution change.
A self-attention block is applied to the smallest resolution for both encoder and decoder.
For calculating perceptual metric, we use the pre-trained 100 epochs ViT-B model from self-supervised method MoCo v3~\cite{chen2021empirical} by default, and the 3rd, 6th, 9th and 12nd layer are selected for calculating the perceptual loss. We also apply the ResNet50~\cite{he2016deep} model with the perceptual similarity calculated at the end of each stage. We set the perceptual loss weight $\lambda$ to 1 without special noting. More different models for calculating perceptual metric are provided in the experiments section.
The input image size is $224 \times 224$, which is consistent with pre-training image input size, the latent codes are in a resolution of $16 \times 16$. We use EMA vector quantizer as the default quantizer algorithm. The learning rate is set $5e^{-5}$ with batchsize 128. We train the PeCo for 100 epochs and warm up the first 5000 iterations to stabilize the training process. The Adam~\cite{kingma2014adam} optimizer is used with $\beta_1$ and $\beta_2$ set to 0.5 and 0.95 respectively.

\noindent \textbf{BERT Pre-training Setup.}
For computation resource consideration, we use the original ViT-B/16~\cite{dosovitskiy2020image} as the basic architecture of our backbone to validate the effectiveness of the learned visual codebook, as in BEiT~\cite{bao2021beit}. The model is pre-trained for 300/800 epochs with the batchsize of 2048. AdamW optimizer is adopted with learning rate, $\beta_1$, $\beta_2$, weight decay set to $1.5e^{-3}$, $0.9$, $0.999$, and 0.05 respectively. We also apply stochastic depth~\cite{huang2016deep} with 0.1 rate, and block-wise masking strategy for obtaining the corrupted images with the same setup as BEiT~\cite{bao2021beit}.

\begin{table}[t]
\centering
\small
\resizebox{1\linewidth}{!}{
\setlength{\tabcolsep}{0.7mm}{
\begin{tabular}{lcccccc}
\toprule
\multirow{2}{*}{ Methods} & pre-train  & pre-train & \multirow{2}{*}{ViT-B} & \multirow{2}{*}{ViT-L} & \multirow{2}{*}{ViT-H} & \multirow{2}{*}{ViT-H$_{448}$}
\\ & dataset & epochs &  \\
\midrule
\multicolumn{7}{l}{\textit{Training from scratch (i.e., random initialization)}} \\
ViT$_{384}$~\cite{dosovitskiy2020image}  & - & - & 77.9 & 76.5 & -- & -- \\
DeiT~\cite{touvron2021training}        & - & - & 81.8 & -- & -- & -- \\
ViT~\cite{he2021masked}           & - & - & 82.3 & 82.6 & 83.1 & --  \\
\midrule
\multicolumn{7}{l}{\textit{Self-Supervised Pre-Training on ImageNet-1K}} \\
DINO~\cite{caron2021emerging}      & IN-$1$K & 300 &  82.8 & -- & -- & -- \\
MoCo v3~\cite{chen2021empirical}   & IN-$1$K & 300 &  83.2 & 84.1 & -- & --\\
BEiT~\cite{bao2021beit}            & IN-$1$K& 800 &  83.2 & 85.2 & -- & -- \\
MAE$^\dag$~\cite{he2021masked}     & IN-$1$K& 1600 & 83.6 & 85.9 & 86.9 & 87.8 \\
PeCo    & IN-$1$K& 800 & \underline{84.5} & \underline{86.5} & \underline{87.5} & \textbf{88.3} \\
\bottomrule
\end{tabular}}}
\vspace{-3mm}
\caption{Image classification accuracy (\%) comparison on ImageNet-$1$K (IN-$1$K) of different self-supervised methods using various backbones. -B, -L, -H stands for using ViT-B, ViT-L, ViT-H model, respectively.  We report Top-1 accuracy and our method PeCo outperforms previous self-supervised methods. $^\dag$MAE is a concurrent work of our PeCo.}
\label{tbl:cls:imagenet}
\vspace{-3mm}
\end{table}

\section{Experiments}

\subsection{Downstream Tasks}
\noindent \textbf{Image Classification} aims to classify a given image into its corresponding class category. We use the popular ImageNet-1K dataset.
To enable classification,
a global average pooling layer is appended after the pre-trained model.
We finetune the model with 100 epochs and a cosine decay learning rate that warmups to $4e^{-3}$ with 20 epochs and decays to 0.
Following \cite{bao2021beit}, the layer-wise learning rate decay is also used and set to 0.65 by default. For more details, please refer to the supplementary materials.

\noindent \textbf{Semantic Segmentation}
is the task of assigning a label to each pixel of the input image.
We compare on the widely used semantic segmentation dataset ADE$20K$ benchmark~\cite{zhou2017scene}.
Here we employ the   Upernet~\cite{xiao2018unified} as the basic framework. For fair comparison, we follow previous works~\cite{bao2021beit} and train Upernet 160k iterations with batch size set as 16, more details are provided in the supplementary material.

\noindent \textbf{Object Detection and Segmentation.}
Object detection is to
locate objects in a given image and identify each object.
We perform fine-tuning on the COCO objection detection and segmentation with the Mask R-CNN~\cite{he2017mask} framework. Specifically, we add four different scale FPNs to scale the feature map into different size following ~\cite{bao2021beit}. The fine-tuning in conducted with ``1x'' (12 training epochs) schedule and single-scale input on the COCO training set and test the performance on COCO validation set, following the strategy used in Swin Transformer ~\cite{liu2021swin}.

\begin{table}[t]
\centering
\small
\resizebox{1\linewidth}{!}{
\setlength{\tabcolsep}{0.7mm}{
\begin{tabular}{lcccc}
\toprule
\multirow{2}{*}{ Methods} & tokenizer  & tokenizer & BERT pre- & IN-1K
\\ & dataset & params & train epoch & Top-1 \\
\midrule

BEiT~\cite{bao2021beit}      & DALLE(400M) & 53.8M & 300/800 &  82.8/83.2 \\
PeCo   & IN-1K(1.3M) & 37.5M & 300/800 &  84.1/84.5 \\
PeCo$_{lite}$   & IN-1K(1.3M) & 25.7M & 300/800 &  84.0/84.5 \\

\bottomrule
\end{tabular}}}
\vspace{-3mm}
\caption{Tokenizer comparison with BEiT. Here we report and tokenizer training dataset and parameters. PeCo$_{lite}$ is a lite-weight version of PeCo that reduces the feature dimension of tokenizer to half.}
\label{tbl:cls:tokenizer}
\vspace{-3mm}
\end{table}

\subsection{Comparison with previous works}
We first compare our PeCo with previous state-of-the-art works. Here we report ImageNet-1K results with various model size. For object detection on CoCo and semantic segmentation on ADE20K, we use ViT-B as the backbone and pre-train on ImageNet-1K with 300 epochs.

\noindent \textbf{Image Classification.}
The Top-1 accuracy on ImageNet-1K classification is reported in Table~\ref{tbl:cls:imagenet}.
We compare our method with 1) ViT~\cite{dosovitskiy2020image} and DeiT~\cite{touvron2021training} that are supervisedly trained from scratch with random initialization;
and 2) MoCo v3~\cite{chen2021empirical}, DINO~\cite{caron2021emerging}, BEiT~\cite{bao2021beit} that are previous state-of-the-art self-supervised learning methods; and 3) the concurrent work MAE~\cite{he2021masked}.
It can be seen that our model (PeCo) significantly improves the performance compared with the models trained from scratch, suggesting the effectiveness of pre-training.
Compared with prior self-supervised pre-training models in base-size,
our model achieves the best performance.

We first report our model pre-trained with 800 epochs, reaching 84.5\% Top-1 accuracy,
1.3\% higher than BEiT and 0.9\% higher than MAE,
This validates that our perceptual codebook is indeed beneficial for pre-training. Furthermore, we validate the effectiveness of our perceptual codebook on large models, \eg ViT-L and ViT-H.
The results are also reported in the Table\ref{tbl:cls:imagenet}, showing significantly better performance than previous counterparts. Concretely,
our model PeCo-H$_{448}$ achieves the best Top-1 accuracy, \textbf{88.3}\%, on ImageNet-1K without external data, outperforming concurrent work MAE by $0.5\%$. This is a new state-of-the-art result using only ImageNet-1K data.

Table.\ref{tbl:cls:tokenizer} compare the tokenizer with BEiT on the training dataset and model size. Here we report the ImageNet-1K pre-training results with both 300 and 800 epochs. We find that our PeCo gets better performance with a small model size and much less training data. More specifically, our model trained with only 300 epochs gets $+1.3 / 0.9\%$ improvement compared with BEiT~\cite{bao2021beit} pre-trained with 300/800 epochs.
We further propose PeCo$_{lite}$, which reduces the channel dimension of the default PeCo tokenizer to half. This reduces the extra time cost introduced by the tokenizer by about $2\times$,  but still gets a similar performance.

\begin{table}[t]
\centering
\small
\begin{tabular}{lccc}
\toprule
\multirow{2}{*}{ Models} & pre-train  & pre-train  &  ADE-$20K$ \\ & dataset & epochs & mIoU\\
\midrule
\multicolumn{3}{l}{\textit{ImageNet-1K supervised pre-training}} & 45.3 \\
\midrule
\multicolumn{4}{l}{\textit{Self-Supervised Pre-Training on ImageNet-1K}} \\
BEiT~\cite{bao2021beit}          & IN-$1$K+DALLE & 300 & 45.7 \\
PeCo(ours)    & IN-$1$K & 300 & \textbf{46.7} \\
\bottomrule
\end{tabular}
\vspace{-3mm}
\caption{Semantic segmentation mIoU (\%) comparison on ADE$20K$ of different self-supervised methods using the same backbone ViT-B. Our method PeCo outperforms previous self-supervised methods.}
\label{tab:ade20k_results}
\vspace{-1mm}
\end{table}

\begin{table}[t]
\centering
\small
\begin{tabular}{lcccc}
\toprule
\multirow{2}{*}{ Models} & pre-train  & pre-train  &  \multicolumn{2}{c}{COCO} \\ & dataset & epochs & $\text{AP}^{\text{bb}}$ & $\text{AP}^{\text{mk}}$\\
\midrule
BEiT~\cite{bao2021beit}          & IN-$1K$+DALLE & 300 & 42.6 & 38.8 \\
PeCo(ours)    & IN-$1K$ & 300 & \textbf{43.9} & \textbf{39.8} \\
\bottomrule
\end{tabular}
\vspace{-3mm}
\caption{Object detection and instance segmentation comparison in terms of box AP ($\text{AP}^{\text{bb}}$) and mask AP ($\text{AP}^{\text{mk}}$) on COCO of different self-supervised methods using the same backbone ViT-B. Our method PeCo outperforms the strong competitor BEiT~\cite{bao2021beit}.}
\label{tab:coco_results}
\vspace{-3mm}
\end{table}

\noindent \textbf{Semantic segmentation.}
We compare our method with 1) supervised pre-train on ImageNet-1K and 2) BEiT~\cite{bao2021beit}, the state-of-the-art self-supervised learning model.
The evaluation metric is mean Intersection of Union (mIoU) averaged over all semantic categories.
The results are given in Table~\ref{tab:ade20k_results}.
We can see that our model (PeCo), without involving any label information during pre-training, achieves better performance than supervised pre-training.
In addition, it can be seen that the proposed model also gets superior performance compared with self-supervised BEiT~\cite{bao2021beit}, further verifying the effectiveness of the perceptual codebook.

\begin{figure*}[t]\centering
\includegraphics[width=0.995\linewidth]{fig/vis5.pdf}\vspace{-.7em}
\caption{Example samples of reconstruction task on ImageNet-$1k$ using BEiT and our PeCo. For each sample,
the first image is the original image, the second one is the corresponding masked image, the third one is the reconstruction from BEiT~\cite{bao2021beit}, the last is the reconstruction from perceptual codebook (PeCo).}
\label{fig:mask_prediction_com}
\vspace{-2mm}
\end{figure*}

\noindent \textbf{Object detection and segmentation.}
We further investigate our transfer performance on object detection and segmentation.
We compare with the strong competitor BEiT~\cite{bao2021beit} on this dataset.
The evaluation metric is box AP for detection and mask AP for segmentation.
The comparison is presented in Table~\ref{tab:coco_results}.
Again, our model (PeCo) achieves better performance due to the learned perceptual codebook.

\begin{figure}[t]\centering
\includegraphics[width=0.995\linewidth]{fig/patch2.pdf}\vspace{-.7em}
\caption{Visualization of image patches that produced the same codeword in different codebooks (from top to bottom): DALL-E, PeCo w/o $\mathcal{L}_{percep}$, and PeCo. We observe that codewords from our PeCo correlated to high semantic patches while two baselines correlated with low-level contents (\eg, texture, color, edge).}
\label{fig:code_correlated_patch}
\vspace{-1mm}
\end{figure}

\subsection{Analysis of Perceptual Codebook}
In this section, we ablate our perceptual codebook by using the setting of self-supervised pre-training on ImageNet-1K. For time consideration,
the number of epochs for both pre-training and fine-tuning are set as 100.

\noindent \textbf{Semantics of the Codewords.}
The most important question would be:
\emph{will the learned perceptual codewords have (more) semantic meanings?}
To answer this, we designed experiments to provide both visual and quantitative results.

First,
we visualize the image patches that are corresponding to the same codeword. We conjecture that such image patches will present the same certain semantic meanings.
We compare with two baselines: (1) a DALL-E~\cite{ramesh2021zero} codebook that is trained on about $250M$ private data and (2) a variant of our model without using the perceptual similarity. We  obtain a collection of codewords from images in ImageNet validation set.
Then we randomly choose some codewords and crop their correlated patch in the original image. The results are shown in Figure~\ref{fig:code_correlated_patch}.
We can see that our codes are highly related to semantic meanings, \eg, comb and wheel shown in the figure, while codes from baselines are often correlated with low-level information (\eg texture, color, edge).

\begin{table}[t]
\centering
\footnotesize
\begin{tabular}{l|cc}
\hline
Methods & LinearP. on codewords  & Classifi. on recon. \\
\hline
DALL-E~\cite{ramesh2021zero} & 6.1    &  18.2 \\
PeCo(w/o $\mathcal{L}_{percep}$) & 10.2  & 17.9  \\
PeCo(ours) & 29.7   &  51.7 \\
\hline
\end{tabular}
\vspace{-3mm}
\caption{Evaluation of the semantics of the codewords
from linear probling accuracy (\%) of codewords on ImageNet-$1$K and classification accuracy (\%) on the reconstructed ImageNet validation images using Deit-T~\cite{touvron2021training}. We compare our method with DALL-E~\cite{ramesh2021zero} and the variant of ours without the perceptual loss. Ours performs the best indicating the semantics of the learned codewords.}
\label{tab:vae_acc}
\vspace{-2mm}
\end{table}

Second,
we quantitatively evaluate the codewords' semantic from two aspects.
(1) We use the codewords of the image as features for classification. An average pooling is conducted over the quantized codewords of the image and we test its linear probing accuracy over ImageNet dataset.
(2) We use an ImageNet-1K supervisedly pre-trained DeiT-T~\cite{touvron2021training} (72.2\% Top1 accuracy on clean ImageNet val set) to test the classification accuracy over the reconstructed images.
We compare with the variant without using the perceptual similarity.
The results are given in Table~\ref{tab:vae_acc}.
We find that our perceptual codewords get much higher accuracy for both linear evaluation on codewords and classification on the reconstructed images.
This indicates that our perceptual codebook has more semantic meanings and benefits the image reconstruction process.
We also provide a visualization of the masked region prediction using BEiT~\cite{bao2021beit} and our PeCo in Figure~\ref{fig:mask_prediction_com}, showing that our PeCo, with the aid of perceptual codebook, is able to make more semantic predictions for the masked region.

\noindent \textbf{Deep Architectures for Perceptual Similarity.}
Another key question would be:
\emph{will the deep architecture from which features are extracted for calculating perceptual similarity affect the perceptual codebook learning and thus affect the pre-training performance?}
Therefore, we investigate two different deep architectures: convolutional-based backbone ResNet50~\cite{he2016deep} and Transformer-based model ViT-B~\cite{dosovitskiy2020image}.
Since our codebook is designed for pre-training, we study the self-supervised models in order to enable pre-training over unlabeled dataset.
The results are reported in Table~\ref{tab:peco_different_percep}.
We can see that in our scenario, using convolution-based or Transformer-based network achieves similar performance.
In addition, we also report the results using the classical supervised (\ie using label) trained VGG~\cite{simonyan2014very} in Table~\ref{tab:peco_different_percep}. It can be seen that using supervised model for perceptual metric achieve comparable performance as self-supervised model.

\noindent \textbf{The Effect of Training Dataset for Perceptual Codebook.}
One strong advantage of our PeCo is that we only use the ImageNet-$1$K with $1.28M$ images for the VQ-VAE training.
This is really data efficient compared with DALL-E using $250M$ images from the Internet for the VQ-VAE training.
To further validate this, we train a perceptual codebook over the
ImageNet-$22$K dataset with $14M$ images following our default setting.
The BERT pre-training is still conducted on ImageNet-$1$K.
We show the transfer performance in Table~\ref{tab:peco_training_dataset_com}.
It can be seen that training perceptual codebook with more data seems not helpful for the following BERT pre-training. This is particularly encouraging as we only need to focus on designing a better perceptual codebook, instead of scaling up the dataset.

\begin{table}[t]
\centering
\footnotesize
\begin{tabular}{c|c}
\hline
Deep models for perceptual similarity & acc. on ImageNet-$1$K \\
\hline
Baseline (w/o perceptual similarity) & 82.6\\
Self-supervised ResNet-50~\cite{chen2021empirical} & 83.3 \\
Self-supervised ViT-B~\cite{chen2021empirical} & 83.5 \\
Supervised VGG~\cite{simonyan2014very} & 83.4 \\
\hline
\end{tabular}
\vspace{-3mm}
\caption{The performance comparison when using different architectures for calculating the perceptual similarity. We evaluate on the downstream ImageNet-$1$K classification (\%).
Using different models achieve comparable performance, all better than the baseline without the perceptual similarity.}
\label{tab:peco_different_percep}
\vspace{-2mm}
\end{table}

\begin{table}[t]
\centering
\footnotesize
\begin{tabular}{c|c}
\hline
Training dataset for perceptual codebook & acc. on ImageNet-$1$K \\
\hline
ImageNet-$1$K & 83.3  \\
ImageNet-$22$K & 83.3  \\
\hline
\end{tabular}
\vspace{-3mm}
\caption{Illustrating the effect of training dataset for learning perceptual codebook. We show fine-tune accuracy (\%) on ImageNet-$1$K.
We can see that enlarging training set for learning perceptual codebook does not bring gain for the transfer performance.}
\label{tab:peco_training_dataset_com}
\vspace{-2mm}
\end{table}

\noindent \textbf{Different Architectures for VQ-VAE.}
Here we investigate the performance when using different architectures for VQ-VAE.
We consider several variants of the network architecture.
For encoder, we explore three models: 1) 16$\times$ down-sample encoder (our default setting); 2) 8$\times$ down sample encoder; 3) ViT-B (16$\times$ down-sample). For the 8$\times$ down-sample encoder, we remove one stage and train it with images of 112$\times$112 resolution.
For decoder, we  use the inversed version of the corresponding decoder.
The results on ImageNet-$1$K dataset are shown in Table~\ref{tab:peco_archi}. We observe that CNN based encoders and decoders achieve better results than vision Transformer.
We further reduce the parameters of decoder by decreasing the channel number or decreasing the depth of the network by half.
Results shown in Table~\ref{tab:peco_archi} suggest that reducing the parameters of decoder may not hurt the fine-tuning performance of PeCo.

\begin{table}[t]
\centering\scriptsize
\begin{tabular}{ll|c}
\hline
Encoder of VQ-VAE & Decoder of VQ-VAE & acc. on ImageNet-$1$K\\
\hline
ViT-B & ViT-B & 82.9 \\
CNN(8x) & CNN(8x) & 83.1 \\
CNN(16x) & CNN(16x) & 83.3 \\
CNN(16x) & CNN(16x) (Half Channel) & 83.2 \\
CNN(16x) & CNN(16x) (Half Depth)   & 83.3 \\

\hline
\end{tabular}
\vspace{-3mm}
\caption{Illustrating the effect of different architectures for training PeCo.  We show fine-tune accuracy (\%) on ImageNet-$1$K.
CNN based encoders and decoders achieve better results than vision Transformer.}
\label{tab:peco_archi}
\end{table}

\noindent \textbf{The Loss Weight of Perceptual Similarity.}
In the experiments, the loss weight $\lambda$ in Eqn~\ref{eq:recloss} is set as 1.
Here we present the performance under various values of $\lambda$ among 0, 0.3, 1, 3, 10.
The results are shown in Table~\ref{tab:peco_different_percep}.
We can see that using perceptual loss yields 83.3\% accuracy outperforming 82.6\% from the model without perceptual loss.
However, further
enlarging the loss weight gets performance drop.
One possible explanation is that large perceptual loss leads the model to pay more attention to semantic while lose some local details, while a good codebook for BERT pre-training needs both semantic and local details.

\begin{table}[t]
\centering\small
\begin{tabular}{l|ccccc}
\hline
$\lambda$    & 0    & 0.3 & 1 & 3 & 10 \\
ImagenNet-$1K$ & 82.6 & 83.3 & 83.3 & 82.9 & 82.8 \\
\hline
\end{tabular}
\vspace{-3mm}
\caption{Illustrating the effect of loss weight of perceptual similarity.
We show fine-tune accuracy (\%) on ImageNet-$1$K.
Enlarging the loss weight can not get consistent improvement, may due to the loss of local details.}
\label{tab:pixel_vs_token}
\end{table}

\subsection{Discussions}
We present several in-depth discussions about the proposed model in this section.

\noindent \textbf{Implicit vs. Explicit.}
The key contribution of our paper is improving the perceptual level of the discrete visual tokens for the subsequent pre-training.
We have successfully demonstrated that through a simple strategy, \ie enforcing perceptual similarity over images.
One may think that it seems quite implicit for learning perceptual codebook by constraining on images instead of directly exploiting some constraint over the codebook.
Indeed, we also experiment in two explicit ways:
1) supervised classification loss over the codewords; 2) constraining a momentum contrastive loss over the quantized codewords through data augmentation in a self-supervised way.
We hope that leveraging those forms of high-level classification objective
may encode some semantics into the codewords.
But empirically we found that such explicit ways are not as effective as the proposed implicit strategy.
The results are reported in Table~\ref{tab:percep_mech}.
We conjecture that the codebook may learn global semantics from the classification/contrastive loss and thus fail to differentiate different codewords, which is not suitable for pre-training.
In contrast, the features from a pre-trained deep model contain rich and local semantics.
However,
we still think that it will be more intuitive to explicitly learn the perceptual codebook and we leave that as future work.

\begin{table}[t]
\centering
\small
\begin{tabular}{c|c}
\hline
Perceptual mechanism  & Top-1 acc. on IN-$1$K\\
\hline
Classification loss on codewords  & 82.7 \\
Contrastive loss on codewords  & 82.7 \\
Perceptual loss on images & 83.3 \\
\hline
\end{tabular}
\vspace{-3mm}
\caption{Performance comparison of our implicit way (perceptual loss on images) and the explicit ways (classification/conrastive loss on codewords) for improving the perceptual level of codebook. We show fine-tune accuracy (\%) on ImageNet-$1$K.}
\label{tab:percep_mech}
\end{table}

\noindent \textbf{Perceptual Loss \emph{vs.} GAN Loss.}
The perceptual loss is widely used in generation tasks with the goal of improving
the image quality. We ask the question that
\emph{is there a positive relation with the image quality and the perceptual level of the codebook.}
In order to explore this, we adopt another technique, adversarial loss in Generative Adversarial Nets(GANs)~\cite{goodfellow2014generative}, which has been proved to be effective in enhancing the reconstructed image.
Specifically,
we add a patch-based discriminator D~\cite{li2016precomputed}, aiming to make the original image and the reconstructed one indistinguishable.
The adversarial loss is,
\begin{align}
	\min_{Enc,\{\mathcal{V}\},Dec}\max_{D}\mathcal{L}_{\text{adv}} = \mathrm{log} D(x) + \mathrm{log} (1 - D(\hat{x}).
	\label{eq:GAN_D_loss}
\end{align}
We add this loss with a suitable weight 0.4 to Eqn~\ref{eq:recloss}  and use the learned codebook for pre-training.
The resulting performance is shown in Table~\ref{tab:adv}.
We can see that adversarial loss brings little gain to the transfer performance of pre-training.
We also show the image reconstruction results in Figure~\ref{fig:imagereconstruction}.
It can be easily seen that with the adversarial loss, the image looks more photo-realistic, presenting finer details.
Therefore, we may arrive at the conclusion that improving the image quality may not necessarily improve the perceptual level of the codebook.

\begin{figure}[t]\centering
\includegraphics[width=1\linewidth]{fig/gan_comparison_v2.pdf}\vspace{-.7em}
\caption{Example samples on image reconstruction results with different loss types. For each sample, the first column is the input, the second to fourth are with different losses. They are pixel loss, pixel loss and perceptual loss, pixel loss and both the perceptual and the adversarial loss, respectively. With an extra adversarial loss, the image quality is further improved (comparing the fourth with the third column). While the
adversarial loss brings little gain to the transfer performance as shown in Table~\ref{tab:adv}.}
\label{fig:imagereconstruction}
\vspace{-3mm}
\end{figure}

\begin{table}[t]
\centering
\begin{tabular}{l|c}
\hline
Loss functions  & Top-1 acc. on IN-$1$K\\
\hline
$\mathcal{L}_{pixel}$ & 82.6 \\
$\mathcal{L}_{pixel}+\mathcal{L}_{percep}$  & 83.3 \\
$\mathcal{L}_{pixel}+\mathcal{L}_{percep} + \mathcal{L}_{adv}$  & 83.3\\
\hline
\end{tabular}
\vspace{-3mm}
\caption{Performance comparison when using different loss functions. We show fine-tune accuracy (\%) on ImageNet-$1$K. Adding an extra adversarial loss which is useful for improving image quality brings little gain to the transfer performance.}
\label{tab:adv}
\vspace{-3mm}
\end{table}

\section{Limitations.}
We are aware that our BERT-style pretrained models still have one key limitation, i.e., the linear probing accuracy is lower than the contrastive learning pretrained counterparts. Similar issues are also observed in BEiT\cite{bao2021beit} and the concurrent work MAE \cite{he2021masked}. And we empirically observe that longer pretraining epochs will alleviate this issue to some extent. However, to fundamental address this issue, better pretraining strategies need to be studied further, e.g., bridge contrastive learning and BERT pretraining.

\section{Conclusion}
The difference between image and language has always been in constant discussion and received a lot of attention in recent years.
Inspired by the observation that the natural representation of images is not a semantic decomposition,
we propose a simple yet effective strategy to obtain perceptually discrete tokens, beneficial for BERT pre-training of vision transformers.
We present extensive comparisons on various downstream tasks. Our results indeed indicate that discretizing images into tokens is not enough, the perceptual level of the visual token also matters. We hope the deep analysis in our work will lead to a broader exploration of this perspective.

\section*{Acknowledgement}
We thank many colleagues at Microsoft for their help and useful discussions, including Hangbo Bao, Li Dong and Furu Wei.

{\small
\bibliographystyle{ieee_fullname}

redpajama_set_name: RedPajamaArXiv


text:
<?php

namespace haqqi\metronic\widgets;

use rmrevin\yii\fontawesome\FA;

class Breadcrumbs extends \yii\widgets\Breadcrumbs
{
    public $separator;

    public $itemTemplate = "
{link} {separator}
\n";

    public $activeItemTemplate = "
{link}
\n";

    public function __construct(array $config = [])
    {
        $this->separator        = FA::icon('angle-right');
        $this->options['class'] = 'page-breadcrumb';

        parent::__construct($config);
    }

    protected function renderItem($link, $template)
    {
        $item = parent::renderItem($link, $template);

        return strtr($item, ['{separator}' => $this->separator]);
    }
}

redpajama_set_name: RedPajamaGithub


I have reflected previously on the fact that the goal of redemption is the resurrection of the physical body to eternal life, that is, to immortality.
John 3:16, though it does not use the term "immortality" when it speaks of "everlasting life" is referring to bodily immortality. Actually, when the Bible speaks of "life," it often is referring to bodily immortality, not to consciousness in a spiritual form other than the body.
I think there is at least one reason why the focus of the Gospel of the Kingdom has been changed to the Gospel of Heaven, with a dismissal of the emphasis on immortality in the physical body.
The reason for the change in the Gospel may have been the influence of other religions that held out the prospect of some kind of paradise after death. Foremost among these would be the religion of Gnosticism, which I believe pervades Christian thinking to the present hour.
Gnosticism is based on the idea that specialized knowledge is the most important factor if we wish to pass from the corrupt material world into a higher form of existence.
The dualism present in Gnostic teaching is that the spiritual world is good and the physical world is bad. I think this idea, that spirit is good and the physical is bad, is embedded in our thinking. The truth is, when God created the physical world He pronounced it very good. The sin and corruption came not from the physical world but from Satan, from the world of spirits.
Obviously if our understanding of the purpose of salvation is to bring us to a more exalted spiritual plane, then the resurrection of our body, or any need to prepare for the resurrection of our body, would be given minor attention or ignored.
The Christian Scriptures do not emphasize a spiritual kingdom but a physical-spiritual kingdom. We begin as physical creatures with a spiritual inward nature. The first step in entering the Kingdom of God is to be born again. This means the Substance of Christ must be born in us.
Then we must, through Christ, overcome many obstacles if we are to continue to abide in Christ. The Substance of Christ, His body and blood in us, responds to our efforts to overcome by growing in the image of Christ until we have a greatly strengthened inner man.
This strengthened inner man makes it possible for us to be filled with all the fullness of God so that when Christ returns, our physical body can be raised from the dead and clothed with a body from Heaven, which has been fashioned by the Holy Spirit as we have pressed forward in Christ.
When the above states that "Christ may dwell in your hearts through faith," the meaning is not that Christ doesn't actually dwell in our heart, we just have faith that He does. It means, rather, that as we in faith keep looking to Christ for our every thought, word, and action, Christ dwells in our heart in actuality.
When Christ next comes to earth, all those who are part of Christ will return with Him. He will come from Heaven with His armies, and pause in the air overhead.
Then the saints of all ages will descend to earth, their bodies, as I have stated, will be assembled and raised, and then their raised bodies will be clothed upon with a body from Heaven that has been manufactured by the Holy Spirit and modified by their behavior on the earth.
If we are alive at the time of the Lord's coming, the blood will be withdrawn from our physical body and replaced with the Spirit of God. Then our body will be clothed with our house from Heaven.
Stop and consider: the saints who return with the Lord Jesus have already been in Heaven—some of them for thousands of years. If residence in Heaven is our goal, and there therefore is no need for a bodily resurrection, why then do these people return with Jesus and pick up their body from the ground or ocean? This makes sense?
Today, in Christian thinking, we emphasize the inward nature of a person, his spirit and soul. What sort of inward nature did Adam have? What did God breathe into him. It may have been blood, because Leviticus tells us the life of the flesh is in the blood. Or it may have been air, because air and blood are closely related in flesh-and-blood life.
In any case, it is likely that Adam had a soul, a personality, and a spirit that could communicate with God.
But when Adam disobeyed God, God did not mention the death of his spirit and soul, but of his body.
This is speaking of Adam's body. It is not likely that Adam's soul and spirit were formed from the dust. So we deduce from this that we do not know where Adam's soul and spirit went when they died. Are they still present in the spirit world?
Also, and relevant to our theme in this essay, the emphasis in Genesis is not on the spirit and soul of Adam but on his body. The moment He sinned, corruption and death began to work in his body.
The Bible does not count us as alive until we are raised from the dead and clothed with Divine Life, the Life of Christ. Since this concept has a bearing on my emphasis in the present essay, let me present two passages that support my assertion.
Now, let us think carefully about the passage above. Adam died physically, and so we all suffer physical death. Christ, the Firstfruits of the Church, died physically. Then Christ was made alive. I believe most of us would assume this means He came alive physically after His physical death.
But the second part states that those who belong to Christ will be made alive when He returns. Will be made alive! Is this saying that we all are dead until Christ comes and raises our body from the grave?
If this is the case, then the Bible is saying that we are dead until our body is resurrected, even though there is eternal life in our inward nature. Would you agree with this?
I think these expressions are referring to the resurrection of the body.
Perhaps, because of the influence of Gnosticism, we are under the impression that God dwells in our inward nature. But it is the body that is the temple of God. I believe, in fact, that the holy city, the new Jerusalem, is fashioned from the resurrected bodies of the saints.
Think about that. We would say the important part of Adam and Eve was their inward nature. This is Gnostic thinking. God said they were dust, ignoring their inward nature. Can you see how heavily our thinking has been warped?
I realize that Paul said sin dwells in our body. This is true. Our sinful nature dwells in our body. In fact, this is the point. It is not the body itself that is sinful, because the body is morally neutral. It is the spirits that dwell in the body that cause the body to have the compulsions that it does.
Paul directed us to cleanse ourselves from all filthiness of the flesh and spirit. The filth of the flesh and spirit are spirits.
When God created Adam and Eve they had no filthiness in their flesh or their spirit. The filth came down from the heavenlies when Satan lied to Eve, persuading her to sin. The original people had no sin nature until Satan managed to persuade the two innocent human beings to sin.
We do not think in these terms of our being dead until we are raised from the dead, but the Apostles knew this to be a fact.
It is true, as Paul said, that our spirit is alive because of righteousness but our body is dead because of sin. Our spirit is alive because it has been touched by the Spirit of God as a result of our obeying God by receiving Jesus Christ as our Savior. It is God's intention, if we put to death the actions of our sinful nature, to make our body alive.
All true life is of the Spirit of God. This is why God regards our present body as dead.
What we have in our inward nature, in addition to the Substance of Christ being formed in it, is a portion of the Holy Spirit. This portion of the Holy Spirit is a sort of down payment on the fullness of the Spirit that will be given to us if we continue obeying Christ. Therefore, that part of us is alive.
But if we do not continue to obey Christ, if we continue to live according to our sinful nature, we will die inwardly. The down payment of the Spirit will be taken from us. Then we will have a dead inward nature and a dead body, as far as God is concerned. The wages of sin for the sinning believer is spiritual death in the inward nature.
Notwithstanding all this, the hope of the Gospel of the Kingdom is not life in our inward nature, it is life in the body—that which we always have understood to be "life" until our thinking was perverted by the influence of the religion of Gnosticism. Our hope is to live again in our body. This is the hope of Easter–we shall live again in our body in a state of immortality.
The entire fifteenth chapter of the Book of First Corinthians is about the resurrecting of our body. This raises the possibility that even as early as Paul's day, people were questioning the resurrection of the body of the believers.
Gnosticism would have nothing to do with the idea that God would send a Savior in the flesh!
So the massive errors of our day, the discounting of the need for believers to prepare themselves for the resurrection, and the view of eternal residence in the spirit world, in Heaven, as being the goal of salvation, may stem in large part from the religion of Gnosticism. I think it may be true that many Christian people regard spiritual as good and physical as evil, this being the message of the religion of Gnosticism.
The "rapture," a totally unscriptural myth, is a humanistic concern for people, teaching them that they need not worry that they might suffer from Antichrist or the Great Tribulation. The reason the believers gladly accept the unscriptural teaching of the rapture is that they wish to escape trouble and pain. The "rapture" has nothing to do with wanting to be with Jesus.
The concept of eternal residence in Heaven in the spirit world, reclining in our mansion with no responsibilities, being the goal of our salvation, has no basis whatever in the Bible. Do you notice that in this pattern there is no place for the resurrection of the body? Who needs a resurrected physical body if we are to live permanently in a mansion in the spirit world?
Needless to say, a great part of our Christian thinking is not in accordance with the Scriptures. It is difficult for me to see how we ever are going to escape the deadly influence of Gnosticism, but I know such a change will take place in the not too distant future. The Lord Jesus prayed that we shall be one in Him in the Father, and with each other, and this shall come to pass—and eventually in physical forms.
Let us compare the two different gospels and decide which is in accordance with the New Testament. One is the Gospel of Heaven. The other is the Gospel of the Kingdom of God, which includes bodily immortality.
Perhaps you can think of additional roles in the Kingdom of God that are especially meaningful to you.
Having set forth God's goals and needs, so we can determine whether the Gospel of Heaven, or the Gospel of the Kingdom, including bodily immortality, is most likely to enable us to satisfy these goals and needs, we now will examine the two gospels to determine what our conclusion might be.
Once we "accept Christ" we need not be overly concerned with godly behavior. We should try to please Christ to show our appreciation for Him and His sacrifice, but even if we fail to put to death the actions of our sinful nature, our eternal residence in Heaven after we die is assured.
We are not saved by works of righteousness we have done. We are saved by a Divine "grace." Grace is an alternative to righteous behavior. Grace, as it is preached today, is one part of a "new dispensation." The new dispensation views our behavior in a different manner as compared with the prior covenants.
All human beings of time past and of today are sinners. All our works are filthy rags. There are none who are righteous, no, not one. Any attempt on our part to live righteously is an affront to the grace of God. It is a legalistic Pharisaism, unworthy of God's endless mercy and grace.
We are eternally saved and are overcomers by virtue of the fact that we have made a profession of belief in Jesus Christ. God holds us to be perpetually righteous as soon as we make the correct profession of faith in Christ.
Believing in Christ does not have to be accompanied by obedience to the commands of Christ or His Apostles. We are found righteous by faith (actually mental assent) and faith alone. Our behavior is not a significant factor in our salvation. It is a free ride to Heaven, we might say.
As far as Heaven is concerned, this is a wonderful Paradise. There is a mansion for every saved person. There are streets of gold. The inhabitants play harps to pass the time. They may praise God for a thousand years. They have no responsibilities other than to praise God. They spend eternity "singing and shouting and dancing about," as one elderly person explained to me.
Notice how often the term "accept Christ" appears when we discuss the grace-Heaven-rapture trilogy. The expression "accept Christ" does not appear in the Bible, as far as I can tell.
Sometimes the doctrine of the rapture is accompanied by the idea that it will occur at "any moment." Also, there is a disagreement whether it will occur before the Great Tribulation; or in the middle of the Great Tribulation; or at the end of the Great Tribulation. In the case of the last view, it would not serve as an escape from the Great Tribulation.
I am sure that there are numerous other subjects that are preached by pastors and evangelists, especially as we are nearing a presidential election. Also, there almost always is great stress on "going forth to save a lost and dying world," or on personal evangelism. However, grace-Heaven-rapture appears to be quite common, according to letters I receive from people who are looking for a church that teaches the Bible.
Now that I have imperfectly described some of the current emphases, let me elaborate a bit on the trilogy from what I believe to be a more scriptural viewpoint. Remember, our purpose here is to determine which of the two gospels, the Gospel of Heaven or the Gospel of the Kingdom, is most likely to accomplish God's goals and needs.
Paul's use of the word "grace" cannot possibly be interpreted as an alternative to godly behavior. In several instances Paul warned the believers that if they continue in sin they would die spiritually and would not inherit the Kingdom of God.
Paul maintained that we are going to reap what we sow. If we sow to the Spirit of God we shall reap eternal life. If however we live according to our sinful nature we are not going to inherit the Kingdom of God.
It is a point of interest that Paul referred to our inheriting the Kingdom of God. Never one time did Paul suggest that if we sinned we would not go to Heaven. It may be that my reader views Heaven and the Kingdom of God as being the same thing. They are not. The Kingdom of God is a government. Heaven is a place. The Kingdom of God governs equally well in Heaven and upon the earth.
When Paul stressed that we are saved by faith and not by works, he was addressing Jews primarily. The problem with the Jews was they always had been taught that righteousness comes only as we obey the Law of Moses. Paul was teaching them that God can assign righteousness to someone who was not under the Law, and Paul used Abraham as an example.
Divine grace means that God will assign righteousness to someone who leaves the Law of Moses and follows Christ. Grace assuredly is not God's way of ascribing righteousness to someone who is continuing to sin. Such a position would be completely contrary to the New Testament.
Although the current employment of the term "grace" ordinarily refers to forgiveness and righteousness apart from righteous behavior, the word is used in the Bible to mean any assistance we receive from God.
Notice in the verse above that God may accept the wicked for a season in the hope they will learn to behave righteously. Isn't that just as true today?
The second doctrine of the trilogy is the idea that we are "saved" so we will go to Heaven when we die. The New Testament, from the time that John the Baptist heralded the Messiah, spoke of the Kingdom of God coming to the earth, not of believers going to Heaven.
I do not know when the Gospel of Heaven began or how it began. It undoubtedly, as I suggested previously, resulted from the religion of Gnosticism.
Since Paul's teaching can be warped to mean that how we behave in the flesh does not matter, which is akin to Gnostic teaching which stresses specialized knowledge (reminiscent of our present-day stress on professing Christ rather than behaving righteously), there may have been numerous Christian believers who converted to Gnosticism, believing it to be much the same religion Paul was teaching.
We need to rethink "Heaven." First of all, the spirit world is a very large area, larger than we have any idea. All the dead from the beginning of the world are there.
Mount Zion, the heavenly Jerusalem, is the home of the elect, the Royal Priesthood. After the final resurrection of the dead, the heavenly Jerusalem will descend through the new sky and be installed forever upon the new earth.
The heavenly Jerusalem is the government of the Kingdom of God. The saints, who will rule from this holy city, will govern the nations of saved people on the new earth. The saved people from the nations will be subjects of the Kingdom of God in that they will be governed by saints who have inherited the Kingdom.
The above two passages tell us about Heaven. There are no mansions. The city is the government of the Kingdom of God and rules the nations of saved people who once were part of the nations of the earth.
The new Jerusalem, when it descends to the earth, may be composed of the resurrected and glorified bodies of the saints. I believe the trees of life are the saints in whom Jesus has been formed and is dwelling. Also, the River of Life is flowing from within the saints. The saints, holy ones, will govern the peoples of the new earth for eternity.
It is of interest that we already have come to Mount Zion, the Bible says. Our inward spiritual nature already is in Mount Zion, in Heaven. Paul says we are dead (referring to our body) and our life is hidden with Christ in God—that is, in Heaven. We already are there with all members of the Body of Christ.
The problem is our body. Actually we are not fighting our way upward but downward. God has given us this wonderful physical body, a possession not shared by any other creature of God, apparently. We, from our heavenly vantage point, are to put to death the actions of our body as it is driven by our sinful nature.
The prize is our body. The Christian fight of faith is to regain possession of our body!
But in the beginning we permitted Satan to advise us. In that day our body died, although it was not evident for a thousand years. Ever since then Satan has actively influenced our body. It is our own fault!
Now, through Jesus Christ, who Himself is the Resurrection and the Life, we have an opportunity to regain that which we forfeited in the beginning.
What does this mean in practical terms to us as we struggle along in our discipleship? It means that we are to prepare ourselves for the coming of Christ, that we might be ready to be resurrected and glorified, and to ascend to meet Him in the air. Christ will be poised in the air with the war stallions, waiting for us to rise to meet Him, mount a war stallion, and descend with Him to bring the Life of God to the dead creation and justice to the nations.
This is the coming of the Kingdom of God to the earth, of which both John the Baptist and the Lord Jesus spoke. We might note that all of the Lord's parables are about the Kingdom of God, not about Heaven.
What does it means to prepare for the coming of Christ? It means to gain victory over our trust in the world, the lusts and passions of our flesh and spirit, and to set aside our own will and ambitions that we may learn to live by the Life of Jesus.
Since you are to appear with Christ, because of this, put to death the actions of your sinful nature. This is how we prepare ourselves to be resurrected when Jesus appears and to work with Him at the task of installing the Kingdom of God upon the earth; and at the task of preparing all the inhabitants of the heavenly Jerusalem for their priestly ministry on the new earth.
And as for the unscriptural "rapture"? It is not included in the Gospel of the Kingdom. It is a modern-day delusion, so totally unscriptural and illogical as to be unworthy of scholarly pursuit.
Now that you have compared the two gospels, the Gospel of Heaven and the Gospel of the Kingdom of God, which of the two do you believe would best prepare us to accomplish God's objectives?
You be the judge. As for me and my house, we believe the Gospel of the Kingdom of God is the best plan for achieving God's purposes, an essential aspect of which is our bodily resurrection unto eternal life.


FEMINIST ART – FEMINIST PERFORMANCES – FEMINIST PUNCH!
Shoebox PR and We Choose Art are pleased to present We Choose Art | A Feminist Perspective 2.0! A group show hosted by guest curators Baha H. Danesh from We Choose Art and Kristine Schomaker of Shoebox PR.
Join us on Saturday, May 21st at MUZEUMM and journey with us through eclectic fine art, photography, sculptures, video installations, and some FEMINIST PUNCH!
This is a FREE event and doors open at 7:00 PM and stay open until 11:00 PM.
Following We Choose Art's philosophy of creating a collaborative community of artistic expression, this unique exhibition continues to document the vibrant and creative culture of modern feminist artists.
Curators Baha Danesh and Kristine Schomaker have created a two-week exhibition to expand on today's modern feminists. They chose both male and female artists whose work continues the feminist dialogue of equality for all. The work presented spans over 40 years with a diversity of artists working in the Los Angeles area. The featured artists are showing works that deal with concepts of race, class, culture, politics, social commentary and representation of women.


text:
Kosciusko County, Indiana - Three And Five-Year-Old Brothers Killed In Tractor-Trailer Accident On Nappanee Farm
By: Paul
Victims Identified As Kenton Hochstetler (3) And Jaylin Hochstetler (5)
Two young brothers tragically lost their lives on a private Nappanee farm the early afternoon of Tuesday, November 19th. Benjamin Sheets, a 26-year-old who was working on the W 1350 N farm with his dad, was the driver of the tractor-trailer involved in the incident.
Reports indicate that the children may have been playing under the tractor-trailer when Sheets turned over the engine and began pulling the truck forward without ever noticing that the boys were there. Sheets indicated he only realized something had happened when he saw the boys in the rear-view mirror of the recently-loaded tractor-trailer.
Emergency crews responded to the scene around 1:45PM but the boys were pronounced dead soon after, with the cause of death being blunt force trauma.
The two victims were part of the family that lives next door to the farm, though the two families are not related to each other. The incident remains under investigation by the Kosciusko County Sheriff's Office.
Hello, I'm attorney Jeff Gibson.
If you or a loved one is a survivor of sexual abuse, I'd be happy to discuss your legal options. Call me on the number below. It costs nothing, and it would be my honor to help you.
A regular contributor to Legal Herald and experienced truck accident attorney Jeff Gibson offered to share the following insight for victims and family members who may have suffered injuries in a similar incident:
"Accidents involving trucks not only change the lives of those directly involved but can often affect an entire family, especially in the instances in which lives are lost. And while it is true that there's nothing that can change the past, victims and their loved ones don't have to suffer the financial burdens that the resulting medical care can entail.
This is because there exists a way for those individuals affected by a truck accident to receive a compensation that covers all the medical treatment: a truck accident lawsuit.
Through a lawsuit, and with the help of a legal representative, victims can fight for a compensation that covers not only medical costs but other damages stemming from the accident that can include loss of companionship and diminished quality of life, among others.
Most victims don't consider filing a lawsuit as their first option, but this path may be the only way for families to feel that justice has been made after a tragedy takes place.
Insurance companies can't be expected to cover all expenses – in fact they usually don't; when the payout limit for a specific policy is reached, victims will stop receiving benefits whether they've been able to get back on their feet or not.
For this reason, those involved in these kinds of incidents should discuss their circumstances with an experienced attorney who will be able to analyze their case, lay out the options, and then fight tirelessly for the rights of their client."
https://wsbt.com/news/local/two-children-dead-after-kosciusko-county-farming-accident
Auto & Truck Accident - Lawyer Commentary

redpajama_set_name: RedPajamaCommonCrawl


text:
Bulb tourism blooms early
Clarissa Place
Published: 6:28 AM March 16, 2015
A civic welcome from Cllr John Hollick was given to a coach party from Bournmouth who came to visit Sidmouth and see the Sid Vale Association's Valley of a Million Bulbs project . Ref shs 0127-10-15AW. Picture: Alex Walton - Credit: Archant
Tourists have been arriving in Sidmouth to see the town's 'Valley of a Million Bulbs'.
A coach of more than 80 gardening enthusiasts travelled down the A35 from Bournemouth to see the daffodils at the start of the flowering season.
After 20 years of tours, UTE operator Neil Goodrich returned to Sidmouth after researching the background to the bulbs for the 2015 tour brochure.
He had previously brought visitors to Connaught Gardens.
The Valley of Million Bulbs was the idea of banker Keith Owen, who left £2.3million to the town in his will.
Mr Goodrich said: "A lot of people know Sidmouth is a nice looking town to visit. There is no vandalism; it comes across a safe, clean area, a smaller version of Bournemouth."
The operator said that to keep the company's nucleus of customers coming back, they organised a visit to Bridport before travelling to Sidmouth.
Mr Goodrich added: "There were only plans for one trip but we had such a lot of interest that we ran a second coach. The combination of a nice collection of violets and an usual bequest of one million bulbs, which isn't far from home, was of interest to our customers."
Sid Vale Association president, the Reverend Handel Bennett joined the tour to show some of the bulb sites. Due to recent frosts and cold weather, the bulbs were not in full flower but visitors have given positive feedback.
He said residents should look out for new areas coming into bloom, including on the verges at Sidford Road; Bickwell Valley; Trow Hill; the parish churchyard; Peak Hill; and at the top of Four Elm Hill.
Mr Goodrich said that to keep the brochure varied, a trip to see the bulbs would not be a yearly occurrence.
Welcoming visitors to the town was Sidmouth Town Council chairman John Hollick. He said that he was 'flattered' by the tourists' interest in the bulbs.
He added: "We who live here are extraordinary fortunate to have so many locals who work as volunteers for the town: The Sid Vale Association, our youth organisations, schools, our Sidmouth in Bloom group and others.
"Tourism is our main industry - our economy benefits and depends on keeping this town in pristine condition."


Check out this video with Scott Bedbury Founder and CEO at Brandstream. He is keynote speaker at Tendensdagen and will lecture in the topic: Creating a Superhuman Brand – Unleashing human potential and meaningful brand storytelling in the New Brand World.
Another speaker I would recommend to check out is: Susan Fournier, Associate Professor of Marketing, Boston University. Speech topic: Getting Inside the "R" in Customer Relationship Management. The speakers are impressive but for me this day which is arranged on the 25th of october this year is one of the best tribe meetings in Scandinavia. Read and mingle more at Tendensdagen.


text:
Digging into Data: Global Currents
Visibility of Knowledge
Detail, Canterbury Psalter (1147)
What can you learn from the visual features of a page?
This is the question that lies at the centre of our digging into data project, the awards for which were announced yesterday. A vast amount of our textual heritage has so far been resistant to large-scale data analysis, whether it is non-western scripts or early- or pre-print documents. These are works that don't lend themselves well to current OCR technology and thus to the usual approaches of data and text mining – which makes data preparation and analysis an extremely long and drawn out process.
Partnering with Mohamed Cheriet at the Synchromedia Lab at the École de technologie supérieure in Montreal and Lambert Schomaker of the Artificial Intelligence Lab at Groningen University in the Netherlands, we will be applying new image-processing techniques to better understand the relations between pages at the visual level. Rather than OCR a text and compare the relations between words, we want to know how it is that pages correlate with one another through their visual features. How much semantic information is contained in the visual dimensions of a page and what other kinds of information is encoded there — whether it be indeces of scribal communities or perhaps styles of ornamentation that marked different periods or cultures? Although we think of texts as things that we read, texts are first and foremost visual objects. Our goal is thus to account for new kinds of texts and new kinds of textual information that have so far been missing from the big data turn.
This is just the first step, however. Our second principal question is: knowing something about the visual relations between pages, can we create larger maps of connections between texts in corpuses of writing that represent different world cultures at different historical junctures? Can we understand the networks of literary exchange that existed and helped define these different cultural formations of the past? To this end, we are bringing together four different databases for our analysis that have been curated by researchers at McGill and Stanford: post-classical Islamic philosophy (Robert Wisnovsky), Chinese Women's Writing from the Ming-Qing Dynasties (Grace Fong), the Anglo-Saxon Middle-Ages (Elaine Treharne), and the European Enlightenment (Andrew Piper/Mark Algee-Hewitt). These collections bring together writings from diverse spans of both time and space, from 1050, the beginnings of Islamic post-classical philosophy and the Anglo-Saxon high middle ages, to 1900, the onset of various global modernisms across China, the Middle East and Europe. Together they comprise 1,194,000 pages. Uniting each of these domains, we would suggest, is the shared sense of being a culture in transition. Our aim is thus as capacious as it is straightforward: how are these different transitional periods and places characterized by networks of shared ideas?
Partnering with Derek Ruths of the Network Dynamics Lab at McGill University, we will be asking some of the following questions regarding our different cultural collections:
what texts were most central to a particular epoch? what do such texts have in common with one another?
What texts play a mediating role between different communities of writing within a corpus? To make the bridge between different clusters of texts, what kinds of writing does one most often pass through?
How are these different cultures themselves comprised of different textual communities? Do we find that different periods or cultures are marked by different degrees of communities (many smaller communities versus fewer strong concentrations)?
How do ideas move across time? Are there strong correlations between works from similar time periods or do we find periods more defined by anachronism or recycling?
These are just some of the questions we hope to answer over the next two years. The value we think is the way this allows us to put into practice a model of comparative globalism, one that places major, often transnational regional cultures from diverse parts of the world in conversation with one another while at the same time preserving the uniqueness of their cultural differences. Our goal is not a flattening of the world into a single, unified cultural account, but the study of the communicative underpinnings that maintain these differences. The quantitative study of literary networks, we argue, allows for a renewed project of comparative inquiry, one that enables artifacts of a very different nature, whether of medium, script, language, or epoch, to be put into conversation with one another. This project would mark the first cross-cultural comparative study of literary networks of its kind.
Digging into Data Literary Networks Visual Language Processing World Literature
Older Post MLA Panel on Macroanalysis
Newer Post Digging into Data Award


Funded with a grant from the Japanese government, DEEP was formally launched in Singapore as one of the events commemorating the ASEF's 10th Anniversary and a year later presented at the 1st Asia-Europe Meeting of Ministers for Education (ASEMME1) in Berlin. Subsequent years proved challenging for the sustainability of the database.
From 2009 – 2012, my team conducted a review of the database to determine its future direction concluding with the report Where to take DEEP?


Interesting, I bought a CZ Shadow II recently for a fun gun. Has a very nice trigger. I can shoot small groups at 10 yards and this bullseye edition has a bushing so should be even better. I'm a red dot guy when it comes to bullseye but for an iron sight 9mm it should be pretty sweet.
Looks to me like it falls into the same category as the USA made Sig Sauer P210 9mm.
I have an older CZ 75 and will have to look into the ACCU bushing. I don't like the fiber optic front sight on either pistol. Seems like a pistol the shooter needs to learn to shoot, like the 1911. It looks good.
I asked a similar question about the 6" Tanfoglio Gold Match extreme pistol. Still very interested in acquiring that pistol in both 9mm and 45acp. Despite the unanimous response of "if you're really serious, get a 1911".
The only absolute negative I've found about the Tanfoglio is the trigger weight around 2.5lb is too light for many competitions so you need to get it higher and could be close for NRA to meet the minimum 2.5lb for "any centerfire".
I have a Shadow II that shoots very straight even in my hands. I looked into the Accu bushing but seeing reports about slide problems because after boring for the bushing there isn't enough metal left. Suggestion was to use the bushing from Cajun Gunworks and it's a simple swap out. Talked to the guy there and he said it would tighten the groups up by maybe an inch at 50yds. The bushing is only $60. Haven't bothered yet since it shoots so well as it is.
On the Tanfo, a spring or two is all that is needed to adjust the trigger weight. Patriot Defense carries a good selection of parts for these. I have a 9mm Stock 3 for Production and a .40 Stock 3 for Limited, both tweaked to have manageable triggers.
I have been using CZ 75 SP-01 with trigger job and CGW bushing as my 9mm BE gun, I also have CZ-75 SAO 9mm as a backup BE gun.
Many shooters in our club switch to CZ-75 after trying mine. The only gun that equals it in our club matches are my CZ-97B (all tuned) in 45ACP. Both capable of less than 2" groups at 50m.


text:
Supplement Feature - February 2012
The ABCs of Aquatic Design
An Alphabetical Stroll Through the Latest Trends
By Dawn Klingensmith
Every year, aquatics facilities gain recognition and win awards for excellence in design. Perhaps the odds of winning are better for places with spectacular or quirky design elements or amenities, such as thrill rides or hot tubs shaped like horseshoes or clovers. Yet apart from these wow factors are straightforward design features that consistently impress judges and, more importantly, win over the public. Foremost among them is a mix of amenities and a design layout that maximizes programming options and appeals to all ages, from toddlers to seniors.
Does your existing or proposed aquatics center have what it takes to be a winner in the eyes of the community? And does it meet objective and subjective measures of what constitutes good design? Here's an A to Z guide of design considerations that streamline operations and make for a better user experience. Also included are aquatics facilities that serve as examples of tried-and-true or progressive design principles.
Access. "There's no need for a public door directly from the outside or the lobby straight onto the pool deck. All access to the pool should be through locker rooms (except for spectator areas, but these should be segregated from the deck anyway)," said Dave Rowland, president of Lutra Aquatics in West Granby, Conn. "Doors leading directly onto the pool deck, which architects seem to love, are just asking for operational problems."
Ages. Design and provide programming with multiple generations in mind. "Our growing senior population means there's more of a need for senior classes as well as wellness and therapy pools," said Melinda Kempfer, business development coordinator, Water Technology Inc., Beaver Dam, Wis. "More and more families are attending together, from toddlers to grandparents."
Bubble Benches. Underwater seating with jets provides an area for parents to relax while supervising children. Bubble benches can be used therapeutically, too, providing a hands-free massage with jets aimed at specific body parts to provide pain relief and muscle relaxation.
Budget. Estimating construction costs for a new facility is difficult, as expenses vary depending on location, availability of materials and other factors. However, the aquatics design firm Counsilman-Hunsaker offers three common ways to prepare an initial budget. First, obtain information from similar operations nearby to get a general idea of cost. Second, use industry averages available in publications that cover construction costs, building standards and other industry data. These resources can provide square-foot cost estimates, and some may take into account local variances. Third, perform a site-specific analysis using a preliminary design and accounting for the types of materials and labor costs in your area. This usually requires a consultant's help.
Chokepoint. Entrances to pools require careful monitoring. "Too many pools, especially school pools, are built with no means to control entry," Rowland said. An access chokepoint helps.
Ceiling height. "High ceilings look nice but cost money for their entire lives. All that air has to be heated or cooled," Rowland said. They can also present maintenance challenges and expenses. At one facility, "the architect didn't include any reasonable way to reach certain lights that are 40 feet above 12 feet of water," Rowland writes on the Lutra Aquatics Web site. "As a result, changing these lamps costs several thousand dollars per bulb."
Aquatics, Design/Architecture
The Air in There
Managing Air Quality in Aquatic Facilities
Feature Article - November 2019
When it's not properly managed, the air in natatoriums and other indoor aquatic facilities can become harmful—especially for lifeguards and frequent users. Learn more about how the industry is trying to tackle this challenge.
inPERSPECTIVE / SURFACES
Eye on Safety
>> New Standard Established for Aquatic Safety Surfaces
Guest Column - November 2019
To ensure you are doing all you can to promote safety and effective pool operations, it is wise to focus on standards for equipment, like the new NSF/ANSI 50 standard, which covers aquatic safety surfaces.
A Splashing Good Time
Inclusive & Multigenerational Splash Play
Supplement Feature - October 2019
Splash play areas continue to evolve, with more ways to make sure people of all ages and abilities can have fun and cool off.
Indoors or Out, Waterparks Offer Fun for All
Whether they are indoors or outdoors, waterparks provide a fun escape for people of all ages. Here, we take a closer look at some of the latest rides and trends in marketing, with ideas to help you bring more people to your own facility.
inPRACTICE / AQUATICS
Safer Air for Swimmers
>> Levine Center for Wellness & Recreation at Queens University in Charlotte, N.C.
Facility Profile - October 2019
At this small North Carolina university, a swimming coach understands the value of good indoor air quality, and as a result, the aquatic facility is a home to champions.


text:
Home » Austrian Election Results Move Nation To The Right Amid Influx Of Muslim Migrants
Austrian Election Results Move Nation To The Right Amid Influx Of Muslim Migrants
Kayla Adanalian
With their traditional Western and Christian culture at stake, Austria held their parliamentary election on Sunday, October 15th. After the coalition government collapsed and the two longtime partner parties turned on one another, the country had to make a decision–patriotism or globalism.
Foreign Minister Sebastian Kurz is the center-right Austrian People's Party candidate who was expected to win yesterday, and he did not disappoint. His opponents were Christian Kern of the Social Democratic Party of Austria and the leader of the Freedom Party of Austria, Heinz-Christian Strache.
Since 2015, Austria has seen a huge increase in Syrian Muslim migrants fleeing their war-torn country. Now, voters are becoming increasingly concerned for their country and its security, and it was all hanging in the balance during this election. Kurz's stance on immigration brings a lot of the same criticism that President Trump's position did in the United States. A Jewish political adviser, Tal Silberstein of the SDP suggested that Kurz was anti-Semitic in social media posts. Silberstein has since been fired and insists that Social Democratic Chancellor Christian Kern knew nothing about the postings.
After seeing the impact of the migration over the last two years, Austrian citizens are prepared to put social justice aside for a new leader who will, in fact, be tougher on immigration and make Austria and it's citizens his number one priority.
Conservative populism continues to spread throughout Europe due to the same rising concerns that Austria is experiencing right now. From Brexit to the popularity of Marine Le Pen in France to the rise of The Freedom Party in The Netherlands, Europe is beginning to wake up to the realities of globalism, and now Austria has joined the movement. The cosmopolitan bureaucrats in Brussels carrying water for the European Union have to be shaking in their boots!
Austrian Elections
Europe migrant problems
The Austrian Approach to Economic History
It is safe to assume that most of the readers of Mises UK are not admirers of the Marxist historiographical school, and...
Dr Matteo Salonia
Refugees, Terror & Institutional Liberalism: How the...
As we have witnessed over the course of the past few months, there is a complete refusal on the part of Western leaders to take any serious steps...
Eurasia can solve most of Europe's problems
On the issue of differences between the West and Russia, I am inclined to agree with Professor Dugin, who argues that Russia is not Europe. Europe...
Refugees and Post-Modern
On June 20, according to the UN decision, World Refugee Day is celebrated. Refugees are an element of archaism, bursting into the well-fed life of...
Berlin Christmas market attack: German Authorities reaction
Let us take a look at how it started yesterday and how our authorities reacted: The first reaction right after the terror attack in Berlin was zero...
Merkel will stay despite the impending storm
There is nothing about responsibility in our immigration policy. Angela Merkel opened the borders and invited the entire world to Germany. And now we...
Austrian Election Results Move Nation To The Right Amid...
With their traditional Western and Christian culture at stake, Austria held their parliamentary election on Sunday, October 15th. After the coalition...
Top 10 Countries With Highest Rape Crime
Here is the list of top 10 countries with highest rape crime. You would be amazed to read that the most developed countries like U.S., Sweden, France...
Alexander Markovics - The Austrian Oligarchy won, but this...
I think that Hofer's loss is the result of the total mobilization of the Austrian establishment, the so-called "swamp". On the one hand, the...


text:
Graham Potter explains why Marc Cucurella's is having a tough start as Chelsea boss gives fresh other injury update
raham Potter has admitted that Marc Cucurella's start to life at Chelsea has been a struggle after substituting him in two of the last three Premier League games.
The £62million signing hasn't thrived so far under the manager he worked with at Brighton, but is hoping to make amends against RB Salzburg in the Champions League on Tuesday.
The Blues boss has explained that illness has been part of why the Spain international hasn't started well at Stamford Bridge, though he is confident that Cucurella has what it takes to come good at the club.
"He's had a couple of issues, a bit of illness, he's been on antibiotics, but he's a lot better," Potter said of the left-back.
"Manchester United was purely tactical, a need to get an extra body in midfield. He was the one to come out probably because of all the things that have happened to him. A bit unlucky there.
"Sometimes when you move clubs it goes really well, and sometimes it can be a little bit of an up-and-down period. Marc's a little bit up and down but he's a top player and he will show his quality."
Kalidou Koulibaly remains out with a minor knee problem, along with longer-term absentees Reece James, N'Golo Kante and Wesley Fofana.
Potter hopes to have Koulibaly back soon and possibly even to face Brighton in the Premier League on Saturday.
"It's not too serious but this game was too soon for him," the Chelsea boss said.
"We are hoping to get him on grass either on Wednesday or Thursday but he is not available for tomorrow."
Chelsea are top of Group E in the Champions League and a win on matchday five would confirm their place in the last 16 as group winners.