text:
\section{Introduction}

Generative Adversarial Networks~\citep[GANs;][]{GAN} are a powerful class of deep generative models, but are known for training difficulties~\citep{salimans2016improved}.
Many approaches have been introduced to improve GAN performance~\citep{wgan, gulrajani2017improved, SNGAN, BigGAN}.
Recent work \citep{improvingimproving,CRGAN} suggests that the performance
of generative models can be improved by introducing consistency regularization
techniques -- which are popular in the semi-supervised learning literature \citep{realisticssl}.
In particular, \citet{CRGAN} show that Generative Adversarial
Networks (GANs) \citep{GAN} augmented with consistency regularization can achieve
state-of-the-art image-synthesis results.
In CR-GAN, real images and their corresponding augmented counterparts are fed into the discriminator.
The discriminator is then encouraged -- via an auxiliary loss term --
to produce similar outputs for an image and its corresponding augmentation.

Though the consistency regularization in CR-GAN is effective, the augmentations are only applied to the real images
and not to generated samples, making the whole procedure somewhat imbalanced.
In particular, the generator can learn these artificial augmentation features and introduce them into generated samples as undesirable artifacts.\footnote{We show examples in Fig.~\ref{fig:artifact_analysis} and discuss further in Section~\ref{sec:discussion:artifact}.}
Further, by regularizing only the discriminator, and by only using augmentations in image space,
the regularizations in \citet{improvingimproving} and \citet{CRGAN} do not act directly on the generator.
By constraining the mapping from the prior to the generated samples, we can achieve further performance gains
on top of those yielded by performing consistency regularization on the discriminator in the first place.

In this work, we introduce \emph{Improved Consistency Regularization}~(ICR) which applies forms
of consistency regularization to the generated images, the latent vector space, and the generator.
First, we address the lack of regularization on the generated samples by introducing
\emph{balanced consistency regularization} (bCR),
where a consistency term on the discriminator is applied to both real images and samples coming from the generator.
Second, we introduce \emph{latent consistency regularization} (zCR), which incorporates regularization terms modulating
the sensitivity of both the generator and discriminator changes in the prior.
In particular, given augmented/perturbed latent vectors, we show it is helpful to encourage the generator to be \emph{sensitive} to the perturbations and the discriminator to be \emph{insensitive}.

When both bCR and zCR are combined into ICR, it yields state-of-the-art image synthesis results.
For unconditional image synthesis on CIFAR-10 and CelebA, our method yields the best known FID scores on various GAN architectures.
For conditional image synthesis on CIFAR-10, we improve the state-of-the-art FID score from 11.48 to 9.21.
Finally, on ImageNet-2012, we apply our technique to the original BigGAN~\citep{BigGAN} model and improve the
FID from 6.66 to 5.38, which is the best score at that model size.

\begin{figure*}[ht]
\begin{center}
\centerline{\includegraphics[width=2\columnwidth]{figures/illustration.png}}
\caption{Illustrations comparing our methods to the baseline.
(1) CR-GAN~\citep{CRGAN} is the baseline, with consistency regularization applied only between real images and their augmentations.
(2) In Balanced Consistency Regularization (bCR-GAN), we also introduce consistency regularization between generated fake images and their augmentations. With consistency regularization on both real and fake images, the discriminator is trained in a balanced way and less augmentation artifacts are generated.
(3) Furthermore, we propose Latent Consistency Regularization (zCR-GAN), where latent $z$ is augmented with noise of small magnitude. Then for the discriminator, we regularize the consistency between corresponding pairs; while for the generator we encourage the corresponding generated images to be more diverse.
Note that
\textcolor{blue}{\{$\rightarrow\leftarrow$\}} indicates a loss term encouraging pairs to be closer together,
while \textcolor{red}{\{$\leftarrow\rightarrow$\}} indicates a loss term pushing pairs apart.
}
\label{fig:illustration}
\end{center}
\end{figure*}

\cut{
In summary, the contributions of this work are:

\begin{itemize}
\item We show that the vanilla CR-GAN~\citep{CRGAN} results in augmentation artifacts
in generated samples.
To address this issue, we propose Balanced Consistency
Regularization (bCR), in which consistency
regularization is also applied to samples from the generator.

\item In order to more thoroughly adapt consistency regularization techniques to the
GAN setting, we propose Latent Consistency Regularization (zCR),
extending the regularization from the discriminator to the generator and from images to latent space.
With draws from the latent prior $z$ perturbed,
the generator is encouraged to be sensitive to those perturbations, while the discriminator is
encouraged to be consistent.

\item Through a combination of these modifications (which we call Improved Consistency Regularization (ICR)),
we show how GANs can be made to achieve state-of-the-art FID results.
For unconditional image synthesis on CIFAR-10 and CelebA, our method yields the best known FID scores on various GAN architectures.
For conditional image synthesis on CIFAR-10, we improve the state-of-the-art FID score from 11.48 to 9.21.
Finally, on ImageNet-2012, we apply our technique to the original BigGAN~\citep{BigGAN} model and improve the FID from 6.66 to 5.38, which is the best score at that model size.
\end{itemize}
}

\section{Background}
\label{sec:background}

\subsection{Generative Adversarial Networks}

A Generative Adversarial Network (GAN)~\citep{GAN} is composed of a Generator model, $G$,
and a Discriminator model, $D$, which are parameterized by
deep neural networks.
The generator is trained to take a latent vector $z \sim p(z)$ from a prior distribution and generate target samples $G(z)$.
The discriminator is trained to distinguish
samples from the target distribution $p_\text{real}(x)$ and samples $G(z)$,
which encourages generator to reduce the discrepancy between the target distribution and $G(z)$.
Both models have respective losses defined as:
\begin{equation*}
\begin{split}
L_D&=-\mathbb{E}_{x \sim p_\text{real}}\left[\log D(x)\right] - \mathbb{E}_{z \sim p(z)}\left[1 - \log D(G(z))\right], \\
L_G&=- \mathbb{E}_{z \sim p(z)}\left[\log D(G(z))\right].
\end{split} \label{eq:ns_gan}
\end{equation*}
This original formulation~\citep{GAN} is known as the non-saturating (NS) GAN.
Extensive research has demonstrated that appropriate re-design of $L_D$ plays an important role in training stability and generation quality.
For example, the hinge loss on the discriminator \citep{lim2017, Tran2017} is defined as:
\begin{equation*}
\begin{split}
L_D = &-\mathbb{E}_{x \sim p_\text{real}}\left[\min(0, D(x)-1)\right] \\
& -\mathbb{E}_{z \sim p(z)}\left[\min(0, -D(G(z))-1)\right], \\
L_G= &- \mathbb{E}_{z \sim p(z)}\left[D(G(z))\right].
\end{split} \label{eq:hinge_gan}
\end{equation*}
The Wassertein GAN (WGAN) \citep{wgan} is another successful reformulation which measures 1-Lipschitz constrained Wasserstein distance \citep{villani2008optimal} between the target distribution and the generated distribution in the discriminator output space.
The loss function of WGAN can be written as:
\begin{equation*}
\begin{split}
L_D&=-\mathbb{E}_{x \sim p_\text{real}}\left[D(x)\right] + \mathbb{E}_{z \sim p(z)}\left[D(G(z))\right], \\
L_G&=- \mathbb{E}_{z \sim p(z)}\left[D(G(z))\right].
\end{split} \label{eq:wass_gan}
\end{equation*}
Follow-up work improves WGAN in multiple ways~\citep{gulrajani2017improved,improvingimproving}.
For instance, \citet{SNGAN} propose spectral normalization to stabilize the training, which is widely used~\citep{SAGAN,BigGAN}
and has become the de-facto weight normalization technique for GANs.

\subsection{Consistency Regularization}

For semi-supervised or unsupervised learning, consistency regularization techniques are effective and have become broadly used recently~\citep{CONSISTENCY, laine2016temporal, CONSISTENCYAVITAL, UDACONSISTENCY, mixmatch2019}.
The intuition behind these techniques is to encode into model training some prior knowledge:
that the model should produce consistent predictions given input instances and their semantics-preserving augmentations.
The augmentations (or transformations) can take many forms, such as image flipping and rotating, sentence back-translating, or even adversarial attacks.
Penalizing the inconsistency can be easily achieved by minimizing $L_2$ loss \citep{CONSISTENCY,laine2016temporal} between instance pairs, or KL-divergence loss \citep{UDACONSISTENCY, miyato2018virtual} between distributions.
In the GAN literature, \citet{improvingimproving} propose a consistency term derived from Lipschitz
continuity considerations to improve the training of WGAN.
Recently, CR-GAN~\citep{CRGAN} applies consistency regularization to the discriminator and achieves substantial improvements.

\section{Improved Consistency Regularization}
This section starts by introducing two new techniques, abbreviated as bCR and zCR, to improve and generalize consistency regularization for GANs.
We denote the combination of both of these techniques as ICR, and we will later show that ICR
yields state-of-the-art image synthesis results in a variety of settings.
Figure~\ref{fig:illustration} shows illustrations comparing our methods to the baseline CR-GAN~\citet{CRGAN}.

\subsection{Balanced Consistency Regularization (bCR)}
\label{sec:bCR}

\begin{algorithm}[tb]
\caption{Balanced Consistency Regularization (bCR)}
\label{alg:bCR}
\begin{algorithmic}
    \STATE {\bfseries Input:} parameters of generator $\theta_G$ and discriminator $\theta_D$, consistency regularization coefficient for real images $\lambda_\text{real}$ and fake images $\lambda_\text{fake}$, number of discriminator iterations per generator iteration $N_D$, augmentation transform $T$ (for images, e.g. shift, flip, cutout, etc).
    \FOR{number of training iterations}
    \FOR{$t=1$ {\bfseries to} $N_D$}
    \STATE Sample batch $z \sim p(z)$, $x \sim p_{\text{real}}(x)$
    \STATE Augment both real $T(x)$ and fake $T(G(z))$ images
    \STATE $L_D \leftarrow D(G(z)) - D(x)$
    \STATE $L_\text{real} \leftarrow \lVert D(x) - D(T(x)) \rVert ^2$
    \STATE $L_\text{fake} \leftarrow \lVert D(G(z)) - D(T(G(z))) \rVert ^2$
    \STATE $\theta_D \leftarrow \text{AdamOptimizer}(L_D + \lambda_{\text{real}} L_\text{real} + \lambda_{\text{fake}} L_\text{fake})$
    \ENDFOR
    \STATE Sample batch $z \sim p(z)$
    \STATE $L_G \leftarrow -D(G(z))$
    \STATE $\theta_G \leftarrow \text{AdamOptimizer}(L_G)$
    \ENDFOR
\end{algorithmic}
\end{algorithm}

Figure~\ref{fig:illustration}(1) illustrates the baseline CR-GAN,
in which a term is added to the discriminator loss function that penalizes its sensitivity to the difference between the original image $x$ and the augmented image $T(x)$.
One key problem with the original CR-GAN is that the discriminator might `mistakenly believe' that the augmentations are
actual features of the target data set, since these augmentations are only performed on the real images.
This phenomenon, which we refer to as consistency imbalance, is not easy to notice for certain types of
augmentation (e.g. image shifting and flipping).
However, it can result in generated samples with explicit augmentation artifacts when augmented samples contain visual artifacts not belonging to real images.
For example, we can easily observe this effect for CR-GAN with cutout augmentation: see the second column in Figure~\ref{fig:artifact_analysis}.
This undesirable effect greatly limits the choice of advanced augmentations we could use.

In order to correct this issue, we propose to also augment generated samples before they are fed into the discriminator,
so that the discriminator will be evenly regularized with respect to both real and fake augmentations and thereby
be encouraged to focus on meaningful visual information.

Specifically, a gradient update step will involve four batches, a batch of real images $x$,
augmentations of these real images $T(x)$, a batch of generated samples $G(z)$,
and that same batch with augmentations $T(G(z))$.
The discriminator will have terms that penalize
its sensitivity between corresponding $\{x, T(x)\}$ and also $\{G(z), T(G(z))\}$,
while the generator cost remains unmodified.

This technique is described in more detail in Algorithm \ref{alg:bCR} and visualized in Figure~\ref{fig:illustration}(2).
We abuse the notation a little in the sense that $D(x)$ denotes the output vector before
activation of the last layer of the discriminator given input $z$.
$T(x)$ denotes an augmentation transform, here for images (e.g. shift, flip, cutout, etc).
The consistency regularization can be balanced by adjusting the
strength of $\lambda_{\text{real}}$ and $\lambda_{\text{fake}}$.
This proposed bCR technique not only removes augmentation artifacts (see third column of
Figure~\ref{fig:artifact_analysis}), but also brings substantial performance improvement
(see Section~\ref{sec:experiments}~and~\ref{sec:discussion}).

\subsection{Latent Consistency Regularization (zCR)}

\begin{algorithm}[tb]
\caption{Latent Consistency Regularization (zCR)}
\label{alg:zCR}
\begin{algorithmic}
    \STATE {\bfseries Input:} parameters of generator $\theta_G$ and discriminator $\theta_D$, consistency regularization coefficient for generator $\lambda_\text{gen}$ and discriminator $\lambda_\text{dis}$, number of discriminator iterations per generator iteration $N_D$, augmentation transform $T$ (for latent vectors, e.g. adding small perturbation noise$\sim \mathcal{N}(0, \sigma_\text{noise})$).
    \FOR{number of training iterations}
    \FOR{$t=1$ {\bfseries to} $N_D$}
    \STATE Sample batch $z \sim p(z)$, $x \sim p_{\text{real}}(x)$
    \STATE Sample perturbation noise $\Delta z \sim \mathcal{N}(0, \sigma_\text{noise})$
    \STATE Augment latent vectors $T(z) \leftarrow z + \Delta z$
    \STATE $L_D \leftarrow D(G(z)) - D(x)$
    \STATE $L_\text{dis} \leftarrow \lVert D(G(z)) - D(G(T(z))) \rVert ^2$
    \STATE $\theta_D \leftarrow \text{AdamOptimizer}(L_D + \lambda_{\text{dis}} L_\text{dis})$
    \ENDFOR
    \STATE Sample batch $z \sim p(z)$
    \STATE $L_G \leftarrow -D(G(z))$
    \STATE $L_\text{gen} \leftarrow -\lVert G(z) - G(T(z)) \rVert ^2$
    \STATE $\theta_G \leftarrow \text{AdamOptimizer}(L_G + \lambda_{\text{gen}} L_\text{gen})$
    \ENDFOR
\end{algorithmic}
\end{algorithm}

In Section~\ref{sec:bCR}, we focus on consistency regularization with respect to augmentations in image space
on the inputs to the discriminator.
In this section, we consider a different question:
Would it help if we enforce consistency regularization with respect to augmentations in latent space~\citep{zhengli2018iclr}?
Given that a GAN model consists of both a generator and a discriminator, it seems reasonable to ask
if techniques that can be applied to the discriminator can also be effectively applied to the generator in certain analogous way.

Towards this end, we propose to augment inputs to the generator
by slightly perturbing draws $z$ from the prior to yield
$T(z) = z + \Delta z, \Delta z \sim \mathcal{N}(0, \sigma_\text{noise})$.
Assuming the added perturbations $\Delta z$ are small enough,
we expect that the output of the discriminator ought not to change much with respect to this perturbation
and modify the discriminator loss by enforcing that $\lVert D(G(z)) - D(G(T(z))) \rVert ^2$ is small.

However, with only this new consistency regularization term added onto the GAN loss,
the generator would be prone to collapse to generating specific samples for any latent $z$,
since that would easily satisfy the constraint above.
To avoid this, we also modify the loss function for the generator
with a term that maximizes the difference between $G(z)$ and $G(T(z))$,
which also encourages generations from similar latent vectors to be diverse.
Though motivated differently, this can be seen as related to the Jacobian Clamping technique from \citet{oboborg} and diversity increase technique in \citet{yang2019diversity}.

This method is described in more detail in Algorithm \ref{alg:zCR} and visualized in Figure~\ref{fig:illustration}(3).
$G(z)$ denotes the output images of the generator given input $z$.
$T(x)$ denotes an augmentation transform, here for latent vectors (e.g. adding small perturbation noise).
The strength of consistency regularization for the discriminator can be adjusted via $\lambda_{\text{dis}}$.
From the view of the generator, intuitively,
the term $L_\text{gen} = -\lVert G(z) - G(T(z)) \rVert ^2$ encourages $\{G(z), G(T(z))\}$ to be diverse.
We have conducted analysis on the effect of $\lambda_\text{gen}$ with experiments in Section~\ref{sec:discussion:zCR}.
This technique substantially improves the performance of GANs, as measured by FID.
We present experimental results in Section~\ref{sec:experiments}~and~\ref{sec:discussion}.

\subsection{Putting it All Together}
Though both Balanced Consistency Regularization and Latent Consistency Regularization improve GAN performance
(see Section \ref{sec:experiments}), it is not obvious that they would work when `stacked on top' of each other.
That is, maybe they are accomplishing the same thing in different ways, and we cannot add up their benefits.
However, validated with extensive experiments,
we achieve the best experimental results when combining Algorithm~\ref{alg:bCR}
and Algorithm~\ref{alg:zCR} together.
We call this combination Improved Consistency Regularization (ICR).
Note that in ICR, we augment inputs in both image and latent spaces,
and add regularization terms to both the discriminator and the generator.
We regularize the discriminator's consistency between corresponding pairs of
$\{D(x), D(T(x))\}$, $\{D(G(z)), D(T(G(z)))\}$, and $\{D(G(z)), D(G(T(z)))\}$;
For the generator, we encourage diversity between $\{G(z), G(T(z))\}$.

\section{Experiments}
\label{sec:experiments}

In this section, we validate our methods on different data sets,
model architectures, and GAN loss functions.
We compare both Balanced Consistency Regularization (Algorithm~\ref{alg:bCR}) and Latent Consistency Regularization (Algorithm~\ref{alg:zCR}) with several baseline methods.
We also combine both techniques (we abbreviate this combination as ICR) and show
that this yields state-of-the-art FID numbers.
We follow the best experimental practices established in \citet{compare_gan},
aggregating all runs and reporting the FID distribution of the top 15\% of trained models.
We provide both quantitative and qualitative results (with more in the appendix).

\subsection{Baseline Methods}

We compare our methods with four GAN regularization techniques:
Gradient Penalty (GP)~\citep{gulrajani2017improved},
DRAGAN (DR)~\citep{kodali2017convergence},
Jensen-Shannon Regularizer (JSR)~\citep{roth2017stabilizing},
and vanilla Consistency Regularization (CR)~\citep{CRGAN}.
The regularization strength $\lambda$ is set to 0.1 for JSR, and 10 for all others.

Following the procedures from \citet{lucic2018gans, compare_gan},
we evaluate these methods across different data sets,
neural architectures, and loss functions.
For optimization, we use the Adam optimizer with batch size of 64 for all experiments.
By default, spectral normalization (SN)~\citep{SNGAN} is used in the
discriminator, as it is the most effective normalization method for
GANs~\citep{compare_gan} and is becoming the
standard for recent GANs~\citep{BigGAN, LOGAN}.

\subsection{Data Sets and Evaluation}

We carry out extensive experiments comparing our methods against the above
baselines on three commonly used data sets in the GAN literature:
CIFAR-10~\citep{CIFAR},
CelebA-HQ-128~\citep{PGGAN},
and ImageNet-2012~\citep{ImageNet}.

For data set preparation, we follow the detailed procedures in \citet{compare_gan}.
CIFAR-10 contains 60K $32 \times 32$ images with 10 labels,
out of which 50K are used for training and 10K are used for testing.
CelebA-HQ-128 (CelebA) consists of 30K $128 \times 128$ facial images,
out of which we use 3K images for testing and train models with the rest.
ImageNet-2012 has approximately 1.2M images with 1000 labels,
and we down-sample the images to $128 \times 128$.
We stop training after 200k generator update steps for CIFAR-10,
100k steps for CelebA, and 250k for ImageNet.

We use the Fréchet Inception Distance (FID)~\citep{FID} as
the primary metric for quantitative evaluation.
FID has been shown to correlate well with human evaluation of image quality
and to be helpful in detecting intra-class mode collapse.
We calculate FID between generated samples and real test images,
using 10K images on CIFAR-10, 3K on CelebA, and 50K on ImageNet.
We also report Inception Scores~\citep{salimans2016improved} in the appendix.

By default, the augmentation transform $T$ on latent vectors $z$ is adding Gaussian
noise $\Delta z \sim \mathcal{N}(0, \sigma_\text{noise})$.
The augmentation transform $T$ on images is a combination of randomly flipping horizontally and shifting by multiple pixels (up to 4 for CIFAR-10 and
CelebA, and up to 16 for ImageNet).
This transform combination results in better performance than alternatives (see \citet{CRGAN}).

There are many different GAN loss functions and we elaborate on several of them in Section~\ref{sec:background}.
Following \citet{CRGAN}, for each data set and model architecture combination,
we conduct experiments using the loss function that achieves the best performance
on baselines.

\subsection{Unconditional GAN Models}
\label{sec:unconditional}

We first test out techniques on the unconditional image
generation task,
which is to model images from an
object-recognition data set without any reference to the underlying
classes.
We conduct experiments on the CIFAR-10 and CelebA data sets,
and use both DCGAN~\citep{DCGAN} and ResNet~\citep{resnet} GAN architectures.

\subsubsection{DCGAN on CIFAR-10}

Figure~\ref{fig:cifar_dcgan_hinge} presents the results of DCGAN on CIFAR-10
with the hinge loss.
Vanilla Consistency Regularization (CR) \citep{CRGAN} outperforms all other baselines.
Our Balanced Consistency Regularization (bCR) technique
improves on CR by more than $3.0$ FID points.
Our Latent Consistency Regularization (zCR) technique
improves scores less than bCR,
but the improvement is still significant compared to the measurement variance.
We set $\lambda_\text{real} = \lambda_\text{fake} = 10$ for bCR,
while using $\sigma_\text{noise} = 0.03$, $\lambda_\text{gen} = 0.5$,
and $\lambda_\text{dis} = 5$ for zCR.

\begin{figure}[hbt]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{figures/cifar_sndcgan_hinge.png}}
\caption{
FID scores for DCGAN trained on CIFAR-10 with the
hinge loss, for a variety of regularization techniques.
Consistency regularization significantly outperforms non-consistency regularizations.
Adding Balanced Consistency Regularization causes a larger improvement than
Latent Consistency Regularization, but both yield improvements much larger
than measurement variances.
}
\label{fig:cifar_dcgan_hinge}
\end{center}
\end{figure}

\begin{figure}[htb!]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{figures/cifar_resnet_ns.png}}
\caption{
FID scores for a ResNet-style GAN trained on CIFAR-10 with the non-saturating loss, for a variety of regularization techniques.
Contrary to the results in Figure~\ref{fig:cifar_dcgan_hinge},
Latent Consistency Regularization outperforms Balanced Consistency Regularization,
though they both substantially surpass all baselines.
}
\label{fig:cifar_resnet_ns}
\end{center}
\end{figure}

\subsubsection{ResNet on CIFAR-10}

DCGAN-type models are well-known and it is
encouraging that our techniques increase performance for those models,
but they have been substantially surpassed in performance by newer techniques.
We then validate our methods on more recent architectures that use residual
connections \citep{resnet}.
Figure~\ref{fig:cifar_resnet_ns} shows unconditional image synthesis results on
CIFAR-10 using a GAN model with residual connections and the non-saturating loss.
Though both of our proposed modifications still outperform all baselines,
Latent Consistency Regularization works better than Balanced Consistency Regularization,
contrary to the results in Figure \ref{fig:cifar_dcgan_hinge}.
For hyper-parameters,
we set $\lambda_\text{real} = 10$ and $\lambda_\text{fake} = 5$ for bCR,
while using $\sigma_\text{noise} = 0.07$, $\lambda_\text{gen} = 0.5$,
and $\lambda_\text{dis} = 20$ for zCR.

\subsubsection{DCGAN on CelebA}

We also conduct experiments on the CelebA data set.
The baseline model we use in this case is a DCGAN model with the non-saturating loss.
We set $\lambda_\text{real} = \lambda_\text{fake} = 10$ for bCR,
while using $\sigma_\text{noise} = 0.1$, $\lambda_\text{gen} = 1$,
and $\lambda_\text{dis} = 10$ for zCR.
The results are shown in Figure~\ref{fig:celeba_dcgan_ns} and are overall similar
to those in Figure~\ref{fig:cifar_dcgan_hinge}.
The improvements in performance for CelebA are not as large as those for
CIFAR-10,
but they are still substantial,
suggesting that our methods generalize across data sets.

\begin{figure}[t]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{figures/celeba_sndcgan_ns.png}}
\caption{
FID scores for DCGAN trained on CelebA with the non-saturating loss, for a variety of regularization techniques.
Consistency regularization significantly outperforms all other baselines.
Balanced Consistency Regularization further improves on Consistency Regularization by more than 2.0 in terms of FID,
while Latent Consistency Regularization improves by around 1.0.
}
\label{fig:celeba_dcgan_ns}
\end{center}
\end{figure}

\subsubsection{Improved Consistency Regularization}
As alluded to above,
we observe experimentally that
combining Balanced Consistency regularization (bCR) and Latent Consistency
Regularization (zCR) (into Improved Consistency Regularization (ICR))
yields results that are better than those given by either method alone.
Using the above experimental results,
we choose the best-performing hyper-parameters to carry out experiments for ICR, regularizing with both bCR and zCR.
Table~\ref{tab:unconditional} shows
that ICR yields the best results for all three unconditional synthesis settings we study.
Moreover, the results of the ResNet model on CIFAR-10 are, to the best of our knowledge,
the best reported results for unconditional CIFAR-10 synthesis.

\begin{table}[tb]
\caption{FID scores for Unconditional Image Synthesis with ICR.
Our ICR achieves the best performance overall.
Baselines are: not using regularization (W/O),
Gradient Penalty (GP)~\citep{gulrajani2017improved},
DRAGAN (DR)~\citep{kodali2017convergence},
Jensen-Shannon Regularizer (JSR)~\citep{roth2017stabilizing},
and vanilla Consistency Regularization (CR)~\citep{CRGAN}.
}
\label{tab:unconditional}
\begin{center}
\begin{tabular}{lccc}
\toprule
        & CIFAR-10  & CIFAR-10  & CelebA \\
Methods & (DCGAN) & (ResNet)  & (DCGAN) \\
\midrule
W/O     & 24.73 & 19.00 & 25.95 \\
GP      & 25.83 & 19.74 & 22.57 \\
DR      & 25.08 & 18.94 & 21.91 \\
JSR     & 25.17 & 19.59 & 22.17 \\
CR      & 18.72 & 14.56 & 16.97 \\
\midrule
ICR (ours)    & \textbf{15.87} & \textbf{13.36} & \textbf{15.43} \\
\bottomrule
\end{tabular}
\end{center}
\end{table}

\subsection{Conditional GAN Models}
\label{sec:conditional}

\begin{table}[tb]
\caption{FID scores for class conditional image generation on CIFAR-10 and ImageNet.
We compare our ICR technique with state-of-the-art GAN models including
SNGAN~\citep{SNGAN}, BigGAN~\citep{BigGAN}, and CR-GAN~\citep{CRGAN}.
The BigGAN implementation we use is from \citet{compare_gan}.
($\ast$)-BigGAN has the exactly same architecture as the publicly available BigGAN and
is trained with the same settings, but with our consistency regularization techniques
added to GAN losses.
On CIFAR-10 and ImageNet, we improve the FID numbers to 9.21 and 5.38 correspondingly,
which are the best known scores at that model size.}
\label{tab:conditional}
\begin{center}
\begin{tabular}{lcc}
\toprule
Models      & CIFAR-10 & ImageNet \\
\midrule
SNGAN       & 17.50 & 27.62 \\
BigGAN      & 14.73 & 8.73 \\
CR-BigGAN   & 11.48 & 6.66 \\
\midrule
ICR-BigGAN (ours)  & \textbf{9.21} & \textbf{5.38} \\
\bottomrule
\end{tabular}
\end{center}
\end{table}

In this section, we apply our consistency regularization techniques to the publicly
available implementation of BigGAN \citep{BigGAN} from \citet{compare_gan}.
We compare it to baselines from \citet{BigGAN,SNGAN,CRGAN}.
Note that the FID numbers from \citet{LOGAN} are based on a larger version of BigGAN
called BigGAN-Deep with substantially more parameters than the original BigGAN,
and are thus not comparable to the numbers we report here.
On CIFAR-10, our techniques yield the best known FID score for conditional synthesis with
CIFAR-10\footnote{
There are a few papers that report lower scores using the PyTorch implementation
of the FID.
That implementation outputs numbers that are much lower,
which are not comparable to numbers from the official
TF implementation,
as explained at
\url{https://github.com/ajbrock/BigGAN-PyTorch\#an-important-note-on-inception-metrics}}: 9.21.
On conditional Image Synthesis on the ImageNet data set,
our technique yields FID of 5.38.
This is the best known score using the same number of parameters as in the original BigGAN model,
though the much larger model from \citet{LOGAN} achieves a better score.
For both setups, we set $\lambda_\text{real} = \lambda_\text{fake} = 10$,
together with $\sigma_\text{noise} = 0.05$, $\lambda_\text{gen} = 0.5$,
and $\lambda_\text{dis} = 20$.

\section{Ablation Studies}
\label{sec:discussion}

\subsection{How do the Hyper-Parameters for Balanced Consistency Regularization Affect Performance?}

In Balanced Consistency Regularization (Algorithm \ref{alg:bCR}),
the cost associated
with sensitivity to augmentations of the real images is weighted by $\lambda_\text{real}$
and the cost associated with sensitivity to augmentations of the generated samples is
weighted by $\lambda_\text{fake}$.
In order to better understand the interplay between these parameters, we train
a DCGAN-type model with spectral normalization on the CIFAR-10 data set with the hinge loss,
for many different values of $\lambda_\text{fake}, \lambda_\text{real}$.
The heat map in Figure \ref{fig:real_fake_ratio} shows that it never pays to set either
of the parameters to zero: this means that Balanced Consistency Regularization always
outperforms vanilla consistency regularization (the baseline CR-GAN).
Generally speaking, setting $\lambda_\text{real}$ and $\lambda_\text{fake}$ similar in
magnitude works well.
This is encouraging, since it means that the performance of bCR
is relatively insensitive to hyper-parameters.

\begin{figure}
\begin{center}
\centerline{\includegraphics[width=0.8\columnwidth]{figures/cifar_sndcgan_real_fake_heatmap.png}}
\caption{
Analysis of the effects of the $\lambda_\text{real}$ and $\lambda_\text{fake}$
hyper-parameters for Balanced Consistency Regularization.
We train DCGAN on CIFAR-10 with hinge loss,
for many different values of $\lambda_\text{fake}, \lambda_\text{real}$.
The results show that Balanced Consistency Regularization essentially always outperforms
vanilla consistency regularization.
Generally speaking, Balanced Consistency Regularization performs best with
$\lambda_\text{fake}$ and $\lambda_\text{real}$ of similar magnitudes.
}
\label{fig:real_fake_ratio}
\end{center}
\end{figure}

\subsection{Examining Artifacts Resulting from `Vanilla' Consistency Regularization}
\label{sec:discussion:artifact}

\begin{figure}[tb]
  \centering
  \subfloat[$8 \times 8$ cutout.]
  {\includegraphics[width=0.3\columnwidth]{figures/cutout_aug_8.png}}  \hfill
  \subfloat[CR samples.]
  {\includegraphics[width=0.3\columnwidth]{figures/cutout_CR_8.png}}  \hfill
  \subfloat[bCR samples.]
  {\includegraphics[width=0.3\columnwidth]{figures/cutout_bCR_8.png}} \hfill
  \subfloat[$16 \times 16$ cutout.]
  {\includegraphics[width=0.3\columnwidth]{figures/cutout_aug_16.png}}  \hfill
  \subfloat[CR samples.]
  {\includegraphics[width=0.3\columnwidth]{figures/cutout_CR_16.png}}  \hfill
  \subfloat[bCR samples.]
  {\includegraphics[width=0.3\columnwidth]{figures/cutout_bCR_16.png}} \hfill
  \subfloat[$32 \times 32$ cutout.]
  {\includegraphics[width=0.3\columnwidth]{figures/cutout_aug_32.png}}  \hfill
  \subfloat[CR samples.]
  {\includegraphics[width=0.3\columnwidth]{figures/cutout_CR_32.png}}  \hfill
  \subfloat[bCR samples.]
  {\includegraphics[width=0.3\columnwidth]{figures/cutout_bCR_32.png}}
\caption{
Illustration of resolving generation artifacts by Balanced Consistency Regularization.
The first column shows CIFAR-10 training images augmented with cutout of different sizes.
The second column demonstrates that the vanilla CR-GAN~\citep{CRGAN} can cause augmentation artifacts to appear in generated samples.
This is because CR-GAN only has consistency regularization on real images passed into the discriminator.
In the last column (our Balanced Consistency Regularization: bCR in Algorithm~\ref{alg:bCR}) this issue is fixed with both real and generated fake images augmented before being fed into the discriminator.
}
\label{fig:artifact_analysis}
\end{figure}

To understand the augmentation artifacts resulting from using vanilla CR-GAN~\citep{CRGAN}, and to validate
that Balanced Consistency Regularization removes those artifacts, we carry out a series of qualitative
experiments using varying sizes for the cutout \citep{cutout} augmentation.
We experiment with cutouts of size $8 \times 8$, $16 \times 16$, and $32 \times 32$,
training both vanilla CR-GANs and GANs with Balanced Consistency Regularization.

The results are shown in Figure~\ref{fig:artifact_analysis}.
Broadly speaking, we observe more substantial cutout artifacts (black rectangles)
in samples from CR-GANs with larger cutout augmentations,
and essentially no such artifacts for GANs trained with
Balanced Consistency Regularization with $\lambda_\text{fake} \geq \lambda_\text{real}$
(we examine several hundred samples from bCR-GANs manually
in order to make this observation).

We do observe a few artifacts when $0 < \lambda_\text{fake} \ll \lambda_\text{real}$,
but much less than those from the vanilla CR-GAN.
We believe that this phenomenon of introducing augmentation artifacts into generations likely
holds for other types of augmentation, but it is much more difficult
to confirm for less visible transforms, and in some cases it may not actually be harmful (e.g. flipping
of images in most contexts).

\subsection{How do the Hyper-Parameters for Latent Consistency Regularization Affect Performance?}
\label{sec:discussion:zCR}

Latent Consistency Regularization (Algorithm \ref{alg:zCR}) has three hyper-parameters:
$\sigma_\text{noise}, \lambda_\text{gen}$ and $\lambda_\text{dis}$, which respectively
govern the magnitude of the perturbation made to the draw from the prior, the weight of
the sensitivity of the generator to that perturbation, and the weight of the sensitivity of
the discriminator to that perturbation.
From the view of the generator, intuitively,
the extra loss term added
$L_\text{gen} = -\lVert G(z) - G(T(z)) \rVert ^2$ encourages
$G(z)$ and $G(T(z))$ to be far away from each other.

We conduct experiments using a ResNet-style GAN on the CIFAR-10 data set with the non-saturating loss
in order to better understand the interplay between these hyper-parameters.
The results in Figure~\ref{fig:z_aug_params} show
that a moderate value for the generator
coefficient (e.g. $\lambda_\text{gen} = 0.5$) works the best (as measured by FID).
This corresponds to encouraging the generator to be sensitive to perturbations
of samples from the prior.
For this experimental setup, perturbations with standard deviation of
$\sigma_\text{noise} = 0.07$ are the best, and higher (but not extremely high) values for the discriminator coefficient $\lambda_\text{dis}$ also perform better.

\begin{figure}[hbt]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{figures/cifar_resnet_z_aug_vary.png}}
\caption{
Analysis on the hyper-parameters of Latent Consistency Regularization.
We conduct experiments using a ResNet-style GAN on CIFAR-10 with  non-saturating loss
in order to better understand the interplay between $\sigma_\text{noise}$, $\lambda_\text{gen}$ and $\lambda_\text{dis}$.
The results show
that a moderate value  for the generator
coefficient (e.g. $\lambda_\text{gen} = 0.5$) works the best.
With the added term $L_\text{gen} = -\lVert G(z) - G(T(z)) \rVert ^2$,
the generator is encouraged to be sensitive to perturbations in latent space.
For this set of experiments, we observe the best performance adding perturbations with standard deviation of
$\sigma_\text{noise} = 0.07$, and higher (but not extremely high) values for the discriminator coefficient $\lambda_\text{dis}$ also improve further.
}
\label{fig:z_aug_params}
\end{center}
\end{figure}

\section{Related Work}
There is so much related work on GANs \citep{GAN} that it is impossible to do it justice
(see \citet{openquestions,compare_gan} for different overviews of the field),
but here we sketch out a few different threads.
There is a several-year-long thread of work on scaling GANs up to do conditional image synthesis on the ImageNet-2012
data set beginning with \citet{ACGAN}, extending through \citet{SNGAN, SAGAN, BigGAN, ylg} and most recently culminating
in \citet{LOGAN} and \citet{CRGAN}, which presently represent the state-of-the-art models at this task
(\citet{LOGAN} uses a larger model size than \citet{CRGAN} and correspondingly report better scores).
There is a separate thread of more `graphics-focused' work on GANs that tends not to use the same benchmarks
and is hard to directly compare with \citep{PGGAN, stylegan, styleganv2}, but nevertheless produces
interesting and impressive results.
Finally, as GANs are known to be hard to train for a variety of reasons, there is a substantial amount of work
\citep{unrolled, wgan, gulrajani2017improved, skillrating, smallgan} dedicated to fixing these
issues, understanding them better, or more accurately measuring the quality of GAN outputs.

Most related work on consistency regularization is from the semi-supervised learning literature,
and focuses on regularizing model predictions to be invariant to small perturbations
\citep{bachman2014learning, CONSISTENCY, laine2016temporal, miyato2018virtual, UDACONSISTENCY} for the purpose
of learning from limited labeled data.
\citet{improvingimproving, CRGAN} apply related ideas to training GAN models and observe initial gains,
which motivates this work.

\section{Conclusion}

Extending the recent success of consistency regularization in GANs~\citep{improvingimproving,CRGAN},
we present two novel improvements:
Balanced Consistency Regularization, in which generator samples are also augmented along with
training data, and Latent Consistency Regularization, in which
draws from the prior are perturbed,
and the sensitivity to those perturbations is discouraged and
encouraged for the discriminator and the generator, respectively.

In addition to fixing a new issue we observe with the vanilla Consistency Regularization (augmentation
artifacts in samples), our techniques yield the best known FID numbers for both unconditional
and conditional image synthesis on the CIFAR-10 data set.
They also achieve the best FID numbers (with the fixed number of parameters used in the
original BigGAN \citep{BigGAN} model) for conditional image synthesis on ImageNet.

These techniques are simple to implement, not particularly computationally burdensome, and relatively
insensitive to hyper-parameters.
We hope they become a standard part of the GAN training toolkit, and that their use allows more interesting usage of GANs to many sorts of applications.

\section*{Acknowledgements}
We would like to thank Colin Raffel, Pouya Pezeshkpour for helpful discussions.

redpajama_set_name: RedPajamaArXiv


Greg supports some of the world's largest governments and companies in developing and executing public affairs strategies. Active in policy forums worldwide, he has assembled an internationally-recognised team capable of managing outcomes of national and multilateral policy processes on five continents.
Before becoming Managing Director of Access Partnership, Greg spent time both in government service and in the commercial arena, where he held responsibilities for leading campaigns to lower trade barriers. Prior to that, Greg worked as a ghost writer of publications covering international affairs and economics.


text:
Joseph May (1816 – 10 February 1890) was a 19th-century Member of Parliament from the Auckland Region in New Zealand.

May was first elected to Auckland Provincial Council for the Mongonui electorate in October 1855, where he served until August 1857. He then represented the Southern Division electorate from November 1859 to September 1861. He then stood in the Raglan electorate and represented it from November 1861 to April 1864, and again from November 1865 to October 1867, and for a third period from December 1869 to October 1873. From October 1874 until the abolition of provincial government on 31 October 1876, he represented the Eden electorate.

From January to November 1856, he was a member of the Auckland Executive Council. He was again appointed to the Executive Council in December 1862, and he served until he and fellow executive councillor William Daldy were on 8 April 1864 replaced by John Anderson Gilfillan and Samuel Jackson. May was appointed to the Executive Council for a third period from December 1869 to December 1870. From June 1870, he was the 7th and last Deputy-Superintendent of Auckland Province.

He represented the Franklin electorate in Parliament from 9 April  to 6 December 1875, when he was defeated.

He died in Mt Roskill, Auckland aged 75.

Notes

External links

References

Members of the New Zealand House of Representatives
1816 births
1890 deaths
Members of the Auckland Provincial Council
Members of Auckland provincial executive councils
New Zealand MPs for North Island electorates
19th-century New Zealand politicians
Unsuccessful candidates in the 1875–1876 New Zealand general election

redpajama_set_name: RedPajamaWikipedia


Creel Basket w Leather catch Shoulder type Vintage Cabin Lodge Free Shipping!!
Stunning Vintage Fishing Creel Basket DATED & INITIALED! 1942! Trout River Wow!
Vintage Creel basket w/ Cooking Utensils Set - Home Decor Collectors - GET NOW!


There are reports of magnetic coatings adding to the acidity of a film base. Some blame it on the coating process itself, others on the magnetic materials, but we all know that anything that seals one or both surfaces of an acetate base will lead to vinegar much more quickly than not.
At least it wasn't a rare IB Tech print or something else precious to your library.
I'm a retired sound editor, and I spent many years at Fox. Our sound effects library was on 35mm mag stripe film, but I noticed that most of the original 35mm fullcoat mag stereo effects (1950's vintage) had a lot of vinegar syndrome, and the odor was REALLY strong. When we found something like that we immediately had the track transferred to new fullcoat film (usually Mylar or polyester base) and discarded the acetate base original. I never encountered any vinegar on newer mag film, just the original masters. I don't know if Fox ever started a restoration program of the mag music tracks from early stereo films. I do know that (for a while, at least) Reel 3 (if memory serves) of THE ROBE was missing (the four track stereo release master. At least the copy in the Fox vault!
Thanks for the info David. The preservation of film is so important, I hope the studios have taken all the efforts they can to save what remains.


text:
Straight From the Source: Anne Nesbet on Writing Historical Fiction
After a childhood spent in California, France, and West Germany, Anne Nesbet went on to study Russian, earn a Ph.D. in Comparative Literature, and become a professor at U. C. Berkeley, where she teaches the history of film, Russian literature and children's literature. Her research has taken her all over the world: she has lived, for example, in Paris, Moscow, Leningrad (when it was still called Leningrad), and East Berlin (when it was still called East Berlin).
Her first two novels for children (THE CABINET OF EARTHS, HarperCollins 2012, and A BOX OF GARGOYLES, HarperCollins 2013) are middle-grade fantasies about a girl who must take on the magical underworld of Paris. Her third fantasy, THE WRINKLED CROWN (HarperCollins 2015) is set in an imaginary, divided world; Linny's breaking of the rule that girls must avoid musical instruments puts her best friend in mortal danger, and Linny must travel to the ends of the world to try to set things right. CLOUD & WALLFISH (Candlewick 2016), is a historical novel set in East Berlin in 1989, and is inspired by the author's experiences in the German Democratic Republic at that time. When she is not teaching or writing, Anne enjoys hiking, playing the violin and viola, and going on adventures with her husband, three daughters, and one very well-traveled (yet still irrepressible) dog.
In the case of Cloud and Wallfish, for some time I had been wondering whether there might be an East German story in me, because my husband and I lived there in 1989, the year the Wall eventually fell, and it was a fascinating, complicated place and time. But it was an idle question that finally got the storytelling juices flowing: I was remembering the old maps we had in East Berlin, in which West Berlin showed up as a blank space, and I wondered, "What would a child do with a map like that?" And suddenly it occurred to me that surely a kid would want to doodle an imaginary world into a map like that. And from that thought, my characters and all the rest of the plot eventually unfolded…
How do you conduct your research?
I was in East Berlin in 1987 and 1989 and 1990, doing research for my dissertation on cultural policies in the Soviet Zone of Germany after World War Two. But I was as fascinated with the German Democratic Republic of the 1980s as I was with the earlier period, so I took copious notes on everyday life, kept a detailed journal, talked to our East German friends, read the newspapers and clipped interesting articles, took photographs of interesting places, bought books about current and recent events, and generally was always "conducting research" all day long, even during the hours I wasn't in the library!
For my WIP, currently called The Orphan Band of Springdale, which is set in a small town in Maine in 1941, I read through that whole year's worth of the local paper and took photographs of various important locations. And asked relatives and friends who grew up on farms questions about when various crops would be planted and how to raise chickens and what breakfast usually consisted of, and the like.
What kinds of sources do you use?
Local newspapers from the period in question, in hardcopy and (when available) online. Guidebooks from the time in question. For Cloud and Wallfish, propaganda materials published by the East German government. Magazines from the time. Historical chronicles of 1989, day by day. My own journals and the ones my husband and I kept together. My notes on everyday life in East Germany in 1989. All of our photographs, and the photographs our friends took. Maps, brochures, East German novels. Everything!
At what point do you feel comfortable beginning to draft? How does your research continue once you begin writing?
I have to have a pretty detailed outline before I begin drafting. That means I need a plot! For both of these historical novels, I sorted through my research notes, noting themes in a plotting notebook. When I finally got the interesting themes condensed into a two-page notebook spread, I started drawing arrows from one thing to the next, figuring out how to get the chain of dominoes going. At the same time I was brainstorming character details on other notebook pages. No matter how carefully I outline beforehand, I'm always caught by surprise by plot holes at some point during the writing of the first draft. Then I have to stop and do emergency research to figure out where to go next or when the school vacations were in 1989 in Berlin–or whatever the question may be!
I love research! It's a kind of treasure hunt, really, but the kind of treasure hunt that threatens to swallow a person up. Sometimes it can be hard to step out of research mode and start writing . . . .
What are some obstacles writing historical fiction brings?
It is a humbling task, writing a historical novel. The more you research, the more you realize you don't know. You learn more and more all the time, and that just makes you ever more aware of the gaps in your knowledge. Of course it is also true that we can't know everything about anything, not even about our own present day! So at some point we have to acknowledge our limits and still take that leap into telling the story.
Good question! I think the most interesting historical topics are filled with gray areas. All that complex gray can deepen a story, but it also takes extra time and pages to portray gray properly. I have tried to maintain some sense of the gray areas in my novels, despite the competing pressure to keep things relatively brief and relatively clear. For one example, I wanted to be sure we got a glimpse of the happy side of Cloud-Claudia's East German childhood, even if our glimpse has to be pretty brief and "in retrospect." And Noah and his mother have a discussion at one point that is really about the complicated ethics of spying . . . .
The world is so large! And so much has happened since the beginning of time! How can we not be absolutely driven to learn as much as we can about as many different human stories as possible? What's more, if we don't understand history, we are moving forward in a fog–and fogs can be dangerous. But I should add quickly that stories from the past aren't just "good for us"–they are also vibrant, fascinating, weird, moving, and lovely.
Filed Under: authors, books and reading, historical fiction, the writing life

redpajama_set_name: RedPajamaCommonCrawl


This beautiful 4 br Holiday Home Rentals is located in Cumbrian Lakes, a gated community sought after for its tranquil setting and proximity to Disney. Cumbrian Lakes is set in a peaceful enclave just off of Poinciana Boulevard, 4 miles south of 192, which has numerous restaurants, shops, and local area attractions. Disney is just 15 minutes away.
Book this 4 Bedrooms Golf Rentals with beautiful Accommodation which Sleeps 5-8 in Kissimmee Holiday Home NOW!!!


text:
Snowsisters (Paperback)
By Tom Wilinsky
Interlude Press, 9781945053528, 264pp.
Tom Wilinsky & Jen Sternick met in high school and started a conversation which, years later, is ongoing through their writing partnership, Never Have I Ever Books.
Tom lives with his partner in New York, where he's an attorney who likes cold weather, old cars, and anything with zombies.
Jen lives in Rhode Island with her husband, two sons, and a cranky, seven-toed cat. She's a former criminal prosecutor who still works in government. She likes theater, travel, and Twitter, an admitted addiction.
They are both avid readers and share books, recipes, music, and strong opinions.
Praise For Snowsisters…
"In a narrative where learning a writer's craft fuels each coming-of-age, the clear literary metaphors for diversity, tradition, and modernity are both thematic and thoroughly satisfying." KIRKUS REVIEWS (12/21/17)
"SNOWSISTERS offers a delightful and much-needed opportunity for LGBTQIA readers to find themselves starring within the pages of a novel, but the message is a universal one: We are the architects of our own stories, and we are the ones who decide when and where those stories are told. A wonderful, important debut."
- Jodi Picoult, NYT bestselling author of SMALL GREAT THINGS
"A compelling portrayal of two young girls from different worlds headed after what they want but slowing down enough to connect and remind us that strength and doing the right thing can be all kinds of messy and complicated—I'm all over that!"
— M-E Girard, author of GIRL MANS UP
"At the heart of this timely, well-crafted novel is a beautifully rendered relationship. Soph and Tess, different as night and day meet at a women's writing retreat in the middle of winter in the middle of nowhere. What happens between them will touch your heart, as all good stories do. A shining and promising debut."
— Sarah Weeks, author of SO B. IT


When you open an account with us, we will ask for your name, address, date of birth, and other information that will allow us to identify you. We may also ask to see your driver's license or other identifying documents.
As a customer of The First National Bank, these restricted transactions are prohibited from being processed through your account or banking relationship with us. If you do engage in an Internet gambling business and open a new account with us, we will ask that you provide evidence of your legal capacity to do so.


text:
Boxing Upcoming Matches and Results
Sauerland Event to Stage Valuev-Chagaev
Sauerland Event will stage Nikolai Valuev´s first mandatory title defense in Germany on April 14. The Berlin-based promoters today came to terms with Universum Box-Promotion, who represent top-ranked challenger Ruslan Chagaev.
The WBA had originally scheduled a purse bid for tomorrow after the two camps had failed to find common ground during their initial negotiations. "This won´t be necessary any more. We have come to terms with Universum today," said Sauerland managing director Chris Meyer. "Of course we wanted to present the clash on one of our fightcards. We were also prepared for the purse bid but after reaching a contractual agreement, there is no need for it. We are staging the fight and we will work closely together with Universum."
The Russian Giant last defended his WBA title in Basle on January 20 when he clinched a third-round TKO victory over Jameel McCline. Chagaev became mandatory challenger back in November following a split decision victory over John Ruiz. It´s the first meeting of the two heavyweights.
Valuev, unbeaten in 47 fights (34 KOs), is just two wins shy of Rocky Marciano´s dream record. Chagaev has won 22 of 23 fights (one draw) along with 17 KOs. He gives up 10.6 inches in height.
The biggest and tallest champion of all time will keep training in St. Petersburg until the end of next week before continuing his preparations in Berlin´s Max-Schmeling-Gym. "I take this fight against Chagaev very seriously," he said. "Everything´s going well in practice. I am in very good shape. I will be ready come April 14."
Boxing Feb 22, 2007


text:
2014 Hundred LDWA - Valleys 100 Section 3 - Maesycwmmer to Trallwn
Valleys 100 report & results
Merchandise Status 31/7/14
Merchandise Closing Date
Route Overview ►
Sections & Photos ►
1. Pontypool to Cwmcarn
2. Cwmcarn to Maesycwmmer
3. Maesycwmmer to Trallwn
4. Trallwn to Llanwonno
5. Llanwonno to Abercynon
6. Abercynon to Gelligaer
7. Gelligaer to Deri
8. Deri to Butetown
9. Butetown to Tredegar
10. Tredegar to Pen y Fan Pond
11. Pen y Fan Pond to Nantyglo
12. Nantyglo to Cwmavon
13. Cwmavon to Keeper's Pond
14. Keepers' Pond to Llanellen
Rules ►
Main Event Rules
Marshals Event Rules
Report & Results
Choice Of Start Time
Qualifying Events ►
Organisation ►
Risk Assessment ►
HQ information
Merchandise Delivery
Information ►
Checkpoint Data ►
HQ Pontypool
CP1. Cwmcarn
CP2. Maesycwmmer
CP3. Trallwn
CP4. Llanwonno
CP5. Moriah Hall
CP5. Abercynon
CP6. Gelligaer
CP7. Deri
CP8. Bute Town
CP9A Bedwellty House
CP9B Georgetown CC
CP10. Pen y Fan Pond
CP11. Nantyglo
Hundred advice
Route Recce
Community Links ►
Valleys 100 Section 3 - Maesycwmmer to Trallwn
Section 3 of The Valleys Hundred leaves Maesycwmmer and heads west via ancient trackways to climb to Mynydd and Cefn Eglwysilan before descending to the Taff Valley and Pontypridd.
From the community hall in Maesycwmmer, the route crosses the Rhymney Valley via ancient trackways. To the north the very impressive Maesycwmmer Viaduct can be seen. The viaduct, which is also known as the Hengoed Viaduct, was built in 1857, but due to a decline in the demand for coal, the viaduct saw the last train cross its impressive structure in 1964. The viaduct was offered for sale for £1 and is now owned by Railway Paths Ltd. Following restoration, the viaduct was reopened in 2000 and is now open to walkers and cyclists.
The route now climbs to the summit of Mynydd Eglwysilan where 360 degree views will be enjoyed before heading south along the Senghenydd Dyke. The dyke is believed to have been created as a land boundary for the Lords of Senghenydd in 1267. The route contours around the Senghenydd Valley. Senghenydd, the town that is below and to the left of the route, saw the worst pit disaster in Great Britain in 1913 when due to an explosion, 439 miners lost their lives.
A small section of rough moorland crossing follows and this section will be flagged for the main event. The route climbs to the summit of Cefn Eglwysilan and as you approach the summit, suddenly the Taff Valley reveals itself with the Rhondda Hills beyond. Views as far as Exmoor as well as the Western and Central Beacons will be enjoyed here.
The route descends from the open moorland to an agricultural area above Pontypridd from where it then descends. A little street walking will then ensure arrival at checkpoint 3 at Trallwn Community Centre.


text:
Q: Lightning pageReference state no longer contains my parameter/attribute in Summer 19 Ran into an issue after the Summer 19 preview in my sandbox. I have a Visualforce page re-directing to a lightning URL Accessible component passing in a parameter, here is the controller code:
PageReference pr = new PageReference('/lightning/cmp/c__SomeComponent?someParam='+this.someParam);

Everything is working fine in production; however, in the Sandbox the PageReference state no longer registers the parameter when the lightning page initializes.
When I do a debug, I see that the parameter is being passed from my VF controller as a part of the URL; however, when the page loads the parameter is no longer listed in the URL. Additionally, I checked what was actually being processed in the state:
console.log(Object.getOwnPropertyNames(component.get('v.pageReference').state));

It did not contain the attribute I was passing.
Color me confused.

A: After some teeth gnashing and head banging, I found this.
I saw that they were including c__ in the parameter's, so I decided that it couldn't hurt to give it a whirl. Lo and behold the parameter passed without issue.
Any parameter I attempted to pass in was being eliminated without including adding c__ to its name.
So long story short, if you parameters are disappearing when calling a URL Accessible Lightning component, add c__ to the front of the parameter name, then adjust your controller to retrieve the parameter with c__ and you should be in business.
//Apex Controller
//before
PageReference pr = new PageReference('/lightning/cmp/c__SomeComponent?someParam='+this.someParam);
//after
PageReference pr = new PageReference('/lightning/cmp/c__SomeComponent?c__someParam='+this.someParam);

//Component Controller
//before
component.get('v.pageReference').state.someParam
//after
component.get('v.pageReference').state.c__someParam

Once you make those quick adjustments, you'll be in business again. This is the first time an SFDC upgrade has broken my code in a long time, so hopefully this will help save folks from much teeth gnashing and headbanging.
Update, a critical update applied the Summer 19 upgrade caused the issue.

redpajama_set_name: RedPajamaStackExchange


text:
Атака в порту Бердянська — удар, який завдали українські війська по кораблях ВМС Росії, пришвартованим у порту Бердянськ, 24 березня 2022 року під час російського вторгнення в Україну. Великий десантний корабель «Саратов» був знищений і два Ropucha-class десантні кораблі отримали пошкодження, але змогли залишити порт. «Саратов» був найбільшим військовим кораблем, затопленим у бою з 1982 року. Це одна з найтяжчих морських втрат, яку зазнала Росія з початку вторгнення, і один із найзначніших успіхів України.

Взяття та окупація Бердянська

26 лютого російські війська захопили порт Бердянськ та аеропорт Бердянськ. Наступного дня російські військові повністю взяли під контроль місто.

Починаючи з 14 березня порт використовувався росіянами як логістичний центр для підтримки їхнього наступу на півдні України й, зокрема, облоги Маріуполя. 21 березня російське ЗМІ «Звезда» повідомило про прибуття до Бердянська десантних транспортів. Офіцер ВМС Росії назвав це «знаковою подією, яка відкриє логістичні можливості Чорноморському флоту».

Атака
Напад стався о 7:45 24 березня. Вогонь на борту Alligator-class Десантний корабель «Саратов» спричинив великий вибух, оскільки судно, очевидно, було навантажене боєприпасами. Внаслідок вибуху пошкоджено два сусідніх Ropucha-class десантні кораблі «Цезарь Куніков» і «Новочеркаськ». Обидва ці кораблі втекли з порту під час гасіння власних пожеж, а згодом повернулися до Криму. Інші пошкодження включали великі резервуари з нафтою на пристані та сусіднє торгове судно, яке було пришвартовано там ще до вторгнення, обидва з яких все ще горіли наступного дня.

Спочатку повідомлялося, що атака була здійснена за допомогою тактичної балістичної ракети ОТР-21 «Точка», але деякі джерела стверджували, що був застосований безпілотник Bayraktar TB2.

Наслідки
Пізніше супутникові знімки підтвердили, що «Саратов» затонув у гавані, а його надбудова видно над поверхнею. Українські джерела стверджували, що на «Цезарі Куникові» загинули 8 членів екіпажу та 3 на «Новочеркаську», але про втрати на «Саратові» не повідомлялося.

Британська розвідка оцінила, що затоплення «Саратова» зашкодить впевненості російського флоту в дії поблизу українського узбережжя. За даними міністерства оборони США, станом на 31 березня жодних спроб поповнити запаси десантних кораблів не спостерігалося.

На початку липня було поширене фото піднятих на поверхню решток корабля, було видно, що він був знищений ущент.

Див. також

 Російський наступ на схід України (2022)

Примітки

Бердянськ
Б
Б
2022 у Запорізькій області
Б
Корабельні аварії під час російсько-української війни


text:
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="ru.stqa.pft.addressbook.tests.GroupDeletionTests" tests="1" skipped="0" failures="0" errors="0" timestamp="2017-08-22T11:18:23" hostname="gpodmorina" time="0.407">
  <properties/>
  <testcase name="testGroupDeletion" classname="ru.stqa.pft.addressbook.tests.GroupDeletionTests" time="0.407"/>
  <system-out><![CDATA[Hibernate: select groupdata0_.group_id as group_id1_2_, groupdata0_.group_footer as group_fo2_2_, groupdata0_.group_header as group_he3_2_, groupdata0_.group_name as group_na4_2_ from group_list groupdata0_
Hibernate: select groupdata0_.group_id as group_id1_2_, groupdata0_.group_footer as group_fo2_2_, groupdata0_.group_header as group_he3_2_, groupdata0_.group_name as group_na4_2_ from group_list groupdata0_
]]></system-out>
  <system-err><![CDATA[]]></system-err>
</testsuite>

redpajama_set_name: RedPajamaGithub


Designed with an exclusive blend of reinforced, woven Thermoplastic and Pure Silicone materials to form the highest quality engine hoses.
This improves water flow and can withstand temperatures of up to 180℃.
We integrated a unique inner liner made from an HDPE coating, that improves and increases the flow of liquids, which offers improved cooling for your race bike.
We have also created "Y" shape kits that eliminate the plastic fitting found on most late model motocross engines that can melt under extreme heat, which can cause engine failure.


Designed by Brittany Pozzi who was named Top Horse Owner of the Decade. RaceX offers advanced technology to aid in lung support in competition settings, and essential digestive enzymes to keep the pH balance of the stomach at optimal level.
Hemicellulose Extract, Salt, Potassium Chloride, Magnesium Oxide, L-Lysine, Menadione Nicotinamide Bisulfate ( Source of Vitamin K Activity and Supplement Niacin), Ascorbic Acid (Source of Vitamin C), Vitamin E Supplement, Citric Acid, Menthol, Guar Gum, Xanthan Gum, Maltodextrins, Silica, Fermentation Products, Flavorings.
Pre-Performance Program: Feed 20cc orally two hours prior to competition.


text:
Road Trip through History: The Colchagua Museum
By Pamela | February 2, 2014 | 0
We might not have gone to the Colchagua Museum in Santa Cruz, Chile, if one of our local hosts hadn't recommended it so strongly. The guidebooks described it as a private collection that had been turned into a museum–something I always approach with the caution. Private collections fueled by a personal passion often create a museum that is not accessible to a viewer who does not share the passion. A wealthy collector is no guarantee that the collection will be coherent or interesting–though it will probably be well-displayed*
The Colchagua Museum focuses on Chilean history, from the prehistoric times to 2010. That's a lot of ground for a moderately sized museum to cover. Not surprisingly, the museum is uneven in its presentation. But when it's good–wow!
The rooms devoted to fossils, paleontology and pre-Columbian history are excellent. The artifacts displayed are some of the finest I've ever seen, interspersed with an occasional clearly identified reproduction needed to illustrate a specific point. The following points in particular grabbed my attention:
A very clear description of the appearance, division, convergence, and re-division of the continents over the millennia. I was aware of Pangaea, the super-continent that broke apart to form the continents as we know them. I wasn't aware that Pangaea was probably preceded by at least one other supercontinent.
The megatherium was an actual prehistoric animal, not a detail made up by E. Nesbit in her wonderful children's novel, Five Children and It. Some times the depths of my own ignorance amaze me.
One of the many failings of American history classes,** is that the only pre-columbian cultures we are taught about are the Aztecs, Mayans, and Incas.*** The Colchagua Museum outlines the range of Andean cultures that preceded the Incas in almost overwhelming detail.
The suggestion that the ancient Japanese may have reached South America, based on a perceived resemblance between artifacts of the Jōmon culture of Japan (10,000 BCE to 300 BCE) and the Valdivia culture of Ecuador (3500 BCE-1800 BCE). This strikes me as being roughly equivalent to thinking the ancient Egyptians settled Central America because the Central American cultures built pyramids. But I don't really know enough about either culture to judge. I'm adding this to my Big List of Questions, but would be happy to hear from any of you who actually know something.
Once we moved past the Incas and into the rooms devoted to the period from the Spanish conquest through the nineteenth century, the exhibits felt more random. They seem to be based on the artifacts at hand rather than a clear sense of narrative.**** By the time we reached the extraordinarily handsome nineteenth century popcorn machine, we were ready to skip ahead to the exhibit that brought us to the museum.
The Great Rescue tells the story of the 33 Chilean miners who were trapped when the San José mine collapsed on August 5, 2010. For 69 days, the world watched as the Chilean government and an international team that included mining engineers, drilling experts and NASA worked to rescue the miners. It was a story that had everything: suspense, human interest, heroic engineering, and a happy ending.
The exhibit tells the story from many angles. The shanty town that grew around the mine entrance where family members waited to hear news. The history of the mine itself. How the miners survived before the rescue team made contact. The moment when a drill bit pulled up the first communication from the miners. The technical details of the rescue. The rescue itself. It ends with video footage of each miner being brought up and reunited with their families. My experience of the exhibit was both visceral and intellectual. I struggled with sympathetic claustrophobia, marveled over the technical brilliance of the solutions, and choked up more than once.
We had planned to go to the Darwin room after we finished the Great Rescue, but decided it would be an anticlimax.
* The Arthur Rubloff paperweight collection at the Art Institute of Chicago comes to mind. Pretty things, but do they really deserve their own gallery?
** By which I mean history taught in the United States, as opposed to classes about American history. Though now that I think about it, the failing is one of classes about American history taught in the United States.
***Indulge me in a small rant, will you? Obviously the only reason we are taught about these cultures is that they were standing on the shores when the Spanish invaded. (We tend to learn about the history of other countries only at the points where it intersects with our own interests.) Worse, from my perspective , our textbooks often describe these cultures as "ancient" even though they are clearly the contemporaries of the Europeans who invaded them. This is one side-step away from terming them "primitive". Which they clearly were not.
****Though I was fascinated by a case that traced the history of printing in Latin America from pre-columbian seals to the nineteenth century printing press.
A few travel notes for anyone who finds themselves in central Chile
If you're interested in eighteenth and nineteenth century history and have even a little Spanish, you're better off going to the national historical museum in Santiago. The exhibits aren't as flashy. They don't have an audio tour (or English signs). But there is a clear narrative, not just one dang thing after another.
If you have several hours to spare, head down the road from Santa Cruz to the small town of Lola and visit the Chilean Craft Museum. Historical examples of traditional Chilean handicrafts shown in contrast with the work of modern artisans in those same crafts. Utterly fascinating, even without any Spanish.
Two words: wine tour. Trust me on this.
Posted in American History, History on Display, Odd Bits, Road Trip Through History and tagged Colchagua museum, Road Trip Through History