text:
\section{Method}

Our goal is to improve the visual realism of a low-quality synthetic image. In particular, we focus on translating images of indoor scenes, $I_{o}$, from the domain of OpenGL-rendered images $\mathcal{O}$, to the domain of real photographs, $\mathcal{R}$. This is an unpaired problem and has to be learned without direct supervision. The translation has to handle two aspects: first, simple rendering engines like OpenGL do not model complex, real-world lighting, and second, synthetic scenes usually do not have realistic real-world materials and textures. We propose handling this by explicitly manipulating the synthetic shading and albedo separately. Moreover, we find that directly translating from OpenGL to real images is challenging due to the large domain gap between them; changing OpenGL shading to real-world shading requires large non-local transformations that are challenging to learn in an unsupervised manner. However, synthetic scenes can be rendered with physically-based renderers that can generate more realistic shading. We leverage this to propose a two-stage translation. First, we translate the OpenGL image $I_{o}$ to the physically-based rendering (PBR) domain, $\mathcal{P}$, by transforming only the shading using paired OpenGL-PBR images. We then translate these PBR images to the real domain, $\mathcal{R}$, by refining both the albedo and shading using two separate networks; this is done in an unsupervised manner. Figure~\ref{fig:framework} shows an overview of our two-stage framework, and we describe them in following sections.

\subsection{OpenGL-to-PBR Image Translation}
While rendering scenes with OpenGL is computationally fast, the resulting images have low visual realism (e.g., see Figure~\ref{fig:teaser}(a)). One of the reasons is that standalone OpenGL only supports simple lighting models (such as directional or point lights) and does not model complex lighting effects like global illumination. In comparison, a physically based renderer can generate images with photorealistic shading by simulating the propagation of light through a scene, albeit at the cost of significantly more processing time. Therefore, we render the same scene with OpenGL and a physically based renderer Mitsuba~\cite{mitsuba} respectively. Both these images have the same geometry and material properties, and differ only in the quality of the shading. We train a neural network on these image pairs to translate the OpenGL image to the PBR domain; this network thus learns to synthesize more realistic shading from a simple OpenGL scene, bypassing the cost of physically-based rendering.

We use paired conditional generative adversarial networks~\cite{isola2017pix2pix, wang2018pix2pixhd} for this task. Let $I_{o}, I_{p}$ be a pair of images in the OpenGL and PBR domains, respectively. Since both the images differ only in their shading components, we decompose them into albedo and shading layers and train a shading generator, $G^S_{{o \rightarrow p}}(\cdot)$, that transforms OpenGL shading, $S_{{o}}$, to PBR shading, $S_{{p}}$. Instead of using only the OpenGL shading as input, we use the auxiliary buffers of the synthetic scene including albedo $A_{o}$ and surface normals, $N_{{o}}$, as the input to the generator. These additional buffers encode semantic (via albedo) and geometric (via normals and shading) information about the synthetic scene and aid in the translation. Finally, we multiply the synthesized PBR shading, $\bar{S}_{p}$, with the original OpenGL albedo, $A_{o}$, to reconstruct the PBR image
$\bar{I}_{{p}}$:
\begin{equation}
    \bar{S}_{{p}} = G_{{o \rightarrow p}}^S(S_{o}, A_{o}, N_{o}),\;\;\; \bar{I}_{{p}} = \bar{S}_{p} * A_{o}
\end{equation}
Our intrinsic image model assumes Lambertian shading. While this is an approximation to real world reflectance, it is sufficient for many regions of indoor images, and is widely used~\cite{bell2014intrinsic,bi2018intrinsic,li2018intrinsic}. To recover high dynamic shading values, we predict the shading in the logarithmic space.

\begin{figure}[t]
  \includegraphics[width=\linewidth]{./images/shading-dis-comp-1.pdf}
  \vspace{-0.6cm}
  \caption{With the shading discriminator, our network is able to predict more accurate shading images
  and get rid of problems such as inconsistent colors.}
    \vspace{-0.6cm}
  \label{fig:shading-dis}
\end{figure}

Similar to pix2pixHD~\cite{wang2018pix2pixhd}, we use a combination of a perceptual reconstruction loss (based on VGG features) and adversarial losses. We utilize adversarial losses via a discriminator on the predicted image, $D^I_{{o \rightarrow p}}$, as well as one on the predicted shading, $D^S_{{o \rightarrow p}}$. This ensures that both the generated images and shadings are aligned with the distribution of PBR images and shading. We used conditional discriminators that also take  albedo and normals as input.

By translating only the shading and multiplying it with the original albedo, we ensure that the textural structure of the albedo is explicitly preserved. Synthesizing the shading separately is also an easier task for the network because the shading component is spatially smooth and does not have the high-frequency details of the image. Moreover, this also allows us to incorporate the shading discriminator, $D^S_{{o \rightarrow p}}$, that provides stronger supervision for training. As shown in Figure~\ref{fig:shading-dis}, without the shading discriminator the network predicts unrealistic illumination with inconsistent colors while our full network avoids this problem.

\subsection{PBR-to-Real Image Translation}

The first stage of our pipeline generates images with more physically accurate shading and global illumination. However, these images may still have a domain shift from real photographs. The albedos are still from the low-quality scenes and may have unrealistic colors. The shading is also still a function of the synthetic scenes and might not align with the intensity, color, contrast and spatial distribution of real-world scenes. In the second stage, we seek to bridge this remaining domain gap to a target real image dataset.

Unlike the first stage, there is no one-to-one correspondence between predicted PBR images and real images. Zhu et al.~\cite{zhu2017cyclegan} introduced the CycleGAN framework to tackle such unpaired problems and we build on their framework for our second stage. Different from the original CycleGAN network which performs the translation in image space, we propose translating the albedo and shading components separately with different generators. This novel design is based on the insight that the albedo contains high frequency details, and should be modified locally to better preserve structural details. On the other hand, shading is a global phenomenon (as it is a function of scene and light source geometry) and should take global context into consideration. Similar observations have been made in~\cite{gershbein1994textures}, which uses different operators for global irradiance and local radiosity.

The output of the first stage is the predicted PBR image, $\bar{I}_{p}$. As noted before, this is the product of the predicted (OpenGL-to-PBR) shading, $\bar{S}_{p}$, and the original scene albedo, $A_{p}$ (which is the same as $A_{o}$). As shown in Figure~\ref{fig:framework}, to translate from the PBR domain to the real domain (${p}\rightarrow{r}$), we use two
generators $G^A_{{p \rightarrow r}}(\cdot)$ and $G^S_{{p \rightarrow r}}(\cdot)$ to synthesize the real albedo and shading, $\bar{A}_{r}$ and $\bar{S}_{r}$, respectively. The final predicted real image, $\bar{I}_{r}$, can then be reconstructed as:
\begin{equation}
    \bar{A}_{r} = G^A_{{p \rightarrow r}}(A_{p}), \; \bar{S}_{r} = G^S_{{p \rightarrow r}}(\bar{S}_{p}), \;\;\; \bar{I}_{r} = \bar{A}_{r} * \bar{S}_{r}
    \label{eqn:stage-2}
\end{equation}

We use different architectures for $G^A_{{p}}$ and $G^S_{p}$. For the albedo, we use a fully convolutional network without downsampling or upsampling blocks. This results in a small receptive field for the network and better preserves the texture details while avoiding large structural changes~\cite{ignatov2017dslr, ignatov2018wespe}. As shown in Figure~\ref{fig:albedo-arch-comp}, allowing downsampling blocks in the albedo generator leads to serious high-frequency artifacts in the textures. Our architecture removes this problem and achieves results with higher quality. In contrast, the shading generator uses downsampling blocks for a larger receptive field in order to allow global changes.

\begin{figure}[t]

    \includegraphics[width=0.98\linewidth]{./images/backcycle-v2.pdf}
    \vspace{-0.3cm}
    \caption{To finish the backward cycle, the real image is first translated to the PBR domain.
    Afterwards we use the pretrained intrinsic decomposition network $H$ to decompose it into its albedo and
shading, which are further fed to corresponding generators. Finally we multiply
the output albedo and shading to reconstruct the original real image.}
    \label{fig:backward}
    \vspace{-0.5cm}
\end{figure}

Similar to CycleGAN, we constrain the forward ${p \rightarrow r}$ translation using a backward ${r \rightarrow p}$ translation. Unlike the PBR domain, we do not have access to albedo and shading layers for real images. Therefore, we use an image-space generator $G_{r \rightarrow p}(\cdot)$ that transforms real images to the PBR domain. We utilize two discriminators, $D_{r}$ and $D_{p}$, to distinguish real and fake samples in real and PBR domains respectively. Here, $D_{r}$ distinguishes between the PBR images translated to the real domain ($\bar{I}_{r}$ from Equation~\ref{eqn:stage-2}) and real images. $D_{p}$ on the other hand discriminates between PBR image (synthesized from the first stage), $\bar{I}_{p}$, and real images translated to the PBR domain, $G_{r \rightarrow p}(I_{r})$. Note that while the generators for the ${p \rightarrow r}$ direction are applied to albedo and shading, the discriminator is applied to the image computed as their product. We train the network by optimizing the standard GAN loss $\mathcal{L}_{\text{GAN}}(G^A_{p \rightarrow r}, G^S_{p \rightarrow r}, D_{r})$ and $\mathcal{L}_{\text{GAN}}(G_{r \rightarrow p}, D_{p})$ for the forward translation ${p \rightarrow r}$ and backward translation ${r \rightarrow p}$, respectively.

Only having the GAN loss is not sufficient for learning meaningful translations because of the lack of pixel-level correspondence~\cite{yi2017dualgan}. Similar to CycleGAN, we also include forward and backward cycle consistency losses. The forward cycle consistency, ${p \rightarrow r \rightarrow p}$ is trivial: we project the predicted real image $\bar{I}_{r}$ back to the PBR domain by feeding it to the generator $G_{{r \rightarrow p}}$, and minimize the $L_1$ loss between the output and the PBR source image $\bar{I}_{p}$:
\begin{align}
    \mathcal{L}_{\text{for}}(G^A_{p \rightarrow r}, G^S_{p \rightarrow r}, G_{r \rightarrow p}) = ||G_{r \rightarrow p}(\bar{I}_{r}) - \bar{I}_{p}||_1 \nonumber \\
    = ||G_{r \rightarrow p}(G^A_{p \rightarrow r}(A_p)*G^S_{p \rightarrow r}(\bar{S}_p)) - \bar{I}_{p}||_1
    \label{eqn:forward}
\end{align}

Specifying the backward cycle is more challenging. We can map a real image $I_{r}$ to the PBR domain as $G_{r \rightarrow p}(I_{r})$. However, translating it back to the real domain via the forward ${p \rightarrow r}$ translation requires an albedo-shading separation that we do not have for these images---we only have them for our original synthetic images. We tackle this by training an intrinsic decomposition network~\cite{li2018intrinsic, bi2018intrinsic, cheng2018intrinsic} to predict the albedo and shading layers for PBR images.

Let $H$ be the intrinsic decomposition network. Given a PBR image $I$ and corresponding albedo $A_I$ and shading $S_I$, $H$ is trained by optimizing the following loss function:
\begin{align}
    \mathcal{L}(H) = ||H_A(I) - A_I||_2^2 +||H_S(I) - S_I||_2^2,
\end{align}
where $H_A(I)$ and $H_S(I)$ are the predicted albedo and shading by the network $H$. We adopt the network architecture used in Li et al.~\cite{li2018intrinsic}, which contains one encoder and two decoders with skip connections. While they use a scale-invariant mean square error (MSE) loss, we use MSE because we require the product of the albedo and shading to be identical to the original image. The intrinsic decomposition network $H$ is pretrained on the predicted PBR images from the first stage, $\bar{I}_{p}$, where we have ground truth albedo and shading. Afterwards, it is fixed during the training of the image translation networks.

We use the intrinsic decomposition network, $H$, to decompose $G_{{r \rightarrow p}}(I_{r})$ into its albedo and shading. Then we can translate each component via the forward ${p \rightarrow r}$ translation to synthesize the result of the full backward cycle, leading to the following backward cycle consistency loss:
\begin{align}
    &I_{p}' = G_{r \rightarrow p}(I_{r}) \nonumber \\
    &I_{r}'' = G^S_{p \rightarrow r}\big(H_S(I_{p}')) * G^A_{p \rightarrow r}(H_A(I_{p}'))\big) \nonumber \\
    &\mathcal{L}_{\text{back}}(G^A_{p \rightarrow r}, G^S_{p \rightarrow r}, G_{{r \rightarrow p}}) = || I_{r}'' - I_{r}||_1
    \label{eqn:backward}
\end{align}
Figure~\ref{fig:backward} shows the formulation of our backward cycle.

Note that our network is asymmetric; the forward translation takes our PBR albedo and shading layers, translates and combines them to construct the result. Our backward translation does the opposite; it first translates real images to the PBR domain and then decomposes them there. This allows us to bypass the requirement for real albedo-shading data, and instead train a PBR albedo-shading decomposition \emph{for which we have ground truth supervision}. As can be seen in Figure~\ref{fig:comp_back_cycle}, utilizing this novel backward consistency cycle significantly reduces artifacts and improves the visual quality of our PBR-to-real translations.

\comment
{
In summary, the final loss function for the PBR-to-real translation is as follows:
\begin{align}
    \mathcal{L}(&G^A_{p}, G^S_{p}, G_{{r}}, D_{{p}}, D_{{r}}) =
  \mathcal{L}_{\text{GAN}}(G^A_{p},G^S_{p}, D_{{r}}) \nonumber \\
  & + \mathcal{L}_{\text{GAN}}(G_{{r}}, D_{{p}})
  + \lambda_{\text{cyc}} \mathcal{L}_{\text{forward}}(G^A_{p},G^S_{p},
  G_{{r}}) \nonumber \\
  & + \lambda_{\text{cyc}} \mathcal{L}_{\text{backward}}(G^A_{p}, G^S_{p}, G_{{r}}),
\end{align}
where $\lambda_{cyc}$ is the weight for the cycle consistency loss.
 }
Our final loss function for PBR-to-real translation is a combination of the two GAN losses (from discriminators $D_{{p}}$ and $D_{{r}}$), and the forward and backward cycle consistency losses (Equations~\ref{eqn:forward} and \ref{eqn:backward}).

\begin{figure}[t]
    \begin{subfigure}{.5\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/no-backward-cycle-comp/26_fake_B_no_cycle.png}
        \subcaption{Without $\mathcal{L}_{\text{back}}$}
    \end{subfigure}%
    \begin{subfigure}{.5\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/no-backward-cycle-comp/26_fake_B_our.png}
        \subcaption{With $\mathcal{L}_{\text{back}}$}
    \end{subfigure}%
    \vspace{-0.3cm}
    \caption{Without the backward cycle, the network tends to generate outputs with undesired new structures.
    Adding the backward cycle with a pretrained intrinsic decomposition network is able to generate images with higher quality.}
    \label{fig:comp_back_cycle}
\end{figure}

\begin{figure}[t]
        \vspace{-0.3cm}
    \begin{subfigure}{.5\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/albedo-arch-comp/c27f207fc10b7580d72ef0b97d2b9e23_000004_with_downsample.png}
        \subcaption{With downsampling}
    \end{subfigure}%
    \begin{subfigure}{.5\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/albedo-arch-comp/c27f207fc10b7580d72ef0b97d2b9e23_000004_without_downsample.png}
        \subcaption{Without downsampling}
    \end{subfigure}%
          \vspace{-0.3cm}
    \caption{Comparison between with and without downsampling blocks in the albedo generator. From the result, we can see that
       the generator without downsampling blocks could better preserve the texture structures of the input.}
       \vspace{-0.3cm}
    \label{fig:albedo-arch-comp}
\end{figure}

\comment{

\section{Method}

Given an indoor image  $I_\mathcal{O}$ in OpenGL domain $\mathcal{O}$, our goal is to
improve its visual realism so that its appearance would
match the target real images. The visual realism improvement mainly comes from two
aspects: more natural material properties such as albedo colors, and more realistic shading.
Direct translation from OpenGL images to
real images is challenging due to large differences in image characteristics.
Considering this, we propose a two-stage translation.
First, we translate the OpenGL image $I_\mathcal{O}$ to the PBR domain $\mathcal{P}$ by predicting physically accurate
shading for the input scene in a supervised setting.  In the second stage, we
further translate it from the PBR to the real domain $\mathcal{R}$ by refining the albedo and
shading using two networks with different architectures.
The overview of our two-stage framework is shown in Figure~\ref{fig:framework}, and
we describe these two stages in following sections.

\subsection{OpenGL to PBR Image Translation}
While rendering synthetic scenes with OpenGL provides an efficient way to
generate abundant images, the image realism is usually low since standalone OpenGL
only supports simple lighting models such as directional or point lights and
lacks global illumination. In comparison, a physically based renderer could
generate images with more realistic shading by simulating the process of light
traversing the scenes and interacting with object surfaces. Based on this
observation, we render the same scene with OpenGL and a physically based renderer
Mitsuba~\cite{mitsuba} respectively, and
train a neural network on these images pairs to translate the OpenGL image to
the PBR domain.

Intuitively, we could train the network by feeding the OpenGL image to
the network and minimizing the difference between the output and corresponding
ground truth PBR image. However, there are two problems with this formulation.
First, since the lighting configurations for the OpenGL rendering and PBR
rendering may be different, each OpenGL image may correspond to multiple
reasonable outputs, and minimizing the pixel-space difference will
result in blurry outputs~\cite{nalbach2017shading}. Second, the entanglement of albeo and
shading also poses a significant hurdle to the translations due to inevitable
information loss when generating OpenGL images.

To solve the problems mentioned above, we use conditional generative
adversarial networks~\cite{isola2017pix2pix, wang2018pix2pixhd} for this task.
Let $I_\mathcal{O}, I_\mathcal{P}$ be a pair of images in OpenGL and
PBR domain.
Instead of using only OpenGL image as input,
we use the auxiliary buffers of the synthetic scene including albedo $A_\mathcal{O}$, normal $N_{\mathcal{O}}$,
as well as OpenGL shading
$S_{\mathcal{O}}$ as the input to the generator $G_{\mathcal{P}}$, and predict a new PBR
shading image $S_{\mathcal{P}}^*$. The albedo includes semantic information about the scene,
and the other two buffers provide geometric information.
After we get the predicted shading, we reconstruct the desired PBR image
$\hat{y}$ by multiplying it by the original albedo:
\begin{align}
    S_{\mathcal{P}}^* = G_{ \mathcal{P}}(M_\mathcal{O})&, M_{\mathcal{O}} = \{A_\mathcal{O}, N_\mathcal{O}, S_\mathcal{O}\}\\
    I_{\mathcal{P}}^* &= S_\mathcal{P}^* \cdot A_\mathcal{O}
\end{align}
To recover high dynamic shading values, we predict the shading in the logarithmic space.
We assume a Lambertian imaging model for surface reflectance. While
it is a rough approximation to the real world reflectance, it is enough to
cover most regions of indoor images, and is widely used to describe the
intrinsic properties of real-world scenes~\cite{li2018intrinsic,
bi2018intrinsic, cheng2018intrinsic}.

We train two discriminators $D^I_{\mathcal{P}}$  and $D^S_{\mathcal{P}}$ to guarantee that the
generated images and shading are aligned with the distribution of PBR images
and shading. The discriminators take the tuple of input $M_\mathcal{O}$ and predicted shading
as fake samples, while the concatenation of $M_\mathcal{O}$ and ground truth PBR shading
are regarded as real samples.
The networks are trained by alternately minimizing generator loss and maximizing discriminator loss.

\begin{figure}[t]
  \includegraphics[width=\linewidth]{./images/shading-dis-comp-1.pdf}
  \caption{With shading discriminator, our network is able to predict more accurate shading image
  and get rid of problems such as inconsistent colors.}
  \label{fig:shading-dis}
\end{figure}

We predict a shading image and multiply it by the albedo to
reconstruct the final image. In comparison to directly  regressing the desired
image, the shading image has a stronger spatially coherency and lies on a
lower dimensional manifold, which reduces the search space of network output values
and avoids possible color shifts.
In addition, it is also easier for the network to capture a low frequency
shading image than reconstructing high frequency details in images.
Moreover, in addition to a single discriminator on images, we are able to train another
discriminator $D_{\mathcal{P}}^S$ to distinguish between real and fake shading, which is
also an easier task than discriminating in image space and provides stronger
supervision for the training. As shown in Figure~\ref{fig:shading-dis}, without shading
discriminator, the network predicts unrealistic illuminations and
inconsistent colors in shading image, while our full network is able to
get rid of this problem.
By disentangling the albedo and shading in the prediction, we are explicitly preserving all the texture
information in the input, and only modifying the shading, which enables our network to
generalize better to new scenes and produce results with higher quality.

\subsection{PBR to Real Image Translation}
In the first stage, we have been able to predict a more physically accurate
shading for the OpenGL image and transform it to the PBR domain.
However, the albedo and shading distribution of corresponding PBR images may not
be well aligned with the target real dataset. For example, there may exist
great difference in terms of shading intensity and color between predicted PBR images
and real images due to different lighting configurations. Therefore, the
performance of models trained on PBR images still degrades on real world images.
Considering this, we propose a second stage to further align the outputs
of the first stage with real images from the target dataset.

Different from the first stage, there is no one-to-one correspondence between predicted
PBR images and real images, which makes it challenging for the network to
preserve important structures of the source image  during the translation.
Instead of using a single generator and discriminator for the task,
Zhu et al.~\cite{zhu2017cyclegan} introduces the CycleGAN framework and proposes to have generators and
discriminators on both domains. It introduces a cycle consistency loss that
requires that the source image projected to target domain and back to source
domain should produce the original image and vice versa. While CycleGAN
is able to preserve overall structures of the source image, it isn't able to
preserve low-level texture structures, as shown in Figure~\ref{fig:teaser}, especially when the domain gap is very large, which
is essential to guarantee the realism in synthetic to real translation.

Our network is based on the CycleGAN framework. Different from the original
CycleGAN network which performs the translation on image space, we propose to
perform translations on albedo and shading components and have separate generators for
each component. This novel design is based on the insight that the albedo
representing the reflectance contains high frequency details, and should be
modified locally to better preserve the
details. In contrast, shading transformation is a global process and should take global context
into consideration. Similar observations have been made in~\cite{gershbein1994textures}, which uses
different operators for computation of global irradiance and local radiosity.

Following the first stage, let $I_\mathcal{P}^*$ be the predicted PBR image, $S_\mathcal{P}^*$ be the predicted shading.
and $A_\mathcal{P}^*$ be its albedo, which is the same as $A_{O}$.
As shown in Figure~\ref{fig:framework}, to translate from PBR to real domain ($\mathcal{P}\rightarrow\mathcal{R}$), we use two
generators $G^A_{\mathcal{P}}$ and $G^S_{\mathcal{P}}$  to translate the albedo and shading separately,
and the final image $I_\mathcal{R}^*$ could be
reconstructed as following:
\begin{align}
    I_\mathcal{R}^* = G^A_{\mathcal{P}}(A_\mathcal{P}^*) \cdot G^S_{\mathcal{P}}(S^*_\mathcal{P})
\end{align}
Specifically, we use different architectures for $G^A_{\mathcal{P}}$ and $G^S_\mathcal{P}$. For the albedo generator, we
use a fully convolutional neural network without  downsampling or
upsampling blocks, which is able to better preserve the texture details and avoid large structure
changes~\cite{ignatov2017dslr, ignatov2018wespe}. In contrast, the shading generator has
a larger receptive field by including two downsampling blocks
so as to make global changes towards the shading. This design is essential to achieve realistic
translation. As shown in Figure~\ref{fig:albedo-arch-comp}, involving downsampling blocks
in albedo generator will change the texture structures of the original input and result in serious
artifacts in the final image, while our proposed architecture is able to get rid of this problem and achieve
results with higher quality.

\begin{figure}[t]

    \includegraphics[width=0.98\linewidth]{./images/backcycle.pdf}
    \caption{To finish the backward cycle, the real image is first translated to the PBR domain.
    Afterwards we use the pretrained intrinsic decomposition network $F$ to decompose it into its albedo and
shading, which are further feeded to corresponding generator. Finally we multiply the translated
the output albedo and shading to reconstruct the original real image.}
    \label{fig:backward}
\end{figure}

In addition, we have a generator $G_\mathcal{R}$ to translate real image $I_\mathcal{R}$
to PBR domain.  Two discriminators $D_\mathcal{R}$ and $D_\mathcal{P}$ are used to distinguish real and fake
samples in real domain and PBR domain. We train the network by optimizing the standard GAN loss
$\mathcal{L}_{\text{GAN}}(G^A_\mathcal{P}, G^S_\mathcal{P}, D_\mathcal{R})$ and $\mathcal{L}_{\text{GAN}}(G_\mathcal{R}, D_\mathcal{P})$
for the forward translation ${p \rightarrow r}$ and backward translation ${r \rightarrow p}$.

Only having GAN loss is not able to preserve the structures of the input images due to lack
of pixel-level correspondence~\cite{yi2017dualgan}.  We also include the forward and backward cycle
consistency loss. The forward cycle consistency is trivial. We
project the predicted real image $I_\mathcal{R}^*$ back to the PBR domain by feeding it to
the generator $G_{\mathcal{R}}$, and minimize the $L_1$ loss between the output
and source image $I_R^*$:
\begin{align}
    \mathcal{L}_{\text{forward}}(G^A_\mathcal{P}, G^S_\mathcal{P}, G_\mathcal{R}) = \mathbb{E}_{I_\mathcal{P}^*}
    ||G_\mathcal{R}(I_\mathcal{R}^*) - I_\mathcal{P}^*||_1
\end{align}

For the backward cycle consistency loss, we first map the real image $I_\mathcal{R}$ to the
PBR domain and get its corresponding PBR image $G_\mathcal{R}(I_\mathcal{R})$. Since
we don't have ground truth albedo and shading for the translated image, we cannot directly map
it  back to the real domain via generators $G_A$ and $G_S$.

To tackle this problem, we train an intrinsic decomposition
network~\cite{li2018intrinsic, bi2018intrinsic, cheng2018intrinsic} to predict their albedo and shading
on PBR images. Figure~\ref{fig:backward} shows the formulation of our backward cycle.
Let $F$ be the intrinsic decomposition network. Given a PBR image $I$ and corresponding albedo $A_I$ and
shading $S_I$, $F$ is trained by optimizing the following loss function:
\begin{align}
    \mathcal{L}(F) = \mathbb{E}_I[||F_A(I) - A_I||_2^2 +||F_S(I) - S_I||_2^2]
\end{align}
where $F_A(I)$ and $F_S(I)$ are the predicted albedo and shading by the
network $F$. We adopt the network architecture used in Li et al.~\cite{li2018intrinsic},
which contains one encoder and two decoders with skip connections. The
difference here is that while Li et al. uses a scale-invariant mean square error(MSE) in the loss
function, we instead use MSE since we require that the product of predicted
albedo and shading is identical to the original image to enforce the cycle consistency loss.
The intrinsic decomposition network $F$ is pretrained on our predicted PBR images in the first stage
where we have ground truth albedo and shading. Afterwards it is fixed during the training of the image translation networks.

With the pretrained intrinsic decomposition network $F$, we decompose
$G_{\mathcal{R}}(I_\mathcal{R})$ into its albedo and shading image. Then we can
translate each component via the generators $G^A_\mathcal{P}$ and $G^S_\mathcal{P}$ respectively and reconstruct
the image  $I_\mathcal{R}'$ mapped back to the real domain by multiplying the albedo and shading after
translation:
\begin{align}
    I_\mathcal{R}' = G^S_\mathcal{P}(F_S(G^S_\mathcal{R}(I_\mathcal{R}))) \cdot G^A_\mathcal{P}(F_A(G_{\mathcal{R}}(I_\mathcal{R})))
\end{align}
Given the reconstructed image $I_\mathcal{R}'$, the consistency loss
$\mathcal{L}_{\text{backward}}(G^A_\mathcal{P}, G^S_\mathcal{P}, G_{\mathcal{R}})$ for the backward cycle
is:
\begin{align}
    \mathcal{L}_{\text{backward}}(G^A_\mathcal{P}, G^S_\mathcal{P}, G_{\mathcal{R}}) = || I_\mathcal{R}' - I_\mathcal{R}||_1
\end{align}
We compare the result with and without the backward cycle in Figure~\ref{fig:comp_back_cycle}. The result
clearly demonstrates that adding backward cycle significantly reduces artifacts and improves the visual quality.

In summary the loss function for the PBR to real translation is as follows:
\begin{align}
    \mathcal{L}(&G^A_\mathcal{P}, G^S_\mathcal{P}, G_{\mathcal{R}}, D_{\mathcal{P}}, D_{\mathcal{R}}) =
  \mathcal{L}_{\text{GAN}}(G^A_\mathcal{P},G^S_\mathcal{P}, D_{\mathcal{R}}) \nonumber \\
  & + \mathcal{L}_{\text{GAN}}(G_{\mathcal{R}}, D_{\mathcal{P}})
  + \lambda_{\text{cyc}} \mathcal{L}_{\text{forward}}(G^A_\mathcal{P},G^S_\mathcal{P},
  G_{\mathcal{R}}) \nonumber \\
  & + \lambda_{\text{cyc}} \mathcal{L}_{\text{backward}}(G^A_\mathcal{P}, G^S_\mathcal{P}, G_{\mathcal{R}})
\end{align}
where $\lambda_{cyc}$ is the weight for the cycle consistency loss. Similar to the first stage,
the networks are optimized by alternately minimizing generator loss and
maximizing discriminator loss during training.

\begin{figure}[t]
    \begin{subfigure}{.5\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/no-backward-cycle-comp/26_fake_B_no_cycle.png}
        \subcaption{Without $\mathcal{L}_{\text{backward}}$}
    \end{subfigure}%
    \begin{subfigure}{.5\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/no-backward-cycle-comp/26_fake_B_our.png}
        \subcaption{With $\mathcal{L}_{\text{backward}}$}
    \end{subfigure}%

    \caption{Without backward cycle, the network tends to generate outputs with undesired new structures.
    Adding backward cycle with pretrained intrinsic decomposition network is able to generate images with higher quality.}
    \label{fig:comp_back_cycle}
\end{figure}

\begin{figure}[t]
    \begin{subfigure}{.5\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/albedo-arch-comp/abd6e9011b58baeda6d826ed6dfe1da2_000014_fake_B.png}
        \subcaption{With downsampling}
    \end{subfigure}%
    \begin{subfigure}{.5\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/albedo-arch-comp/abd6e9011b58baeda6d826ed6dfe1da2_000014_fake_B_no_downsample.png}
        \subcaption{Without downsampling}
    \end{subfigure}%
    \caption{Comparison between with and without downsampling blocks in albedo generator. From the result, we can see that
        generator without downsampling blocks could better preserve the textures structures of the input.}
    \label{fig:albedo-arch-comp}
\end{figure}
}

\section{Conclusion}
\vspace{-0.2cm}
We propose a novel two-stage framework to translate synthetic OpenGL images to the real domain. We achieve this by manipulating the albedo and shading layers of an image: we first translate them to the PBR domain by training on paired data,
followed by an unsupervised translation to the real domain. We have demonstrated that our approach leads to translations with higher visual quality and better performance for domain adaptation on scene inference tasks. We believe that operating on the albedo-shading decomposition is a way to incorporate physical structure into generative models and would like to explore applications of this idea to other tasks like image synthesis, image editing and style transfer.

\boldstart{Acknowledgements}
This work was supported in part by ONR grant N000141712687, Adobe
Research, a Samsung GRO grant, and the UC San Diego Center for Visual
Computing. Part of this work was done while Sai Bi was an intern at Adobe Research.

\section{Experiment Results}
\label{sec:exp}

\section{Implementation}\label{sec:implementation}
\boldstart{OpenGL-to-PBR.} Our first stage network architecture is based on pix2pixHD~\cite{wang2018pix2pixhd}. We use a $70 \times 70$ PatchGAN~\cite{isola2017pix2pix} for the discriminator.  The generator $G^S_{o \rightarrow p}$ contains two sub-nets including a global and a local network each with $9$ and $3$ residual blocks as proposed by Wang et al.~\cite{he2015deep}.
While the shading directly predicted by the generator is reasonable, it may have noise, discontinuities or typical GAN blocking artifacts, which degrade the finale image quality. We leverage the inherent spatial smoothness of the shading and remove potential artifacts by applying a \textit{guided filter layer}~\cite{wu2018fast} after the output layer of the shading generator. The guided filter layer $h(\cdot)$ takes the OpenGL shading, $S_{o}$, and translated OpenGL shading, $G^S_{{o \rightarrow p}}(S_{o})$, as input, and outputs the predicted shading, $\bar{S}_{p}$. We set radius to $4$ and regularization parameter to $0.01$.

\comment{
The global generator takes the downsampled inputs (shading, albedo, normals) and predicts low-resolution shading so as to recover its overall structure. The local generator takes the original inputs as well as the output of the global generator and refines the details. During training, the global generator is first trained for $20$ epochs, and afterwards we train both networks jointly.

\boldstartspace{Guided filter.} While the shading predicted by the generator above is reasonable, it may have noise, discontinuities or typical GAN blocking artifacts, which degrade the final image quality. We leverage the inherent spatial smoothness of the shading and remove potential artifacts by applying a guided filter layer~\cite{wu2018fast} after the output layer of the shading generator. The guided filter layer $h(\cdot)$ takes the OpenGL shading, $S_{o}$, and translated OpenGL shading, $G^S_{{o \rightarrow p}}(S_{o})$, as input, and outputs the predicted shading, $\bar{S}_{p}$. We set radius to $4$ and regularization parameter to $0.01$.
}

\comment
{
:
\begin{align}
    a, b & = g_{r, \sigma}(S_{o}, G_{o \rightarrow p}(S_{o})) \\
    \bar{S}_{{o}, i} & = a_k * S_{{o}, i} + b_k, \forall i \in \omega_k \label{equ:gf}
\end{align}
\Sai{where $w_k$ is a local $r \times r$ window centered at pixel $i$, $\sigma$ is a scalar that controls the smoothing extent, and $a$, $b$ are computed by minimizing the reconstruction loss between $S_o$ and $G_{{o \rightarrow p}}(S_o)$ subject to the local linear model in Equation~\ref{equ:gf}}.  The guided filter performs edge-aware smoothing~\cite{he2010guided}, and by using the OpenGL shading image as guidance, we ensure that the filter can effectively
smooth the predicted shading without blurring across important edges. In our experiments, we set $r = 4$ and $\sigma=0.01$.
}

\boldstart{PBR-to-Real.} For our second stage, we use the same PatchGAN architecture for the discriminator. Both the shading generator $G^S_{p \rightarrow r}$ and real-to-PBR generator $G_{r \rightarrow p}$ have $2$ convolutional blocks with stride of $2$ to downsample the input, followed by $4$ residual blocks and $2$ convolutional blocks with upsampling layers. The downsampling/upsampling gives the shading generator a large receptive field allowing it to capture global information about the scene and make global changes such as adjusting shading colors and intensity. The albedo generator $G^A_{p \rightarrow r}$ has a similar architecture as $G^S_{p \rightarrow r}$, except that it does not have any downsampling/upsampling. This keeps the receptive field small and forces the albedo generator to only modify the albedo locally thereby preserving textural details.

\boldstart{Training data.} For the OpenGL and PBR images we use the synthetic dataset from Li et al.~\cite{li2018intrinsic}, which contains about $20000$ $480 \times 640$ images of indoor scenes from the SUNCG dataset~\cite{song2017suncg}, rendered with both OpenGL and Mitsuba~\cite{mitsuba}, a physically based renderer.  We use $5000$ image pairs for training the first stage. Then the first-stage network is used to translate another $5000$ OpenGL images to the PBR domain, which are used for second-stage training. Two real image datasets are used in the second stage: the real estate dataset from Poursaeed et al.~\cite{poursaeed2018vision} and the NYUv2 dataset~\cite{silberman2012nyu}. The real estate dataset contains high quality real images of indoor scenes, and we use it for comparison on image quality (Section~\ref{sec:visual}). The NYUv2 dataset is used for domain adaptation experiments (Section~\ref{sec:domain}). Finally an extra $5000$ OpenGL images are used for testing the full pipeline. We select the training and testing OpenGL images from different scenes to avoid overlap.

\boldstart{Training details.}
Both stages are trained separately. We use Adam~\cite{kingma2015adam} with an initial learning rate of $0.0002$ for training both stages. The networks are trained for $100$ epochs, with the first $50$ epochs trained with the initial learning rate, and the remaining $50$ epochs with a linearly decaying learning rate. We randomly crop patches from the input images for training with a patch size of $400 \times 400$ for the first stage and $256 \times 256$ for the second stage.

\section{Introduction}

\begin{figure}[t]
  \centering
  \includegraphics[width=0.98\linewidth]{./images/teaser.pdf}

  \vspace{-0.3cm}
  \caption{Our two-stage adversarial framework translates an OpenGL rendering (a) to
     a realistic image (c). Compared to single-stage prediction
      with CycleGAN (b), our result has more realistic illumination and better preserves texture details, as shown in the insets. (Best viewed in digital).}
        \vspace{-0.6cm}
\label{fig:teaser}
\end{figure}

Deep learning-based image synthesis methods are generating images with increasingly higher visual quality~\cite{goodfellow2014gan,isola2017pix2pix,karras2018progressive,wang2018pix2pixhd, wang2016generative} even from minimal input like latent codes or semantic segmentation maps. While impressive, one challenge with these approaches is the lack of fine-grained control over the layout and appearance of the synthesized images. On the other hand, it is possible to compose 3D scenes with a desired layout and appearance and render them to create photo-realistic images. However, this requires high-quality scene assets (geometries, materials, lighting) and compute-heavy physically-based rendering.

The goal of this work is to combine the advantages of these two approaches. Given a low-quality synthetic image of a scene---the coarse models in the SUNCG indoor scene dataset~\cite{song2017suncg} rendered with a simple rendering engine like OpenGL---we train a deep neural network to translate it to a high-quality realistic image. One approach to this problem would be to train an unpaired image-to-image translation network, like CycleGAN~\cite{zhu2017cyclegan}, from synthetic OpenGL images to real photographs. However, the unpaired nature of this problem---it is not clear how we would create a dataset of synthetic images with their ``real" counterparts---makes it challenging and results in images with signficiant artifacts like implausible lighting and texture distortions (Fig.~\ref{fig:teaser}(b)). In contrast, our result retains the layout and coarse appearance of the original image, but introduces realistic lighting including global illumination, improves the quality of the scene textures, and removes effects like aliasing (Fig.~\ref{fig:teaser}(c)).

Improving the realism of a synthetic image requires improving the quality of both illumination and texture. Moreover, these two aspects need to be handled in different ways: illumination needs to be synthesized globally while textures can be modified locally. To this end, we propose disentangling a synthetic image into its constituent shading and albedo layers---i.e., an intrinsic image decomposition~\cite{barrow1978intrinsic}---and training \emph{separate} translation networks for each of them.

We leverage the intrinsic images disentanglement to change this problem from a purely unpaired setting to a two-stage paired-unpaired setting. We render the synthetic scenes with a physically-based renderer (``PBR") to simulate realistic illumination and create paired OpenGL-PBR shading data. In the first stage of our pipeline, we use this data to train an OpenGL-to-PBR shading translation network that synthesizes realistic shading. We combine this new shading with the original OpenGL textures to reconstruct our intermediate PBR images.

In the second stage, we translate these PBR images to the real image domain in an unsupervised manner using a CycleGAN-like network. We train individual PBR-to-real generators for the shading and albedo layers; we use an encoder-decoder architecture for shading to increase global context, and a purely convolutional network, with no downsampling/unsampling for albedo. As in CycleGAN, the quality of this translation is best with a backward real-to-PBR cycle, and a cycle-consistency loss. In the absence of the disentangled shading-albedo layers for real images, we accomplish this with an asymmetric architecture and a PBR-domain intrinsic image decomposition network.

While our focus is on improving visual quality, our method can be used for domain adaptation. Deep networks can be trained on large-scale, labeled synthetic datasets~\cite{richter2016gta,zhang2017pbr} and prior work has looked at adapting them to improve their performance on real data~\cite{gecer2018face, shrivastava2017learning, mueller2018hands}. Many of these methods impose a task-specific loss on this adaptation~\cite{hoffman2018cycada,murez2018domain,zheng2018t2net}. In contrast, we show that by improving the overall visual realism of synthetic data, we can achieve similar improvements in real image performance on tasks such as normal and depth estimation without the need for task-specific losses, as demonstrated in Table~\ref{table:normal} and Table~\ref{table:depth}.

\comment
{
In summary, we make the following contributions:
\begin{itemize}
    \item a disentangled shading-albedo space synthetic-to-real translation method that produces high-quality realistic images.
    \item a two-stage pipeline that combines paired OpenGL-to-PBR shading translation with unpaired PBR-to-real translation.
    \item
\end{itemize}
}

\comment
{
    The use of deep learning-based methods have led to significant improvements on a variety of scene understanding problems, ranging from semantic segmentation~\cite{long15fcn, chen2018deeplab, he2017mask} to depth and normal reconstruction~\cite{eigen15depth, li2015normal,wang2015normal}. A central challenge in training these approaches is to collect sufficient labeled data for these problems --- a task that is especially challenging for scene understanding/reconstruction tasks where human annotations may not be reliable.

    As a result, recent methods has proposed leveraging synthetically generated data (which has accurate, detailed labels) for such tasks~\cite{shrivastava2017learning}. This has been aided by the introduction of large-scale synthetic scene datasets~\cite{song2017suncg,richter2016gta}. However, synthetically generated images often differ in appearance from real images; this can be because synthetic scenes do not adequately model the complexity of real world scenes (in terms of complex material properties, lighting, geometry, camera settings) or because the methods used to render the images do not capture real-world lighting effects (e.g., the use of simple rendering engines like OpenGL). This visual domain shift leads to significant drops in performance when neural networks trained on synthetic data are applied to real images.

    Deep learning has shown great capability and capacity in solving a variety of
problems in indoor scene understanding, including semantic
segmentation~\cite{long15fcn, chen2018deeplab, he2017mask}, surface normal
estimation~\cite{li2015normal, wang2015normal} and layout
estimation~\cite{tulsiani2018factoring, huang2018layout}, which play an
essential role in many real world tasks such as robot path planing and
scene monitoring~\cite{zhang2017pbr}. The introduction of large scale
datasets such as SUNCG~\cite{song2017suncg, zhang2017pbr} also facilitates the
training of neural network models by providing abundant synthetic images with
rich annotations.

While synthetic data has many advantages, neural networks trained on it usually
have poor performance on real world images due to the difference in image
characteristics. This difference comes from two aspects.
First, synthetic images have poor realism when compared to real images,
as a result of the use of simple rendering engines such as OpenGL and poor
modelling of the real world including low-quality textures and material properties.
Second, the data distribution of the synthetic dataset does not match
the real dataset. For example, synthetic images from the SUNCG dataset usually have gray-scale
lighting, while real images from the NYUv2 dataset~\cite{silberman2012nyu} usually
have colored lighting. These two factors result in the well-known domain shift problem
when we apply CNN models trained on synthetic images to real-world images.

In this paper, we propose a novel framework for translating synthetic indoor images
to real images by learning an image translation network to bridge the gap
between them.
Our input is an image rendered with OpenGL that has simple albedo and shading.
Different from previous approaches~\cite{zheng2018t2net} that go directly from OpenGL to real images with
generative adversarial networks~\cite{goodfellow2014gan},
we propose to improve the realism of OpenGL images in two stages by first
translating them to the
physically based rendering domain with supervised training, followed by
refining their appearance
to match with real images in an unsupervised manner. From Figure x we can see
that by combining supervised and unsupervised training, our approach is able to generate much better
translations, and create more realistic final images.

In the first stage, to overcome the issue of lack of global illumination for OpenGL images, we utilize a network to predict the global illumination
by training on paired images of the same scene rendered with OpenGL and
a physically based renderer (PBR) respectively. Previous image-to-image translation approaches
directly predict the desired output images. In comparison, our network takes the albedo,
normal as well as OpenGL shading image of the synthetic scene as input, and
predicts a new shading image that contains global lighting effects. Afterwards
we multiply the new shading by the original albedo to reconstruct the desired
output image.
As we can see from Figure x,
by keeping the albedo fixed and only predicting the low frequency shading,
we could achieve better translations with much higher quality and fewer
artifacts than na\"ively regressing the PBR images that have more
complex spatial variations.

While adding global illumination makes the synthetic images more realistic,
properties such as lighting distributions of the synthetic images may not
be aligned with the real dataset. Therefore, in the second stage, we
further bridge the gap by performing an unsupervised adaptation to translate
the predicted PBR images from the first stage to the real domain. Previous
approaches such as~\cite{gecer2018face, mueller2018hands} directly perform this translation on image space
using networks with global non-linear transformations, which usually results in
blurry outputs and undesired large structure change in textures. We propose
to use two generators to translate the albedo and shading separately, based on
the observation that the albedo transformations should be local while the shading
transformations are global.
Therefore, we use a fully convolutional neural network with small recpetive
fields to translate the albedo, and use an encoder-decoder network to translate
the shading. From Figure x, we can clearly see that our proposed framework is
able to achieve more photo-realistic translations compared to direct
image space translations. In summary, we make the following contributions:
\begin{itemize}
    \item We propose a novel two-stage framework to translate synthetic
        images to real images. We first translate the OpenGL image
        to the PBR domain by predicting its global illumination, and
        then translate the PBR image to the real domain.
    \item We propose to do the translation on intrinsic image components
        including albedo and shading separately. By designing different
        architectures for albedo and shading, our framework is able to better
        preserve the structures of the input and achieve
        high quality translations with many fewer artifacts.
\end{itemize}

}

\section{Related Work}

\boldstart{Image-to-Image Translation.}
To improve the realism of synthetic images, Johnson et al.~\cite{johnson2010cg2real}
retrieve similar patches to the input from real image collections to synthesize realistic imagery.
Recently deep neural networks have been widely used for this task.
When paired training data is available, previous methods have proposed training conditional generative models with a combination of supervised reconstruction losses and adversarial losses (pix2pix~\cite{isola2017pix2pix}, StackGAN~\cite{zhang2017stackgan}). Such mappings are challenging to learn in the unsupervised setting with only adversarial losses, and prior work has utilized cycle-consistency losses (CycleGAN~\cite{zhu2017cyclegan}) or a shared latent space for the two domains (UNIT~\cite{liu2017unit}). These approaches have been extended to handle multi-modal output (BicyleGAN~\cite{zhu2017multimodal}, MUNIT~\cite{huang2017multimodal}), multiple domains~\cite{choi2018stargan}, higher-resolution images (pix2pix-HD~\cite{wang2018pix2pixhd}), and videos~\cite{wang2018vid2vid}. These methods---especially the unsupervised approaches---can introduce undesired structural changes and artifacts when there is a large domain difference, as there is between synthetic OpenGL images and real images. We handle this by working in a disentangled shading-albedo space. This allows us to a) use a two-stage pipeline that goes from OpenGL images to PBR images and then to the real domain, and b) design separate shading and albedo networks to avoid artifacts.

\boldstart{Domain adaptation.} Domain adaptation methods seek to generalize the performance of a ``task'' network, trained on one domain, to another domain; for example, to train networks on large-scale labeled synthetic datasets and apply them on real images. Feature-space domain adaptation methods either match the distributions of source and target domain features~\cite{sun2016coral} or learn to produce domain-agnostic features using feature-space adversarial losses~\cite{ganin2015unsupervised,ganin2016domain,tzeng2017adversarial}.

Instead, image-space domain adaptation methods seek to match image distributions. The key challenge here is to avoid changing image content in ways that will impair the performance of the task network. This can be handled by using paired source-target data to regularize the translation~\cite{gecer2018face}. In the unpaired setting, prior work has constrained the translated images to be close to the source images~\cite{shrivastava2017learning} but this only works for small domain shifts. Most current methods use a combination of task-specific losses (i.e., preserving the task network's output after translation)~\cite{mueller2018hands}, image-space and feature-space adversarial losses, cycle-consistency losses, and semantic losses (i.e., preserving the semantics of the image after translation)~\cite{hoffman2018cycada,murez2018domain,zheng2018t2net}.

Our contributions are orthogonal to this direction of work. We demonstrate that translating images in the shading-albedo space leads to higher visual quality, which improves performance on real data. We do this without using task-specific or semantic losses and are thus not constrained to a specific task. Our work can be combined with other domain adaptation ideas to further improve results.

\comment
{
    More recently, some works train a generative adversarial
network~\cite{goodfellow2014gan} to replace
the domain classifier and discriminate feature representations of different
domains. Tzeng et al.~\cite{tzeng2017adversarial} learns a mapping from source
domain features to target domain features by fooling the domain discriminator.

Multiple works have exploited the idea of translating synthetic images to the real
domain and improving their visual realism so as to boost the performance of
network models on real world data.
As mentioned above, different from general image-to-image translation tasks,
the key challenge in this problem is to avoid introducing
content changes that will destroy the annotation information from the source
domain. The lack of paired training data makes the problem even more challenging.
Shrivastava et al.~\cite{shrivastava2017learning} applies a per-pixel $L_1$
loss to prevent the translated image from drifting too much from
the source image, and the simple $L_1$ regularizer is not able to handle large domain
shift and avoid structure changes.
Mueller et al.~\cite{mueller2018hands} enforce the geometry
consistency of synthetic and translated hand images by minimizing the
difference in the silhouette. Gecer et al.~\cite{gecer2018face} uses
a small number of paired synthetic and real face images to regularize the
translation process, which doesn't apply to our task since there is no paired
data between OpenGL images and real images. Hoffman et
al.~\cite{hoffman2018cycada} pretrains a semantic segmentation network and uses
it to enforce the semantic consistency of synthetic images and translated
images, which requires additional label information from the synthetic data,
and the high level semantic information cannot capture low-level texture
changes. Zheng et al.~\cite{zheng2018t2net} also feeds the real image to the
synthetic generator and enforces the generator to make minimal changes to the
real image so as to constrain the translation, which provides little
guidance for synthetic images when the domain gap is very large.
Compared to previous approaches that use some ad hoc losses to regularize the
translations, we propose to exploit different properties of albedo and
shading. That is, we use a fully convolutional neural network to locally
modify the albedo, and use a global encoder-decoder network to transform the
shading.  In this way, we are able to better preserve the shapes, textures and
semantics of the synthetic image.
}

\section{Results}
In this section, we first show the comparison on visual quality of translations against
baseline methods (Section~\ref{sec:visual}).  Afterwards we show that our two stage pipeline could
boost the performance of network models on real images when trained with our translated images (Section~\ref{sec:domain}).

\begin{figure}[t]
    \centering
    \begin{subfigure}{.495\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/first-stage-comp/1-opengl.png}
        \subcaption{OpenGL image}
    \end{subfigure}%
    \begin{subfigure}{.495\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/first-stage-comp/1-gt.png}
        \subcaption{PBR image}
    \end{subfigure}%

    \begin{subfigure}{.33\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/first-stage-comp/1-i.png}
        \subcaption{p2pHD-OpenGL}
    \end{subfigure}%
    \begin{subfigure}{.33\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/first-stage-comp/1-s+a+n.png}
        \subcaption{p2pHD-S+A+N}
    \end{subfigure}%
    \begin{subfigure}{.33\linewidth}
        \centering
        \includegraphics[width=0.98\linewidth]{./images/first-stage-comp/1-our.png}
        \subcaption{Our result}
    \end{subfigure}%

    \vspace{-0.3cm}
    \caption{OpenGL to PBR comparison.
    A PBR renderer like Mitsuba needs around 20 mins. to render a noise-free image as in (b). In comparison our network can generate  a high quality result in $0.03$ secs. (e).
    Compared to the pix2pix-HD
    framework that directly
    predicts the output images using OpenGL (c) or auxiliary buffers (d), with inconsistent shading such as abrupt highlights on the cabinet, our method generates images
    with much higher visual realism.
    }
       \vspace{-0.3cm}
  \label{fig:first-stage}
\end{figure}

\subsection{Comparison on visual quality}\label{sec:visual}
We compare the results of our method against different baselines for each separate stage and the whole pipeline.
In addition to qualitative comparisons, we also quantitatively measure the visual realism of images
generated with different methods with two metrics. The first is the FID score~\cite{heusel2017gans
which has been shown to be effective in measuring
the distance between two distributions and is consistent with image noise and distortion.
In addition, we also conduct a human perceptual study on Amazon Mechanical Turk. For each task,
workers are shown the images output by different methods, and asked to select the most realistic result with the fewest artifacts. For each task, we have
$100$ images, each evaluated by $10$ different workers, for a total of $1000$ judgements. We discuss each stage in detail below.

\boldstart{OpenGL to PBR translation.} Our proposed network takes the albedo, normal and
OpenGL shading as input to  predict the PBR shading, which is multiplied by the albedo
to reconstruct the desired PBR image. We compare our method with two baselines:
the pix2pix-HD network~\cite{wang2018pix2pixhd} that predicts
the PBR image conditioned on the OpenGL image only (denoted as \textit{p2pHD-OpenGL} in Table~\ref{table:score} ), and the same
network that takes the same input as ours
but directly predicts the PBR images (\textit{p2pHD-S+A+N}). Both baselines are trained with the same generator architecture and
parameter settings as ours. The only differences are the generator inputs/outputs and the lack of shading discriminators.

We calculate the FID scores between the predicted PBR images and ground truth PBR images, and we also
conduct a user study to ask workers to select the most visually realistic PBR predictions among outputs of
three methods. From the results in Table~\ref{table:score} ($\mathcal{O}\rightarrow \mathcal{P}$), we can clearly see that our method achieves a much lower
FID score compared to other baselines, implying that our predictions are more aligned with the distribution
of the ground truth PBR images. In addition, we obtain a user preference rate of $60.6\%$,
much higher than the other two baselines, demonstrating that doing predictions in shading
space generates images with much higher quality.
A visual comparison is shown in Figure~\ref{fig:first-stage}.
\comment{Figure~\ref{fig:first-stage} shows a visual comparison
between our approach and other baselines. From the figure we can see that our result has more realistic
shading and fewer artifacts. In comparison, directly predicting PBR images with either
OpenGL or auxiliary buffers as input produces inconsistent shading across different regions.}

\begin{table}[t]
\centering
\renewcommand{\arraystretch}{1.0}
\begin{tabular}{c|c|rc}
\hline
\multicolumn{2}{l|}{} &  \makecell{FID } & \makecell{User \\Preference} \\ \hline
\multirow{3}{*}{$\mathcal{O \rightarrow P}$}
    & p2pHD-OpenGL~\cite{wang2018pix2pixhd}  & 21.01 &  10.2\%\\
    & p2pHD-S+A+N~\cite{wang2018pix2pixhd} &  11.63 &  29.2\%\\
    & \textbf{Ours} &  \textbf{7.40} &  \textbf{60.6\%}\\ \hline
\multirow{2}{*}{$\mathcal{P \rightarrow R}$}
& CycleGAN~\cite{zhu2017cyclegan} &  54.80 & 27.9 \% \\
    & \textbf{Ours} & \textbf{53.48} &  \textbf{72.1\%} \\ \hline
\multirow{3}{*}{$\mathcal{O \rightarrow R}$}
    & CycleGAN~\cite{zhu2017cyclegan} & 59.42  &  16.9\% \\
    & T\textsuperscript{2}Net~\cite{zheng2018t2net}&  65.33 & 4.7\%  \\
    & \textbf{Ours}& \textbf{53.48}  &  \textbf{78.4\%}\\ \hline
\end{tabular}
\vspace{-0.3cm}
\caption{Comparison on FID and user preference score.}
\label{table:score}
\end{table}

\begin{figure}[t]
 \vspace{-0.4cm}
  \includegraphics[width=\linewidth]{./images/second-stage-comp-v2.pdf}
  \vspace{-0.6cm}
  \caption{PBR to Real comparison. Given the predicted PBR image (a) in the first stage, we further
  translate it to real domain. Compared to the original CycleGAN~\cite{zhu2017cyclegan}
    that performs the translation in image space, our method could better preserve the
structure of the textures and generate images with many fewer artifacts.}
  \label{fig:second-stage}
  \vspace{-0.5cm}
\end{figure}

\begin{figure*}[t]
  \centering
  \vspace{-0.5cm}
  \includegraphics[width=\textwidth]{./images/main_comp_1.pdf}
    \vspace{-0.6cm}
  \caption{Comparison on the full pipeline from OpenGL to Real against single stage translation
      with CycleGAN and T\textsuperscript{2}Net.}
  \label{fig:full-pipeline}
  \vspace{-0.5cm}
\end{figure*}

\boldstart{PBR to Real translation.} In this stage, we train the network to translate the output of
the first stage to the real domain. We compare to the na\"ive CycleGAN framework that performs the translations
in image space. FID score is calculated between
the translated images and real images from the real estate dataset.  Similarly, from Table~\ref{table:score} we can see that
disentangled translations on albedo and shading with our proposed framework achieves
lower FID scores and much higher user preference rate. As shown in Figure~\ref{fig:second-stage},
our network is able to modify the albedo and shading of the input while preserving the texture structures.
In contrast, the original CycleGAN network introduces obvious artifacts which degrade
image quality.

\boldstart{Full pipeline.} We also compare our two-stage translations to previous methods
that translate directly from OpenGL to real: the CycleGAN method and T\textsuperscript{2}Net~\cite{zheng2018t2net}.
T\textsuperscript{2}Net uses a single generator and discriminator for synthetic to real translation. In addition to standard GAN loss,
it introduces an identity loss to guarantee that when real images are fed to synthetic generator, the outputs are
similar to the inputs. The results show that our full pipeline significantly outperform the single-stage
translations used in both baselines. The visual comparison in Figure~\ref{fig:full-pipeline}
shows that single-stage translation is not able to produce realistic shading and generates results with
noise and artifacts due to the large gap between OpenGL images and real images, which our proposed method addresses satisfactorily.

\begin{table}[t]

\renewcommand{\arraystretch}{1.25}
\resizebox{\linewidth}{!}{%
\begin{tabular}{l|c|ccccc}
\hline
\multicolumn{2}{l|}{} & \multicolumn{2}{c|}{Lower is better} &  \multicolumn{3}{c}{Higher is better(\%)}  \\ \cline{3-7}
\multicolumn{2}{l|}{} & Mean & \multicolumn{1}{l|}{Median}  &  $ < 11.25$ &  $ < 22.5$ & $ < 30$  \\ \hline
\multirow{2}{*}{$\mathcal{O \rightarrow P}$}
       & OpenGL  &  37.73 &  31.91 & 17.93 & 38.36 & 48.52 \\
       & \textbf{Ours-PBR} &  34.82 & 27.66  & 21.16  &  42.82 & 53.25 \\ \hline
\multirow{1}{*}{$\mathcal{P \rightarrow R}$}
       & CycleGAN~\cite{zhu2017cyclegan}&  33.90 &  27.24 &  22.28 & 43.99 & 54.38 \\ \hline

\multirow{3}{*}{$\mathcal{O \rightarrow R}$}
       & CycleGAN~\cite{zhu2017cyclegan}&  36.33 & 30.25 & 19.15  & 39.19  & 50.15 \\
       & T\textsuperscript{2}Net~\cite{zheng2018t2net} &  36.93 & 30.93 & 18.93 & 39.23 & 50.49 \\
       & \textbf{Ours-full} & \textbf{33.15} & \textbf{26.27} & \textbf{23.52} & \textbf{44.95} & \textbf{55.28} \\
       & Real\textsuperscript{*} & 28.18 & 21.95 & 28.14 &52.21 & 62.35\\ \hline

\end{tabular}
}
\vspace{-0.3cm}
\caption{Normal estimation results on NYUv2 dataset.}
\label{table:normal}
\end{table}

\begin{table}[t]

\renewcommand{\arraystretch}{1.25}
\resizebox{\linewidth}{!}{%
\begin{tabular}{l|c|ccccc}
\hline
\multicolumn{2}{l|}{} & \multicolumn{2}{c|}{Lower is better} &  \multicolumn{3}{c}{Higher is better (\%)}  \\ \cline{3-7}
\multicolumn{2}{l|}{} & RMSE & \multicolumn{1}{l|}{RMSE-log}  &  $ < 1.25$ &  $ < 1.25^2$ & $ < 1.25^3$  \\ \hline
\multirow{2}{*}{$\mathcal{O \rightarrow P}$}
       & OpenGL  &  1.0770 &  0.3873 & 43.53 & 73.60 & 89.75 \\
       & \textbf{Ours-PBR} &  1.0293 & 0.3514  & 46.23  &  75.41 & 91.35 \\ \hline
\multirow{1}{*}{$\mathcal{P \rightarrow R}$}
       & CycleGAN~\cite{zhu2017cyclegan}&  0.9824 &  0.3394 &  48.24 &  78.61& 92.25 \\ \hline

\multirow{3}{*}{$\mathcal{O \rightarrow R}$}
       & CycleGAN~\cite{zhu2017cyclegan}&  1.0328 & 0.3726 & 45.46 & 75.49 & 91.13 \\
       & T\textsuperscript{2}Net~\cite{zheng2018t2net} &  1.0085 & 0.3548 & 47.49 & 77.57 &  91.94\\
       & \textbf{Our-full} &  \textbf{0.9774} & \textbf{0.3328} &  \textbf{49.59} & \textbf{79.54} & \textbf{93.14} \\
       & Real\textsuperscript{*} & 0.7070 &  0.2516 & 67.76 & 88.75 &  96.35 \\ \hline
\end{tabular}
}
\vspace{-0.3cm}
\caption{Depth estimation results on NYUv2 dataset. \textit{Real\textsuperscript{*}} is the model trained on 5000 real images from NYUv2.}
\label{table:depth}
\vspace{-0.5cm}
\end{table}

\subsection{Domain adaptation}\label{sec:domain}
Network models trained with synthetic images often performance poorly on real images due to the domain gap.
Our two-stage network improves the visual realism of synthetic images, thereby boosting the performance
of network models trained with the translated images.
We compare the performance of network models trained with images generated with different methods on two tasks including normal estimation and
depth estimation.
We train the task networks on $5000$ images from SUNCG dataset after translation to the real domain, and evaluate them on the test dataset
in NYUv2, which contains $1449$ images with ground truth normal and depth.

\boldstart{Normal estimation.}
We use a U-Net~\cite{ronneberger2015u} with $7$ downsampling blocks for the normal estimation task, and apply
the inverse dot product between ground truth normal and predicted normal as the loss function. The network
is trained with an Adam optimizer for 200 epochs with a learning rate of $2\times10^{-4}$ for the first 100 epochs
and a linearly decaying rate for the remaining 100 epochs. We test the network on the NYUv2 dataset.
In Table~\ref{table:normal}, we report the mean and median angle between predicted normal and ground truth normal,
as well as the percentage of pixels where the angle is below a certain threshold. From the table we can see compared to
the model trained on OpenGL images, training on our predicted PBR images significantly reduces the average angle, and translation to the real domain further reduces it to $33.15$, which demonstrates
that both stages help reduce the domain gap and help improve network performance.
In addition, compared to the na\"ive CycleGAN in image space for the second stage, our disentangled design is also able to achieve
higher accuracy. Finally, compared to going directly from OpenGL to real images
with methods such as CycleGAN and T\textsuperscript{2}Net, our two-stage translation significantly outperforms
them on all metrics.

\vspace{-0.2cm}
\boldstartspace{Depth estimation.}
We use the network architecture in Zheng et al.~\cite{zheng2018t2net} for depth estimation, and
adopt the same training protocol. We evaluate the performance of different approaches on the
NYUv2 dataset using metrics such as relative mean squared error (RMSE) between the prediction and  ground truth, RMSE in log space as well as the percentage of
pixels whose ratios to ground truth are below a threshold~\cite{eigen2014depth}. Table~\ref{table:depth}
summarizes the scores of different network models. From the table we can see that training
with our predicted PBR images achieves much higher accuracy than training
on the synthetic OpenGL images, and our full pipeline further improves the accuracy by translating
the PBR images to real domain. Our two-stage translation is also able to outperform single-stage translation
with T\textsuperscript{2}Net and CycleGAN and leads to a lower error.

redpajama_set_name: RedPajamaArXiv


Alef Bookstores are generally easy to find with large pictures of their logo displayed prominently at street level. A recent visit to the Zamalek branch confirmed what we know about Alef Bookstores: they're located throughout the city, often in wealthier neighborhoods; they have a wide-ranging selection of books, in both English and Arabic; the stores are comfortable and often have air conditioning and a café; and the books are organized by genre, though with all of the browsing that happens, the organization becomes a bit chaotic within each sub-section. Employees at Alef Bookstores tend to be just that, rather than the book aficionados you find at non-chain bookstores who are making a career out of bookselling and thus can offer recommendations about titles, authors, or other places to track down the book you're searching for. That being said, with their computer system, employees at Alef can tell you immediately if a specific title or author is available at their branch or at one of the other Alef branches.
If you're looking for current and popular titles in Arabic, classic works like those by Naguib Mahfouz and Tawfiq al-Hakim, popular English-language titles, or children's books (in Arabic and English), head to a branch of Alef Bookstores. You can also check their searchable, usually up-to-date, bilingual website before your visit, though we recommend calling to confirm a title is at a specific branch before making your way there in person. Different branches hold book signings and release parties; check the "Events" section of their website for details.


Irresistibly pleasant is how the European country Belgium could be best described and it conforms to this adjective not just in the climatic sense but social, economical and political too. Indeed, how can a relatively small country which gave the world French Fries and Tintin be anything other than imaginative, innovative and courteous to the core?
When you visit Belgium, the first point that would strike you pertains to diversity – geographical, linguistic and culinary. To its north, this country is flanked by beckoning beaches, around the central portion there are rolling hills and the south is predominantly mountainous. Officially three languages that rule the roost here are Dutch, French and German but in spite of this the local population would be more than willing to converse in English on being given a chance.
Being home to NATO head-quarters has ensured that it remains in the eye of the storm but Belgium does boast of an equally tumultuous past. The fact that Napoleon's winning spree was truncated at its borders is indicative of the determination of Belgians as also the fact that this country has often found itself repulsing its more predatory neighbors.
Life here is a wonderful mix of culture and science with both emerging equally victorious and thanks to all these eclectic ingredients the outcome is delectable, tempting and as balanced as can be imagined.


UHT pateurization machine for Asia market small size beverage juice sterilize machinery This machine adopts the present most advanced welding technology, selects the import high quality polish stainless steel pipeline. If completing with the high-pressure even machine, it also can be used for thick material. It is suitable for the sterilizing of liquid and unpacked product such as milk and other beverage.
6 YRS Foshan Longmai Kitchen Equipment Co., Ltd.
10 YRS Guangzhou Mingke Packaging Machine Co., Ltd.
6 YRS Shanghai Joylong Industry Co., Ltd.
11 YRS Shantou Xinchuang Food Packing Machinery Co., Ltd.
5, Good after-sales service offered, please get back to us if you got any questions. 2.we could send our engineers to your country, they are all experienced in repairing machine and working abroad. is located in the west industrial area of Huangyan, which is well known as the Mold kingdom of China.
Tubular Uht Sterilization Machine for Milk and Juice 1. 1.high efficiency 2,fully automatic 3,also can used as milk juice beverage sterilizer 1 , plate super high temperature sterilizing machine: 2 , plate sterilizer machine is widely used for liquid food "sterilization" heat treatment of automatic system, and is suitable for the milk, juice, tea beverages or similar products for continuous sterilization. The automatic control system, the control elements adopt famous brand products, including PLC program controller and engineering control screen, steam pressure reducing valve, the regulator, the purge valve, materials and heat pump water pump. Other valves and fittings etc all fluid fittings choose high quality standard products 4 , The material outlet temperature can be adjustable according to the final filling style . 2. .
6 YRS Anhui Sanmee Glassware Co., Ltd.
2 YRS Chaozhou Chaoan Happiness Foods Co., Ltd.
3 YRS Wuhan Dulin Technology Development Co., Ltd.
disposible cup plastic for milk Material: Food Grade PET Capacity: 1OZ - 32OZ according to request Shape: Round shape, lids and sleeve optional Size: Different size based on capacity Logo: Without logo, our brand logo,or customized logo Color: According to request We make different material disposable plastic cups from 1oz to 32oz. More detailed specification information, please contact our sales directly. 5. About order Welcome to order online to use Alibaba Trade Assurance.
Storage condition The aprons shall maintain their properties when stored in a dry condition at temperature between 10 to 40 degree. 5. With more than 16 years experience, 260 skilled works, 30 foreign trade salesmen, we have good cooperations with importers, dealers, wholesalers. Thanks for visting our website, if you are interestes in this Anti water blue color custom disposable plastic shoe covers, please contact with me freely.
1 YRS Shaoxing Dingchi Textile Ltd.
11 YRS Jiangyin Aluminum Foil Packaging East Asia Co., Ltd.
It is designed and built by our They use 16 sets vibrating sieve to screen the powder. Customer interview From Sri Lanka Customers After visiting our factory, They bought 4 sets of M1500-3S vibrating screen. From Pakistan Customers They bought 2 sets of leaf filter.
We're not limited to the products listed on website because we have professional engineer and experienced R&D team. We take credit and quality first as the pursuit, grasping the new tendency of the international cosmetics. As a modern factory, it has fully reached the international standards of the cosmetics production enterprise .
Why purchase from us9 Since our establishment, we have considered "products quality first" as our principle. 1. Why choose you9 We have professional team of workers , service and inspection, and we have our own subfactories. Our services 1>Sample service: We can provide 1-2pcs free samples,the customers need pay the shipping cost.
2 YRS Anhui Langxu Glassware Manufacturing Co., Ltd.
6 YRS Shantou Kingyang Foods Co., Ltd.
Alibaba.com offers 2,683 asia milk products. About 4% of these are chocolate, 1% are face cream & lotion, and 1% are dairy processing machines.
A wide variety of asia milk options are available to you, such as halal, haccp, and iso. You can also choose from box, bulk, and bag. As well as from free samples, paid samples.
There are 2,683 asia milk suppliers, mainly located in Asia. The top supplying country is China (Mainland), which supply 100% of asia milk respectively.
Asia milk products are most popular in North America, Western Europe, and South America.
You can ensure product safety by selecting from certified suppliers, including 1,104 with Other, 633 with ISO9001, and 105 with ISO14001 certification.


With Cozymeal's Cooking Classes in Tampa, you will learn a new way to prepare your favorite cuisine, or try something totally out of the box. Here in the sunshine state, we have a vast array of options available. Have an intimate date night where you cook and devour a luscious meal, grab a group for the night of your lives as you learn new skills, or fly solo to a class and make new friends while enjoying delectable cuisines. We have something for every style, taste, and need. Book today!


Shingles is a viral infection of the nerves that supply certain areas of the skin. Usually affects people over 50 years of age.
Symptoms: Usually affects area of skin supplied by the affected nerve on one side of the body or face. Severe burning pain in the affected area often precedes the rash. After about one to four days a painful, itchy rash of small crusting blisters appears. Over the next few days the blisters dry out, scab over and drop off. There may be small scars which fade away over three to four weeks. The eye may also be affected. This is a very serious condition.
Causes: Reactivation of the Varicella zoster (chicken pox) virus. The virus lies dormant in the nerve cells for many years after chicken pox infection. Only people who have had chicken pox develop shingles. Emotional stress and /or physical illness may trigger factors.
Possible complications: Persistent pain after healing of the rash (postherpetic pain).Corneal ulcers and infections if the eyes are infected, Possibility of blindness in the affected eye if left untreated.
Treatment in the home: Take analgesics in recommended doses to relieve the pain. Apply calamine lotion to the skin rash to relieve the itching.
When to consult a doctor: As soon as shingles are suspected. Antiviral drugs will reduce the severity of the active stage and minimize nerve damage, provided that treatment starts immediately the rash appears. If treatment is delayed beyond early stages of the disease, little can be done to influence its course or to the likelihood of postherpetic pain.
What the doctor may do: Prescribe antiviral therapy to relieve symptoms. Prescribe a lotion to lessen weeping and itching. Prescribe oral antibiotics if there is an associated bacterial infection. Refer you to an ophthalmologist if an eye is affected.


text:
Nii Ashitey Nsotse (* 19. April 1955 bei Accra, Ghana) ist ein ghanaischer Musiker und Trommellehrer. Er lebt in Wiesloch.

Biografie
Durch seine Familie kam Nii Ashitey schon früh in Berührung mit traditioneller Musik, sein Vater brachte ihm bereits mit zehn Jahren das Trommeln bei. Ashitey beschloss sich am National Folcloristic Company Arts Councel of Ghana zum Berufsmusiker ausbilden zu lassen. Ashitey studierte dort Musik, Trommeln und Tanz sowie Trommelbau. Er schloss seine Ausbildung mit dem Titel National Drummer ab, heute trägt der Musiker den Titel Master Drummer. Nii Ashitey war Mitglied der National Folkloric Company, die Ghana bei den Commonwealth Games in Kanada repräsentierte und ist Mitbegründer des ghanaischen Musikfestivals Nafac-Festival. Als Musiker, solo und mit seiner Band Nokoko, der er 1972 beitrat, führte ihn sein Weg auch über die Landesgrenzen hinaus, unter anderem nach Nigeria, Togo und Benin. 1983 kam er nach Deutschland, um hier Trommel- und Tanzworkshops anzubieten. 1985 ging die Band Nokoko auf ihre erste Europa-Tournee. Seit 1986 entstanden zusammen mit der Gruppe Nokoko Yé (Umbenennung) drei Studioalben. Das erfolgreichste Album davon ist wohl Bushtaxi from Bamako to Accra, das 2001 veröffentlicht wurde (ft. Kassoum Traoré) und mit ungewöhnlicher Instrumentierung punktet.

Weblinks
 jazzpromotion.net
 Nsotse: Deutschland ist meine zweite Heimat

Musiker (Ghana)
Ghanaer
Geboren 1955
Mann
Musiker (Deutschland)

redpajama_set_name: RedPajamaWikipedia


A driving force in communications, Myriam Achard has more than 20 years of experience in public relations. For the past 12 years, she has worked with Phoebe Greenberg to develop and promote the Canadian and international avant-garde arts scene. Myriam has been a Director of PR & Communications at the Phi Centre since 2012 where she is also responsible for the VR Garden's programming and the immersive / interactive exhibits.


Perfect for hiding risky pics.
By now, most of us probably have years of Instagram photos that we would like to hide or forget all together. All those food pics we thought looked appetizing in 2012 can probably go now, if we're being honest. Luckily for all of us, Instagram is officially rolling out a feature that will let you hide your cake and keep it too.
Instagram archive is the latest in a series of user friendly features that allows Instagrammers to tailor their experience. With archive, users can now hide/temporarily remove photos from their feeds with the option of placing them back in the future.
You won't have to permanently delete those super unprofessional party pictures. Just archive them for now and put them back on your feed later.
Once a photo is archived, it will no longer appear in your feed and followers will not see them. However, if you choose to unarchive them, the photos will become available again.
The feature has not been rolled out to all users yet but looks like it is likely to be a permanent addition to Instagram's growing list of features.
BRB hiding all my sepia toned dinner pics from 2012.


Historically, the beginning of October marks the start of great things to come. Think about it: Columbus Day weekend (a day off with zero responsibilities!), leaves starting to turn, Halloween, Thanksgiving fun things on the horizon.
In sports it's a time where you are overwhelmed with options. NFL and NCAA football go full tilt, the MLB regular season comes to an end and playoffs begin - but this year, for me the options are limited, and there is no one to blame but the Boston Red Sox. Playoffs? Not this year - not by a long shot.
What the Red Sox have done in 2012 is such an epic failure that it is actually comical. You have to look at it that way otherwise it will make you very, very angry. Trust me. The season begins; there is an added Wild Card this year giving another team a chance at post season play. Boston has a payroll just south of $200 million, breathing that rarified air only certain teams can afford to breathe. Optimism ran rampant through Red Sox Nation. What ensued was a sight so ugly it rivals the Too Hot for TV pictures of Roseanne my friend Slippery Pete e-mailed me last week!
It starts at the top. The Sox owners were focused on football, or as Americans know it, soccer. Soccer! You have to be kidding me! While they were asleep at the wheel they let Theo leave for Chicago and got nothing in return, hired his sidekick Ben Cherington and then forced a manager Cherington didn't want, Bobby Valentine, down his throat. It's like Bill Parcells once said: If you charge someone with the task of cooking a meal you should at least allow them to buy the groceries. And as the season would play out, there would end up being a few rotten tomatoes in the fridge.
Here is the Red Sox starting lineup for Sunday against the Orioles.
It's not easy to make chicken salad out of that collection of talent. It's like the Bad News Bears without any of the things that made them endearing - and there's also no chance they play a good game with the Yankees on the last day of the season.
What lies ahead? Hard to say. By dumping Carl Crawford and Adrian Gonzalez, Boston has a ton of money to spend this off-season. But spending for the sake of spending is what got them in a tough spot to start. John Lackey is still on the roster. The line up above will be nothing like the one that takes the field to start 2013. The off-season will no doubt be a busy one; I just wish it was the post season that we were talking about. Maybe next year.


text:
910 Traffic Ave Sumner WA 98390
Visit Research Center
Online Trade Appraisal
Warranty Protection For Life
Oil Changes For Life
Maintenance For Life
Right From The Start
About Sunset
2017 Sell Down
2018 Chevy Silverado 1500 Tacoma WA
2018 Chevy Silverado For Sale at Sunset Chevrolet
The Chevrolet Silverado is recognizable everywhere you go and is the best of the best when it comes to pickup trucks! The Silverado is offered in tons of configurations with three cab layouts, three engine choices and three-bed sizes, so there is a perfect Silverado for everyone! Read the rest of our review to learn more about this incredible full-size pickup!
4.3-Liter V6 VVT DI with Active Fuel Management Engine
Hydra-Matic 6L80 Six-Speed Automatic Transmission
Towing Capacity of 12,500 Pounds
StabiliTrak Electronic Stability Control
Safety Alert Driver Seat
IntelliBeam Headlamps with Automatic High Beam Control
20-Inch Chrome Wheels
What kind of standard features do the trim levels of the Silverado come with?
The 2018 Silverado offers a ton of bed and cab combinations as well as trim levels and packages full of comfort and convenience features! There are three cab styles for 2018; two-door regular cab, four-door extended cab and the crew cab. The regular cab can seat up to three people and can be equipped with the 6.5 foot or 8-foot bed. The extended cab can fit up to six passengers and is equipped with 6.5-foot bed only and lastly the crew cab seats six as well but gets full-size rear doors and a 5'8" bed. Then you get to choose the trim level! There are a total of five for 2018; the base Work Truck, the LS, Custom, LT, LTZ and the top of the line High Country. There is also an off-roading package the Z71 available for the LT and LTZ models. The base Work Truck models come with a ton of great features such as xenon headlights, 17-inch steel wheels, black front and rear bumpers, LED daytime running lights, a bed light, and manually adjustable mirrors. The interior of the Work Truck features includes air conditioning, power door locks, a tilt-only steering column, vinyl floor covering, cruise control, driver information display and front bench seat with 40/20/40 split. You'll also find some great standard technology features inside the Silverado Work Truck such as an auxiliary jack, smartphone compatibility, Bluetooth, a six-speaker sound system and Chevrolet's MyLink interface. If you are looking for the best of the best features in the Silverado, look no further than the High Country model! The High Country includes 20-inch chrome wheels, chrome side step rails, a spray on bedliner, cargo box tie downs, a unique grilled and a trailer brake controller. The interior of the High Country includes features like a heated steering wheel, a navigation system, a Bose audio system with seven speakers, power adjustable pedals, ventilated front bucket seats and even wireless phone charging! Most of the features from the upper models can be added to lower models with through additional packages.
What kind of powertrain options does the Chevrolet Silverado offer?
There are three engine options for the 2018 Chevrolet Silverado! The first and standard engine is a 4.3-liter V6 VVT DI with Active Fuel Management that produces 285 horsepower and 305 pound-feet of torque. The first engine option is a 5.3-liter V8 VVT DI with Active Fuel Management that outputs 355 horsepower and 383 pound-feet of torque. The last engine option is a 6.2-liter V8 VVT DI with Active Fuel Management that generates 420 horsepower and 460 pound-feet of torque! The 4.3-liter engine is paired with a Hydra-Matic 6L80 six-speed automatic transmission while the 6.2-liter engine is matched with Hydra-Matic 8L90 eight-speed automatic transmission. The 5.3-liter engine can be paired with both transmission options! The Silverado tops the crew cab segment's towing capacity at a really impressive 12,500 pounds!
What kind of safety features are available for the 2018 Silverado?
The list of safety and security standard and available security and safety features is endless! The Silverado comes with six standard airbags such as head curtain side airbags, seat-mounted side airbags, and frontal airbags. StabiliTrak electronic stability control with rollover mitigation technology, hill start assist and trailer sway control are also standard safety features. The Enhanced Driver Alert Package is available for the LT, and LTZ models (standard on the High Country model) adds Safety Alert Driver Seat, Forward Collision Alert, Lane Keep Assist, Front and Rear Park Assist and IntelliBeam headlamps with automatic high beam control to the truck!
Final Thoughts on the 2018 Chevrolet Silverado
There is no other truck on the market like the 2018 Chevrolet Silverado. It can haul and tote anything you could ever need to with the comfort and convenience of a luxury vehicle! We know that we have the perfect Silverado on our lot for you so come into Sunset Chevrolet today to test out all the features and even take it for a spin!
Sunset Chevrolet
The Sunset Community Hours & Directions Privacy Policy

redpajama_set_name: RedPajamaCommonCrawl


text:
Government Is Managing COVID-19 Pandemic
by Sharon Austin | Jan 12, 2021 | Corona Virus, Top Stories
COVID-19 update and press conference – January 12th, 2021. (PMO)
Government is continuing to manage the COVID-19 pandemic and the situation is not out of control.
Minister of Health and Wellness, Lieutenant Colonel Jeffrey Bostic, reported an additional 84 new cases during today's COVID-19 update and press conference, but stressed that there was no need for alarm.
Lt. Colonel Bostic told the media: "We have not lost control because we still have the capacity to deal with what we are dealing with, even though in one or two areas, which we would have mentioned, that capacity has been stretched.
"Stretched, but not broken; broken but can be fixed, and we are fixing those areas. So, I am saying to you that we are very much managing the situation. There is no need for alarm at all, or panic. But, we are not going to rest until we are absolutely satisfied that we have it fully under control. And when we have it fully under control, is when the numbers are going to be on the decline in the way that we anticipate."
The Minister said he would not report that the situation was completely under control because there were some issues officials were grappling with.
"In relation, for example, to the testing and the return of results, we admitted that we have been struggling with that and … I am very hopeful because of the steps that we've taken, and what I'm seeing happening now, that we are going to get out of this in the coming days," he stated.
He noted that officials were doing everything possible to contain the situation, and explained that the positive cases being seen were predominantly linked to the clusters identified.
During the conference, Minister Bostic emphasised the importance of contact tracing, saying that was the only way to contain the spread by finding those primary contacts.
Senior Medical Officer, Dr. Anton Best, urged persons to cooperate with the Ministry of Health officials who were engaging in COVID-19 investigations. (PMO)
"If we do not do … the contact tracing with the primary contacts, in order to stop the spread, we will not win this battle …. Contact tracing is vitally important to what we are doing now, and the teams are out there doing a fantastic job to find all of those primary contacts.
"I can tell you, up to yesterday … there were about 253 primary contacts that we have been investigating, and this is really germane to the effort to contain the situation, so that we can get back to some degree of normalcy that we enjoyed for nine or 10 months," he stated.
Senior Medical Officer, Dr. Anton Best, also underscored the importance of contact tracing and urged persons to cooperate with the Ministry of Health officials who were engaging in that investigation.
He pointed out that it was necessary for people to be honest, since it would help in containing the spread of the virus.
"So, we do have scenarios in which persons do not want to reveal all of the information and if they don't reveal the information to the investigators, we are not going to be able to say, well this person is part of this cluster; if they're hiding something from us.
"For instance, if somebody went on the bus crawl but for whatever reason, did not say, yes I was on the bus crawl, which everybody's talking about, then we're going to have a person that we have a question mark over, so how did this person get COVID," Dr. Best explained.
sharon.austingill-moore@barbados.gov.bb
Tags: Clusters, Contact tracing, COVID -19, Dr. Anton Best, Lieutenant Colonel Jeffrey Bostic, Minister of Health and Wellness, Senior Medical Officer of Health, testing


text:
Franklin Servan-Schreiber
Long Article in the Neue Zürcher Zeitung
Wonderful long form coverage in Switzerland in the two most presitigious newspapers: in German in the NZZ.ch, and in French in LeTemps.ch. Also featured in the top news magazine in France, Le Point, which in turn was featured as the opening story of the National radio news' press review of France Inter.
A wonderful way to finish the year.
Long Article in Le Figaro - 29 November 2021
New Logo for our New Phase of Development.
TRANSMUTEX
© 2021 by Transmutex SA


text:
using System.Configuration;
using System.Xml;

namespace FantasySimulator.DebugConsole.Config
{
    public class SoccerSimulatorConfigurationSectionHandler : IConfigurationSectionHandler
    {
        public object Create(object parent, object configContext, XmlNode section)
        {
            return section;
        }
    }
}

redpajama_set_name: RedPajamaGithub


text:
Q: Pandas - Merging and Comparing two DataFrames (one unique column) I get that this is a common question but there are so many variations on it in pandas that I couldn't find a solution to my problem.
I have two DataFrames - one for today and one for yesterday and would like to create two new df's using both as inputs.
Based on my single Unique column - the comparison between the two df's can have one of three outputs.

*

*The Unique Key is in both today and yesterday

*The Unique Key is in today but not yesterday

*The Unique Key is not today but is in yesterday
The first df I would like to merge the df's such that it outputs #1 & #2, with the values in yesterday taking precedence AND adding any additional columns from yesterday.
The second output I simply want to output #3.
Using some examples:
today
index c1 c2 unique
1     x  y  14A
2     x  y  15A
4     x  y  17A
5     x  y  18A
6     x  y  19A

yesterday
index c1 c2 unique c3
1     a  b  14A    z
2     a  b  15A    z
3     a  b  16A    z
4     a  b  17A    z

output1 (#1 & #2)
index c1 c2 unique c3
1     a  b  14A    z
2     a  b  15A    z
4     a  b  17A    z
5     x  y  18A
6     x  y  19A

output2 (#3)
index c1 c2 unique c3
3     a  b  16A    z

A few extra things:

*

*I have been trying to use pd.merge for #1, I would really like to not have _x and _y duplicate columns when merging.

*I tried yest[~yest['UniqueKey'].isin(today['UniqueKey'])] for #3 and I think it works?

Any help is much appreciated.
Thanks.

A: For output #3, you may try with :
DataFrame.join(other_Dataframe, how = 'right')

You can follow: Pandas Documentation

A: My solution for first requirement(output1 & output2)
case1 = yesterday[yesterday['unique'].isin(today['unique'])]

case2 = today[~today['unique'].isin(yesterday['unique'])]

output_1_2 = case1.append(case2, ignore_index = True)
output_1_2.replace(np.nan, "", regex = True) # to replace NaN

For second requirement(output3), your approach works.

A: output 1(#1 & #2) can be obtained using pandas.DataFrame.combine_first
if not already done, set the unique column as the data frame's index
today = today.set_index('unique')
yesterday = yesterday.set_index('unique')

then,
yesterday[yesterday.index.isin(today.index)].combine_first(today).reset_index()
  unique c1 c2   c3  index
0    14A  a  b    z    1.0
1    15A  a  b    z    2.0
2    17A  a  b    z    4.0
3    18A  x  y  NaN    5.0
4    19A  x  y  NaN    6.0

output 2(#3) can be obtained using the set difference of the two indices & .loc, or better yet using negated isin & boolean indexing
yesterday[~yesterday.index.isin(today.index)]
# or: yesterday.loc[list(set(yesterday.index) - set(today.index))]
# both produce the following output:
  unique  index c1 c2 c3
0    16A      3  a  b  z

redpajama_set_name: RedPajamaStackExchange


Ask TaxGuru » Individual » Rental Real-Estate » Are landlords required to include advance rent paid by tenants are as rental income?
The IRS specifically defines that "advance rent" is any amount of rent that a landlord receives before the period that it covers.
Landlords are required to include advance rent in their rental income in the year it is received it regardless of the period covered or the method of accounting the landlord is employing for that particular tax year.