text:
\section{Related work}
\label{sec:realted-work}

The original architecture of a GAN model consisted of a single generator, whose objective is to synthesize data samples that are indistinguishable from real ones, and an adversarial discriminator, which attempts to distinguish between real data samples and generated ones \cite{Goodfellow_2014_GAN, Goodfellow_2016_NIPSTutorial}. Both models are trained simultaneously and in an adversary. When successfully trained, the generator tends to synthesize realistic samples, whereas the discriminator is expected to become maximally confused. Sampling in this architecture consists of sampling a latent variable from a simple distribution (e.g., standard Gaussian) and mapping its value to the sample space using the generator's function. Despite the simplicity of such a process, it is unrestrained over the whole target data manifold. In other words, there is no control on the samples being synthesized by the generator.

In \cite{Radford_2015_DCGAN}, it was observed that the generator tends to structure the latent space, and the complex transformations in the sample space (e.g., face pose) can be controlled via a simple linear interpolation on the latent variable. This feature was exploited in \cite{Chen_2016_InfoGAN} by discretizing a subset of the latent variable attributes in order to control the sampling process. This approach, however, limits the trained generator to a single estimation of the density function whose output can be controlled by conditioning on the selected attributes. In a like manner, Conditional GANs (cGAN) \cite{Mirza_2014_cGAN, Sricharan_2017_SScGAN} controlled this sampling process by conditioning the generator on an observed variable in addition to the latent one. Examples of conditioning variables include text embeddings \cite{Reed_2016_Text2Image}, foreign domain image \cite{Isola_2016_Pix2Pix, Denton_2015_LaplacianPyramids}, or class attributes \cite{Mirza_2014_cGAN}. However, cGAN models require the additional information found in the conditioning variable and are incapable of learning hidden structures of the target distribution in an unsupervised fashion.

Coupled GAN models \cite{Liu_2016_CoGAN} introduced a multi-generator multi-discriminator architecture whose objective is to learn the joint distribution of multi-domain images from their marginal ones. For a given realization of the latent variable, each generator synthesizes the same sample but in different domains. While this approach eliminates the need for inter-domain knowledge, it requires the complete knowledge of each domain in advance.

Categorical GANs \cite{Springenberg_2015_CatGAN} resemble one of the promising approaches to unravel hidden structures in unlabeled data manifolds. The binary logistic discriminator in traditional GAN models is replaced by a multi-class one whose objective is to maintain high confidence classification scores for the real data samples while being maximally uncertain when provided with synthesized images. As a result, the discriminator becomes capable of clustering the real data samples in a completely unsupervised fashion. Nevertheless, this approach adopts a single generator which is still uncontrolled in its sampling process.

Our approach differs in that its objective is the controlled generation process rather than discrimination. The proposed GAN model attempts to control the sampling process by training several generators that collectively resemble the targeted distribution while maintaining distinct modes among these generators. One can selectively sample from a certain sub-population by sampling from the corresponding generator only.

\section{Conclusion}
\label{sec:conclusion}

We proposed a more detailed representation learning model based on a multi-generator extension to the GAN architecture. The generators tend to cooperatively resemble the target data distribution but each generator tends to produce only a distinct mode. This architecture provides selective sampling from a certain cluster of the target without the need of any labeling information.
\section{Introduction and Related Work}
\label{sec:introduction-and-related-work}

One of the popular unsupervised learning methods is generative modeling, and in the recent years a promising architecture, referred to as \ul{G}enerative \ul{A}dversarial \ul{N}etworks (GAN) \cite{Goodfellow_2014_GAN}, has attracted a wide interest due to their impressive success in various applications domains, including text-to-image generation \cite{Reed_2016_Text2Image}, image-to-image translation \cite{Isola_2016_Pix2Pix}, multi- and super-resolution image synthesis \cite{Denton_2015_LaplacianPyramids}, video generation \cite{Vondrick_2016_GeneratingVideos}, in addition to a variety of  unsupervised and semi-supervised learning tasks \cite{Shrivastava_2016_SimulatedGANApple, Springenberg_2015_CatGAN, Zhu_2017_CycleConsistentGAN}. The common principle behind all variants of the GAN architecture is to exploit the discriminative power of neural networks in training generative counterparts. GAN models represent a compromise between ease of sampling and perceptual quality of synthesized samples.

Similar to other generative models, the ultimate objective of GANs is to model a target distribution from a limited set of observations.

\section{Introduction}
\label{sec:introduction}

Over the last few years, a promising generative modeling architecture, referred to as \underline{G}enerative \underline{A}dversarial \underline{N}etwork (GAN) \cite{Goodfellow_2014_GAN}, has attracted a wide interest due to their impressive success in various application domains, including text-to-image generation \cite{Reed_2016_Text2Image}, compressive sensing \cite{Quan_2017_RefineGAN_CS_MRI}, image-to-image translation \cite{Isola_2016_Pix2Pix}, multi- and super-resolution image synthesis \cite{Denton_2015_LaplacianPyramids}, video generation \cite{Vondrick_2016_GeneratingVideos}, human driver modeling \cite{Kuefler_2017_GAIL_DriverGAN} in addition to a variety of  unsupervised and semi-supervised learning tasks \cite{Shrivastava_2016_SimulatedGANApple, Springenberg_2015_CatGAN, Zhu_2017_CycleConsistentGAN}. The common principle behind all variants of the GAN architecture is to exploit the discriminative power of neural networks in training generative counterparts. GAN models represent a compromise between ease of sampling and perceptual quality of synthesized samples.

Similar to other generative models, the ultimate objective of GANs is to model a target distribution from a limited set of observations. To this end, a basic GAN architecture consists of a cascade of two sub-models, a \emph{generator} $g$ whose objective is to synthesize samples $\hat{\underline{x}}$ indistinguishable from real observations $\underline{x}$, and a \emph{discriminator} $h$ whose objective is to train the generator by estimating a \emph{distance} between the generator's induced distribution $\mathbb{P}_{\hat{\underline{x}}}$ and the target one $\mathbb{P}_{{\underline{x}}}$. In spite of several difficulties in their training process, GAN models proved to be successful in synthesizing plausible, realistic samples while maintaining ease of sampling even for high-dimensional densities \cite{Arjovsky_2017_WassersteinGAN}. Consequently, extensive research has been carried out in order to understand and improve the training process of GAN models \cite{Salimans_2016_ImprovedGAN, Arjovsky_2017_PrincipledMethods, Arjovsky_2017_WassersteinGAN, Mao_2017_LSGAN, Yi_2017_DualGAN}, introduce this model to novel application domains \cite{Isola_2016_Pix2Pix, Zhu_2017_CycleConsistentGAN, Reed_2016_Text2Image, Zhu_2016_iGAN, Li_2016_TextureGAN}, and improve the basic formulation for more advanced learning tasks \cite{Springenberg_2015_CatGAN, Denton_2015_LaplacianPyramids, Wang_2016_S2GAN, Mirza_2014_cGAN, Makhzani_2015_AdversarialAE}.

In this paper, we propose an extension to the basic GAN architecture that targets a more detailed representation of the real data manifold. More specifically, in the proposed architecture, multiple generators cooperate in resembling a target data distribution while maintaining distinct manifolds. All generators are trained using a single adversarial discriminator but are also constrained to distinct distributions via a set of supplementary discriminators. The ultimate objective of the architecture is a set of generative functions learned from a finite set of data samples that jointly model the target data distribution but individually resemble distinct modes in it.

\section{Experiments and results}
\label{sec:experiments-and-results}

\subsection{Experimental setup}
\label{subsec:experimental-setup}
We test and evaluate the proposed model on the MNIST dataset \cite{LeCun_1998_LeNet} and with the two-model case\footnote{We delegate the many model $K>2$ case to a future work. Our initial experiments show that the derived value function is unbalanced (i.e., divergences are unequally bounded and weighted) which results in degradation of the perceptual quality as the number of models increase.} $K=2$. In each test case, we select images from a two-digit combination (e.g, $\{0,\,1\}$) and train the randomly initialized model on the unlabeled dataset. We expect that each of the two generators will synthesize images of a unique digit.

The latent random vector is uniformly distributed $\underline{z}\sim \mathcal{U}(\texttt{-}\,1, 1)$ in a 100-dimensional space $\underline{z}\in \mathbb{R}^{100}$ and it seeds two generators $g_1$ and $g_2$ where each is a 2-layer, feed-forward, fully connected neural network. Both the observed variable and the synthesized one belong to the same space $\underline{x},\hat{\underline{x}} \in \mathbb{R}^{784}$ and both are used to train the adversarial discriminator\footnote{A single binary discriminator for $g_1$ petted against $g_2$.} $h$. We use a single supplementary discriminator $h_1$ whose input is a batch of the synthesized images $\hat{\underline{x}}$. The two discriminators $h$ and $h_1$ are identically structured and have exactly the inverse architecture of the generators (except for the output layer). The output layer of each model uses logistic sigmoidal activation whereas its hidden layers use rectified linear units \cite{Nair_2010_ReLU_RBM}. The whole model is 4-layer deep in its longest path and comprises 0.8M trainable parameters. Finally, all models are trained using the Adam optimizer \cite{Kingma_2014_ADAM} with a learning rate of $10^{-3}$ and each sub-model gains a single parameter-update step in each iteration.

We highlight that instead of adhering to the aforementioned theoretical analysis in which training alternates between gradient-descent and -ascent steps, we rather flip the desired output $y\leftarrow 1-y$ and use only gradient-descent. This technique has been proposed in several previous works \cite{Goodfellow_2016_NIPSTutorial, Mao_2017_LSGAN} with conceivable reasoning\footnote{More recent studies proved that such an approach indeed eliminates the vanishing gradient problem but rather introduces critical sources of instability in the training process \cite{Arjovsky_2017_PrincipledMethods}.} but rather violates the presented analytical study of the architecture since each sub-model retains a different objective function.

\begin{figure}[t]
	\begin{tikzpicture}[every node/.style={inner sep=0,outer sep=0}]
	\node (a) {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS01_DIS0_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS17_DIS0_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS24_DIS0_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS08_DIS1_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS26_DIS1_EPOCH0002999_019.png}};
	\end{tikzpicture}
	\caption{Two-digit experiments on the proposed architecture using the MNIST dataset. The top-most row of each experiment shows
		a sample of the real images whereas the two lower rows depict synthesized samples from each generator.}
	\label{fig:two-digit-success}
	\vspace{-2.5mm}
\end{figure}
\begin{figure}[!ht]
	\begin{tikzpicture}[every node/.style={inner sep=0,outer sep=0}]
	\node (a) {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS04_DIS1_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS19_DIS0_EPOCH0004999_019.png}};
	\end{tikzpicture}
	\caption{Examples of failure cases.}
	\label{fig:two-digit-failure}
	\vspace{-2.5mm}
\end{figure}

\subsection{Results and discussion}
\label{subsec:dicussion}

Fig. \ref{fig:two-digit-success} shows the output of each generator in various experiments (each with a distinct two-digit combination). As observed, the generators produce samples of moderate to high perceptual quality but each one synthesizes a different digit from the given combination. On the other hand, few failure cases were observed and are shown in Fig. \ref{fig:two-digit-failure}. In these experiments, we still observe that the generators tend to produce, to some extent, different representations of the same digit rather than distinct digits.

Training adversarial networks, in general, is known to be unstable \cite{Arjovsky_2017_PrincipledMethods}, and the alternating gradient-based parameter updates (rather than training to completion) between the two sub-models in the original model is one of the sources for their instability \cite{Arjovsky_2017_PrincipledMethods}. In the proposed architecture, these alternating parameter updates are extended to a larger number of sub-models and, as a result, expected to amplify this instable training behavior. However, we observed in our experiments that the generators maintain distinct manifolds even in the early training steps and only collapse to the same digit when over-trained.

Furthermore, the objective of the supplementary discriminators is to provide a simple means for exploiting the powerful back-propagation algorithm in pushing the generators' distributions apart from each other. This seems to be valid as long as these models are weak classifiers. If they are provided with enough training capacity, they begin to deliver their discrimination task all by themselves and their driving force on the generators' distributions disappears. As a results, the generators tend to collapse to the same mode which is highly-rewarded by the adversarial discriminator rather than adhering to distinct classes. In fact, the precise amount of training, even for traditional GAN models, is a challenging hyper-parameter to optimize for reasons explained in \cite{Arjovsky_2017_PrincipledMethods}.

\section{Introduction}
\label{sec:intro}

These guidelines include complete descriptions of the fonts, spacing, and
related information for producing your proceedings manuscripts. Please follow
them and if you have any questions, direct them to Conference Management
Services, Inc.: Phone +1-979-846-6800 or email
to \\\texttt{papers@2018.ieeeicassp.org}.

\section{Formatting your paper}
\label{sec:format}

All printed material, including text, illustrations, and charts, must be kept
within a print area of 7 inches (178 mm) wide by 9 inches (229 mm) high. Do
not write or print anything outside the print area. The top margin must be 1
inch (25 mm), except for the title page, and the left margin must be 0.75 inch
(19 mm).  All {\it text} must be in a two-column format. Columns are to be 3.39
inches (86 mm) wide, with a 0.24 inch (6 mm) space between them. Text must be
fully justified.

\section{PAGE TITLE SECTION}
\label{sec:pagestyle}

The paper title (on the first page) should begin 1.38 inches (35 mm) from the
top edge of the page, centered, completely capitalized, and in Times 14-point,
boldface type.  The authors' name(s) and affiliation(s) appear below the title
in capital and lower case letters.  Papers with multiple authors and
affiliations may require two or more lines for this information. Please note
that papers should not be submitted blind; include the authors' names on the
PDF.

\section{TYPE-STYLE AND FONTS}
\label{sec:typestyle}

To achieve the best rendering both in printed proceedings and electronic proceedings, we
strongly encourage you to use Times-Roman font.  In addition, this will give
the proceedings a more uniform look.  Use a font that is no smaller than nine
point type throughout the paper, including figure captions.

In nine point type font, capital letters are 2 mm high.  {\bf If you use the
smallest point size, there should be no more than 3.2 lines/cm (8 lines/inch)
vertically.}  This is a minimum spacing; 2.75 lines/cm (7 lines/inch) will make
the paper much more readable.  Larger type sizes require correspondingly larger
vertical spacing.  Please do not double-space your paper.  TrueType or
Postscript Type 1 fonts are preferred.

The first paragraph in each section should not be indented, but all the
following paragraphs within the section should be indented as these paragraphs
demonstrate.

\section{MAJOR HEADINGS}
\label{sec:majhead}

Major headings, for example, "1. Introduction", should appear in all capital
letters, bold face if possible, centered in the column, with one blank line
before, and one blank line after. Use a period (".") after the heading number,
not a colon.

\subsection{Subheadings}
\label{ssec:subhead}

Subheadings should appear in lower case (initial word capitalized) in
boldface.  They should start at the left margin on a separate line.

\subsubsection{Sub-subheadings}
\label{sssec:subsubhead}

Sub-subheadings, as in this paragraph, are discouraged. However, if you
must use them, they should appear in lower case (initial word
capitalized) and start at the left margin on a separate line, with paragraph
text beginning on the following line.  They should be in italics.

\section{PRINTING YOUR PAPER}
\label{sec:print}

Print your properly formatted text on high-quality, 8.5 x 11-inch white printer
paper. A4 paper is also acceptable, but please leave the extra 0.5 inch (12 mm)
empty at the BOTTOM of the page and follow the top and left margins as
specified.  If the last page of your paper is only partially filled, arrange
the columns so that they are evenly balanced if possible, rather than having
one long column.

In LaTeX, to start a new column (but not a new page) and help balance the
last-page column lengths, you can use the command ``$\backslash$pagebreak'' as
demonstrated on this page (see the LaTeX source below).

\section{PAGE NUMBERING}
\label{sec:page}

Please do {\bf not} paginate your paper.  Page numbers, session numbers, and
conference identification will be inserted when the paper is included in the
proceedings.

\section{ILLUSTRATIONS, GRAPHS, AND PHOTOGRAPHS}
\label{sec:illust}

Illustrations must appear within the designated margins.  They may span the two
columns.  If possible, position illustrations at the top of columns, rather
than in the middle or at the bottom.  Caption and number every illustration.
All halftone illustrations must be clear black and white prints.  Colors may be
used, but they should be selected so as to be readable when printed on a
black-only printer.

Since there are many ways, often incompatible, of including images (e.g., with
experimental results) in a LaTeX document, below is an example of how to do
this \cite{Lamp86}.

\section{FOOTNOTES}
\label{sec:foot}

Use footnotes sparingly (or not at all!) and place them at the bottom of the
column on the page on which they are referenced. Use Times 9-point type,
single-spaced. To help your readers, avoid using footnotes altogether and
include necessary peripheral observations in the text (within parentheses, if
you prefer, as in this sentence).

\begin{figure}[htb]

\begin{minipage}[b]{1.0\linewidth}
  \centering
  \centerline{\includegraphics[width=8.5cm]{./figures/image1}}
  \centerline{(a) Result 1}\medskip
\end{minipage}
\begin{minipage}[b]{.48\linewidth}
  \centering
  \centerline{\includegraphics[width=4.0cm]{./figures/image3}}
  \centerline{(b) Results 3}\medskip
\end{minipage}
\hfill
\begin{minipage}[b]{0.48\linewidth}
  \centering
  \centerline{\includegraphics[width=4.0cm]{./figures/image4}}
  \centerline{(c) Result 4}\medskip
\end{minipage}
\caption{Example of placing a figure with experimental results.}
\label{fig:res}
\end{figure}

\section{COPYRIGHT FORMS}
\label{sec:copyright}

You must submit your fully completed, signed IEEE electronic copyright release
form when you submit your paper. We {\bf must} have this form before your paper
can be published in the proceedings.

\section{RELATION TO PRIOR WORK}
\label{sec:prior}

The text of the paper should contain discussions on how the paper's
contributions are related to prior work in the field. It is important
to put new work in  context, to give credit to foundational work, and
to provide details associated with the previous work that have appeared
in the literature. This discussion may be a separate, numbered section
or it may appear elsewhere in the body of the manuscript, but it must
be present.

You should differentiate what is new and how your work expands on
or takes a different path from the prior studies. An example might
read something to the effect: "The work presented here has focused
on the formulation of the ABC algorithm, which takes advantage of
non-uniform time-frequency domain analysis of data. The work by
Smith and Cohen \cite{Lamp86} considers only fixed time-domain analysis and
the work by Jones et al \cite{C2} takes a different approach based on
fixed frequency partitioning. While the present study is related
to recent approaches in time-frequency analysis [3-5], it capitalizes
on a new feature space, which was not considered in these earlier
studies."

\section{REFERENCES}
\label{sec:refs}

List and number all bibliographical references at the end of the
paper. The references can be numbered in alphabetic order or in
order of appearance in the document. When referring to them in
the text, type the corresponding reference number in square
brackets as shown at the end of this sentence \cite{C2}. An
additional final page (the fifth page, in most cases) is
allowed, but must contain only references to the prior
literature.
\section{Model Architecture}
\label{sec:model-architecture}

Let $\underline{x} \in \mathcal{X}$ be the observed data of a continuous random vector $\underline{x}$ from the space $\mathcal{X}$ of observations. The notation $\underline{x} \sim p_{\underline{x}}$ is used to denote the probability density function (pdf) $p_{\underline{x}}$ of $\underline{x}$ of the real data distribution $\mathbb{P}_{\underline{x}}$.

The proposed architecture is depicted in Fig. \ref{fig:model-architecture} and consists of $K$ generative models (generators) $\{g_k\}_{k=1}^{K}$, each of which maps a latent variable $\underline{z}_k\sim p_{\underline{z}}$ to a synthesized sample $\smash{\hat{\underline{x}}_k = g_k(\underline{z}_k)}\sim p_k$ and induces a continuous\footnote{For simplicity, and for illustrative purposes, we shall assume that all distributions are continuous with non-disjoint support.} distribution $\mathbb{P}_k$. All synthesized samples $\hat{\underline{x}}_k$ belong to the same space $\mathcal{X}$. Let $p_{\underline{x}}$ be the density function of the real data distribution $\smash{\mathbb{P}}_{\underline{x}}$. Furthermore, let $\hat{\underline{x}}$ be a random vector equi-probably sampled from $\{g_k\}_{k=1}^{K}$. This results in a mixture pdf $p_{\hat{\underline{x}}}$ defined as
\begin{equation}
p_{\hat{\underline{x}}}(\underline{x}) =\; \frac{1}{K} \sum_{k=1}^K p_k(\underline{x})
\label{eq:mixture-model}
\end{equation}%
We further introduce $K$ binary logistic models (discriminators) $\{h_k\}_{k=1}^{K}$ each of which maps samples from the input data space $\hat{\underline{x}}\sim\mathcal{X}$ to a probability $h_k: \mathcal{X} \rightarrow [0, 1]$ where $h_k(\hat{\underline{x}})$ resembles the probability that $\hat{\underline{x}}$ is sampled from the $k^\text{th}$ model $g_k$ as opposed to all other components in the mixture. The opposite probability that $\hat{\underline{x}}$ is equi-probably sampled from the remaining set of generators $g_j,\,j\neq k$ is
\begin{equation}
p_{\bar{k}}(\underline{x}) = \frac{1}{K-1} \sum_{\substack{j=1\\j\neq k}}^{K} p_j(\underline{x})
\end{equation}
which also gives another formulation for the mixture pdf
\begin{equation}
p_{\hat{\underline{x}}}(\underline{x}) =\; \frac{1}{K} \sum_{k=1}^K p_{\bar{k}}(\underline{x})
\label{eq:complement-mixture-model}
\end{equation}
The loss function of each of the $k^\text{th}$ discriminator becomes
{\small
\begin{align*}
\mathcal{L}_{h_k}
& =                                    \mathbb{E}_{\underline{z} \sim p_{\underline{z}}         }             \log\left(     h_k(g_k(\underline{z})) \right) +
    \sum_{\substack{j=1\\j\neq k}}^{K} \mathbb{E}_{\underline{z} \sim p_{\underline{z}}         }             \log\left( 1 - h_k(g_j(\underline{z})) \right) \\
& =                                    \mathbb{E}_{\underline{x} \sim p_k         }             \log\left(     h_k(    \underline{x})  \right) +
    \sum_{\substack{j=1\\j\neq k}}^{K} \mathbb{E}_{\underline{x} \sim p_j         }             \log\left( 1 - h_k(    \underline{x})  \right) \\
& =                                    \mathbb{E}_{\underline{x} \sim p_k         }             \log\left(     h_k(    \underline{x})  \right) +
                                       \mathbb{E}_{\underline{x} \sim p_{\bar{k}} } \left[(K-1) \log\left( 1 - h_k(    \underline{x})  \right) \right] \\
\end{align*}}%

Moreover, an additional adversarial discriminator $h$ distinguishes samples of the real mixture distribution $\mathbb{P}_{\underline{x}}$ from those of the generators' mixture $\mathbb{P}_{\hat{\underline{x}}}$. The loss function of this discriminator is similar to the traditional GAN loss
{\small \begin{align*}
	\mathcal{L}_{h}
	& =\mathbb{E}_{\underline{x} \sim p_{     \underline{x}}}\log\left(     h(\underline{x}) \right) +
	\mathbb{E}_{\underline{x} \sim p_{\hat{\underline{x}}}}\log\left( 1 - h(\underline{x}) \right) \\
	\end{align*}}%

Let $V$ be the value function of the \emph{mini-minimax} game defined as
{\small \begin{align}
	\min_{\{g_k\}} \min_{\{h_k\}} \max_{h} V = \mathcal{L}_{h} - \sum_{k=1}^{K} \mathcal{L}_{h_k}

	\end{align}}%
The adversarial GAN game in this case becomes a \emph{minimax} game between two teams. The first team consists of a single player (the adversarial discriminator $h$) whose objective is to \emph{maximize} the value function. The second team consists of all generators $\{g_k\}_{k=1}^{K}$ and supplementary discriminators $\{h_k\}_{k=1}^{K}$ where each player in this team is tasked with \emph{minimizing} the same value function. Each model optimizes the value function with respect to its own parameters (assuming all other models are fixed).

In the non-parametric limit, and assuming each model can train to completion, the resulting discriminators reach their global optimum at (proof can be found in \cite{Goodfellow_2014_GAN})
{\small \begin{equation*}
h_k^\star(\underline{x}) =\, \frac{p_k(\underline{x})}{p_k(\underline{x}) + (K-1) \cdot p_{\bar{k}}(\underline{x})} =\, \frac{p_k(\underline{x})}{K \cdot p_{\hat{\underline{x}}}(\underline{x})}
\end{equation*}}%
and
{\small \begin{equation*}
h^\star(\underline{x})   =\, \frac{p_{{\underline{x}}}(\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})\; +\; p_{{\underline{x}}}(\underline{x})}
\end{equation*}}%
Substituting these optimal values in the value function $V$ results in
{\small \begin{alignat*}{3}
V && =\; &&&                       \mathbb{E}_{\underline{x} \sim p_{\underline{x}}      }        \log\left(     h^\star  (\underline{x}) \right)
           +                       \mathbb{E}_{\underline{x} \sim p_{\hat{\underline{x}}}}        \log\left( 1 - h^\star  (\underline{x}) \right)  \\
  && -\; &&& \sum_{k=1}^{K} \left( \mathbb{E}_{\underline{x} \sim p_k                    }        \log\left(     h_k^\star(\underline{x}) \right) +
                                   \mathbb{E}_{\underline{x} \sim p_{\bar{k}}            }        \log\left( 1 - h_k^\star(\underline{x}) \right) \right) \\
  && =\; &&&                       \mathbb{E}_{\underline{x} \sim p_{\underline{x}}      }        \log\left(\frac{p_{{\underline{x}}}(\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})\; +\; p_{{\underline{x}}}(\underline{x})}     \right) \\
  && +\; &&&                       \mathbb{E}_{\underline{x} \sim p_{\hat{\underline{x}}}}        \log\left(\frac{p_{\hat{\underline{x}}}(\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})\; +\; p_{{\underline{x}}}(\underline{x})} \right) \\
  && -\; &&& \sum_{k=1}^{K}        \mathbb{E}_{\underline{x} \sim p_k                    }  \left[\log\left(\frac{p_k                    (\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})}  \right) + \log\left(\frac{1}{K}\right)                      \right] \\
  && -\; &&& \sum_{k=1}^{K}        \mathbb{E}_{\underline{x} \sim p_{\bar{k}}            }  \left[\log\left(\frac{p_{\bar{k}            } (\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})} \right) + \log\left(\frac{K-1}{K}\right)                    \right] \\
\end{alignat*}\vspace{-10mm}}%

All terms in the previous equation resemble different, non-symmetric Kullback-Leibler (KL) divergences resulting in
{\small \begin{alignat}{3}
V && =\; &&&                \text{D}_\text{KL} \left( \mathbb{P}_{\underline{x}}        \;\|\;  \mathbb{P}_{\hat{\underline{x}}} \texttt{+}\; \mathbb{P}_{{\underline{x}}}    \right) +
                            \text{D}_\text{KL} \left( \mathbb{P}_{\hat{\underline{x}}}  \;\|\;  \mathbb{P}_{\hat{\underline{x}}} \texttt{+}\; \mathbb{P}_{{\underline{x}}}    \right) \nonumber \\
  && -\; &&& \sum_{k=1}^{K} \text{D}_\text{KL} \left( \mathbb{P}_k                      \;\|\;  \mathbb{P}_{\hat{\underline{x}}}                                         \right) -
             \sum_{k=1}^{K} \text{D}_\text{KL} \left( \mathbb{P}_{\bar{k}}              \;\|\;  \mathbb{P}_{\hat{\underline{x}}}                                         \right) \nonumber \\
  && -\; &&& K\cdot\log\left(\frac{K-1}{K^2}\right)
\label{eq:generator-value-function-kl}
\end{alignat}}%
It can be further simplified to symmetric Jensen-Shannon (JS) divergences
{\small \begin{alignat}{3}
V  && =\; &&& 2\cdot\text{D}_\text{JS} \left( \mathbb{P}_{\hat{\underline{x}}},  \mathbb{P}_{{\underline{x}}}   \right) \nonumber - K\cdot \text{D}_\text{JS} \left( \mathbb{P}_{{1}}, \dots, \mathbb{P}_{{K}}  \right) \nonumber \\
   && -\; &&&  K\cdot\text{D}_\text{JS} \left( \mathbb{P}_{\bar{1}}, \dots, \mathbb{P}_{\bar{K}} \right) - K\cdot\log\frac{K-1}{K^2} - \log 4
\label{eq:generator-value-function}
\end{alignat}}%
where the $\pi$-generalized JS divergence for a finite mixture of distributions is defined as \cite{Lin_1991_ShannonDivergences}
{\small \begin{equation}
    \text{D}_\text{JS}^{\pi_1, \dots, \pi_K}(\mathbb{P}_{1}, \dots, \mathbb{P}_K) =\; \sum_{k=1}^{K}\pi_k\cdot\text{D}_\text{KL}\left(\mathbb{P}_k\;\Big\|\;\sum_{j=1}^{K}\pi_j\cdot\mathbb{P}_j\right)
    \label{eq:jensen-shannon-mixture}
\end{equation}}%
where $\pi_1,\dots,\pi_K$ are the distributions' weights and are assumed in this work to be $1/K$. Eq. \ref{eq:jensen-shannon-mixture} is applicable to the two distributions $\mathbb{P}_{{\underline{x}}}$ and $\mathbb{P}_{\hat{\underline{x}}}$ by defining an average distribution $\mathbb{P}_A = (\mathbb{P}_{\hat{\underline{x}}} + \mathbb{P}_{{\underline{x}}}) / 2$. It is also applicable within the mixture model $\mathbb{P}_{\hat{\underline{x}}}$ using Eq. \ref{eq:mixture-model} and \ref{eq:complement-mixture-model}.
Minimizing Eq. \ref{eq:generator-value-function} is equivalent to minimizing the \emph{difference} between the real data distribution and the estimated one while maximizing the dissimilarity between the components of the latter. This, in turn, induces each generator in the model to maintain distinct observational claims of the real data distribution.

\begin{figure}[!t]
	\input{./figures/model-architecture.tex}
	\vspace{-5.0mm}
\end{figure}

Eq. \ref{eq:generator-value-function-kl} represents a very interesting objective function. The first two terms resemble the symmetric JS divergence between the real distribution and the estimated mixture which is non-negative and retains its minimum when $\mathbb{P}_{\underline{x}} = \mathbb{P}_{\hat{\underline{x}}}$. This term is the common measure in traditional GAN models \cite{Goodfellow_2014_GAN}. The third KL-divergence prevents any of the component models $g_k$ from prevalence (representing the whole mixture all by itself), whereas the last term prevents the same component from vanishing.

In practice, each model (either a discriminator or a generator) is a parametric Multi-Layer Perceptron (MLP). This, of course, limits the family of density functions representable by each generator. Nevertheless, each one retains its own parameters, and as a result, the estimated distribution represents a heterogeneous mixture model. Training is straightforward and, similar to traditional GANs, consists of alternating gradient-based parameter updates. Gradient-ascent is utilized by the adversarial discriminator while the supplementary discriminators and the generators use gradient-descent.

\section{Related work}
\label{sec:realted-work}

The original architecture of a GAN model consisted of a single generator, whose objective is to synthesize data samples that are indistinguishable from real ones, and an adversarial discriminator, which attempts to distinguish between real data samples and generated ones \cite{Goodfellow_2014_GAN, Goodfellow_2016_NIPSTutorial}. Both models are trained simultaneously and in an adversary. When successfully trained, the generator tends to synthesize realistic samples, whereas the discriminator is expected to become maximally confused. Sampling in this architecture consists of sampling a latent variable from a simple distribution (e.g., standard Gaussian) and mapping its value to the sample space using the generator's function. Despite the simplicity of such a process, it is unrestrained over the whole target data manifold. In other words, there is no control on the samples being synthesized by the generator.

In \cite{Radford_2015_DCGAN}, it was observed that the generator tends to structure the latent space, and the complex transformations in the sample space (e.g., face pose) can be controlled via a simple linear interpolation on the latent variable. This feature was exploited in \cite{Chen_2016_InfoGAN} by discretizing a subset of the latent variable attributes in order to control the sampling process. This approach, however, limits the trained generator to a single estimation of the density function whose output can be controlled by conditioning on the selected attributes. In a like manner, Conditional GANs (cGAN) \cite{Mirza_2014_cGAN, Sricharan_2017_SScGAN} controlled this sampling process by conditioning the generator on an observed variable in addition to the latent one. Examples of conditioning variables include text embeddings \cite{Reed_2016_Text2Image}, foreign domain image \cite{Isola_2016_Pix2Pix, Denton_2015_LaplacianPyramids}, or class attributes \cite{Mirza_2014_cGAN}. However, cGAN models require the additional information found in the conditioning variable and are incapable of learning hidden structures of the target distribution in an unsupervised fashion.

Coupled GAN models \cite{Liu_2016_CoGAN} introduced a multi-generator multi-discriminator architecture whose objective is to learn the joint distribution of multi-domain images from their marginal ones. For a given realization of the latent variable, each generator synthesizes the same sample but in different domains. While this approach eliminates the need for inter-domain knowledge, it requires the complete knowledge of each domain in advance.

Categorical GANs \cite{Springenberg_2015_CatGAN} resemble one of the promising approaches to unravel hidden structures in unlabeled data manifolds. The binary logistic discriminator in traditional GAN models is replaced by a multi-class one whose objective is to maintain high confidence classification scores for the real data samples while being maximally uncertain when provided with synthesized images. As a result, the discriminator becomes capable of clustering the real data samples in a completely unsupervised fashion. Nevertheless, this approach adopts a single generator which is still uncontrolled in its sampling process.

Our approach differs in that its objective is the controlled generation process rather than discrimination. The proposed GAN model attempts to control the sampling process by training several generators that collectively resemble the targeted distribution while maintaining distinct modes among these generators. One can selectively sample from a certain sub-population by sampling from the corresponding generator only.

\section{Conclusion}
\label{sec:conclusion}

We proposed a more detailed representation learning model based on a multi-generator extension to the GAN architecture. The generators tend to cooperatively resemble the target data distribution but each generator tends to produce only a distinct mode. This architecture provides selective sampling from a certain cluster of the target without the need of any labeling information.
\section{Introduction and Related Work}
\label{sec:introduction-and-related-work}

One of the popular unsupervised learning methods is generative modeling, and in the recent years a promising architecture, referred to as \ul{G}enerative \ul{A}dversarial \ul{N}etworks (GAN) \cite{Goodfellow_2014_GAN}, has attracted a wide interest due to their impressive success in various applications domains, including text-to-image generation \cite{Reed_2016_Text2Image}, image-to-image translation \cite{Isola_2016_Pix2Pix}, multi- and super-resolution image synthesis \cite{Denton_2015_LaplacianPyramids}, video generation \cite{Vondrick_2016_GeneratingVideos}, in addition to a variety of  unsupervised and semi-supervised learning tasks \cite{Shrivastava_2016_SimulatedGANApple, Springenberg_2015_CatGAN, Zhu_2017_CycleConsistentGAN}. The common principle behind all variants of the GAN architecture is to exploit the discriminative power of neural networks in training generative counterparts. GAN models represent a compromise between ease of sampling and perceptual quality of synthesized samples.

Similar to other generative models, the ultimate objective of GANs is to model a target distribution from a limited set of observations.

\section{Introduction}
\label{sec:introduction}

Over the last few years, a promising generative modeling architecture, referred to as \underline{G}enerative \underline{A}dversarial \underline{N}etwork (GAN) \cite{Goodfellow_2014_GAN}, has attracted a wide interest due to their impressive success in various application domains, including text-to-image generation \cite{Reed_2016_Text2Image}, compressive sensing \cite{Quan_2017_RefineGAN_CS_MRI}, image-to-image translation \cite{Isola_2016_Pix2Pix}, multi- and super-resolution image synthesis \cite{Denton_2015_LaplacianPyramids}, video generation \cite{Vondrick_2016_GeneratingVideos}, human driver modeling \cite{Kuefler_2017_GAIL_DriverGAN} in addition to a variety of  unsupervised and semi-supervised learning tasks \cite{Shrivastava_2016_SimulatedGANApple, Springenberg_2015_CatGAN, Zhu_2017_CycleConsistentGAN}. The common principle behind all variants of the GAN architecture is to exploit the discriminative power of neural networks in training generative counterparts. GAN models represent a compromise between ease of sampling and perceptual quality of synthesized samples.

Similar to other generative models, the ultimate objective of GANs is to model a target distribution from a limited set of observations. To this end, a basic GAN architecture consists of a cascade of two sub-models, a \emph{generator} $g$ whose objective is to synthesize samples $\hat{\underline{x}}$ indistinguishable from real observations $\underline{x}$, and a \emph{discriminator} $h$ whose objective is to train the generator by estimating a \emph{distance} between the generator's induced distribution $\mathbb{P}_{\hat{\underline{x}}}$ and the target one $\mathbb{P}_{{\underline{x}}}$. In spite of several difficulties in their training process, GAN models proved to be successful in synthesizing plausible, realistic samples while maintaining ease of sampling even for high-dimensional densities \cite{Arjovsky_2017_WassersteinGAN}. Consequently, extensive research has been carried out in order to understand and improve the training process of GAN models \cite{Salimans_2016_ImprovedGAN, Arjovsky_2017_PrincipledMethods, Arjovsky_2017_WassersteinGAN, Mao_2017_LSGAN, Yi_2017_DualGAN}, introduce this model to novel application domains \cite{Isola_2016_Pix2Pix, Zhu_2017_CycleConsistentGAN, Reed_2016_Text2Image, Zhu_2016_iGAN, Li_2016_TextureGAN}, and improve the basic formulation for more advanced learning tasks \cite{Springenberg_2015_CatGAN, Denton_2015_LaplacianPyramids, Wang_2016_S2GAN, Mirza_2014_cGAN, Makhzani_2015_AdversarialAE}.

In this paper, we propose an extension to the basic GAN architecture that targets a more detailed representation of the real data manifold. More specifically, in the proposed architecture, multiple generators cooperate in resembling a target data distribution while maintaining distinct manifolds. All generators are trained using a single adversarial discriminator but are also constrained to distinct distributions via a set of supplementary discriminators. The ultimate objective of the architecture is a set of generative functions learned from a finite set of data samples that jointly model the target data distribution but individually resemble distinct modes in it.

\section{Experiments and results}
\label{sec:experiments-and-results}

\subsection{Experimental setup}
\label{subsec:experimental-setup}
We test and evaluate the proposed model on the MNIST dataset \cite{LeCun_1998_LeNet} and with the two-model case\footnote{We delegate the many model $K>2$ case to a future work. Our initial experiments show that the derived value function is unbalanced (i.e., divergences are unequally bounded and weighted) which results in degradation of the perceptual quality as the number of models increase.} $K=2$. In each test case, we select images from a two-digit combination (e.g, $\{0,\,1\}$) and train the randomly initialized model on the unlabeled dataset. We expect that each of the two generators will synthesize images of a unique digit.

The latent random vector is uniformly distributed $\underline{z}\sim \mathcal{U}(\texttt{-}\,1, 1)$ in a 100-dimensional space $\underline{z}\in \mathbb{R}^{100}$ and it seeds two generators $g_1$ and $g_2$ where each is a 2-layer, feed-forward, fully connected neural network. Both the observed variable and the synthesized one belong to the same space $\underline{x},\hat{\underline{x}} \in \mathbb{R}^{784}$ and both are used to train the adversarial discriminator\footnote{A single binary discriminator for $g_1$ petted against $g_2$.} $h$. We use a single supplementary discriminator $h_1$ whose input is a batch of the synthesized images $\hat{\underline{x}}$. The two discriminators $h$ and $h_1$ are identically structured and have exactly the inverse architecture of the generators (except for the output layer). The output layer of each model uses logistic sigmoidal activation whereas its hidden layers use rectified linear units \cite{Nair_2010_ReLU_RBM}. The whole model is 4-layer deep in its longest path and comprises 0.8M trainable parameters. Finally, all models are trained using the Adam optimizer \cite{Kingma_2014_ADAM} with a learning rate of $10^{-3}$ and each sub-model gains a single parameter-update step in each iteration.

We highlight that instead of adhering to the aforementioned theoretical analysis in which training alternates between gradient-descent and -ascent steps, we rather flip the desired output $y\leftarrow 1-y$ and use only gradient-descent. This technique has been proposed in several previous works \cite{Goodfellow_2016_NIPSTutorial, Mao_2017_LSGAN} with conceivable reasoning\footnote{More recent studies proved that such an approach indeed eliminates the vanishing gradient problem but rather introduces critical sources of instability in the training process \cite{Arjovsky_2017_PrincipledMethods}.} but rather violates the presented analytical study of the architecture since each sub-model retains a different objective function.

\begin{figure}[t]
	\begin{tikzpicture}[every node/.style={inner sep=0,outer sep=0}]
	\node (a) {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS01_DIS0_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS17_DIS0_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS24_DIS0_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS08_DIS1_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS26_DIS1_EPOCH0002999_019.png}};
	\end{tikzpicture}
	\caption{Two-digit experiments on the proposed architecture using the MNIST dataset. The top-most row of each experiment shows
		a sample of the real images whereas the two lower rows depict synthesized samples from each generator.}
	\label{fig:two-digit-success}
	\vspace{-2.5mm}
\end{figure}
\begin{figure}[!ht]
	\begin{tikzpicture}[every node/.style={inner sep=0,outer sep=0}]
	\node (a) {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS04_DIS1_EPOCH0002999_019.png}};
	\node (a) [below = 2mm of a.south, anchor=north] {\includegraphics[width=\linewidth, totalheight=0.13\linewidth]{./figures/DIGITS19_DIS0_EPOCH0004999_019.png}};
	\end{tikzpicture}
	\caption{Examples of failure cases.}
	\label{fig:two-digit-failure}
	\vspace{-2.5mm}
\end{figure}

\subsection{Results and discussion}
\label{subsec:dicussion}

Fig. \ref{fig:two-digit-success} shows the output of each generator in various experiments (each with a distinct two-digit combination). As observed, the generators produce samples of moderate to high perceptual quality but each one synthesizes a different digit from the given combination. On the other hand, few failure cases were observed and are shown in Fig. \ref{fig:two-digit-failure}. In these experiments, we still observe that the generators tend to produce, to some extent, different representations of the same digit rather than distinct digits.

Training adversarial networks, in general, is known to be unstable \cite{Arjovsky_2017_PrincipledMethods}, and the alternating gradient-based parameter updates (rather than training to completion) between the two sub-models in the original model is one of the sources for their instability \cite{Arjovsky_2017_PrincipledMethods}. In the proposed architecture, these alternating parameter updates are extended to a larger number of sub-models and, as a result, expected to amplify this instable training behavior. However, we observed in our experiments that the generators maintain distinct manifolds even in the early training steps and only collapse to the same digit when over-trained.

Furthermore, the objective of the supplementary discriminators is to provide a simple means for exploiting the powerful back-propagation algorithm in pushing the generators' distributions apart from each other. This seems to be valid as long as these models are weak classifiers. If they are provided with enough training capacity, they begin to deliver their discrimination task all by themselves and their driving force on the generators' distributions disappears. As a results, the generators tend to collapse to the same mode which is highly-rewarded by the adversarial discriminator rather than adhering to distinct classes. In fact, the precise amount of training, even for traditional GAN models, is a challenging hyper-parameter to optimize for reasons explained in \cite{Arjovsky_2017_PrincipledMethods}.

\section{Introduction}
\label{sec:intro}

These guidelines include complete descriptions of the fonts, spacing, and
related information for producing your proceedings manuscripts. Please follow
them and if you have any questions, direct them to Conference Management
Services, Inc.: Phone +1-979-846-6800 or email
to \\\texttt{papers@2018.ieeeicassp.org}.

\section{Formatting your paper}
\label{sec:format}

All printed material, including text, illustrations, and charts, must be kept
within a print area of 7 inches (178 mm) wide by 9 inches (229 mm) high. Do
not write or print anything outside the print area. The top margin must be 1
inch (25 mm), except for the title page, and the left margin must be 0.75 inch
(19 mm).  All {\it text} must be in a two-column format. Columns are to be 3.39
inches (86 mm) wide, with a 0.24 inch (6 mm) space between them. Text must be
fully justified.

\section{PAGE TITLE SECTION}
\label{sec:pagestyle}

The paper title (on the first page) should begin 1.38 inches (35 mm) from the
top edge of the page, centered, completely capitalized, and in Times 14-point,
boldface type.  The authors' name(s) and affiliation(s) appear below the title
in capital and lower case letters.  Papers with multiple authors and
affiliations may require two or more lines for this information. Please note
that papers should not be submitted blind; include the authors' names on the
PDF.

\section{TYPE-STYLE AND FONTS}
\label{sec:typestyle}

To achieve the best rendering both in printed proceedings and electronic proceedings, we
strongly encourage you to use Times-Roman font.  In addition, this will give
the proceedings a more uniform look.  Use a font that is no smaller than nine
point type throughout the paper, including figure captions.

In nine point type font, capital letters are 2 mm high.  {\bf If you use the
smallest point size, there should be no more than 3.2 lines/cm (8 lines/inch)
vertically.}  This is a minimum spacing; 2.75 lines/cm (7 lines/inch) will make
the paper much more readable.  Larger type sizes require correspondingly larger
vertical spacing.  Please do not double-space your paper.  TrueType or
Postscript Type 1 fonts are preferred.

The first paragraph in each section should not be indented, but all the
following paragraphs within the section should be indented as these paragraphs
demonstrate.

\section{MAJOR HEADINGS}
\label{sec:majhead}

Major headings, for example, "1. Introduction", should appear in all capital
letters, bold face if possible, centered in the column, with one blank line
before, and one blank line after. Use a period (".") after the heading number,
not a colon.

\subsection{Subheadings}
\label{ssec:subhead}

Subheadings should appear in lower case (initial word capitalized) in
boldface.  They should start at the left margin on a separate line.

\subsubsection{Sub-subheadings}
\label{sssec:subsubhead}

Sub-subheadings, as in this paragraph, are discouraged. However, if you
must use them, they should appear in lower case (initial word
capitalized) and start at the left margin on a separate line, with paragraph
text beginning on the following line.  They should be in italics.

\section{PRINTING YOUR PAPER}
\label{sec:print}

Print your properly formatted text on high-quality, 8.5 x 11-inch white printer
paper. A4 paper is also acceptable, but please leave the extra 0.5 inch (12 mm)
empty at the BOTTOM of the page and follow the top and left margins as
specified.  If the last page of your paper is only partially filled, arrange
the columns so that they are evenly balanced if possible, rather than having
one long column.

In LaTeX, to start a new column (but not a new page) and help balance the
last-page column lengths, you can use the command ``$\backslash$pagebreak'' as
demonstrated on this page (see the LaTeX source below).

\section{PAGE NUMBERING}
\label{sec:page}

Please do {\bf not} paginate your paper.  Page numbers, session numbers, and
conference identification will be inserted when the paper is included in the
proceedings.

\section{ILLUSTRATIONS, GRAPHS, AND PHOTOGRAPHS}
\label{sec:illust}

Illustrations must appear within the designated margins.  They may span the two
columns.  If possible, position illustrations at the top of columns, rather
than in the middle or at the bottom.  Caption and number every illustration.
All halftone illustrations must be clear black and white prints.  Colors may be
used, but they should be selected so as to be readable when printed on a
black-only printer.

Since there are many ways, often incompatible, of including images (e.g., with
experimental results) in a LaTeX document, below is an example of how to do
this \cite{Lamp86}.

\section{FOOTNOTES}
\label{sec:foot}

Use footnotes sparingly (or not at all!) and place them at the bottom of the
column on the page on which they are referenced. Use Times 9-point type,
single-spaced. To help your readers, avoid using footnotes altogether and
include necessary peripheral observations in the text (within parentheses, if
you prefer, as in this sentence).

\begin{figure}[htb]

\begin{minipage}[b]{1.0\linewidth}
  \centering
  \centerline{\includegraphics[width=8.5cm]{./figures/image1}}
  \centerline{(a) Result 1}\medskip
\end{minipage}
\begin{minipage}[b]{.48\linewidth}
  \centering
  \centerline{\includegraphics[width=4.0cm]{./figures/image3}}
  \centerline{(b) Results 3}\medskip
\end{minipage}
\hfill
\begin{minipage}[b]{0.48\linewidth}
  \centering
  \centerline{\includegraphics[width=4.0cm]{./figures/image4}}
  \centerline{(c) Result 4}\medskip
\end{minipage}
\caption{Example of placing a figure with experimental results.}
\label{fig:res}
\end{figure}

\section{COPYRIGHT FORMS}
\label{sec:copyright}

You must submit your fully completed, signed IEEE electronic copyright release
form when you submit your paper. We {\bf must} have this form before your paper
can be published in the proceedings.

\section{RELATION TO PRIOR WORK}
\label{sec:prior}

The text of the paper should contain discussions on how the paper's
contributions are related to prior work in the field. It is important
to put new work in  context, to give credit to foundational work, and
to provide details associated with the previous work that have appeared
in the literature. This discussion may be a separate, numbered section
or it may appear elsewhere in the body of the manuscript, but it must
be present.

You should differentiate what is new and how your work expands on
or takes a different path from the prior studies. An example might
read something to the effect: "The work presented here has focused
on the formulation of the ABC algorithm, which takes advantage of
non-uniform time-frequency domain analysis of data. The work by
Smith and Cohen \cite{Lamp86} considers only fixed time-domain analysis and
the work by Jones et al \cite{C2} takes a different approach based on
fixed frequency partitioning. While the present study is related
to recent approaches in time-frequency analysis [3-5], it capitalizes
on a new feature space, which was not considered in these earlier
studies."

\section{REFERENCES}
\label{sec:refs}

List and number all bibliographical references at the end of the
paper. The references can be numbered in alphabetic order or in
order of appearance in the document. When referring to them in
the text, type the corresponding reference number in square
brackets as shown at the end of this sentence \cite{C2}. An
additional final page (the fifth page, in most cases) is
allowed, but must contain only references to the prior
literature.
\section{Model Architecture}
\label{sec:model-architecture}

Let $\underline{x} \in \mathcal{X}$ be the observed data of a continuous random vector $\underline{x}$ from the space $\mathcal{X}$ of observations. The notation $\underline{x} \sim p_{\underline{x}}$ is used to denote the probability density function (pdf) $p_{\underline{x}}$ of $\underline{x}$ of the real data distribution $\mathbb{P}_{\underline{x}}$.

The proposed architecture is depicted in Fig. \ref{fig:model-architecture} and consists of $K$ generative models (generators) $\{g_k\}_{k=1}^{K}$, each of which maps a latent variable $\underline{z}_k\sim p_{\underline{z}}$ to a synthesized sample $\smash{\hat{\underline{x}}_k = g_k(\underline{z}_k)}\sim p_k$ and induces a continuous\footnote{For simplicity, and for illustrative purposes, we shall assume that all distributions are continuous with non-disjoint support.} distribution $\mathbb{P}_k$. All synthesized samples $\hat{\underline{x}}_k$ belong to the same space $\mathcal{X}$. Let $p_{\underline{x}}$ be the density function of the real data distribution $\smash{\mathbb{P}}_{\underline{x}}$. Furthermore, let $\hat{\underline{x}}$ be a random vector equi-probably sampled from $\{g_k\}_{k=1}^{K}$. This results in a mixture pdf $p_{\hat{\underline{x}}}$ defined as
\begin{equation}
p_{\hat{\underline{x}}}(\underline{x}) =\; \frac{1}{K} \sum_{k=1}^K p_k(\underline{x})
\label{eq:mixture-model}
\end{equation}%
We further introduce $K$ binary logistic models (discriminators) $\{h_k\}_{k=1}^{K}$ each of which maps samples from the input data space $\hat{\underline{x}}\sim\mathcal{X}$ to a probability $h_k: \mathcal{X} \rightarrow [0, 1]$ where $h_k(\hat{\underline{x}})$ resembles the probability that $\hat{\underline{x}}$ is sampled from the $k^\text{th}$ model $g_k$ as opposed to all other components in the mixture. The opposite probability that $\hat{\underline{x}}$ is equi-probably sampled from the remaining set of generators $g_j,\,j\neq k$ is
\begin{equation}
p_{\bar{k}}(\underline{x}) = \frac{1}{K-1} \sum_{\substack{j=1\\j\neq k}}^{K} p_j(\underline{x})
\end{equation}
which also gives another formulation for the mixture pdf
\begin{equation}
p_{\hat{\underline{x}}}(\underline{x}) =\; \frac{1}{K} \sum_{k=1}^K p_{\bar{k}}(\underline{x})
\label{eq:complement-mixture-model}
\end{equation}
The loss function of each of the $k^\text{th}$ discriminator becomes
{\small
\begin{align*}
\mathcal{L}_{h_k}
& =                                    \mathbb{E}_{\underline{z} \sim p_{\underline{z}}         }             \log\left(     h_k(g_k(\underline{z})) \right) +
    \sum_{\substack{j=1\\j\neq k}}^{K} \mathbb{E}_{\underline{z} \sim p_{\underline{z}}         }             \log\left( 1 - h_k(g_j(\underline{z})) \right) \\
& =                                    \mathbb{E}_{\underline{x} \sim p_k         }             \log\left(     h_k(    \underline{x})  \right) +
    \sum_{\substack{j=1\\j\neq k}}^{K} \mathbb{E}_{\underline{x} \sim p_j         }             \log\left( 1 - h_k(    \underline{x})  \right) \\
& =                                    \mathbb{E}_{\underline{x} \sim p_k         }             \log\left(     h_k(    \underline{x})  \right) +
                                       \mathbb{E}_{\underline{x} \sim p_{\bar{k}} } \left[(K-1) \log\left( 1 - h_k(    \underline{x})  \right) \right] \\
\end{align*}}%

Moreover, an additional adversarial discriminator $h$ distinguishes samples of the real mixture distribution $\mathbb{P}_{\underline{x}}$ from those of the generators' mixture $\mathbb{P}_{\hat{\underline{x}}}$. The loss function of this discriminator is similar to the traditional GAN loss
{\small \begin{align*}
	\mathcal{L}_{h}
	& =\mathbb{E}_{\underline{x} \sim p_{     \underline{x}}}\log\left(     h(\underline{x}) \right) +
	\mathbb{E}_{\underline{x} \sim p_{\hat{\underline{x}}}}\log\left( 1 - h(\underline{x}) \right) \\
	\end{align*}}%

Let $V$ be the value function of the \emph{mini-minimax} game defined as
{\small \begin{align}
	\min_{\{g_k\}} \min_{\{h_k\}} \max_{h} V = \mathcal{L}_{h} - \sum_{k=1}^{K} \mathcal{L}_{h_k}

	\end{align}}%
The adversarial GAN game in this case becomes a \emph{minimax} game between two teams. The first team consists of a single player (the adversarial discriminator $h$) whose objective is to \emph{maximize} the value function. The second team consists of all generators $\{g_k\}_{k=1}^{K}$ and supplementary discriminators $\{h_k\}_{k=1}^{K}$ where each player in this team is tasked with \emph{minimizing} the same value function. Each model optimizes the value function with respect to its own parameters (assuming all other models are fixed).

In the non-parametric limit, and assuming each model can train to completion, the resulting discriminators reach their global optimum at (proof can be found in \cite{Goodfellow_2014_GAN})
{\small \begin{equation*}
h_k^\star(\underline{x}) =\, \frac{p_k(\underline{x})}{p_k(\underline{x}) + (K-1) \cdot p_{\bar{k}}(\underline{x})} =\, \frac{p_k(\underline{x})}{K \cdot p_{\hat{\underline{x}}}(\underline{x})}
\end{equation*}}%
and
{\small \begin{equation*}
h^\star(\underline{x})   =\, \frac{p_{{\underline{x}}}(\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})\; +\; p_{{\underline{x}}}(\underline{x})}
\end{equation*}}%
Substituting these optimal values in the value function $V$ results in
{\small \begin{alignat*}{3}
V && =\; &&&                       \mathbb{E}_{\underline{x} \sim p_{\underline{x}}      }        \log\left(     h^\star  (\underline{x}) \right)
           +                       \mathbb{E}_{\underline{x} \sim p_{\hat{\underline{x}}}}        \log\left( 1 - h^\star  (\underline{x}) \right)  \\
  && -\; &&& \sum_{k=1}^{K} \left( \mathbb{E}_{\underline{x} \sim p_k                    }        \log\left(     h_k^\star(\underline{x}) \right) +
                                   \mathbb{E}_{\underline{x} \sim p_{\bar{k}}            }        \log\left( 1 - h_k^\star(\underline{x}) \right) \right) \\
  && =\; &&&                       \mathbb{E}_{\underline{x} \sim p_{\underline{x}}      }        \log\left(\frac{p_{{\underline{x}}}(\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})\; +\; p_{{\underline{x}}}(\underline{x})}     \right) \\
  && +\; &&&                       \mathbb{E}_{\underline{x} \sim p_{\hat{\underline{x}}}}        \log\left(\frac{p_{\hat{\underline{x}}}(\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})\; +\; p_{{\underline{x}}}(\underline{x})} \right) \\
  && -\; &&& \sum_{k=1}^{K}        \mathbb{E}_{\underline{x} \sim p_k                    }  \left[\log\left(\frac{p_k                    (\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})}  \right) + \log\left(\frac{1}{K}\right)                      \right] \\
  && -\; &&& \sum_{k=1}^{K}        \mathbb{E}_{\underline{x} \sim p_{\bar{k}}            }  \left[\log\left(\frac{p_{\bar{k}            } (\underline{x})}{p_{\hat{\underline{x}}}(\underline{x})} \right) + \log\left(\frac{K-1}{K}\right)                    \right] \\
\end{alignat*}\vspace{-10mm}}%

All terms in the previous equation resemble different, non-symmetric Kullback-Leibler (KL) divergences resulting in
{\small \begin{alignat}{3}
V && =\; &&&                \text{D}_\text{KL} \left( \mathbb{P}_{\underline{x}}        \;\|\;  \mathbb{P}_{\hat{\underline{x}}} \texttt{+}\; \mathbb{P}_{{\underline{x}}}    \right) +
                            \text{D}_\text{KL} \left( \mathbb{P}_{\hat{\underline{x}}}  \;\|\;  \mathbb{P}_{\hat{\underline{x}}} \texttt{+}\; \mathbb{P}_{{\underline{x}}}    \right) \nonumber \\
  && -\; &&& \sum_{k=1}^{K} \text{D}_\text{KL} \left( \mathbb{P}_k                      \;\|\;  \mathbb{P}_{\hat{\underline{x}}}                                         \right) -
             \sum_{k=1}^{K} \text{D}_\text{KL} \left( \mathbb{P}_{\bar{k}}              \;\|\;  \mathbb{P}_{\hat{\underline{x}}}                                         \right) \nonumber \\
  && -\; &&& K\cdot\log\left(\frac{K-1}{K^2}\right)
\label{eq:generator-value-function-kl}
\end{alignat}}%
It can be further simplified to symmetric Jensen-Shannon (JS) divergences
{\small \begin{alignat}{3}
V  && =\; &&& 2\cdot\text{D}_\text{JS} \left( \mathbb{P}_{\hat{\underline{x}}},  \mathbb{P}_{{\underline{x}}}   \right) \nonumber - K\cdot \text{D}_\text{JS} \left( \mathbb{P}_{{1}}, \dots, \mathbb{P}_{{K}}  \right) \nonumber \\
   && -\; &&&  K\cdot\text{D}_\text{JS} \left( \mathbb{P}_{\bar{1}}, \dots, \mathbb{P}_{\bar{K}} \right) - K\cdot\log\frac{K-1}{K^2} - \log 4
\label{eq:generator-value-function}
\end{alignat}}%
where the $\pi$-generalized JS divergence for a finite mixture of distributions is defined as \cite{Lin_1991_ShannonDivergences}
{\small \begin{equation}
    \text{D}_\text{JS}^{\pi_1, \dots, \pi_K}(\mathbb{P}_{1}, \dots, \mathbb{P}_K) =\; \sum_{k=1}^{K}\pi_k\cdot\text{D}_\text{KL}\left(\mathbb{P}_k\;\Big\|\;\sum_{j=1}^{K}\pi_j\cdot\mathbb{P}_j\right)
    \label{eq:jensen-shannon-mixture}
\end{equation}}%
where $\pi_1,\dots,\pi_K$ are the distributions' weights and are assumed in this work to be $1/K$. Eq. \ref{eq:jensen-shannon-mixture} is applicable to the two distributions $\mathbb{P}_{{\underline{x}}}$ and $\mathbb{P}_{\hat{\underline{x}}}$ by defining an average distribution $\mathbb{P}_A = (\mathbb{P}_{\hat{\underline{x}}} + \mathbb{P}_{{\underline{x}}}) / 2$. It is also applicable within the mixture model $\mathbb{P}_{\hat{\underline{x}}}$ using Eq. \ref{eq:mixture-model} and \ref{eq:complement-mixture-model}.
Minimizing Eq. \ref{eq:generator-value-function} is equivalent to minimizing the \emph{difference} between the real data distribution and the estimated one while maximizing the dissimilarity between the components of the latter. This, in turn, induces each generator in the model to maintain distinct observational claims of the real data distribution.

\begin{figure}[!t]
	\input{./figures/model-architecture.tex}
	\vspace{-5.0mm}
\end{figure}

Eq. \ref{eq:generator-value-function-kl} represents a very interesting objective function. The first two terms resemble the symmetric JS divergence between the real distribution and the estimated mixture which is non-negative and retains its minimum when $\mathbb{P}_{\underline{x}} = \mathbb{P}_{\hat{\underline{x}}}$. This term is the common measure in traditional GAN models \cite{Goodfellow_2014_GAN}. The third KL-divergence prevents any of the component models $g_k$ from prevalence (representing the whole mixture all by itself), whereas the last term prevents the same component from vanishing.

In practice, each model (either a discriminator or a generator) is a parametric Multi-Layer Perceptron (MLP). This, of course, limits the family of density functions representable by each generator. Nevertheless, each one retains its own parameters, and as a result, the estimated distribution represents a heterogeneous mixture model. Training is straightforward and, similar to traditional GANs, consists of alternating gradient-based parameter updates. Gradient-ascent is utilized by the adversarial discriminator while the supplementary discriminators and the generators use gradient-descent.

redpajama_set_name: RedPajamaArXiv


text:
Home › Archive for Volume 12, Issue 2, 2018
From airports to airport territories: expansions, potentials, conflicts
Author: Yves Boquet
Airports, as nodes in world transport networks, have been studied mostly in their role as hubs in the spatial strategies of airlines. This paper examines the role of airports as engines of growth in their local surroundings and the complexity of the spatial system of the airport at the local and regional scale. It studies how to ensure an efficient movement of planes of the airside and an efficient access point to airports on the landside while taking into account the interests of people living next to airports, who are heavily impacted the noise generated by aircraft, as well as considering where should new airports be built.
Keywords: accessibility, airport, congestion, local economy, NIMBY, noise
The reproductive behaviour of the female population in the only Roma governed community in Europe
Author: Biljana Apostolovska Toshevska, Mirjanka Madjevikj, Marija Ljakoska and Petar Sokoloski
This paper puts a special accent on the Roma female reproductive population and its behaviour concerning the formation and size of the family. A survey was conducted in the area of Shuto Orizari, the only Roma governed community in Europe, on respondents from the female reproductive population divided into three age groups. For the analysis of the variance of the means of the three chosen groups were used the standard statistical tests and the one-way ANOVA test was used for testing the eight hypotheses for the equality of the means of the three independent specimens. A chi-squared test of independence between the two variables was made on the whole sample. The research and the gained results about the factors that determine the demographic behaviour of the Roma female reproductive population are expected to find a way to improve the overall situation and to use it in creating demographic policies.
Keywords: children, demographic policies, reproduction, Roma female population, Shuto Orizari
Housing and resettlement of Jakarta's urban poor: case study of Kampung Pulo's slum revitalisation in Jakarta, Indonesia
Author: Astrid Meilasari-Sugiana, Deffi Ayu Puspito Sari and Rini Anggraini
This paper analyses the interface between policy and practice in housing Jakarta's urban poor and revitalising the Kampung Pulo neighbourhood along the Ciliwung riverbank. The paper discusses issues faced by policymakers in securing land and housing for Jakarta's poor, in relocating the majority of Kampung Pulo's residents into housing flats in Central Jakarta and in revitalising the remaining slum areas into semi-permanent settlements with basic facilities. The paper combines top-down policy analysis and bottom-up ethno-methodology to illustrate best practices and contribute to the body of knowledge relating to sustainable urban governance.
Keywords: collective action, landscape governance, revitalisation, urban environmental management
Pedestrians' crossing behaviour in south-western Nigeria
Author: Moses Olaniran Olawole and Adewale Mukhtar Olayiwola
This study examined the pedestrians' safe and unsafe behaviour before and during street crossing in three cities in South Western Nigeria. 1,214 pedestrians were observed at crossing sites using direct observation approach. The data were analysed using descriptive statistics, chi-square, Analysis of Variance (ANOVA) and logistic regression. In all, 730 (60.1%) of the observed pedestrians were male, and 484 (39.9%) were female. The analysis of the data showed that safe crossing practices were generally high, as over 50% of pedestrians at different stages of the crossing process were categorized as exhibiting safe observation behaviour. In terms of crossing tempo, Chi-square test revealed significant difference between male (65.1%) than female (56.0%) pedestrians (χ2=10.12, d=1, p=0.000). While ANOVA showed significant variation in the risky crossing behaviour among the cities (F (2, 1211) =71.93, P< 0.01). Logistic regression analysis showed that adult pedestrians are 0.79 times more likely to exhibit safe crossing behaviour than younger pedestrians and pedestrians are 0.42 times more likely to exhibit safe crossing when vehicles are moving on one side of the road than when there is no vehicular movement. The study suggests measures to enhance safe street crossing behaviour in the sampled cities and other cities in Nigeria.
Keywords: crossing behaviour, head movements, Nigeria, pedestrians, safety, street
Assessment of the religious-tourism potential in Romania
Author: Giușcă Mădălina-Cristiana, Aurel Gheorghilas and Liliana Dumitrache
This paper aims to provide a model for the assessment of the religious-tourism potential in Romania. The religious practice has been in a continuous change over the past 20 years, making the pilgrimage phenomenon to have an upward trend. In the same time, many religious edifices are becoming more attractive to visitors, as they are included in package tours, and visited by large numbers of tourists. In Romania, there are many religious places visited both, by tourists or pilgrims; it is difficult to assess their numbers and also to distinguish between religious practices and touristic visits as no studies have been conducted in this direction. They are centres of religious (spiritual) power, in which people regularly spend money and time while around them it has been developed a real infrastructure, from roads or construction of facilities to more complex services that include accommodation, stores or commercial stalls, selling souvenirs or items more or less related to religious practice. Therefore, the objectives of this paper aimed at evaluating the tourist valences of religious edifices in Romania as well as the degree of their valorisation; in order to meet these objectives, three dimensions were considered and assessed: the historical-cultural dimension, the religious dimension, and the economic dimension. Many relevant elements for these dimensions have been quantified, giving them scores based on unitary criteria. The results help to see better the differentiation/association between the tourist value and the religious value of religious edifices in Romania, highlighting particularities at the national level, which will be the subject of further analysis.
Keywords: monasteries, pilgrimage destinations, Religion, religious tourism potential, Romania

redpajama_set_name: RedPajamaCommonCrawl


Costa de' Este is a spectacular four-bedroom, four and a half bath beachfront villa situated on the east coast of St. Croix with straight on Buck Island views. The sandy beachfront is just steps from the sunny veranda and swimming pool. The villa combines the splendors of historic St. Croix by incorporating a Danish inspired Sugar Mill into the architectural design with high-end contemporary comforts. An expansive gallery frames the sun drenched terrace and swimming pool with poolside bar overlooking the blues of the Caribbean Sea. Step onto the white-sand beach to take a dip into the sea or enjoy a sunset stroll while watching the sailboats glide past on their way back from Buck Island. Gather for alfresco feasts under the shade of the gallery or in the screened in dining area with a custom built wet bar topped with granite located right off the kitchen.
The great room includes a gourmet kitchen with wine cooler, dining area and generous living space with soaring ceilings that open out to the outdoors creating a seamless transition ideal for entertainment and relaxation. Let the breeze wash over you as you sip wine on the gallery or cook a delicious meal in the beautiful kitchen. The finishes throughout are exquisite, including Mahogany wood accents containing the winding spiral staircase, Travertine, marble and coral stone flooring and gorgeous kitchen granite countertops that compliment the mahogany cabinetry.
A spacious loft floats above the great room located at the top of the spiral, mahogany staircase. The upper-level loft, looking out to endless views of the sea and down to the living room, is currently designed as a cozy, TV room and classy office. The loft wraps around leading to one of the master bedroom suites and opens to a balcony that looks out over the pool and Buck Island.
The entrance to the upper-level master suite is quite grand with its arched mahogany doorway complete with intricately etched glass panels. The en suite bath is outfitted with beautiful finishes, a curved shower with natural light, double sinks and a beautiful Caribbean inspired mural graces the bathroom ceiling.
The lower-level master suite is located on the east side of the home with a private entrance off of the gallery and a grand bath-en suite. The two guest bedrooms with baths en suite, are situated on the opposite end of the home on the west side, can be enter from the covered gallery. All three have beautiful sea views and easy access to the gallery, veranda and swimming pool. One of the guest bedrooms opens directly to the gallery and pool veranda.
Additional features include a three-car garage, half bath off of great room, laundry room, impact glass windows and doors with wood Caribbean shutters, two cistern chambers, each 20,000 gallons, reverse osmosis system, two wells (one on the property and one on the adjacent property parcels) and 25 KW automatic, diesel generator. The property includes plots 5K and 5L (see documents) for added privacy between you and the other homes.
Beyond the private paradise of Costa de' Este, you're within easy reach to fine dining at Duggan's Reef and many other options in downtown Christiansted, hiking to gorgeous snorkeling spots, catching a charter from The St. Croix Yacht Club or Green Cay Marina or enjoying luxury, resort amenities at The Buccaneer including spa treatments, world-class golf or tennis clinics with tennis pros.


text:
Country Intelligence
Offsets / IC
United States of America(3,325/week)
Germany(1,092/week)
France(1,054/week)
Poland(677/week)
Spain(345/week)
View all (9,028/week)
industrie(91/week)
Aviation(77/week)
aéronautique(37/week)
Countries(58/week)
Jun 19, 2019 SOURCE: BusinessWire
Chisholm Oil and Gas, LLC and Gastar Exploration LLC Announce Strategic Combination to Create the Leading STACK E&P Company
Chisholm Oil and Gas, LLC ("Chisholm") of Tulsa, Oklahoma and Gastar Exploration LLC ("Gastar") of Houston, Texas announced today that they have entered into definitive agreements to effect a merger between the two companies. The combined company, which will operate under the name Chisholm Oil and Gas and be headquartered in Tulsa, will have net production of approximately 20,000 Boe per day and a combined acreage position of approximately 165,000 net acres, primarily in Kingfisher County, Oklahoma.
Chisholm is currently running three rigs and has a dedicated frac crew. The parties believe that the Gastar assets will be highly synergistic to their upstream operations as well as to Chisholm's ownership stake in Great Salt Plains Midstream Holdings, LLC and its wholly-owned salt water disposal subsidiary, Cottonmouth SWD, LLC.
The transaction is expected to close in the 3rd quarter of 2019, subject to satisfaction of customary closing conditions.
Chisholm was formed in 2017 with backing from funds managed by certain affiliates of Apollo Global Management, LLC (NYSE: APO) and management. Gastar is owned through private equity funds affiliated with Ares Management Corporation (NYSE: ARES).
Chisholm was advised by Citigroup Inc., as financial advisor, and Vinson & Elkins LLP and Paul, Weiss, Rifkind, Wharton & Garrison, as legal advisors. Evercore and Tudor Pickering Holt & Co. provided financial advisory services to Gastar in connection with the transaction, and Kirkland & Ellis served as legal advisor to Gastar.
About Chisholm Oil and Gas, LLC
Chisholm is a Tulsa, Oklahoma-based exploration and production company focused on acquiring and developing oil and gas assets throughout the STACK play in Blaine, Canadian, Kingfisher, Logan and Major Counties, Oklahoma.
About Gastar Exploration LLC
Gastar is engaged in the exploration, development and production of oil, condensate, natural gas and natural gas liquids in the United States. Specifically, Gastar is focused on the exploration and development of its Mid-Continent assets in the STACK Play in Oklahoma, which includes the Hunton Limestone, Meramec Shale/Mississippi Lime, Woodford Shale, and Osage and Oswego formations.
About Apollo Global Management, LLC
Apollo is a leading global alternative investment manager with offices in New York, Los Angeles, San Diego, Houston, Bethesda, London, Frankfurt, Madrid, Luxembourg, Mumbai, Delhi, Singapore, Hong Kong, Shanghai and Tokyo. Apollo had assets under management of approximately $303 billion as of March 31, 2019 in private equity, credit and real assets funds invested across a core group of nine industries where Apollo has considerable knowledge and resources. For more information about Apollo, please visit www.apollo.com.
About Ares Management Corporation
Ares Management Corporation is a publicly traded, leading global alternative asset manager with approximately $137 billion of assets under management as of March 31, 2019 and 18 offices in the United States, Europe, Asia and Australia. Since its inception in 1997, Ares has adhered to a disciplined investment philosophy that focuses on delivering strong risk-adjusted investment returns throughout market cycles. Ares believes each of its three distinct but complementary investment groups in Credit, Private Equity and Real Estate is a market leader based on assets under management and investment performance. Ares was built upon the fundamental principle that each group benefits from being part of the greater whole. For more information, visit www.aresmgmt.com.
This Press Release contains forward-looking statements based on Chisholm's current expectations that involve a number of risks and uncertainties. Generally, forward-looking statements do not relate strictly to historical or current facts and may include words such as "believes," "will," "expects," "anticipates," "intends" or similar words or phrases. Forward-looking statements in this press release include, but are not limited to, statements regarding expectations (i) that the acquisition will close on the anticipated closing date or at all, (ii) as to the productivity of the acquired assets, and (iii) with respect to the realization of operating efficiencies and synergies from the combined assets. No forward-looking statement can be guaranteed. Numerous risks, uncertainties and other factors may cause actual results to differ materially from those expressed in any forward-looking statement.
OIL/GAS
GASTAR EXPLORATION LLC
ARES MANAGEMENT
APOLLO GLOBAL MANAGEMENT
Gastar Exploration Announces Appointment of New Independent Directors
Gastar Exploration Announces Second Quarter 2018 Results
Gastar Exploration Announces New Independent Director
Gastar Exploration Announces Third Quarter 2018 Results
Northwoods Energy, a Portfolio Company of Funds Affiliated with Apollo Global Management, Announces Its Acquisition of SM Energy's Powder River Basin Assets for $500 Million
Gastar Exploration Announces Senior Management Change
Gastar Exploration Announces First Quarter 2018 Results
Related Procurements
Provision of catering and laundry services to be undertaken at the CPE (Centre Polyvalent de l'Enfance) crèche of the European Parliament in Luxembourg Kirchberg
The Working Time and Childcare Facilities Unit of the European Parliament manages, on behalf of the European Institutions in Luxembourg, the CPE 2 Crèche located at 3, rue Albert Borschette...
Isolating switches
L'appel d'offre a pour objet la fourniture de sectionneurs à fermeture verticale à installer dans le réseau 220 kV géré par Creos:— 4 (+1) jeux sectionneurs de barres tripolaires 245...
Deadline: Sep 2, 2019
Architectural and related services
Planungswettbewerb für den Bau des neuen SEO-Sitzes (+/- 80 Arbeitsplätze, diverse Sitzungssäle, Kantine usw.).
View all Procurements
GreeceAthens
DEFEA - Defence Exhibition Athens 2020, 22-24 June, Metropolitan Expo, Athens, Greece
Α high-profile international defence exhibition where the leading Greek and international companies will present land,...
PortugalLisbon
Military Airlift and Air-to-Air Refuelling 2019, 3-4 December, Lisbon, Portugal
SMi Group are proud to announce that the 20th annual Military Airlift and Air-to-Air Refuelling conference and...
United KingdomLondon
Maritime Information Warfare 2019, 18-19 November, London, UK
SMi's Maritime Information Warfare conference is returning to London, UK on 18 – 19 November 2019 for its 3rd year,...
LuxSpace Sarl
STERIA PSF Luxembourg SA
HITEC Luxembourg S.A.
to Epicos.com
Please register in order to have access to personalized daily information on world-wide A&D industry developments and business opportunities.
for Epicos.com Newsletter
Sign up for our free newsletters and receive twice a week news and intelligence regarding the Aerospace & Defence Industry, meticulously selected by our editorial team.
© 2019 EPICOS


I blogged yesterday about Learner Profile posters specific to PE that I created for my PE environment. When making these posters, I used pictures of my own students. These posters are great for my own teaching space and I am more than happy to share these posters with you. If you would like a copy, please let me know. Ideally, having pictures of your own students on the posters is best, so what I can do, is send you the template posters with the text but minus the pictures. If you have good photos of your students that match the specific learner profile attribute, you can drag them on to the poster and size them up to fit nicely with the text.
Let me know if you would like either the templates or the originals.
We are a new school and I would love to be able to give your wonderful templates to our PE teacher.
We are also a new school and would love to use your templates in our gym.
I would love a copy of your templates if possible. Thanks!
I would love a copy of your Learner in Action templates. I have enjoyed reading your blog....great ideas! I can't wait to incorporate some of these in my classes this year.


text:
Was the medieval era dark and backwards as modern scholarship asserts?
asked 18 Apr 2022 in category Other History
The "Middle Ages" were neither dark, primitive, uncivilised or backward as Western medievalist scholarship would like us to believe. Rather, it was an era of High Ages for the Islamic civilization. The researcher Philip K. Science writes: "It is clear that the Middle Ages were not really dark in the sense that certain narrow medievalists report. Those historians have erred in their pronouncements on the course of scientific thought in the Middle Ages, showing us only the darkest side of the period. An exaggerated emphasis upon the least progressive elements and exclusive preoccupation with the limit domain of western thought are responsible for this grave injustice. The truth of the matter is that that stretch of history was not as dark as our ignorance of it. Simply because its greatest achievement were made by Easterners is no valid excuse for its deprecation. The unbiased verdict of history decrees that from the second half of the eighth to the end of eleventh century, Arabic was the scientific language of mankind"... (The course of Arab scientific thought, The American Journal of Islamic Social Science, Volume 8 Number 3, December 1991, "footnote on page 412 - 413")
answered 19 Apr 2022 anonymous
The "Middle Ages" were neither dark, primitive, uncivilised or backward as Western medievalist scholarship would like us to believe.
Rather, it was an era of High Ages for the Islamic civilization.
The researcher Philip K. Science writes:
"It is clear that the Middle Ages were not really dark in the sense that certain narrow medievalists report. Those historians have erred in their pronouncements on the course of scientific thought in the Middle Ages, showing us only the darkest side of the period. An exaggerated emphasis upon the least progressive elements and exclusive preoccupation with the limit domain of western thought are responsible for this grave injustice. The truth of the matter is that that stretch of history was not as dark as our ignorance of it. Simply because its greatest achievement were made by Easterners is no valid excuse for its deprecation. The unbiased verdict of history decrees that from the second half of the eighth to the end of eleventh century, Arabic was the scientific language of mankind"... (The course of Arab scientific thought, The American Journal of Islamic Social Science, Volume 8 Number 3, December 1991, "footnote on page 412 - 413")
How did the West rise in power and the Muslim World decline? (2 answers)
Was Winston Churchill a racist? (1 answer)
What would have happened if Muslims had won the Battle of Tours? (1 answer)
Did classical scholarship end soon after Ghazali and the Abbasid era? (1 answer)
If the Qur'an is eloquent, so was Shakespeare, so why is Qur'an divine? (1 answer)
Is Islam in its Dark Ages? (0 answers)
According to many Muslims, literary excellence of the Qur'an proves divine inspiration, so was Shakespeare or Virgil divinely inspired too? (0 answers)


Turn the paper pouch upside down in your hand and let the coin gently fall into your palm where you can conceal it from the audience. 7. For a finale, you can tear the paper pouch in two, where the audience see that the coin appears to have vanished.... Bright Bugz are a super cool trick that makes light jump from one hand to the other! You can even swallow the light then pull it out of your friend's ear! Mystify your friends by learning all the tricks such as making light go through your head.
When I move the mouse over anything on the screen it flickers. When I move it across a hyperlink it flickers from an arrow to a hand. This flickering is so quick and constant that it has become an annoyance and is enough to drive me back to using Safari full time.... Sixteen visual standup coin effects: Invisiblair - The ultimate invisible coin gimmick is explained in its entirety. Invisihop - Use Invisiblair to make a coin jump from hand to hand.
26/04/2013 · A gimmick free way to make a coin fly upwards from one hand to the other. This is called the muscle pass and a very advanced trick. Please subscribe. A gimmick free way to make a coin fly upwards... Learn how to make a rubberband jump from your pinkie and ring fingers to the first and middle fingers of the same hand and then back again. A rubber band mysteriously jumps from your pinkie and ring fingers to the first and middle fingers of the same hand and then back again. All you need is a rubberband. .
This video will show you how to perform a really cool, easy coin trick with only two different coins and your hands. The effect is that you throw one of the coins up in the air, catch it, and reveal the other coin, which the audience has not seen yet.
You hold the coin in your hand and slowly around the cup. Suddenly, in your hands coin has pass through the cup. Classic close-up magic. Crossing the magic. Easy to learn and extraordinary effect. Suddenly, in your hands coin has pass through the cup.
I used my pliers to open the jump ring and slid the coin onto it. (Jessyratfink has a great tutorial on this you can find here .) Before I closed the jump ring I attached it to the chain since the closures on the chain I got were too small to fit through the jump ring after it was closed.