How soon will I receive my planner?
Planner in house processing is typically between 3-5 business days. All planners are sent via USPS Priority mail and take roughly 2-3 business days shipping time, depending on the location it is shipping to (outside of the US takes longer). This time may take longer if there is a holiday or inclement weather.
Yes. If you or friends are purchasing multiple planners together at 1 time and they are shipping to the same address, you can ship up to 2 planners together in the same box for the same low shipping price.
I reside outside of the USA. Are there other shipping options available other than Priority?
How long is the planner dated for?
All planners are 12 month and are dated 01/01/17 – 12/31/17.
How big are the planners?
All planners are 9″x8.5″, including the wire coils. They easily fit within most tote bags and some larger bags and purses.


text:
Choosing Wisely: Texas Emergency Physician is Choosing Wisely
Gerad Troutman, MD, an emergency physician at Lubbock's University Medical Center (UMC), is using Choosing Wisely when symptoms of dehydration occur.
"When I first encountered our specialty's Choosing Wisely list," Dr Troutman said, "my first thought was how I could apply this to pre-hospital medicine?"
Choosing Wisely promotes conversations between physicians and patients based on evidence provided by dozens of national specialty societies, including the American College of Emergency Physicians (ACEP). These specialty societies help physicians "choose wisely" by identifying tests or procedures commonly used in their field whose necessity physicians and patients should question and discuss.
One suggestion from the ACEP list stood out to Dr. Troutman: "Avoid instituting intravenous (IV) fluids before doing a trial of oral rehydration therapy in uncomplicated emergency department cases of mild to moderate dehydration in children."
Dr. Troutman began his research and pulled data from UMC EMS system of any patient age 2-18 who had a complaint of nausea and vomiting and had an IV started in the field.
"I found that many of these patients had normal vital signs and appeared to have benign exams," Dr. Troutman said. "It would have been appropriate to consider oral rehydration in some of these patients."
Dr. Troutman plans to use Choosing Wisely to tweak protocols and encourage UMC's training department to put an emphasis on abnormal vital signs and severe signs or symptoms of dehydration as the only need for an IV in these patients.
Texas College of Emergency Physicians (TCEP) is encouraging its members to use the Choosing Wisely lists. The TCEP board of directors has approved the ACEP Choosing Wisely list to mirror their own suggestions for Texas emergency physicians to consider.
The four other ACEP Choosing Wisely recommendations are:
1.Avoid computed tomography (CT) scans of the head in emergency department patients with minor head injury who are at low risk based on validated decision rules.
2.Avoid placing indwelling urinary catheters in the emergency department for either urine output monitoring in stable patients who can void, or for patient or staff convenience.
3.Don't delay engaging available palliative and hospice care services in the emergency department for patients likely to benefit.
4.Avoid antibiotics and wound cultures in emergency department patients with uncomplicated skin and soft tissue abscesses after successful incision and drainage and with adequate medical follow-up.
The ABIM Foundation awarded TMA and its philanthropic arm, the TMA Foundation, a grant to advance the Choosing Wisely campaign among Texas physicians. Nine Texas county medical societies, six state medical specialty societies, and the Texas Osteopathic Medical Association are participating in the program with TMA. Support for the grant program comes from the Robert Wood Johnson Foundation.
For more information, including links to all of the specialty societies' Choosing Wisely lists, see the TMA website at www.texmed.org/choosingwisely.
Texas Medicine, May 2014

redpajama_set_name: RedPajamaCommonCrawl


Etymology: "Zelkova" comes from "Dzelva", its name in the Caucasus. "carpinifolia" reminds that it has leaves similar to those of charm.
Origin: Caucasus, introduced into Europe in the XVIIIth century (in Paris, in 1782).
Habitat: Zelkova appreciates the sunny zones.
Form (insulated tree): straight crown. Upright branches, as the charm.
Its bark exfoliates like that of the plane tree to reveal an orange layer (cambium).
Its harsh and toothed leaves resemble those of the Elm in smaller (5 cm) and more sharpened, and those of Hornbeam in more oval. Their base is slightly asymmetrical. They turn to yellow-orange at fall.
Fruits are small drupes (nucules), in the axil of leaves.
It is very resistant to cold (zone 3:-40 °).
Height: 20 m tall. Very slender leaves.
Zelkova of China, Zelkova sinensis, with less sharpened leaves.


text:
\section{Introduction}

Variational autoencoders (VAEs) are generative models with the useful feature of learning representations
of input data in their latent space. A VAE comprises of a prior (the probability distribution of the latent space), a decoder and an encoder (also referred to as the approximating posterior or the inference network). There have been efforts devoted to making each of these components more powerful.
The decoder can be made richer by using autoregressive methods such as pixelCNNs, pixelRNNs \citep{oord2016pixel} and MADEs \citep{germain2015made}. However, VAEs tend to ignore the latent code (in the sense described by \citet{yeung2017tackling})  in the presence of powerful decoders \citep{chen2016variational, gulrajani2016pixelvae, goyal2017z}. There are also a myriad of works strengthening the encoder distribution \citep{kingma2016improved, rezendeICML15, salimans2015markov}.
Improving the priors is manifestly appealing, since it directly translates into a more powerful generative model. Moreover, a rich structure in the latent space is one of the main purposes of VAEs.  \citet{chen2016variational} observed that a more powerful autoregressive prior and a simple encoder is commensurate with a powerful  inverse autoregressive approximating posterior and a simple prior.

Boltzmann machines (BMs) are known to represent intractable and multi-modal distributions~\citep{le2008representational}, ideal for priors in VAEs, since they can lead to a more expressive generative model. However, BMs contain discrete variables which are incompatible with the reparameterization trick, required for efficient propagation of gradients through stochastic units. It is desirable to have discrete latent variables in many applications such as semi-supervised learning \citep{kingma2014semi}, semi-supervised generation \citep{maaloe2017semi} and hard attention models \citep{serra2018overcoming, gregor2015draw}, to name a few. Many operations, such as choosing between models or variables are naturally expressed using discrete variables \citep{yeung2017tackling}.

\citet{rolfe2016discrete} proposed the first model to use a BM in the prior of a VAE, i.e., a discrete VAE (dVAE). The main idea is to introduce auxiliary continuous variables (Fig.~\ref{fig:dvae}) for each discrete variable through a ``smoothing distribution''. The discrete variables are marginalized out in the autoencoding term by imposing certain constraints on the form of the relaxing distribution. However, the discrete variables cannot be marginalized out from the remaining term in the objective (the KL term). Their proposed approach relies on properties of the smoothing distribution to evaluate these terms.
In Appendix~\ref{appen:reinforce}, we show that this approach is equivalent to REINFORCE when dealing with some parts of the KL term (\emph{i.e.}, the cross-entropy term).
\citet{vahdat2018dvae++} proposed an improved version, dVAE++, that uses a modified distribution for the smoothing variables, but has the same form for the autoencoding part (see Sec.~\ref{sec:VAE}). The qVAE, \citep{khoshaman2018quantum}, expanded the dVAE to operate with a quantum Boltzmann machine (QBM) prior \citep{Amin:2016qd}.
A major shortcoming of these methods is that they are unable to have multi-sampled (importance-weighted) estimates of the objective function during training, which can improve the performance.

To use the reparameterization trick directly with discrete variables (without marginalization), a continuous and differentiable proxy is required. The Gumbel (reparameterization) trick, independently developed by \citet{jang2016categorical} and \citet{maddison2016concrete}, achieves this by relaxing discrete distributions. However, BMs and in general discrete random Markov fields (MRFs) are incompatible with this method. Relaxation of discrete variables (rather than distributions) for the case of factorial categorical prior (Gumbel-Softmax) was also investigated in both works. It is not obvious whether such relaxation of discrete variables would work with BM priors.

The contributions of this work are as follows:
  we propose the GumBolt, which extends the Gumbel trick to BM and MRF priors and is significantly simpler than previous models that marginalize discrete variables. We show that BMs are compatible with relaxation of discrete variables (rather than distributions) in Gumbel trick. We propose an objective using such relaxation and show that the main limitations of previous models with BM priors can be circumvented; we do not need marginalization of the discrete variables, and can have an importance-weighted objective.
GumBolt considerably outperforms the previous works in a wide series of experiments on permutation invariant MNIST and OMNIGLOT datasets, even without the importance-weighted objective (Sec.~\ref{sec:exps}). Increasing the number of importance weights can further improve the performance. We obtain the state-of-the-art results on these datasets among models with only discrete latent variables.

\section{Background}
\subsection{ Variational autoencoders}
\label{sec:VAE}
Consider a generative model involving observable variables $x$ and latent variables $z$. The joint probability distribution can be decomposed as $p_{\theta}(x, z) = p_{\theta} (z) p_{\theta}(x|z)$. The first and second terms on the right hand side are the prior and decoder distributions, respectively, which are parametrized by $\theta$.
 Calculating the marginal $p_{\theta}(x)$ involves performing intractable, high dimensional sums or integrals.
 Assume an element $x$ of the dataset $\mathcal{X}$ comprising of $N$ independent samples from an unknown underlying distribution is given. VAEs operate by introducing a family of approximating posteriors $q_\phi(z |x)$ and maximize a lower bound (also known as the ELBO), $\ELBO{}$ , on the log-likelihood $\log p_{\theta}(x)$ \citep{kingmaICLR15}:
 \begin{equation}
 \begin{split}
     \log p_{\theta}(x) \ge \ELBO{} &= \underset{q_\phi(z |x)}{\mathbb{E}} \left[\log \frac{p_{\theta}(x, z)}{q_\phi(z |x)}\right]  \\
     &= \underset{q_\phi(z |x)}{\mathbb{E}} \left[\log p_\theta(x | z)\right] - \KLD{q_\phi(z|x)}{p_\theta(z)},
 \end{split}
 \label{eq:ELBO}
 \end{equation}
where the first term on the right-hand side is the autoencoding term and $D_\text{KL}$ is the Kullback-Leibler divergence \citep{bishop_pattern_2011}. In VAEs, the parameters of the distributions (such as the means in the case of Bernoulli variables) are calculated using neural nets. To backpropagate through latent variables $z$, the {reparameterization trick} is used; $z$ is reparametrized as a deterministic function $ f(\phi, x, \rho)$, where the stochasticity of $z$ is relegated to another random variable, $\rho$, from a distribution that does not depend on $\phi$. Note that it is impossible to backpropagate if $z$ is discrete, since $f$ is not differentiable.
\subsection{Gumbel trick}
\label{sec:gumbel}
The non-differentiability of $f$ can be resolved by finding a relaxed proxy for the discrete variables. Assume a binary unit, $z$, with mean $\bar{q}$ and logit $l$; \emph{i.e.},   $p(z=1)=\bar{q}=\sigma(l)$, where $\sigma(l) \equiv \frac{1}{1+\exp(-l)}$ is the sigmoid function. Since $\sigma(l)$ is a monotonic function, we can reparametrize $z$ as $ z=\mathcal{H}(\rho - (1-\bar q)) = \mathcal{H}\left(l + \sigma^{-1}(\rho)\right)$ \citep{maddison2016concrete}, where $\mathcal{H}$ is the Heaviside function, $\rho\sim \mathcal{U}$ with $\mathcal{U}$ being a uniform distribution in the range $[0,1]$, and $\sigma^{-1} (\rho) = \log(\rho)-\log(1-\rho)$ is the inverse sigmoid or logit function. This transformation results in a non-differentiable reparameterization, but can be smoothed when the Heaviside function is replaced by a sigmoid function with a temperature $\tau$, \emph{i.e.}, $\mathcal{H}(\dots) \xrightarrow{}\sigma(\frac{\dots}{\tau})$. Thus, we introduce the continuous proxy \citep{maddison2016concrete}:
\begin{equation}
    f(\phi, x, \rho)= \zeta = \sigma\left({\frac{l(\phi, x) + \sigma^{-1}(\rho)}{\tau}}\right).
    \label{eq:Gumbel}
\end{equation}
The continuous $\zeta$ is differentiable and is equal to the discrete $z$ in the limit $\tau\xrightarrow{}0$.

\subsection{Boltzmann machine}
\label{sec:gen}

Our goal is to use a BM as prior. A BM is a probabilistic energy model described by
\begin{equation}
    p_{\theta}(z)= \frac{e^{{-E_{\theta}(z)}}}{Z_\theta},
    \label{eq:discrete_prior}
\end{equation}
where $E_{\theta}(z)$ is the energy function, and $Z_\theta=\sum_{\{z\}}{e^{{-E_{\theta}(z)}}}$ is the partition function; $z$ is a vector of binary variables. Since finding $p_{\theta}(z)$ is typically intractable, it is common to use sampling techniques to estimate the gradients.
To facilitate MCMC sampling using Gibbs-block technique, the connectivity of latent variables is assumed to be bipartite; \emph{i.e.}, $z$ is decomposed as $[z_1, z_2]$ giving
\begin{equation}
    -E_\theta(z)= a^T z_1 + b^T z_2+ z_2^T W z_1,
    \label{eq:energy}
\end{equation}
where $a$, $b$ and $W$ are the biases (on $z_1$ and $z_2$, respectively) and weights. This bipartite structure is known as the restricted Boltzmann Machine.

\section{Proposed approach}
\begin{figure}[!htbp]
  \centering
  \subfigure[d(q)VAE(++)]{
    \label{fig:dvae}
    \includegraphics[scale=0.35, page=3]{mod_figs.pdf}
  }
  \subfigure[Concrete, Gumbel-Softmax]{
    \label{fig:concrete}
    \includegraphics[scale=0.35, page=2]{mod_figs.pdf}
  }
  \subfigure[GumBolt]{
    \label{fig:gumbolt}
    \includegraphics[scale=0.35, page=1]{mod_figs.pdf}
  }

  \caption[]{
  Schematic of the discussed models with discrete variables in their latent space. The dashed red and solid blue arrows represent the inference network, and the generative model, respectively. (a) dVAE, qVAE \citep{khoshaman2018quantum} and dVAE++ have the same structure. They involve auxiliary continuous variables, $\zeta$, for each discrete variable, $z$, provided by the same conditional probability distribution, $r(\zeta|z)$, in both the generative and approximating posterior networks. (b) Concrete and Gumbel-Softmax apply the Gumbel trick to the discrete variables to obtain the $\zeta$s that appear in both the inference and generative models. (c) GumBolt only involves discrete variables in the generative model, and the relaxed $\zeta$s are used in the inference model during training. Note that during evaluation, the temperature is set to zero, leading to $\zeta=z$.
    }
  \label{fig:models}
\end{figure}

The importance-weighted or multi-sampled objective of a VAE with BM prior can be written as:
 \begin{equation}
 \begin{split}
      \log p_{\theta}(x) \ge \ELBO{k} &=
      \underset{\prod_i q_\phi(z^i |x)}{\mathbb{E}}
      \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{p_{\theta}(z^i)p_{\theta}(x|z^i)}{q_\phi(z^i |x)}\right]  \\
      &=
      \underset{\prod_i q_\phi(z^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(z^i)} p_{\theta} (x|z^i)}{q_\phi(z^i |x)}\right] -\log{Z_{\theta}},
 \end{split}
 \label{eq:ELBO_disc}
 \end{equation}
where $k$ is the number of samples or importance-weights over which the Monte Carlo objective is calculated,  \citep{mnih2016variational}. $z^i$ are independent vectors sampled from $q_\phi(z^i |x)$. Note that we have taken out $Z_\theta$ from the argument of the expectation value, since it is independent of $z$. The partition function is intractable but its derivative can be estimated using sampling:
\begin{equation}
    \nabla_{\theta}\log{Z_{\theta}} = \nabla_{\theta} \log {\sum_{\{z\}}e^{{-E_{\theta}(z)}}} =
     -\frac{{\sum_{\{z\}}\nabla_{\theta}  E_{\theta}(z)e^{{-E_{\theta}(z)}}}}{\sum_{\{z\}}e^{{-E_{\theta}(z)}}}
     =
    -\underset{p_{\theta} (z)}{\mathbb{E}}
    \left[ \nabla_{\theta} E_{\theta}(z) \right].
\label{eq:pf}
\end{equation}

Here, $\sum_{\{z\}}$ involves summing over all possible configurations of the binary vector $z$.
The objective $\ELBO{k}$ cannot be used for training, since it involves non-differentiable discrete variables. This can be resolved by relaxing the distributions:
 \begin{equation}
  \begin{split}
      \log p_{\theta}(x) \ge \Fcont{k} &= \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{p_\theta(\zeta^i) p_{\theta} (x|\zeta^i)}{q_\phi(\zeta^i |x)}\right] \nonumber \\
      &= \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{q_\phi(\zeta^i |x)}\right] -\log{\tilde Z_{\theta}}.
 \label{eq:ELBO_cont_full}
  \end{split}
 \end{equation}
Here, $\zeta^i$ is a continuous variable sampled from  Eq.~\ref{eq:Gumbel}, which is consistent with the Gumbel probability $q_\phi(\zeta^i |x)$ defined in \citep{maddison2016concrete}, and $p_\theta(\zeta) \equiv e^{-E_{\theta}(\zeta)}/\tilde{Z_\theta}$, where $\tilde{Z_\theta} \equiv \int d\zeta e^{-E_{\theta}(\zeta)}$.
The expectation distribution is the joint distribution over independent $z^i$ samples.
Notice that $\log \tilde{Z}_\theta$ is different from $\log Z_\theta$, therefore its derivatives cannot be estimated using discrete samples from a BM, making this method inapplicable for BM priors. The derivatives could be estimated using samples from a continuous distribution, which is very different from the BM distribution.
Analytical calculation of the expectations, suggested for Bernoulli prior by \citet{maddison2016concrete} is also infeasible for BMs, since it requires exhaustively summing over all possible configurations of the binary units.

\subsection{GumBolt probability proxy}
\label{sec:gumbolt}
To replace $\log \tilde{Z}_\theta$ with $\log Z_\theta$, we introduce a proxy probability distribution:
\begin{equation}
  \breve{p}_{\theta}(\zeta) \equiv \frac{e^{{-E_{\theta}(\zeta)}}}{Z_\theta} .
    \label{eq:gumbolt_p}
\end{equation}

Note that $\breve{p}_{\theta}(\zeta)$ is not a true (normalized) probability density function, but $\breve{p}_{\theta}(\zeta) \to {p}_{\theta}(z)$ as $\tau \to 0$.

Now consider the following theorems (see Appendix~\ref{appen:proofgum} for proof):
 \begin{theorem}
\label{theo:optima_cont_vert}
For any polynomial function $E_\theta (z)$ of $n_z$ binary variables $z \in \{0,1\}^{n_z}$, the extrema of the relaxed function $E_\theta(\zeta)$ with $\zeta \in [0,1]^{n_z}$ reside on the vertices of the hypercube, i.e., $\zeta_{\rm extr} \in \{0,1\}^{n_z}$.
\end{theorem}

 \begin{theorem}
\label{theo:gumbel}
For any polynomial function $E_\theta (z)$ of $n_z$ binary variables $z \in \{0,1\}^{n_z}$, the proxy probability $\breve{p}_{\theta}(\zeta) \equiv e^{-E_\theta (\zeta)}/Z_\theta$, with $\zeta \in [0,1]^{n_z}$, is a lower bound to the true probability ${p}_{\theta}(\zeta)\equiv e^{-E_\theta (\zeta)}/\tilde Z_\theta$, \emph{i.e.}, $\breve{p}_{\theta}(\zeta) \le {p}_{\theta}(\zeta)$, where $Z_{\theta} \equiv {\sum_{\{z\}}{e^{{-E_{\theta}(z)}}}}$ and $\tilde{Z}_\theta \equiv \int\limits_{\{\zeta\}} d\zeta e^{-E_{\theta}(\zeta)}$ .
\end{theorem}

 Therefore, according to theorem (2), replacing ${p}_{\theta}(\zeta)$ with $\breve{p}_{\theta}(\zeta)$, we obtain a lower bound on $\Fcont{k}$:
 \begin{equation}
      \F{k} = \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{q_\phi(\zeta^i |x)}\right] -\log{Z_{\theta}} \le \Fcont{k}.
 \label{eq:ELBO_cont}
 \end{equation}

This allows reparameterization trick, while making it possible to use sampling to estimate the gradients. The structure of our model with a BM prior is portrayed in Figure~\ref{fig:gumbolt}, where both continuous and discrete variables are used. Notice that in the limit $\tau \to 0$, $\breve{p}_{\theta}(\zeta^i)$ becomes a probability mass function (pmf) $p_\theta(z^i)$, while $q_\phi(\zeta^i |x)$ remains as a probability density function (pdf). To resolve this inconsistency, we replace $q_\phi(\zeta^i |x)$ with the Bernoulli pmf: $\breve{q}_\phi(\zeta^i |x) = \zeta^i \log \bar q^i + (1-\zeta^i) \log (1-\bar q^i)$, which approaches  $q_\phi(z^i |x)$ when $\tau \to 0$. The training objective
 \begin{equation}
      \Fgumbolt{k} = \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{\breve{q}_\phi(\zeta^i |x)}\right] -\log Z_\theta
 \label{eq:ELBO_cont}
 \end{equation}
becomes $\ELBO{k}$ at $\tau=0$ as desired (see Fig.~\ref{fig:losses} for the relationship among different objectives). This is the analog of the Gumbel-Softmax trick \citep{jang2016categorical} when applied to BMs.
During training, $\tau$ should be kept small for the continuous variables to stay close to the discrete variables, a common practice with Gumbel relaxation \citep{tucker2017rebar}. For evaluation, $\tau$ is set to zero, leading to an unbiased evaluation of the objective function.
For generation, discrete samples from BM are directly fed into the decoder to obtain the probabilities of each input feature.

\subsection{Compatibility of BM with discrete variable relaxation}
\label{sec:stab}
The term in the discrete objective that involves the prior distribution is $\mathbb{E}_{q_\phi(z|x)}\left[ \log p_\theta(z)\right]$. When $z$ is replaced with $\zeta$, there is no guarantee that the parameters that optimize $\mathbb{E}_{q_\phi(\zeta|x)}\left[ \log p_\theta(\zeta)\right]$ would also optimize the discrete version. This happens naturally for Bernoulli distribution used in \citep{jang2016categorical}, since the extrema of the prior term in the objective $\log (p(\zeta)) = \zeta \log (\bar p) + (1- \zeta) \log(1-\bar p)$ occur at the boundaries (\emph{i.e.}, when $\zeta=1$ or $\zeta=0$). This means that throughout the training, the values of $\zeta$ are pushed towards the boundary points, consistent with the discrete objective.

In the case of a BM prior, according to theorem (1) (proved in Appendix~\ref{appen:proofgum}),
the extrema of $\log p_{\theta} (\zeta) \propto -E_{\theta}(z)$ also occur on the boundaries; this shows that having a BM rather than a factorial Bernoulli distribution does not exacerbate the training of GumBolt.

\section{Related works}
\label{sec:related}
Several approaches have been devised to calculate the derivative of the expectation of a function with respect to the parameters of a Bernoulli distribution, $\mathcal{I} \equiv \nabla_\phi \mathbb{E}_{q_\phi(z)}\left[ f(z)\right]$:
\begin{enumerate}
    \item \label{it:anal} Analytical method: for simple functions, \emph{e.g.}, $f(z)=z$, one can analytically calculate the expectation and obtain $\mathcal{I}= \nabla_\phi \mathbb{E}_{q_\phi(z)}\left[z\right] = \nabla_\phi \bar q,$ where $\bar q$ is the mean of the Bernoulli distribution. This is a non-biased estimator with zero variance, but can only be applied to very simple functions. This approach is frequently used in semi-supervised learning \citep{kingma2014semi} by summing over different categories.
    \item Straight-through method: continuous proxies are used in backpropagation to evaluate derivatives, but discrete units are used in forward propagation \citep{bengio2013estimating, raiko2014techniques}.
    \item \label{it:reinf} REINFORCE trick: $\mathcal{I}= \mathbb{E}_{q_\phi(z)}\left[ f(z) \nabla_\phi \log q_\phi(z) \right]$, although it has high variance, which can be reduced by variance reduction techniques \citep{williams1992simple}.
    \item \label{it:reparam} Reparameterization trick: this method, as delineated in Sec.~\ref{sec:VAE}-\ref{sec:gumbel}, is biased except in the limit where the proxies approach the discrete variables.
    \item \label{it:marg} Marginalization: if possible, one can marginalize the discrete variables out from some parts of the loss function \citep{rolfe2016discrete}.
\end{enumerate}

NVIL \citep{mnih2014neural} and its importance-weighted successor VIMCO \citep{mnih2016variational} use (\ref{it:reinf}) with input-dependent signals obtained from neural networks to subtract from a baseline in order to reduce the variance of the estimator. REBAR \citep{tucker2017rebar} and its generalization, RELAX \citep{grathwohl2017backpropagation} use (\ref{it:reinf}) and employ (\ref{it:reparam}) in their control variates obtained using the Gumbel trick.
DARN \citep{gregor2013deep} and MuProp \citep{gu2015muprop} apply the Taylor expansion of the function $f(z)$ to synthesize baselines.
dVAE and dVAE++ (Fig.~\ref{fig:dvae}), which are the only works with BM priors, operate primarily based on (\ref{it:marg}) in their autoencoding term and use a combination of (1-4) for their KL term. In Appendix~\ref{appen:reinforce}, we show that dVAE has elements of REINFORCE in calculating the derivative of the KL term.
Our approach, GumBolt, exploits (\ref{it:reparam}), and does not require marginalizing out the discrete units.

\section{Experiments}
\label{sec:exps}
In order to explore the effectiveness of the GumBolt, we present the results of a wide set of experiments conducted on standard feed-forward structures that have been used to study models with discrete latent variables \citep{maddison2016concrete, tucker2017rebar, vahdat2018dvae++}. At first, we evaluate GumBolt against dVAE and dVAE++ baselines, all in the same framework and structure. We also demonstrate empirically that the GumBolt objective, Eq.~\ref{eq:ELBO_cont}, faithfully follows the  non-differentiable discrete objective throughout the training.
We then note on the relation between our model and other models that involve discrete variables. We also gauge the performance advantage GumBolt obtains from the BM by removing the couplings of the BM and re-evaluating the model.
\subsection{Comparison against dVAE and dVAE++}
\begin{table}[]
\centering
\caption{Test-set log-likelihood of the GumBolt compared against dVAE and dVAE++. $k$ represents the number of samples used to calculate the objective during \emph{training}. Note that dVAE and dVAE++ are only consistent with $k=1$. See the main text for more details.}
\label{tab:gumvsdvae}
\begin{tabular}{@{}lcrrrrr@{}}
\toprule
\rowcolor[HTML]{FFFFFF}
 & \multicolumn{1}{l}{\cellcolor[HTML]{FFFFFF}} & dVAE & dVAE++ & \multicolumn{1}{c}{\cellcolor[HTML]{FFFFFF}}  GumBolt & &  \\ \midrule
\rowcolor[HTML]{FFFFFF}
 & \multicolumn{1}{l}{\cellcolor[HTML]{FFFFFF}} & $k=1$ & $k=1$ & $k=1$ & $k=5$ & $k=20$ \\
 & $-$ & 90.11 & 90.40 & 88.88 & 88.18 & \cellcolor[HTML]{FFFFFF}\textbf{87.45} \\
 & $\sim$ & 85.72 & 85.41 & 84.86 & 84.31 & \cellcolor[HTML]{FFFFFF}\textbf{83.87} \\
\multirow{-2}{*}{\textbf{MNIST}} & $-\,-$ & 85.71 & 87.35 & 85.42 & 84.65 & \cellcolor[HTML]{FFFFFF}\textbf{84.46} \\
 & $\sim \, \sim$ & 84.33 & 84.75 & 83.28 & 83.01 & \cellcolor[HTML]{FFFFFF}\textbf{82.75} \\ \midrule
 & $-$ & 106.83 & 106.01 & 105.00 & 103.99 & \cellcolor[HTML]{FFFFFF}\textbf{103.69} \\
 & $\sim$ & 102.85 & 101.97 & 101.61 & 101.02 & \cellcolor[HTML]{FFFFFF}\textbf{100.68} \\
\multirow{-2}{*}{\textbf{OMNIGLOT}} & \multicolumn{1}{l}{$-\, -$} & 101.98 & 102.62 & 100.62 & 99.38 & \cellcolor[HTML]{FFFFFF}\textbf{99.36} \\
 & $\sim \,\sim$ & 101.75 & 100.70 & 99.82 & 99.32 & \cellcolor[HTML]{FFFFFF}\textbf{98.81} \\ \bottomrule
\end{tabular}
\end{table}

We compare the models on statically binarized MNIST \citep{salakhutdinov2008dbn} and OMNIGLOT datasets \citep{lake2015human} with the usual compartmentalization into the training, validation, and test-sets.
The $4000$-sample estimation of log-likelihood \citep{burda2015importance} of the models are reported in Table~\ref{tab:gumvsdvae}.
The structures used are the same as those of \citep{vahdat2018dvae++}, which were in turn adopted from \citep{tucker2017rebar} and \citep{maddison2016concrete}.
 We performed experiments with dVAE, dVAE++ and GumBolt on the same structure, and set the temperature to zero during evaluation (the results reported in \citep{vahdat2018dvae++} are calculated using non-zero temperatures). The inference network is chosen to be either factorial or have two hierarchies (Fig.~ \ref{fig:gumbolt}). In the case of two hierarchies, we have:
 $$q_{\phi}(z| x)=q_{\phi}(z_1, z_2 |x)= q_{\phi}(z_1 |x) q_{\phi}(z_2 |z_1, x)$$
 , where $z = [z_1, z_2].$

The meaning of the symbols in Table~\ref{tab:gumvsdvae} are as follows: $-$, and $\sim$ represent linear and nonlinear layers in the encoder and decoder neural networks. The number of stochastic layers (hierarchies) in the encoder is equal to the number of symbols. The dimensionality of the latent space is $200$ times the number of symbols; \emph{e.g.}, $\sim \,\sim$ means two stochastic layers (just as in Fig.~\ref{fig:gumbolt}), with $2$  hidden layers (each one containing $200$ deterministic units) in the encoder. The dimensionality of each stochastic layer is equal to $200$ in the encoder network; the generative network is a $200 \times 200$ RBM (a total of $400$ stochastic units), for $\sim \,\sim$ and $- \, -$, whereas, for $-$ and $\sim$, it is a $100 \times 100$ RBM. Note that in the case of $-\,-$, only one layer of $200$ deterministic units is used in each one of the two hierarchies.
The decoder network receives the samples from the RBM and probabilistically maps them into the input space using one or two layers of deterministic units. Since the RBM has bipartite structure, our model has two stochastic layers in the generative model.
The chosen hyper-parameters are as follows: $1M$ iterations of parameter updates using the ADAM algorithm \citep{kingma2014adam}, with the default settings and batch size of $100$ were carried out. The initial learning rate is $3\times 10^{-3}$ and is subsequently reduced by
$0.3$ at $60 \%$, $75\%$, and $95\%$ of the total iterations. KL annealing \citep{sonderby2016ladder} was used via a linear schedule during $30 \%$ of the total iterations. The value of temperature, $\tau$ was set to $\frac{1}{7}$ for all the experiments involving GumBolt, $\frac{1}{5}$ for experiments with dVAE, and $\frac{1}{10}$ and $\frac{1}{8}$ for dVAE++ on the MNIST and OMNIGLOT datasets, respectively; these values were cross-validated from $\{\frac{1}{10}, \frac{1}{9}\dots ,\frac{1}{5} \}$. The GumBolt shows the same average performance for temperatures in the range $\{\frac{1}{9}, \frac{1}{8} ,\frac{1}{7} \}$. The reported results are the averages from performing the experiments $5$ times. The standard deviations in all cases are less than $0.15$; we avoid presenting them individually to keep the table less cluttered.
We used the batch-normalization algorithm \citep{ioffe2015batch} along with $\texttt{tanh}$ nonlinearities. Sampling the RBM was done by performing $200$ steps of Gibbs updates for every mini-batch, in accordance with our baselines,using persistent contrastive divergence (PCD) ~\citep{tieleman2008training}.
We have observed that by having $2$ and $20$ PCD steps, the performance of our best model on MNIST dataset is deteriorated by $0.35$ and $0.12$ nats on average, respectively.
In order to estimate the log-partition function, $\log Z_\theta$, a GPU implementation of parallel tempering algorithm with bridge sampling was used \citep{desjardins2010tempered, bennett1976efficient, shirts2008statistically}, with a set of parameters to ensure the variance in   $\log Z_\theta$ is less than 0.01: $20K$ burn-in steps were followed by $100K$ sweeps,  $20$ times (runs), with a pilot run to determine the inverse temperatures (such that the replica exchange rates are approximately $0.5$).

We underscore several important points regarding Table~\ref{tab:gumvsdvae}.
First, when one sample is used in the training objective ($k=1$), GumBolt outperforms dVAE and dVAE++ in all cases. This can be due to the efficient use of reparameterization trick and the absence of REINFORCE elements in the structure of GumBolt as opposed to dVAE (Appendix~\ref{appen:reinforce}).
 Second, the previous models do not apply when $k>1$. GumBolt allows importance weighted objectives according to Eq.~\ref{eq:ELBO_cont}. We see that in all cases, by adding more samples to the training objective, the performance of the model is enhanced.

Fig.~\ref{fig:tr_steps} depicts the $k=20$ estimation of the GumBolt and discrete objectives on the training and validation sets during training. It can be seen that over-fitting does not happen since all the objectives are improving throughout the training. Also, note that the differentiable GumBolt proxy closely follows the non-differentiable discrete objective. Note that the kinks in the learning curves are caused by our stepwise change of the learning rate and is not an artifact of the model.

\begin{figure}[!htbp]
  \centering
  \subfigure[]{
    \label{fig:losses}
    \includegraphics[scale=.5]{losses_cropped.pdf}
  }
  \subfigure[]{
    \label{fig:tr_steps}
    \includegraphics[scale=0.40]{tr_steps.pdf}
  }

  \caption[]{ (a) Relationship between the different objectives. Note that functional dependence on $\theta$ and $\phi$ have been suppressed for brevity. (b) The values of the discrete ($\mathcal{L}$) and GumBolt ($\mathcal{F}$) objectives (with $k=20$ for all objectives) throughout the training on the training and validation sets (of MNIST dataset) for a GumBolt on $\sim \,\sim$ structure during $10^6$ iterations of training. The subscripts ``val'' and ``tr'' correspond to the validation and training sets, respectively.
  The abrupt changes are caused by stepwise annealing of the learning rate. This figure signifies that the differentiable GumBolt objective faithfully follows the non-differentiable discrete objective, leading to no overfitting caused by following a wrong objective. We did not use early stopping in our experiments.
    }
  \label{fig:res}
\end{figure}
\subsection{Comparison with other discrete models and the importance of powerful priors}

 If the BM prior is replaced with a factorial Bernoulli distribution, GumBolt transforms into CONCRETE \citep{maddison2016concrete} (when continuous variables are used inside discrete pdfs) and Gumbel-Softmax \citep{jang2016categorical}. This can be achieved by setting the couplings ($W$) of the BM to zero, and keeping the biases. Since the performance of CONCRETE and Gumbel-Softmax has been extensively compared against other models \citep{maddison2016concrete, jang2016categorical, tucker2017rebar, grathwohl2017backpropagation}, we do not repeat these experiments here; we note however that CONCRETE performs favorably to other discrete latent variable models in most cases.

\begin{table}[!htbp]
\centering
\caption{Performance of GumBolt (test-set log-likelihood) in the presence and absence of coupling weights. -nW in the second column signifies that the elements of the coupling matrix $W$ are set to zero, \emph{throughout} the training rather than just during evaluation. Removing the weights significantly degrades the performance of GumBolt.}
\label{tab:other_models}
\begin{tabular}{@{}lcrrrrrl@{}}
\toprule
 & \multicolumn{1}{l}{} & \begin{tabular}[c]{@{}r@{}}GumBolt\\ $k=20$\end{tabular} & \begin{tabular}[c]{@{}r@{}}GumBolt-nW\\ $k=20$\end{tabular}
 \\
 & $-$ & \textbf{87.45} & 99.94 &   \\
\textbf{MNIST} & $\sim$ & \textbf{83.87} & 93.50   \\

 & $\sim \, \sim$ & \textbf{82.75} & 88.01   \\ \midrule
 & $-$ & \textbf{103.69} & 112.21   \\
\textbf{OMNIGLOT} & $\sim$ & \textbf{100.68} & 107.221    \\
 & $\sim \,\sim$ & \textbf{98.81} & 105.01
 \\ \bottomrule
\end{tabular}
\end{table}

An interesting question is studying how much of the performance advantage of GumBolt is caused by powerful BM priors. We have studied this in Table~\ref{tab:other_models} by setting the couplings of the BM to $0$ throughout the training (denoted by GumBolt-nW). The GumBolt with couplings significantly outperforms the GumBolt-nW. It was shown in \citep{vahdat2018dvae++} that dVAE and dVAE++ outperform other models with discrete latent variables (REBAR, RELAX, VIMCO, CONCRETE and Gumbel-Softmax) on the same structure. By outperforming the previuos models with BM priors, our model  achieves state-of-the-art performance in the scope of models with discrete latent variables.

Another important question is if some of the improved performance in the presence of BMs can be salvaged in the GumBolt-nW by having more powerful neural nets in the decoder. We observed that by making the decoder's neural nets wider and deeper, the performance of the GumBolt-nW does not improve. This predictably suggests that the increased probabilistic capability of the prior cannot be obtained by simply having a more deterministically powerful decoder.

\vspace{-3mm}
\section{Conclusion}
\vspace{-3mm}
In this work, we have proposed the GumBolt that extends the Gumbel trick to Markov random fields and BMs. We have shown that this approach is effective and on the entirety of a  wide host of structures outperforms the other models that use BMs in their priors. GumBolt is much simpler than previous models that require marginalization of the discrete variables and achieves state-of-the-art performance on MNIST and OMNIGLOT datasets in the context of models with only discrete variables.

\vspace{-3mm}

\bibliographystyle{apalike}

\subsection{GumBolt probability proxy}
\label{sec:gumbolt}
To replace $\log \tilde{Z}_\theta$ with $\log Z_\theta$, we introduce a proxy probability distribution:
\begin{equation}
  \breve{p}_{\theta}(\zeta) \equiv \frac{e^{{-E_{\theta}(\zeta)}}}{Z_\theta} .
    \label{eq:gumbolt_p}
\end{equation}

Note that $\breve{p}_{\theta}(\zeta)$ is not a true (normalized) probability distribution, but $\breve{p}_{\theta}(\zeta) \to {p}_{\theta}(z)$ as $\tau \to 0$.

Now consider the following theorem (see Appendix~\ref{appen:proofgum} for a proof):
 \begin{theorem}
\label{theo:gumbel}
For any polynomial function $E_\theta (z)$ of binary variables $z \in \{0,1\}^{n_z}$, the proxy probability $\breve{p}_{\theta}(\zeta) \equiv e^{-E_\theta (\zeta)}/Z_\theta$, with $\zeta \in [0,1]^{n_z}$, is a lower bound to the true probability ${p}_{\theta}(\zeta)\equiv e^{-E_\theta (\zeta)}/\tilde Z_\theta$, \emph{i.e.}, $\breve{p}_{\theta}(\zeta) \le {p}_{\theta}(\zeta)$, where $Z_{\theta} \equiv {\sum_{\{z\}}{e^{{-E_{\theta}(z)}}}}$ and $\tilde{Z}_\theta \equiv \int\limits_{\{\zeta\}} d\zeta e^{-E_{\theta}(\zeta)}$ .
\end{theorem}

 Therefore, replacing ${p}_{\theta}(\zeta)$ with $\breve{p}_{\theta}(\zeta)$, we obtain a lower bound on $\Fcont{k}$ :
 \begin{equation}
      \F{k} = \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{q_\phi(\zeta^i |x)}\right] -\log{Z_{\theta}} \le \Fcont{k}.
 \label{eq:ELBO_cont}
 \end{equation}

This allows reparameterization trick, while making it possible to use sampling to estimate the gradients. The structure of our model with a BM prior is portrayed in Figure~\ref{fig:gumbolt}, where both continuous and discrete variables are used. Notice that in the limit $\tau \to 0$, $\breve{p}_{\theta}(\zeta^i)$ becomes a probability mass function (pmf) $p_\theta(z^i)$, while $q_\phi(\zeta^i |x)$ remains as a probability density function (pdf). To resolve this inconsistency, we replace $q_\phi(\zeta^i |x)$ with the Bernoulli pmf: $\breve{q}_\phi(\zeta^i |x) = \zeta^i \log \bar q^i + (1-\zeta^i) \log (1-\bar q^i)$, which approaches  $q_\phi(z^i |x)$ when $\tau \to 0$. The training objective
 \begin{equation}
      \Fgumbolt{k} = \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{\breve{q}_\phi(\zeta^i |x)}\right] -\log Z_\theta
 \label{eq:ELBO_cont}
 \end{equation}
becomes $\ELBO{k}$ at $\tau=0$ as desired (see Fig.~\ref{fig:losses} for the relationship among different objectives). This is the analog of the Gumbel-Softmax trick \citep{jang2016categorical} when applied to BMs.
During training, $\tau$ should be kept small for the continuous variables to stay close to the discrete variables, a common practice with Gumbel relaxation \citep{tucker2017rebar}. For evaluation, $\tau$ is set to zero, leading to an unbiased evaluation of the objective function.
For generation, discrete samples from BM are directly fed into the decoder to obtain the probabilities of each input feature.

\subsection{Compatibility of BM with discrete variable relaxation}
\label{sec:stab}
The term in the discrete objective that involves the prior distribution is $\mathbb{E}_{q_\phi(z|x)}\left[ \log p_\theta(z)\right]$. By replacing $z$ with $\zeta$, there is no guarantee that the parameters that optimize $\mathbb{E}_{q_\phi(\zeta|x)}\left[ \log p_\theta(\zeta)\right]$ would also optimize the discrete version. This happens naturally for Bernoulli distribution used in \citep{jang2016categorical}, since the maximum or minimum of $\log (p(\zeta)) = \zeta \log (\bar p) + (1- \zeta) \log(1-\bar p)$ occur at the boundaries (\emph{i.e.}, when $\zeta=1$ or $\zeta=0$). This means that throughout the training, the values of $\zeta$ are pushed towards the boundary points, consistent with the discrete objective.

In the case of a BM prior, consider the following theorem (proved in Appendix~\ref{appen:proofgum}):
\begin{theorem}
\label{theo:optima_cont_vert}
For any polynomial function $E_\theta (z)$ of binary variables $z \in \{0,1\}^{n_z}$, the extrema of the relaxed function $E_\theta(\zeta)$ with $\zeta \in [0,1]^{n_z}$ reside on the vertices of the hypercube, i.e., $\zeta_{\rm opt} \in \{0,1\}^{n_z}$.
\end{theorem}
Therefore, the maximum or minimum of $\log p_{\theta} (\zeta) \propto -E_{\theta}(z)$, also occur on the boundaries; this shows that having a BM rather than a factorial Bernoulli distribution does not exacerbate the training of GumBolt.

\subsection{GumBolt probability proxy}
\label{sec:gumbolt}
To replace $\log \tilde{Z}_\theta$ with $\log Z_\theta$, we introduce a proxy probability distribution:
\begin{equation}
  \breve{p}_{\theta}(\zeta) \equiv \frac{e^{{-E_{\theta}(\zeta)}}}{Z_\theta} .
    \label{eq:gumbolt_p}
\end{equation}

Note that $\breve{p}_{\theta}(\zeta)$ is not a true (normalized) probability distribution, but $\breve{p}_{\theta}(\zeta) \to {p}_{\theta}(z)$ as $\tau \to 0$.

Now consider the following theorem (see Appendix~\ref{appen:proofgum} for a proof):
 \begin{theorem}
\label{theo:gumbel}
For any polynomial function $E_\theta (z)$ of binary variables $z \in \{0,1\}^{n_z}$, the proxy probability $\breve{p}_{\theta}(\zeta) \equiv e^{-E_\theta (\zeta)}/Z_\theta$, with $\zeta \in [0,1]^{n_z}$, is a lower bound to the true probability ${p}_{\theta}(\zeta)\equiv e^{-E_\theta (\zeta)}/\tilde Z_\theta$, \emph{i.e.}, $\breve{p}_{\theta}(\zeta) \le {p}_{\theta}(\zeta)$, where $Z_{\theta} \equiv {\sum_{\{z\}}{e^{{-E_{\theta}(z)}}}}$ and $\tilde{Z}_\theta \equiv \int\limits_{\{\zeta\}} d\zeta e^{-E_{\theta}(\zeta)}$ .
\end{theorem}

 Therefore, replacing ${p}_{\theta}(\zeta)$ with $\breve{p}_{\theta}(\zeta)$, we obtain a lower bound on $\Fcont{k}$ :
 \begin{equation}
      \F{k} = \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{q_\phi(\zeta^i |x)}\right] -\log{Z_{\theta}} \le \Fcont{k}.
 \label{eq:ELBO_cont}
 \end{equation}

This allows reparameterization trick, while making it possible to use sampling to estimate the gradients. The structure of our model with a BM prior is portrayed in Figure~\ref{fig:gumbolt}, where both continuous and discrete variables are used. Notice that in the limit $\tau \to 0$, $\breve{p}_{\theta}(\zeta^i)$ becomes a probability mass function (pmf) $p_\theta(z^i)$, while $q_\phi(\zeta^i |x)$ remains as a probability density function (pdf). To resolve this inconsistency, we replace $q_\phi(\zeta^i |x)$ with the Bernoulli pmf: $\breve{q}_\phi(\zeta^i |x) = \zeta^i \log \bar q^i + (1-\zeta^i) \log (1-\bar q^i)$, which approaches  $q_\phi(z^i |x)$ when $\tau \to 0$. The training objective
 \begin{equation}
      \Fgumbolt{k} = \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{\breve{q}_\phi(\zeta^i |x)}\right] -\log Z_\theta
 \label{eq:ELBO_cont}
 \end{equation}
becomes $\ELBO{k}$ at $\tau=0$ as desired (see Fig.~\ref{fig:losses} for the relationship among different objectives). This is the analog of the Gumbel-Softmax trick \citep{jang2016categorical} when applied to BMs.
During training, $\tau$ should be kept small for the continuous variables to stay close to the discrete variables, a common practice with Gumbel relaxation \citep{tucker2017rebar}. For evaluation, $\tau$ is set to zero, leading to an unbiased evaluation of the objective function.
For generation, discrete samples from BM are directly fed into the decoder to obtain the probabilities of each input feature.

\subsection{Compatibility of BM with discrete variable relaxation}
\label{sec:stab}
The term in the discrete objective that involves the prior distribution is $\mathbb{E}_{q_\phi(z|x)}\left[ \log p_\theta(z)\right]$. By replacing $z$ with $\zeta$, there is no guarantee that the parameters that optimize $\mathbb{E}_{q_\phi(\zeta|x)}\left[ \log p_\theta(\zeta)\right]$ would also optimize the discrete version. This happens naturally for Bernoulli distribution used in \citep{jang2016categorical}, since the maximum or minimum of $\log (p(\zeta)) = \zeta \log (\bar p) + (1- \zeta) \log(1-\bar p)$ occur at the boundaries (\emph{i.e.}, when $\zeta=1$ or $\zeta=0$). This means that throughout the training, the values of $\zeta$ are pushed towards the boundary points, consistent with the discrete objective.

In the case of a BM prior, consider the following theorem (proved in Appendix~\ref{appen:proofgum}):
\begin{theorem}
\label{theo:optima_cont_vert}
For any polynomial function $E_\theta (z)$ of binary variables $z \in \{0,1\}^{n_z}$, the extrema of the relaxed function $E_\theta(\zeta)$ with $\zeta \in [0,1]^{n_z}$ reside on the vertices of the hypercube, i.e., $\zeta_{\rm opt} \in \{0,1\}^{n_z}$.
\end{theorem}
Therefore, the maximum or minimum of $\log p_{\theta} (\zeta) \propto -E_{\theta}(z)$, also occur on the boundaries; this shows that having a BM rather than a factorial Bernoulli distribution does not exacerbate the training of GumBolt.

\section{Introduction}

Variational autoencoders (VAEs) are generative models with the useful feature of learning representations
of input data in their latent space. A VAE comprises of a prior (the probability distribution of the latent space), a decoder and an encoder (also referred to as the approximating posterior or the inference network). There have been efforts devoted to making each of these components more powerful.
The decoder can be made richer by using autoregressive methods such as pixelCNNs, pixelRNNs \citep{oord2016pixel} and MADEs \citep{germain2015made}. However, VAEs tend to ignore the latent code (in the sense described by \citet{yeung2017tackling})  in the presence of powerful decoders \citep{chen2016variational, gulrajani2016pixelvae, goyal2017z}. There are also a myriad of works strengthening the encoder distribution \citep{kingma2016improved, rezendeICML15, salimans2015markov}.
Improving the priors is manifestly appealing, since it directly translates into a more powerful generative model. Moreover, a rich structure in the latent space is one of the main purposes of VAEs.  \citet{chen2016variational} observed that a more powerful autoregressive prior and a simple encoder is commensurate with a powerful  inverse autoregressive approximating posterior and a simple prior.

Boltzmann machines (BMs) are known to represent intractable and multi-modal distributions~\citep{le2008representational}, ideal for priors in VAEs, since they can lead to a more expressive generative model. However, BMs contain discrete variables which are incompatible with the reparameterization trick, required for efficient propagation of gradients through stochastic units. It is desirable to have discrete latent variables in many applications such as semi-supervised learning \citep{kingma2014semi}, semi-supervised generation \citep{maaloe2017semi} and hard attention models \citep{serra2018overcoming, gregor2015draw}, to name a few. Many operations, such as choosing between models or variables are naturally expressed using discrete variables \citep{yeung2017tackling}.

\citet{rolfe2016discrete} proposed the first model to use a BM in the prior of a VAE, i.e., a discrete VAE (dVAE). The main idea is to introduce auxiliary continuous variables (Fig.~\ref{fig:dvae}) for each discrete variable through a ``smoothing distribution''. The discrete variables are marginalized out in the autoencoding term by imposing certain constraints on the form of the relaxing distribution. However, the discrete variables cannot be marginalized out from the remaining term in the objective (the KL term). Their proposed approach relies on properties of the smoothing distribution to evaluate these terms.
In Appendix~\ref{appen:reinforce}, we show that this approach is equivalent to REINFORCE when dealing with some parts of the KL term (\emph{i.e.}, the cross-entropy term).
\citet{vahdat2018dvae++} proposed an improved version, dVAE++, that uses a modified distribution for the smoothing variables, but has the same form for the autoencoding part (see Sec.~\ref{sec:VAE}). The qVAE, \citep{khoshaman2018quantum}, expanded the dVAE to operate with a quantum Boltzmann machine (QBM) prior \citep{Amin:2016qd}.
A major shortcoming of these methods is that they are unable to have multi-sampled (importance-weighted) estimates of the objective function during training, which can improve the performance.

To use the reparameterization trick directly with discrete variables (without marginalization), a continuous and differentiable proxy is required. The Gumbel (reparameterization) trick, independently developed by \citet{jang2016categorical} and \citet{maddison2016concrete}, achieves this by relaxing discrete distributions. However, BMs and in general discrete random Markov fields (MRFs) are incompatible with this method. Relaxation of discrete variables (rather than distributions) for the case of factorial categorical prior (Gumbel-Softmax) was also investigated in both works. It is not obvious whether such relaxation of discrete variables would work with BM priors.

The contributions of this work are as follows:
  we propose the GumBolt, which extends the Gumbel trick to BM and MRF priors and is significantly simpler than previous models that marginalize discrete variables. We show that BMs are compatible with relaxation of discrete variables (rather than distributions) in Gumbel trick. We propose an objective using such relaxation and show that the main limitations of previous models with BM priors can be circumvented; we do not need marginalization of the discrete variables, and can have an importance-weighted objective.
GumBolt considerably outperforms the previous works in a wide series of experiments on permutation invariant MNIST and OMNIGLOT datasets, even without the importance-weighted objective (Sec.~\ref{sec:exps}). Increasing the number of importance weights can further improve the performance. We obtain the state-of-the-art results on these datasets among models with only discrete latent variables.

\section{Background}
\subsection{ Variational autoencoders}
\label{sec:VAE}
Consider a generative model involving observable variables $x$ and latent variables $z$. The joint probability distribution can be decomposed as $p_{\theta}(x, z) = p_{\theta} (z) p_{\theta}(x|z)$. The first and second terms on the right hand side are the prior and decoder distributions, respectively, which are parametrized by $\theta$.
 Calculating the marginal $p_{\theta}(x)$ involves performing intractable, high dimensional sums or integrals.
 Assume an element $x$ of the dataset $\mathcal{X}$ comprising of $N$ independent samples from an unknown underlying distribution is given. VAEs operate by introducing a family of approximating posteriors $q_\phi(z |x)$ and maximize a lower bound (also known as the ELBO), $\ELBO{}$ , on the log-likelihood $\log p_{\theta}(x)$ \citep{kingmaICLR15}:
 \begin{equation}
 \begin{split}
     \log p_{\theta}(x) \ge \ELBO{} &= \underset{q_\phi(z |x)}{\mathbb{E}} \left[\log \frac{p_{\theta}(x, z)}{q_\phi(z |x)}\right]  \\
     &= \underset{q_\phi(z |x)}{\mathbb{E}} \left[\log p_\theta(x | z)\right] - \KLD{q_\phi(z|x)}{p_\theta(z)},
 \end{split}
 \label{eq:ELBO}
 \end{equation}
where the first term on the right-hand side is the autoencoding term and $D_\text{KL}$ is the Kullback-Leibler divergence \citep{bishop_pattern_2011}. In VAEs, the parameters of the distributions (such as the means in the case of Bernoulli variables) are calculated using neural nets. To backpropagate through latent variables $z$, the {reparameterization trick} is used; $z$ is reparametrized as a deterministic function $ f(\phi, x, \rho)$, where the stochasticity of $z$ is relegated to another random variable, $\rho$, from a distribution that does not depend on $\phi$. Note that it is impossible to backpropagate if $z$ is discrete, since $f$ is not differentiable.
\subsection{Gumbel trick}
\label{sec:gumbel}
The non-differentiability of $f$ can be resolved by finding a relaxed proxy for the discrete variables. Assume a binary unit, $z$, with mean $\bar{q}$ and logit $l$; \emph{i.e.},   $p(z=1)=\bar{q}=\sigma(l)$, where $\sigma(l) \equiv \frac{1}{1+\exp(-l)}$ is the sigmoid function. Since $\sigma(l)$ is a monotonic function, we can reparametrize $z$ as $ z=\mathcal{H}(\rho - (1-\bar q)) = \mathcal{H}\left(l + \sigma^{-1}(\rho)\right)$ \citep{maddison2016concrete}, where $\mathcal{H}$ is the Heaviside function, $\rho\sim \mathcal{U}$ with $\mathcal{U}$ being a uniform distribution in the range $[0,1]$, and $\sigma^{-1} (\rho) = \log(\rho)-\log(1-\rho)$ is the inverse sigmoid or logit function. This transformation results in a non-differentiable reparameterization, but can be smoothed when the Heaviside function is replaced by a sigmoid function with a temperature $\tau$, \emph{i.e.}, $\mathcal{H}(\dots) \xrightarrow{}\sigma(\frac{\dots}{\tau})$. Thus, we introduce the continuous proxy \citep{maddison2016concrete}:
\begin{equation}
    f(\phi, x, \rho)= \zeta = \sigma\left({\frac{l(\phi, x) + \sigma^{-1}(\rho)}{\tau}}\right).
    \label{eq:Gumbel}
\end{equation}
The continuous $\zeta$ is differentiable and is equal to the discrete $z$ in the limit $\tau\xrightarrow{}0$.

\subsection{Boltzmann machine}
\label{sec:gen}

Our goal is to use a BM as prior. A BM is a probabilistic energy model described by
\begin{equation}
    p_{\theta}(z)= \frac{e^{{-E_{\theta}(z)}}}{Z_\theta},
    \label{eq:discrete_prior}
\end{equation}
where $E_{\theta}(z)$ is the energy function, and $Z_\theta=\sum_{\{z\}}{e^{{-E_{\theta}(z)}}}$ is the partition function; $z$ is a vector of binary variables. Since finding $p_{\theta}(z)$ is typically intractable, it is common to use sampling techniques to estimate the gradients.
To facilitate MCMC sampling using Gibbs-block technique, the connectivity of latent variables is assumed to be bipartite; \emph{i.e.}, $z$ is decomposed as $[z_1, z_2]$ giving
\begin{equation}
    -E_\theta(z)= a^T z_1 + b^T z_2+ z_2^T W z_1,
    \label{eq:energy}
\end{equation}
where $a$, $b$ and $W$ are the biases (on $z_1$ and $z_2$, respectively) and weights. This bipartite structure is known as the restricted Boltzmann Machine.

\section{Proposed approach}
\begin{figure}[!htbp]
  \centering
  \subfigure[d(q)VAE(++)]{
    \label{fig:dvae}
    \includegraphics[scale=0.35, page=3]{mod_figs.pdf}
  }
  \subfigure[Concrete, Gumbel-Softmax]{
    \label{fig:concrete}
    \includegraphics[scale=0.35, page=2]{mod_figs.pdf}
  }
  \subfigure[GumBolt]{
    \label{fig:gumbolt}
    \includegraphics[scale=0.35, page=1]{mod_figs.pdf}
  }

  \caption[]{
  Schematic of the discussed models with discrete variables in their latent space. The dashed red and solid blue arrows represent the inference network, and the generative model, respectively. (a) dVAE, qVAE \citep{khoshaman2018quantum} and dVAE++ have the same structure. They involve auxiliary continuous variables, $\zeta$, for each discrete variable, $z$, provided by the same conditional probability distribution, $r(\zeta|z)$, in both the generative and approximating posterior networks. (b) Concrete and Gumbel-Softmax apply the Gumbel trick to the discrete variables to obtain the $\zeta$s that appear in both the inference and generative models. (c) GumBolt only involves discrete variables in the generative model, and the relaxed $\zeta$s are used in the inference model during training. Note that during evaluation, the temperature is set to zero, leading to $\zeta=z$.
    }
  \label{fig:models}
\end{figure}

The importance-weighted or multi-sampled objective of a VAE with BM prior can be written as:
 \begin{equation}
 \begin{split}
      \log p_{\theta}(x) \ge \ELBO{k} &=
      \underset{\prod_i q_\phi(z^i |x)}{\mathbb{E}}
      \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{p_{\theta}(z^i)p_{\theta}(x|z^i)}{q_\phi(z^i |x)}\right]  \\
      &=
      \underset{\prod_i q_\phi(z^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(z^i)} p_{\theta} (x|z^i)}{q_\phi(z^i |x)}\right] -\log{Z_{\theta}},
 \end{split}
 \label{eq:ELBO_disc}
 \end{equation}
where $k$ is the number of samples or importance-weights over which the Monte Carlo objective is calculated,  \citep{mnih2016variational}. $z^i$ are independent vectors sampled from $q_\phi(z^i |x)$. Note that we have taken out $Z_\theta$ from the argument of the expectation value, since it is independent of $z$. The partition function is intractable but its derivative can be estimated using sampling:
\begin{equation}
    \nabla_{\theta}\log{Z_{\theta}} = \nabla_{\theta} \log {\sum_{\{z\}}e^{{-E_{\theta}(z)}}} =
     -\frac{{\sum_{\{z\}}\nabla_{\theta}  E_{\theta}(z)e^{{-E_{\theta}(z)}}}}{\sum_{\{z\}}e^{{-E_{\theta}(z)}}}
     =
    -\underset{p_{\theta} (z)}{\mathbb{E}}
    \left[ \nabla_{\theta} E_{\theta}(z) \right].
\label{eq:pf}
\end{equation}

Here, $\sum_{\{z\}}$ involves summing over all possible configurations of the binary vector $z$.
The objective $\ELBO{k}$ cannot be used for training, since it involves non-differentiable discrete variables. This can be resolved by relaxing the distributions:
 \begin{equation}
  \begin{split}
      \log p_{\theta}(x) \ge \Fcont{k} &= \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{p_\theta(\zeta^i) p_{\theta} (x|\zeta^i)}{q_\phi(\zeta^i |x)}\right] \nonumber \\
      &= \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{q_\phi(\zeta^i |x)}\right] -\log{\tilde Z_{\theta}}.
 \label{eq:ELBO_cont_full}
  \end{split}
 \end{equation}
Here, $\zeta^i$ is a continuous variable sampled from  Eq.~\ref{eq:Gumbel}, which is consistent with the Gumbel probability $q_\phi(\zeta^i |x)$ defined in \citep{maddison2016concrete}, and $p_\theta(\zeta) \equiv e^{-E_{\theta}(\zeta)}/\tilde{Z_\theta}$, where $\tilde{Z_\theta} \equiv \int d\zeta e^{-E_{\theta}(\zeta)}$.
The expectation distribution is the joint distribution over independent $z^i$ samples.
Notice that $\log \tilde{Z}_\theta$ is different from $\log Z_\theta$, therefore its derivatives cannot be estimated using discrete samples from a BM, making this method inapplicable for BM priors. The derivatives could be estimated using samples from a continuous distribution, which is very different from the BM distribution.
Analytical calculation of the expectations, suggested for Bernoulli prior by \citet{maddison2016concrete} is also infeasible for BMs, since it requires exhaustively summing over all possible configurations of the binary units.

\subsection{GumBolt probability proxy}
\label{sec:gumbolt}
To replace $\log \tilde{Z}_\theta$ with $\log Z_\theta$, we introduce a proxy probability distribution:
\begin{equation}
  \breve{p}_{\theta}(\zeta) \equiv \frac{e^{{-E_{\theta}(\zeta)}}}{Z_\theta} .
    \label{eq:gumbolt_p}
\end{equation}

Note that $\breve{p}_{\theta}(\zeta)$ is not a true (normalized) probability density function, but $\breve{p}_{\theta}(\zeta) \to {p}_{\theta}(z)$ as $\tau \to 0$.

Now consider the following theorems (see Appendix~\ref{appen:proofgum} for proof):
 \begin{theorem}
\label{theo:optima_cont_vert}
For any polynomial function $E_\theta (z)$ of $n_z$ binary variables $z \in \{0,1\}^{n_z}$, the extrema of the relaxed function $E_\theta(\zeta)$ with $\zeta \in [0,1]^{n_z}$ reside on the vertices of the hypercube, i.e., $\zeta_{\rm extr} \in \{0,1\}^{n_z}$.
\end{theorem}

 \begin{theorem}
\label{theo:gumbel}
For any polynomial function $E_\theta (z)$ of $n_z$ binary variables $z \in \{0,1\}^{n_z}$, the proxy probability $\breve{p}_{\theta}(\zeta) \equiv e^{-E_\theta (\zeta)}/Z_\theta$, with $\zeta \in [0,1]^{n_z}$, is a lower bound to the true probability ${p}_{\theta}(\zeta)\equiv e^{-E_\theta (\zeta)}/\tilde Z_\theta$, \emph{i.e.}, $\breve{p}_{\theta}(\zeta) \le {p}_{\theta}(\zeta)$, where $Z_{\theta} \equiv {\sum_{\{z\}}{e^{{-E_{\theta}(z)}}}}$ and $\tilde{Z}_\theta \equiv \int\limits_{\{\zeta\}} d\zeta e^{-E_{\theta}(\zeta)}$ .
\end{theorem}

 Therefore, according to theorem (2), replacing ${p}_{\theta}(\zeta)$ with $\breve{p}_{\theta}(\zeta)$, we obtain a lower bound on $\Fcont{k}$:
 \begin{equation}
      \F{k} = \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{q_\phi(\zeta^i |x)}\right] -\log{Z_{\theta}} \le \Fcont{k}.
 \label{eq:ELBO_cont}
 \end{equation}

This allows reparameterization trick, while making it possible to use sampling to estimate the gradients. The structure of our model with a BM prior is portrayed in Figure~\ref{fig:gumbolt}, where both continuous and discrete variables are used. Notice that in the limit $\tau \to 0$, $\breve{p}_{\theta}(\zeta^i)$ becomes a probability mass function (pmf) $p_\theta(z^i)$, while $q_\phi(\zeta^i |x)$ remains as a probability density function (pdf). To resolve this inconsistency, we replace $q_\phi(\zeta^i |x)$ with the Bernoulli pmf: $\breve{q}_\phi(\zeta^i |x) = \zeta^i \log \bar q^i + (1-\zeta^i) \log (1-\bar q^i)$, which approaches  $q_\phi(z^i |x)$ when $\tau \to 0$. The training objective
 \begin{equation}
      \Fgumbolt{k} = \underset{\prod_i q_\phi(\zeta^i |x)}{\mathbb{E}} \left[\log \frac{1}{k} \sum_{i=1}^{k} \frac{e^{-E_{\theta}(\zeta^i)} p_{\theta} (x|\zeta^i)}{\breve{q}_\phi(\zeta^i |x)}\right] -\log Z_\theta
 \label{eq:ELBO_cont}
 \end{equation}
becomes $\ELBO{k}$ at $\tau=0$ as desired (see Fig.~\ref{fig:losses} for the relationship among different objectives). This is the analog of the Gumbel-Softmax trick \citep{jang2016categorical} when applied to BMs.
During training, $\tau$ should be kept small for the continuous variables to stay close to the discrete variables, a common practice with Gumbel relaxation \citep{tucker2017rebar}. For evaluation, $\tau$ is set to zero, leading to an unbiased evaluation of the objective function.
For generation, discrete samples from BM are directly fed into the decoder to obtain the probabilities of each input feature.

\subsection{Compatibility of BM with discrete variable relaxation}
\label{sec:stab}
The term in the discrete objective that involves the prior distribution is $\mathbb{E}_{q_\phi(z|x)}\left[ \log p_\theta(z)\right]$. When $z$ is replaced with $\zeta$, there is no guarantee that the parameters that optimize $\mathbb{E}_{q_\phi(\zeta|x)}\left[ \log p_\theta(\zeta)\right]$ would also optimize the discrete version. This happens naturally for Bernoulli distribution used in \citep{jang2016categorical}, since the extrema of the prior term in the objective $\log (p(\zeta)) = \zeta \log (\bar p) + (1- \zeta) \log(1-\bar p)$ occur at the boundaries (\emph{i.e.}, when $\zeta=1$ or $\zeta=0$). This means that throughout the training, the values of $\zeta$ are pushed towards the boundary points, consistent with the discrete objective.

In the case of a BM prior, according to theorem (1) (proved in Appendix~\ref{appen:proofgum}),
the extrema of $\log p_{\theta} (\zeta) \propto -E_{\theta}(z)$ also occur on the boundaries; this shows that having a BM rather than a factorial Bernoulli distribution does not exacerbate the training of GumBolt.

\section{Related works}
\label{sec:related}
Several approaches have been devised to calculate the derivative of the expectation of a function with respect to the parameters of a Bernoulli distribution, $\mathcal{I} \equiv \nabla_\phi \mathbb{E}_{q_\phi(z)}\left[ f(z)\right]$:
\begin{enumerate}
    \item \label{it:anal} Analytical method: for simple functions, \emph{e.g.}, $f(z)=z$, one can analytically calculate the expectation and obtain $\mathcal{I}= \nabla_\phi \mathbb{E}_{q_\phi(z)}\left[z\right] = \nabla_\phi \bar q,$ where $\bar q$ is the mean of the Bernoulli distribution. This is a non-biased estimator with zero variance, but can only be applied to very simple functions. This approach is frequently used in semi-supervised learning \citep{kingma2014semi} by summing over different categories.
    \item Straight-through method: continuous proxies are used in backpropagation to evaluate derivatives, but discrete units are used in forward propagation \citep{bengio2013estimating, raiko2014techniques}.
    \item \label{it:reinf} REINFORCE trick: $\mathcal{I}= \mathbb{E}_{q_\phi(z)}\left[ f(z) \nabla_\phi \log q_\phi(z) \right]$, although it has high variance, which can be reduced by variance reduction techniques \citep{williams1992simple}.
    \item \label{it:reparam} Reparameterization trick: this method, as delineated in Sec.~\ref{sec:VAE}-\ref{sec:gumbel}, is biased except in the limit where the proxies approach the discrete variables.
    \item \label{it:marg} Marginalization: if possible, one can marginalize the discrete variables out from some parts of the loss function \citep{rolfe2016discrete}.
\end{enumerate}

NVIL \citep{mnih2014neural} and its importance-weighted successor VIMCO \citep{mnih2016variational} use (\ref{it:reinf}) with input-dependent signals obtained from neural networks to subtract from a baseline in order to reduce the variance of the estimator. REBAR \citep{tucker2017rebar} and its generalization, RELAX \citep{grathwohl2017backpropagation} use (\ref{it:reinf}) and employ (\ref{it:reparam}) in their control variates obtained using the Gumbel trick.
DARN \citep{gregor2013deep} and MuProp \citep{gu2015muprop} apply the Taylor expansion of the function $f(z)$ to synthesize baselines.
dVAE and dVAE++ (Fig.~\ref{fig:dvae}), which are the only works with BM priors, operate primarily based on (\ref{it:marg}) in their autoencoding term and use a combination of (1-4) for their KL term. In Appendix~\ref{appen:reinforce}, we show that dVAE has elements of REINFORCE in calculating the derivative of the KL term.
Our approach, GumBolt, exploits (\ref{it:reparam}), and does not require marginalizing out the discrete units.

\section{Experiments}
\label{sec:exps}
In order to explore the effectiveness of the GumBolt, we present the results of a wide set of experiments conducted on standard feed-forward structures that have been used to study models with discrete latent variables \citep{maddison2016concrete, tucker2017rebar, vahdat2018dvae++}. At first, we evaluate GumBolt against dVAE and dVAE++ baselines, all in the same framework and structure. We also demonstrate empirically that the GumBolt objective, Eq.~\ref{eq:ELBO_cont}, faithfully follows the  non-differentiable discrete objective throughout the training.
We then note on the relation between our model and other models that involve discrete variables. We also gauge the performance advantage GumBolt obtains from the BM by removing the couplings of the BM and re-evaluating the model.
\subsection{Comparison against dVAE and dVAE++}
\begin{table}[]
\centering
\caption{Test-set log-likelihood of the GumBolt compared against dVAE and dVAE++. $k$ represents the number of samples used to calculate the objective during \emph{training}. Note that dVAE and dVAE++ are only consistent with $k=1$. See the main text for more details.}
\label{tab:gumvsdvae}
\begin{tabular}{@{}lcrrrrr@{}}
\toprule
\rowcolor[HTML]{FFFFFF}
 & \multicolumn{1}{l}{\cellcolor[HTML]{FFFFFF}} & dVAE & dVAE++ & \multicolumn{1}{c}{\cellcolor[HTML]{FFFFFF}}  GumBolt & &  \\ \midrule
\rowcolor[HTML]{FFFFFF}
 & \multicolumn{1}{l}{\cellcolor[HTML]{FFFFFF}} & $k=1$ & $k=1$ & $k=1$ & $k=5$ & $k=20$ \\
 & $-$ & 90.11 & 90.40 & 88.88 & 88.18 & \cellcolor[HTML]{FFFFFF}\textbf{87.45} \\
 & $\sim$ & 85.72 & 85.41 & 84.86 & 84.31 & \cellcolor[HTML]{FFFFFF}\textbf{83.87} \\
\multirow{-2}{*}{\textbf{MNIST}} & $-\,-$ & 85.71 & 87.35 & 85.42 & 84.65 & \cellcolor[HTML]{FFFFFF}\textbf{84.46} \\
 & $\sim \, \sim$ & 84.33 & 84.75 & 83.28 & 83.01 & \cellcolor[HTML]{FFFFFF}\textbf{82.75} \\ \midrule
 & $-$ & 106.83 & 106.01 & 105.00 & 103.99 & \cellcolor[HTML]{FFFFFF}\textbf{103.69} \\
 & $\sim$ & 102.85 & 101.97 & 101.61 & 101.02 & \cellcolor[HTML]{FFFFFF}\textbf{100.68} \\
\multirow{-2}{*}{\textbf{OMNIGLOT}} & \multicolumn{1}{l}{$-\, -$} & 101.98 & 102.62 & 100.62 & 99.38 & \cellcolor[HTML]{FFFFFF}\textbf{99.36} \\
 & $\sim \,\sim$ & 101.75 & 100.70 & 99.82 & 99.32 & \cellcolor[HTML]{FFFFFF}\textbf{98.81} \\ \bottomrule
\end{tabular}
\end{table}

We compare the models on statically binarized MNIST \citep{salakhutdinov2008dbn} and OMNIGLOT datasets \citep{lake2015human} with the usual compartmentalization into the training, validation, and test-sets.
The $4000$-sample estimation of log-likelihood \citep{burda2015importance} of the models are reported in Table~\ref{tab:gumvsdvae}.
The structures used are the same as those of \citep{vahdat2018dvae++}, which were in turn adopted from \citep{tucker2017rebar} and \citep{maddison2016concrete}.
 We performed experiments with dVAE, dVAE++ and GumBolt on the same structure, and set the temperature to zero during evaluation (the results reported in \citep{vahdat2018dvae++} are calculated using non-zero temperatures). The inference network is chosen to be either factorial or have two hierarchies (Fig.~ \ref{fig:gumbolt}). In the case of two hierarchies, we have:
 $$q_{\phi}(z| x)=q_{\phi}(z_1, z_2 |x)= q_{\phi}(z_1 |x) q_{\phi}(z_2 |z_1, x)$$
 , where $z = [z_1, z_2].$

The meaning of the symbols in Table~\ref{tab:gumvsdvae} are as follows: $-$, and $\sim$ represent linear and nonlinear layers in the encoder and decoder neural networks. The number of stochastic layers (hierarchies) in the encoder is equal to the number of symbols. The dimensionality of the latent space is $200$ times the number of symbols; \emph{e.g.}, $\sim \,\sim$ means two stochastic layers (just as in Fig.~\ref{fig:gumbolt}), with $2$  hidden layers (each one containing $200$ deterministic units) in the encoder. The dimensionality of each stochastic layer is equal to $200$ in the encoder network; the generative network is a $200 \times 200$ RBM (a total of $400$ stochastic units), for $\sim \,\sim$ and $- \, -$, whereas, for $-$ and $\sim$, it is a $100 \times 100$ RBM. Note that in the case of $-\,-$, only one layer of $200$ deterministic units is used in each one of the two hierarchies.
The decoder network receives the samples from the RBM and probabilistically maps them into the input space using one or two layers of deterministic units. Since the RBM has bipartite structure, our model has two stochastic layers in the generative model.
The chosen hyper-parameters are as follows: $1M$ iterations of parameter updates using the ADAM algorithm \citep{kingma2014adam}, with the default settings and batch size of $100$ were carried out. The initial learning rate is $3\times 10^{-3}$ and is subsequently reduced by
$0.3$ at $60 \%$, $75\%$, and $95\%$ of the total iterations. KL annealing \citep{sonderby2016ladder} was used via a linear schedule during $30 \%$ of the total iterations. The value of temperature, $\tau$ was set to $\frac{1}{7}$ for all the experiments involving GumBolt, $\frac{1}{5}$ for experiments with dVAE, and $\frac{1}{10}$ and $\frac{1}{8}$ for dVAE++ on the MNIST and OMNIGLOT datasets, respectively; these values were cross-validated from $\{\frac{1}{10}, \frac{1}{9}\dots ,\frac{1}{5} \}$. The GumBolt shows the same average performance for temperatures in the range $\{\frac{1}{9}, \frac{1}{8} ,\frac{1}{7} \}$. The reported results are the averages from performing the experiments $5$ times. The standard deviations in all cases are less than $0.15$; we avoid presenting them individually to keep the table less cluttered.
We used the batch-normalization algorithm \citep{ioffe2015batch} along with $\texttt{tanh}$ nonlinearities. Sampling the RBM was done by performing $200$ steps of Gibbs updates for every mini-batch, in accordance with our baselines,using persistent contrastive divergence (PCD) ~\citep{tieleman2008training}.
We have observed that by having $2$ and $20$ PCD steps, the performance of our best model on MNIST dataset is deteriorated by $0.35$ and $0.12$ nats on average, respectively.
In order to estimate the log-partition function, $\log Z_\theta$, a GPU implementation of parallel tempering algorithm with bridge sampling was used \citep{desjardins2010tempered, bennett1976efficient, shirts2008statistically}, with a set of parameters to ensure the variance in   $\log Z_\theta$ is less than 0.01: $20K$ burn-in steps were followed by $100K$ sweeps,  $20$ times (runs), with a pilot run to determine the inverse temperatures (such that the replica exchange rates are approximately $0.5$).

We underscore several important points regarding Table~\ref{tab:gumvsdvae}.
First, when one sample is used in the training objective ($k=1$), GumBolt outperforms dVAE and dVAE++ in all cases. This can be due to the efficient use of reparameterization trick and the absence of REINFORCE elements in the structure of GumBolt as opposed to dVAE (Appendix~\ref{appen:reinforce}).
 Second, the previous models do not apply when $k>1$. GumBolt allows importance weighted objectives according to Eq.~\ref{eq:ELBO_cont}. We see that in all cases, by adding more samples to the training objective, the performance of the model is enhanced.

Fig.~\ref{fig:tr_steps} depicts the $k=20$ estimation of the GumBolt and discrete objectives on the training and validation sets during training. It can be seen that over-fitting does not happen since all the objectives are improving throughout the training. Also, note that the differentiable GumBolt proxy closely follows the non-differentiable discrete objective. Note that the kinks in the learning curves are caused by our stepwise change of the learning rate and is not an artifact of the model.

\begin{figure}[!htbp]
  \centering
  \subfigure[]{
    \label{fig:losses}
    \includegraphics[scale=.5]{losses_cropped.pdf}
  }
  \subfigure[]{
    \label{fig:tr_steps}
    \includegraphics[scale=0.40]{tr_steps.pdf}
  }

  \caption[]{ (a) Relationship between the different objectives. Note that functional dependence on $\theta$ and $\phi$ have been suppressed for brevity. (b) The values of the discrete ($\mathcal{L}$) and GumBolt ($\mathcal{F}$) objectives (with $k=20$ for all objectives) throughout the training on the training and validation sets (of MNIST dataset) for a GumBolt on $\sim \,\sim$ structure during $10^6$ iterations of training. The subscripts ``val'' and ``tr'' correspond to the validation and training sets, respectively.
  The abrupt changes are caused by stepwise annealing of the learning rate. This figure signifies that the differentiable GumBolt objective faithfully follows the non-differentiable discrete objective, leading to no overfitting caused by following a wrong objective. We did not use early stopping in our experiments.
    }
  \label{fig:res}
\end{figure}
\subsection{Comparison with other discrete models and the importance of powerful priors}

 If the BM prior is replaced with a factorial Bernoulli distribution, GumBolt transforms into CONCRETE \citep{maddison2016concrete} (when continuous variables are used inside discrete pdfs) and Gumbel-Softmax \citep{jang2016categorical}. This can be achieved by setting the couplings ($W$) of the BM to zero, and keeping the biases. Since the performance of CONCRETE and Gumbel-Softmax has been extensively compared against other models \citep{maddison2016concrete, jang2016categorical, tucker2017rebar, grathwohl2017backpropagation}, we do not repeat these experiments here; we note however that CONCRETE performs favorably to other discrete latent variable models in most cases.

\begin{table}[!htbp]
\centering
\caption{Performance of GumBolt (test-set log-likelihood) in the presence and absence of coupling weights. -nW in the second column signifies that the elements of the coupling matrix $W$ are set to zero, \emph{throughout} the training rather than just during evaluation. Removing the weights significantly degrades the performance of GumBolt.}
\label{tab:other_models}
\begin{tabular}{@{}lcrrrrrl@{}}
\toprule
 & \multicolumn{1}{l}{} & \begin{tabular}[c]{@{}r@{}}GumBolt\\ $k=20$\end{tabular} & \begin{tabular}[c]{@{}r@{}}GumBolt-nW\\ $k=20$\end{tabular}
 \\
 & $-$ & \textbf{87.45} & 99.94 &   \\
\textbf{MNIST} & $\sim$ & \textbf{83.87} & 93.50   \\

 & $\sim \, \sim$ & \textbf{82.75} & 88.01   \\ \midrule
 & $-$ & \textbf{103.69} & 112.21   \\
\textbf{OMNIGLOT} & $\sim$ & \textbf{100.68} & 107.221    \\
 & $\sim \,\sim$ & \textbf{98.81} & 105.01
 \\ \bottomrule
\end{tabular}
\end{table}

An interesting question is studying how much of the performance advantage of GumBolt is caused by powerful BM priors. We have studied this in Table~\ref{tab:other_models} by setting the couplings of the BM to $0$ throughout the training (denoted by GumBolt-nW). The GumBolt with couplings significantly outperforms the GumBolt-nW. It was shown in \citep{vahdat2018dvae++} that dVAE and dVAE++ outperform other models with discrete latent variables (REBAR, RELAX, VIMCO, CONCRETE and Gumbel-Softmax) on the same structure. By outperforming the previuos models with BM priors, our model  achieves state-of-the-art performance in the scope of models with discrete latent variables.

Another important question is if some of the improved performance in the presence of BMs can be salvaged in the GumBolt-nW by having more powerful neural nets in the decoder. We observed that by making the decoder's neural nets wider and deeper, the performance of the GumBolt-nW does not improve. This predictably suggests that the increased probabilistic capability of the prior cannot be obtained by simply having a more deterministically powerful decoder.

\vspace{-3mm}
\section{Conclusion}
\vspace{-3mm}
In this work, we have proposed the GumBolt that extends the Gumbel trick to Markov random fields and BMs. We have shown that this approach is effective and on the entirety of a  wide host of structures outperforms the other models that use BMs in their priors. GumBolt is much simpler than previous models that require marginalization of the discrete variables and achieves state-of-the-art performance on MNIST and OMNIGLOT datasets in the context of models with only discrete variables.

\vspace{-3mm}

\bibliographystyle{apalike}

redpajama_set_name: RedPajamaArXiv


With the vast growth of computer users of today, we cannot deny the fact that energy usage requirements in every household has drastically grown. We all know that this will not only cost us big amounts but will also hurt the environment in so many ways. Chang-Su Lee of Design Hara is Seoul, South Korea has submitted an eco-friendly power PC to Igreenspot that many of the computer savvy people will love to hear. Hailed as the Green HTPC, this small yet powerful PC has a chassis built with recyclable green materials. it is also compatible with small standard motherboards and consumes energy that is 30% lower to your traditional desktop PCs.Another green feature of this cute little powerful PC is its ability to reduce carbon dioxide emission. The Green HTPC's side panels are also made of harmless materials like natural wood. The HTPC's parts are connected by bolts and nuts making it easy to be recycled when it useful life ends. In addition, each HTPC will be unique as this is a full custom handmade product in accordance with the customer's preferences.The customer will also be given the chance to choose their own high-performance internal components his or her HTCP can have like the motherboard, RAM, and more.


Formed in 1983 when Brian Marshall left the Prudential to trade as an Independent Insurance Broker, with many of his existing clients following him. His brother, Stuart joined the following year (also ex-Prudential) as the business rapidly expanded. Initially working from home, the Company moved in 1984 to the offices in Greyhound Lane that they continue to occupy today.
In 1985 the Company expanded into the Estate Agency market both Sales & Letting properties locally. Doug Ardley was engaged to manage the business (retired 2011) with the later assistance of family member Glenn Brown, who succeeded him as Manager. Following the rapid expansion of the Property Management portfolio, it was decided to exit the sales market, to specialize on rental management.
We are proud to say that many clients remain with us from the early days and that business growth has largely been via recommendation only. We feel we still offer the same service and values that our clients have valued across the years.
Upon the anticipated retirement of Brian (in mid 2016) Stuart & Glenn will manage the Companies jointly.
Ovalmere (1993) Limited is authorized and regulated by the Financial Conduct Authority.


text:
Group D Round 3 Preview and World Cup Prediction
Group D has been an exciting one so far. England have already been eliminated, but the remaining three teams are still in it with a chance. Costa Rica are guaranteed a spot in the round of 16, but have not yet clinched top spot.
Italy v Uruguay – Tuesday, June 24 @ 12pm:
Uruguay have an excellent chance to progress with a win, however, Italy only needs a draw to go through due to their better goal difference. Both teams should take notes on what Costa Rica has done considering that they have already beaten both teams in this World Cup.
My prediction is a Uruguay win. Luis Suarez is going to be the key player in this one. He looked lively and much closer to being fit than anyone expected in their previous match against England. Even if he is marked there are enough other attacking players to damage Italy.
Italy do have a potent offence, and one that could wreak havoc against a sometimes struggling defense. The difference here is that Uruguay seemed to have gotten themselves together following the 3-1 loss to Costa Rica and look much stronger throughout the pitch.
Costa Rica v England – Tuesday, June 24 @ 12pm:
England are out and Costa Rica are through already. The Three Lions are playing only for pride and hope of leaving with at least a point. They are going to struggle to do this. Costa Rica play as a team and have already shown what they can do in defeating both Uruguay and Italy.
England will need to show something more than they have in their previous losses. Defensive lapses, lack of leadership, and a lack of creativity in front of goal have been massive
My prediction is another Costa Rica win to be able to top the group. England will be making changes, and these could be enough to get a result, but the Ticos have been relentless in their quest to progress and will capitalize on the errors that seem to always crop up in the English team.
Round 3 World Cup Predictions
← Group C Round 3 Preview and World Cup Prediction
World Cup Predictions: Group E Round 2 Preview →
Home » 2014 World Cup, Matchday Previews, Predictions » Group D Round 3 Preview and World Cup Prediction


Fact-finding requires good interviews. There are circumstances that for a variety of reasons, so-called investigative interviews best be carried out by an experienced, and above all, independent individual.
Interviews held by a third party will significantly lessen the chance that not all facts surface due to internal relationships, loyalty or peer pressure.
These days a more professional approach towards all types of investigative interviewing is a not to be underestimated requirement to uphold your reputation and that a case is handled fair and well-managed from day one. Additional training on top of that minimize the chance of human error during the interview and makes information that comes forward more reliable.


Do we have a flash programmer on S32DS PA?
My target is MPC560xB, and I would like to download elf or s19 to d-flash, where can I find the flash programmer on S32DS PA?
Also how can I dump the internal flash including C/D flash via S32DS PA?
lastly, how can I type the censorship password via S32DS PA?
currently you can use a S32DS debugger configuration to program your custom .elf/.srec file into the target MCU.
In coming release v1.2 there should be the flash programming support available in IDE that makes it easier.
Memory View does support import/export- dump of the target MCU memory.
S32DS how to properly dump RAM in debug session?