The impressive performance and plasticity of convolutional neural networks to solve different vision problems are shadowed by their black-box nature and its consequent lack of full understanding. To reduce this gap we propose to describe the activity of individual neurons by quantifying their inherent selectivity to specific properties. Our approach is based on the definition of feature selectivity indexes that allow the ranking of neurons according to specific properties. Here we report the results of exploring selectivity indexes for: (a) an image feature (color); and (b) an image label (class membership). Our contribution is a framework to seek or classify neurons by indexing on these selectivity properties. It helps to find color selective neurons, such as a red-mushroom neuron in layer conv4 or class selective neurons such as dog-face neurons in layer conv5, and establishes a methodology to derive other selectivity properties. Indexing on neuron selectivity can statistically draw how features and classes are represented through layers at a moment when the size of trained nets is growing and automatic tools to index can be helpful.
1 INTRODUCTION
In parallel with the success of CNNs to solve vision problems, there is a growing interest in developing methodologies to understand and visualize the internal representations of these networks. How the responses of a trained CNN encode the visual information is a fundamental question for computer and eventually for human vision.
Several works have proposed different methodologies to address the understanding problem. Recently, in Li et al. (2016) two main groups of works are mentioned. On one side those works that deal with the problem from a theoretical point of view. These are works such as Montavon et al. (2011) where kernel sequences are used to conclude that deep networks create increasingly better representations as the number of layer increases, Paul & Venkatasubramanian (2014) which explains why a deep learning network learns simple features first and that the representation complexity increases as the layers get deeper, Goodfellow et al. (2014) where an explanation for why an adversarial example created for one network is still valid in many others and they usually assign it the same (wrong) class, or Arora et al. (2014) that presents algorithms for training certain deep generative models with provable polynomial running time. On the other side, an empirical point of view, which comprises approaches that pursuit methodologies to visualize intermediate features in the image space, or approaches that analyze the effect of modifying a given feature map in a neuron activation. Our work is framed in the first subset of empirical approaches.
Visualizing intermediate features seeks to describe the activity of individual neurons. This description is the basis of this work hypothesis that is based on the idea that a proper understanding of the activity of the individual neurons allow us to draw a map of the CNN behavior. This behavior can be understood either in terms of relevant image features or in terms of the discriminative power of the neurons across the full architecture.
The first and most obvious way to describe the activity of a single neuron is given by the inherent set of weights of the learned filters. These weights can be used to compare neurons between them, either
within the same layer or versus neurons in similar CNNs which have been trained under different initialization conditions, as it is proposed by Li et al. (2016). A direct visualization of these weights is intuitive when they belong to neurons of a first convolutional layer. However, when layers are stacked, that intuition disappears and the capability to understand the neuron activity is lost.
A second method to describe neuron activity is projecting the filter weights into the image space, trying to get the inherent feature that maximally activates the filter. The projection can be computed by composing the inversion of the layer operators under a specific neuron towards the image space: this was called a Decoded Filter (DF) in Rafegas & Vanrell (2016). The resulting image represents an estimation of the feature that should highly activate such neuron. The disentangling algorithm that inverts the filter would give a good estimation of the feature image if most of the layer operators were invertible. However, when the number of non-invertible operators increases, the estimation becomes unintelligible. The appearance of the DFs can be seen in Fig. 1 of Rafegas & Vanrell (2016). They have also been explored by Springenberg et al. (2015) for architectures with no pooling layers since pooling is the less invertible operator. They point out the interest of obtaining such a representation, since it would allow the understanding of neuron activity independently of the input image. However, the majority of proficient CNNs contain pooling layers.
A third way to describe neuron activity is by exploring the images that maximally activate the neuron. One of the most relevant works pursuing the visualization of intermediate features, is the one proposed by Zeiler & Fergus (2014), where they project intrinsic features of neurons from the image that have provoked a maximum spike to a certain neuron, the network representation is projected into the image space by isolating them in the deconvolution approach Zeiler et al. (2010). By observing different projections that maximally activate a certain neuron they get the intuition about the main features learned on the network. Later on, in Springenberg et al. (2015) the guided backpropagation improves the deconvolution approach by a new way of inverting rectified linear (ReLu) nonlinearities, achieving better visualizations of the activations. These approaches present a main drawback, their feature visualization is image-specific, since the maximum activation of a neuron not always generalize the intrinsic feature of the neuron. To solve this problem, in some works instead of using the image that provokes the maximum activation, they use optimization techniques to generate an image that maximizes the activation. The key point of these works is using an appropriate regularization in the generation process, otherwise, the resulting image appearance is unrealistic and difficult to understand. Simonyan et al. (2014) propose a method to generate an image which is representative of a certain class by maximizing the score of this image to be classified in a certain class (or highly activates the specified neuron) with an L2-regularization. A similar work was performed afterwards in Yosinski et al. (2015) but taking advantage of combining three different regularizations to achieve more recognizable images. Although they have explored different regularizations to achieve more realistic intrinsic feature representations, their visualizations present important artifacts that complicate the understanding of the intrinsic property.
Finally, other works focus on proposing approaches able to reconstruct the input image given a feature map, going further of analyzing the individual neuron activity. Mahendran & Vedaldi (2015) make use of optimization algorithms to search for an image whose feature map best matches a given feature map by incorporating natural image priors. Contrary, in Dosovitskiy & Brox (2015), the authors propose to reconstruct the input image from its feature maps of a given convolutional network by training a new deconvolutional network to learn filter weights that minimize the image reconstruction error when these filters are applied to the image feature maps. With this approach they are also able to get an image reconstruction with natural priors.
In the second subset of empirical approaches, Alexey Dosovitskiy (2015) train a generative deconvolutional network to create images from neuron activations. With this methodology, the variation of the activations enables the visualization of the differences in the generated images. A similar analysis is done by Aubry & Russell (2015), but instead of forward-propagate different activations to the image space and comparing them, they observe the changes on neuron activations when similar computer-generated images with different scene factors are introduced into a CNN. These works contribute in giving a deeper understanding on the internal CNN behavior. Both works conclude that there are specific neurons which are sensitive to color changes, point of views, scale or lighting configurations.
Likewise, in Zeiler & Fergus (2014) in this work we pursuit visualizing the intrinsic feature of a neuron by analyzing the images that maximally activates a specific neuron. However, to avoid the
lack of generality of this approach, we define the Neuron Feature which is not based on a single maximum activation. The Neuron Feature is a weighted average version of a set of maximum activation images that capture the essential properties shared by the most important activations and makes it not to be image-specific. Additionally, our Neuron Feature overcomes the problem of unrealistic representation we metnioned earlier, by directly averaging on the image space. In this way we achieve two main advantages: (a) keeping the properties of the natural images, and (b) providing a very straightforward approach to compute it.
Afterwards, we introduce the concept of neuron selectivity index, that is used in human vision research to characterize the response of specific cells to specific stimuli (Shapley & Hawken (2011)). This concept allows to achieve a higher level of abstraction in the understanding of a single neuron. In this work we provide two selectivity indexes which are different in their essence: a color selectivity index that quantifies the degree of response of a neuron to a specific color; a class selectivity index that quantifies the degree of response of a neuron to a specific class label. Indexes are derived from the neuron feature or directly from the set of images with maximum activations. We analyze both indexes on a VGG-M network (Chatfield et al. (2014)) trained on ImageNet (Deng et al. (2009)) and we confirm their flexibility to cluster neurons according to their index values and extract conclusions in terms of their task in the net. By selecting color selective neurons we are able to outline how color is represented by the network. Curiously we found some parallelism between color representation in the first convolutional layer and known evidences about the representation in the human visual system. Color selective-neurons also show some preferences towards specific colors which coincide with ImageNet color biases. Indexing on class selectivity neurons we found highly class selective neurons like digital-clock at conv2, cardoon at conv3 and ladybug at conv5, much before the fully connected layers.
2 NEURON FEATURE
As we mentioned in the previous section we propose to visualize the image feature that activates a neuron, whenever is possible, by directly computing a weighted average of the N -th first images that maximally activate this neuron. We will refer to it as the Neuron Feature (NF).
In order to build the NF we need to calculate the activations associated to each individual neuron. They need to be accordingly ranked with the rest of activations of the layer. For each neuron we select the set of images that achieve a minimum normalized activation value but constrained to a maximum number of images for practical reasons. By normalized activation we mean the value of the maximum activation of a neuron for a specific input image, which is normalized by the maximum of these values achieved by the same neuron over all the images in the dataset.
In Fig. 1 we can see the behavior of the ranked normalized responses of a subset of neurons for every convolution layer of the VGG-M CNN trained on ImageNet by Chatfield et al. (2014). The y-axis represents the normalized activation value of a single neuron to an image of the dataset. Images are ranked on the x-axis according with their activation value, from highest to lowest activation (we just plot the first 400 images for each neuron). Therefore, the first relative activation value is always 1 for all neurons and then the normalized activation values decrease monotonically. This
conv1
conv2
conv3
conv4
conv5
(a) (b)
Figure 3: Examples of NFs for each convolutional layer of the network VGG-M (see section 4.1. (a) 20 examples of structured NF, (b), blurred NF. Although sizes of NF increments through layers, we scale them into the same size. Original sizes are: 7x7x3 , 27x27x3, 75x75x3, 107x107x3 and 139x139x3 for conv1, conv2, conv3, conv4 and conv5, respectively.
normalization allows to compare different neuron behaviors, from neurons which are activated by most of the images (flatter behavior), to neurons that highly activates only for a subset of images and have very little activation for the rest (steeper behavior). In this figure we also provide the percentage of area for each plotted curve. This percentage is computed over the area of the neuron that presents the maximum AUC in the entire architecture. We can observe different behaviors in all layers. In general, we can state that in deeper layers the behavior of the neurons is steeper (lower AUC), i.e. neurons highly spike for a small number of images. However, in shallower layers the behavior is flatter, i.e. neurons highly spike for a lot of images. This is an expected behavior, since the image features spiking neurons in first layers (e.g. oriented edges) are shared by almost all the images, while the features spiking shallow neurons are more selective features (e.g. faces) that only spike for specific images. The observation of the responses confirms the adequacy of our assumption to fix a minimum value for the activation and a maximum number of images to capture the most important activations for all the neurons. Similar observations have been made for other networks like VGG-S and VGG-F Chatfield et al. (2014) 1.
Thus, the NF is computed as:
NF (nL,i) = 1
Nmax Nmax∑ j=1 wj,i,LIj (1)
wherewj,i,L is the relative activation of the j-th cropped image, denoted as Ij , of the i-th neuron nL,i at layer L. The relative activation is the activation aj,i of a neuron, given a input image, with respect to its maximum activation obtained for any image, wj,i,L =
aj,i amax,i where amax,i = max ak,i,∀k.
In Fig. 2 we can see some NFs and their corresponding set of first 100 maximum activations, and in Fig. 3 (a) we can see a selected subset of 20 NF per layer. In this image we can identify specific shapes that display the intrinsic property that fires a single neuron. At first glance, we can see how in this particular network the first two layers are devoted to basic properties. Oriented edges of different frequencies and in different colors in the first layer; textures, blobs, bars and more specific curves in the second layer. The rest of the layers seem to be devoted to more complex objects. We can see that dog and human faces, cars and flowers are detected at different scales in different layers, since the size of the NF and their corresponding cropped images increase with depth. This visualization of the neuron activity can be seen as a way to visualize a trained vocabulary of the CNN that opens multiple ways to analyze the global behavior of the network from its single units. However, not all neurons present such a clear tuning to an identifiable shape. Some neurons present a blurred version of NF, such as, those in Fig. 3(b). The level of blurring is directly related to a high variability between the maximally activated images for a neuron.
At this point, we want to make a short parenthesis to relate the previous representational observations with the scientific problem about neural coding that is focus of attention in visual brain research (Kriegeskorte & Kreiman (2011)). We are referring to the hypothesis about distributed representations that encode object information in neuron population codes, that co-exist with strong evidences of neurons which are only activated by a very specific object. In line with this idea, we invite to speculate about neurons presenting a highly structured NF could be closer to localist code neurons while neurons with a blurred NF as closer to a distributed code. We return on this discussion later on at sections 4.3 and 5.
Finally, we want to add a further analysis about how neuron feature is related to the neuron activity is representing. In Fig. 4 we plot the level of the neuron responses when the input image is its own NF. We can observe a high degree of activation (in green) between the NF and the response of the net to this feature. However we have some disagreements between the NF and the neuron activations: an important example is shown in layer 2, that is curiously bigger than in layer 3 and 4. This is explained by the high number of dead neurons2 and also by a higher presence of texture selective neurons, that is observed in Fig. 3. Another example, which is more understandable, is the clear increase of disagreement that happens through layers 3, 4 and 5, that seems to be explained by an increase in invariance that is obvious when the size of the image increases.
1The results are shown for a maximum number of images equal to Nmax = 100 and a minimum activation value over a 70% of the maximum activation. We plot these values on Fig. 1
2By dead neurons we mean neurons whose activation for any input image is not enough to drive the subsequent ReLU.
3 NEURON SELECTIVITY INDEX
In this section we propose to describe neurons by their inherent response to a specific property, using an index. The index has to allow to rank them in a proportional order between their response and the existence of the property in the input image. Therefore, we translate the problem of describing neuron activity to the problem of proposing methods which are able to quantify specific image facets that correlate with the degree of activation of the neuron holding such a property. A selectivity index of a single unit is a flexible an independent method for discriminating or clustering between neurons inside the same network. Selectivity indexes can be defined either for image features or for image labels. In what follows, we propose two selectivity indexes one on each group.
3.1 COLOR SELECTIVITY INDEX
Color selectivity is a property that can be proved in specific neurons of the human brain. The level of activation of the neuron when the observer is exposed to a stimulus with a strong color bias, and its corresponding low activation when the color is not present, is the object of attention in vision research that pursuits the understanding of how color is coded in the human visual system (Shapley & Hawken (2011),Conway & Tsao (2009)).
Here we propose a method to compute a color selectivity index for neurons in artificial neural networks. We propose to base it directly on the image properties of the NF we have defined above. We quantify the selectivity to a specific chromaticity directly from the color distribution of the NF. We define this index as the angle between the first principal component (v) of the color distribution of the NF and the intensity axis (b) of the Opponent Color Space (OPP). To compute (v) we use a weighted Principal Component Analysis Delchambre (2014) that allows to strengthen the selectivity of small color areas. Weights are applied to each pixel in order to reinforce those pixels that are shared by most cropped images and that highly contribute to the NF. Therefore, the weights are the inverse of the standard deviation. In this way, a NF defined by cropped images with different colors will tend to be represented by a grayish image and its principal component will be close to the intensity axis in the OPP color space and it will receive a low selectivity index. We formulate this index (in degrees) as follows:
α(nL,i) = 1
90 arccos ( b · v ‖b‖‖v‖ ) (2)
Other selectivity indexes that can be derived from this, are those related to color attributes. We can easily extract color name labels using a color naming approach such as Benavente et al. (2008) and directly define color selectivity to basic names such as red, or green, among others.
3.2 CLASS SELECTIVITY INDEX
Class selectivity is a property of a neuron that can help to establish its discriminative power for one specific class or can allow to cluster neurons accordingly with the ontological properties of their class labels.
We propose a method to compute a class selectivity index for individual neurons by compiling the class labels of the images that maximally activates this neuron in a single descriptor. We define class selectivity from the set of class labels of the N images used to build the NF. To quantify this index we build the class label distribution of the full set of images. As in the color selectivity index, we weight the significance of a class label by the relative activation of its image. Thus, the relative frequency of each class c for a certain neuron is defined as:
fc(n i,L) = ∑Nc j wj,i,L∑N l wl,i,L
(3)
where Nc refers to the number of images, among the N cropped images activating this neuron, that belong to class c.
Given the densities for all the classes. Finally, our class selectivity index is defined as follows:
γ(nL,i) = N −M N − 1
(4)
whereM is the minimum number of classes that covers a pre-fixed ratio, th, of the neuron activation, this can be denoted as ∑M c fc ≥ th. This threshold allow to avoid considering class labels with very small activation weight. Jointly with the index value the selectivity provides the set of M classes that describe the neuron selectivity and their corresponding relative frequency values.
Therefore, a low class selectivity index indicates a poor contribution of this neuron to a single class (minimum is 0 when M = N ), while a high value (maximum is 1) indicates a strong contribution of this neuron to a single class. In between we can have different degrees of selectivity to different number of classes. Obviously, this index is irrelevant for the last fully connected layers in a CNN, but it allows to group related neurons across different convolutional layers.
Here we want to point out, that this index can also contribute to give some insights about the problem of how information is coded through layers, in the debate of localist and distributed neural codes we mentioned before (Kriegeskorte & Kreiman (2011)). Neurons with high class selectivity index should be in line with a localist code, while neurons with low class selectivity index should be part of a distributed code. This way the index is defined allow a large range of interpretations in between these two kinds of coding as it has been outlined in the visual coding literature.
4 RESULTS
In this section we report some empirical results to show how the proposed selectivity indexes perform and what representational conclusions we can extract from the subsets of neurons sharing indexed properties.
4.1 EXPERIMENTAL SETUP
In this paper we analyze the neurons of a CNN architecture trained on ImageNet ILSVRC dataset Deng et al. (2009) (using a subset of 1.2M images classified in 1.000 categories). We report the results for the VGG-M CNN that was trained by Chatfield et al. (2014) for a generic visual task of object recognition. The details of the CNN architecture are given in table 1. We selected this network since it has a similar structure to those which have been reported as having a representational performance that competes with human performance (as was proved in Cadieu et al. (2014)). Nevertheless, we have obtained similar results for VGG-F and VGG-S that are provided in Chatfield et al. (2014). We used the Matconvnet library provided by Vedaldi & Lenc (2015) for all the experiments.
4.2 COLOR SELECTIVITY
General purpose CNN architectures are usually trained on RGB color images. However there is a strong belief in the computer vision community that color is a dispensable property. The results we obtain by indexing color selective neurons make us conclude that there is no basis for such a belief. Results show that color is strongly entangled at all levels of the CNN representation. In a preliminary experiment we have tested a subset of ImageNet images with VGG-M in their original color and the same subset in a gray scale representation. Classification results show a considerable decrease: while original RGB images are classified with a 27.50% top-1 error and 10.14% top-5 error, gray scale image versions present 51.12% and 26.37% errors, top-1 and top-5 errors respectively.
In a first experiment we extract how many NFs are related to color in each convolutional layer using the proposed color selectivity index. The bars in Fig. 8 plot the relative quantity of neurons that are color selective compared to those that are not. Grey represents the ratio of neurons that do not spike for the presence of a color and reddish represent neurons that are highly activated by the presence of a color. In the graphic we can observe that shallow layers are the main responsible for the color representation on the images: 50% and 40% of neurons are color selective in layers conv1 and conv2, respectively. Nevertheless, we also still found around 25% of color selective neurons in in deeper layers. Therefore, although neurons in deeper layers tend to be color invariant, an important part of the representation is devoted to color, that reinforces the discriminative power of color in object recognition. In Fig. 6 we show some examples of NFs with different degrees of color selectivity at different layers of the network and showing the corresponding cropped images.
Regarding color representation in layer 1 we want to point out two more observations derived from the NFs (see Fig. 5): (a) selectivity to different spatial-frequencies is only tackled by gray-level neurons; and (b) four main color axis emerge (black-white, blue-yellow, orange-cyan and cyanmagenta). Curiously, these two observations correlate with evidences in the human visual system (Shapley & Hawken (2011)).
In a second experiment, we analyze how color selective neurons from all layers cover the color space. Figure 7 displays the distribution of color selective neurons with α ≥ 0.40. Each NF is plotted on the hue angle that represents the projection of its first principal component on the OPP chromaticity plane (red-green and blue-yellow components). Dashed rings identify different convolutional layers from conv1 (inner ring) to conv5 (outer ring) linking the NFs that belong to the same layer. We can appreciate the emergence of an axis (from orange to cyan) that connects a crowded area of color selective neurons. We can add a low population of NFs in the magenta area, that becomes more crowded on the opposite side where green and yellow selectivity has several neurons. The interest of this explanation relies on the fact that a similar distribution appears in the ImageNet color distribution, that is plotted at the bottom of the same images, where a similar interpretation in terms of emergent axes can be done. A more in depth study is required to prove this correlation, but we illustrate how neuron selectivity helps in the understanding of how a specific property is represented by the CNN.
4.3 CLASS SELECTIVITY
Following with the analysis of ranking neurons by their response to a certain property, here we focus on the proposed selectivity index that relates to image labels instead of to an image property, is the class selectivity index, which only applies for classification networks. We report the results of different experiments where we have fixed th = 1, which means we consider all the class labels for the N = 100 images that maximally activates the neuron. As we mentioned before, this index can enlighten how classes are encoded through the net layers, that again it can be related to the scientific problem of how general object recognition is encoded in the human brain. Here we hypothesize that the difference between localist or distributed codes could correlate with the idea of neurons highly selective to a single class and neurons highly selective to several classes, we resume on this later at section 5.
In a first experiment we analyze how many neurons present different degrees of class selectivity through layers. The bars in Fig. 9 plot the relative quantity of neurons that are class selective compared to those that are not. Grey represents the ratio of neurons that are not activated by a single class and bluish represent neurons that are highly activated by a single class. Opposite to what we showed about color selectivity, we found most of class selective neurons in deeper layers, and no class selectivity in shallow layers, as expected. We have moved from a very basic image property, color, to a very high level property, class label. This fact corroborates the idea that CNNs start by defining basic feature detectors that are share by most of the classes, and the neurons become more specialized when they belong to deeper layers representing larger areas in the image space and therefore more complex shapes. We start to have neurons with relevant class selectivity in layer conv3, where a 5% of neurons is quite class selective and we found some neurons with a degree of selective close to 1. These ratios progressively increase up to layer conv5 where we have more than a 50% of neurons with a class selectivity index greater than 0.6, that means that we have less than 40 different classes activating this neuron, which is a very selective ratio considering the number of classes of the ImageNet dataset. In the same layer a 20% of neurons present a high class selectivity index, than means less than 20 different classes. Further experiments should explore how this graphic evolves by moving from current class labels which are on the leaves of the ImageNet ontology towards higher nodes with more generic classes.
Secondly, we have visualized the properties of a set of images presenting different degrees of class selectivity in Fig. 10 for different levels of depth. We visualize each neuron with their NF visualization and the corresponding cropped images. We also show two tag clouds of each neuron. They visualize the importance of each class label. With an orange frame we plot the leave classes of the ImageNet ontology, while in the green frame we plot generic classes. This second analysis could help finding neurons that are specialized to a general semantic concept that different final classes share. Note that neurons with high class selectivity index have a set of cropped images that we can identify as belonging to the same class.
Finally we stress the utility of ranking images by selectivity indexes in Fig. 11, where we show interesting neurons in different convolutional layers that present high values for both selectivity indexes, neurons which are both, color and class selective.
5 CONCLUSIONS
In this paper we propose a framework to analyze a trained CNN by dissecting individual neurons using their indexes of selectivity to specific properties. We have proposed two properties of different nature: (a) color, that is a low-level image property that we have shown to be entangled in all the representations levels of the net; (b) class label, that is a high-level image property that can be analyzed at different levels of abstraction. We have shown that while the number of color selective neurons decreases with depth, the number of class selective neuron increases. In this line of describing the activity of individual images, we have also proposed to visualize the activity with what we have called the neuron feature (NF), that allows to arise interesting structures that are shared by the images that highly activate a neuron.
The proposed work have made us to speculate about two different ways to address the coding properties of individual neurons (localist versus distributed). Firstly, we have mentioned the possibility that a blurred NF, i.e. without a clear structure, belongs to a neuron that can be part of a distributed code
where the neuron does not represent a selectivity to a single shape, maybe to diverse shapes than can be part of a code in deeper neurons. Secondly, we speculate about the possibility that neurons with high class selective index can represent a localist code, and part of a distributed when is low. In parallel, the analysis of the color selective neurons have made to arise some parallelism between color representation in the 1st convolutional layer and known evidences about the representation in the human visual system.
As further work we need to fully exploit the potential of the indexes in different CNN architectures, and defining new selectivity indexes like shape or texture, that could be a perfect complement to current ones.
ACKNOWLEDGMENTS
Project funded by MINECO Ref. TIN (TIN2014-61068-R)

The authors analyze trained neural networks by quantifying the selectivity of individual neurons in the network for a variety of specific features, including color and category.
Pros:
   * The paper is clearly written and has good figures.
   * I think they executed their specific stated goal reasonably well technically.   E.g. the various indexes they use seem well-chosen for their purposes.
Cons:
   * I must admit that I am biased against the whole enterprise of this paper.   I do not think it is well-motivated or provides any useful insight whatever.   What I view their having done is produced, and then summarized anecdotally, a catalog of piecemeal facts about a neural network without any larger reason to think these particular facts are important.  In a way, I feel like this paper suffers from the same problem that plagues a typical line of research in neurophysiology, in which a catalog of selectivity distributions of various neurons for various properties is produced -- full stop.  As if that were in and of itself important or useful information.   I do not feel that either the original neural version of that project, or this current model-based virtual electrophysiology, is that useful.   Why should we care about the distribution of color selectivities?   Why does knowing distribution as such constitute "understanding"?    To my mind it doesn't, at least not directly.
Here's what they could have done to make a more useful investigation:

     (a) From a neuroscience point of view, they could have compared the properties that they measure in models to the same properties as measured in neurons the real brain.   If they could show that some models are better matches on these properties to the actual neural data than others, that would be a really interesting result.   That is is to say, the two isolated catalogs of selectivities (from model neurons and real neurons)  alone seem pretty pointless.  But if the correspondence between the two catalogs was made -- both in terms of where the model neurons and the real neurons were similar, and (especially importantly) where they were different --- that would be the beginning of nontrivial understanding.   Such results would also complement a growing body of literature that attempts to link CNNs to visual brain areas.  Finding good neural data is challenging, but whatever the result, the comparison would be interesting.
and/or
    (b) From an artificial intelligence point of view, they could have shown that their metrics are *prescriptive* constraints.   That is, suppose they had shown that the specific color and class selectivity indices that they compute, when imposed as a loss-function criterion on an untrained neural network, cause the network to develop useful filters and achieve significantly above-chance performance on the original task the networks were trained on.     This would be a really great result, because it would not only give us a priori reason to care about the specific property metrics they chose, but it would also help contribute to efforts to find unsupervised (or semi-supervised) learning procedures, since the metrics they compute can be estimated from comparatively small numbers of stimuli and/or high-level semantic labels.    To put this in perspective, imagine that they had actually tested the above hypothesis and found it to be false:  that is, that their metrics, when used as loss function constraints, do not improve performance noticeably above chance performance.  What would we then make of this whole investigation?  It would then be reasonable to think that the measured properties were essentially epiphenomenal and didn't contribute at all to the power of neural networks in solving perceptual tasks.  (The same could be said about neurophysiology experiments doing the same thing.)
     [--> NB: I've actually tried things just like this myself over the years, and have found exactly this disappointing result.  Specifically,  I've found a number of high-level generic statistical property of DNNs that seem like they might potentially "interesting", e.g. because they apparently correlate with complexity or appear to illustrate difference between low, intermediate and high layers of DNNs.  Every single one of these, when imposed as optimization constraints, has basically lead nowhere on the challenging tasks (like ImageNet) that cause the DNNs to be interesting in the first place.  Basically, there is to my mind no evidence at this point that highly-summarized generic statistical distributions of selectivities, like those illustrated here, place any interesting constraints on filter weights at all.   Of course, I haven't tried the specific properties the authors highlight in these papers, so maybe there's something important there.]
I know that both of these asks are pretty hard, but I just don't know what else to say -- this work otherwise seems like a step backwards for what the community ought to be spending its time on.


BOOSTED GENERATIVE MODELS
We propose a new approach for using boosting to create an ensemble of generative models, where models are trained in sequence to correct earlier mistakes. Our algorithm can leverage many existing base learners, including recent latent variable models. Further, our approach allows the ensemble to leverage discriminative models trained to distinguish real from synthetic data during sample generation. We show theoretical conditions under which incorporating a new model to the ensemble will improve the fit and empirically demonstrate the effectiveness of boosting on density estimation, sample generation, and unsupervised feature learning on real and synthetic datasets.
1 INTRODUCTION
Many of the recent successful applications of machine learning in computer vision, speech recognition, and natural language processing are based on discriminative models. Learning generative models has proven to be much more difficult. Deep architectures, including latent variable models such as Boltzmann machines (Smolensky, 1986), variational autoencoders (Kingma & Welling, 2014), and generative adversarial networks (Goodfellow et al., 2014), have recently shown great success. Despite significant progress, existing generative models cannot fit complex distributions with a sufficiently high degree of accuracy.
In this paper, we propose a technique for ensembling (imperfect) generative models to improve their overall performance. Our meta-algorithm is inspired by boosting, a powerful technique used in supervised learning to construct ensembles of weak classifiers (e.g., decision stumps or trees), which individually might not perform well on a given classification task. The boosting algorithm will attempt to learn a classifier to correct for the mistakes made and repeat this procedure recursively. Under some conditions on the weak classifiers’ effectiveness, the boosting meta-algorithm can drive the (training) error to zero (Freund et al., 1999). Boosting can also be thought as a feature learning algorithm, where at each round a new feature is learned by training a classifier on a re-weighted version of the original dataset. In practice, algorithms based on boosting (such as boosted trees) perform extremely well in machine learning competitions (Caruana & Niculescu-Mizil, 2006).
We show that a similar procedure can be applied to generative models. Given an initial generative model that provides an imperfect fit to the data distribution, we construct a second model to correct for the error, and repeat recursively. The second model is also a generative one, which is trained on a re-weighted version of the original training set. Our meta algorithm is general and can construct ensembles of many existing generative models such as restricted Boltzmann machines and variational autoencoders. Surprisingly, our method can even leverage discriminative models, which have been shown to perform extremely well in practice (Krizhevsky et al., 2012; LeCun et al., 2015). Specifically, we train a discriminator to distinguish true data samples from “fake” ones generated by
the current model and provide a principled way to include this discriminator in the ensemble. We also provide conditions under which adding a model to the ensemble is guaranteed to improve the fit and recover the true data distribution under idealized conditions.
To evaluate our algorithmic framework, we learn several ensembles of weakly-trained generators and discriminators and test them on popular use cases of generative models: density estimation, sample generation, and unsupervised feature learning. We show how boosted generative models can outperform baseline models without any additional computation cost.
2 BOOSTING GENERATIVE MODELS
Boosting is an ensembling technique for supervised learning, providing an algorithmic formalization of the hypothesis that a sequence of weak learners can create a single strong learner (Schapire & Freund, 2012). In this section, we propose a framework that extends boosting to unsupervised settings for learning joint distributions using generative models. For ease of presentation, all distributions are w.r.t. any arbitrary x ∈ Rd, unless otherwise specified. Formally, we consider the following maximum likelihood estimation (MLE) setting. Given some data points X = {xi ∈ Rd}mi=1 sampled i.i.d. from an unknown distribution with p.d.f. p, we provide a model class Q parametrizing the distributions that can be represented by the generative model and minimize the KL-divergence w.r.t. the true distribution,
min q∈Q
DKL(p ‖ q). (1)
In practice, we only observe samples from p and hence, maximize the log-likelihood of the observed data X . Selecting the model class for maximum likelihood learning is non-trivial; the maximum likelihood estimate w.r.t. a small class can be very far from the true distribution whereas a large class poses the risk of overfitting.
2.1 FACTORED LIKELIHOODS FOR UNSUPERVISED BOOSTING
In unsupervised boosting, we factorize the joint distribution specified by a generative model as a geometric average of T +1 intermediate model distributions {ht}Tt=0, each assigned an exponentiated weight αt,
qT =
∏T t=0 h αt t
ZT where the partition function ZT = ∫ ∏T t=0 h αt t dx. Because a joint optimization over all the intermediate model distributions and weights is computationally prohibitive, we instead perform a greedy optimization at every round. The joint distribution of a boosted generative model (BGM) can be recursively expressed as,
q̃t = h αt t · q̃t−1 (2)
where q̃t is the unnormalized BGM distribution (at round t). The base model distribution h0 is learned using the maximum likelihood principle. Given suitable weights, we now derive conditions on the intermediate model distributions that allow us to make “progress” in every round of boosting.
Theorem 1. Let δtKL(ht, αt) = DKL(p ‖ qt−1) − DKL(p ‖ qt) denote the reduction in KL divergence at the tth round of boosting. Then, for all 0 ≤ αt ≤ 1, the following conditions hold,
1. Sufficient: If Ep[log ht] ≥ logEqt−1 [ht], then δtKL(ht, αt) ≥ 0.
2. Necessary: If δtKL(ht, αt) ≥ 0, then Ep[log ht] ≥ Eqt−1 [log ht].
Proof. See Appendix A.1.1.
Algorithm 1 DiscBGM(X = {xi}mi=1, T rounds) Initialize D0(xi) = 1/m for all i = 1, 2, . . . ,m. Set (unnormalized) density estimate q̃0 = h0 Train generative model h0 to maximize Exi∼D0 [log h0(xi)]
for t = 1, . . . , T do • Generate k negative samples from qt−1 • Train discriminative model dt to maximize Exi∼D0 [log dt] + Exi∼qt−1 [log(1− dt)]. • Set ht = γ · dt1−dt where γ = k/m. • Choose αt. • Set (unnormalized) density estimate q̃t = hαtt · q̃t−1. end for
Estimate ZT = ∫ q̃T (x)dx. return qT = q̃T /ZT
The equality in the above conditions holds true for αt = 0 which corresponds to the trivial case where the intermediate model distribution in the current round is ignored in the BGM distribution in Eq. (2). For all other valid αt > 0, the non-degenerate versions of the sufficient inequality guarantees progress towards the true data distribution. Note that the intermediate models increase the overall capacity of the BGM at every round.
From the necessary condition, we see that a “good” intermediate density ht necessarily assigns a better-or-equal log-likelihood under the true desired distribution as opposed to the BGM distribution in the previous round, qt−1. This condition suggests two learning objectives for intermediate models which we discuss next.
2.2 DISCRIMINATIVE APPROACH FOR BOOSTING GENERATIVE MODELS
In the discriminative approach for boosting generative models, the intermediate model distribution is specified as the odds ratio of a binary classifier. Specifically, consider the following binary classification problem: we observe m samples drawn i.i.d. from the true data distribution p (w.l.o.g. assigned the label y = +1), and k samples drawn i.i.d. from the BGM distribution in the previous round qt−1 (assigned the label y = −1). The objective of the binary classifier is to learn a conditional distribution dt ∈ Dt that maximizes the cross-entropy,
max dt∈Dt
Ex∼p[log dt] + Ex∼qt−1 [log(1− dt)]. (3)
Definition 1. If ut denotes the joint distribution over (x, y) at round t, then a binary classifier with density dt is Bayes optimal iff,
dt(x) = ut(y = +1 | x). Theorem 2. If a binary classifier dt trained to optimize Eq. (3) is Bayes optimal, then the BGM distribution at the end of the round will immediately converge to the true data distribution if we set αt = 1 and
ht = γ · dt
1− dt (4)
where γ = k/m.
Proof. See Appendix A.1.2.
In practice, a classifier with limited capacity trained on a finite dataset will not generally be Bayes optimal. The above theorem, however, suggests that a good classifier can provide a “direction of improvement”. Additionally, if the intermediate model distribution ht obtained using Eq. (4) satisfies the conditions in Theorem 1, it is guaranteed to improve the BGM distribution.
Algorithm 2 GenBGM(X = {xi}mi=1, T rounds)
Initialize D0(xi) = 1/m for all i = 1, 2, . . . ,m. Train generative model h0 to maximize Exi∼D0 [log h0(xi)]. Set (unnormalized) density estimate q̃0 = h0
for t = 1, 2, . . . , T do • Update Dt using Eq. (5). • Train generative model ht to maximize Exi∼Dt [log ht(xi)]. • Choose αt. • Set (unnormalized) density estimate q̃t = q̃t−1 · hαtt . end for
Estimate ZT = ∫ q̃T (x)dx. return qT = q̃T /ZT .
The pseudocode for the corresponding boosting algorithm DiscBGM is given in Algorithm 1. At every round of boosting, we train a binary classifier to optimize the objective in Eq. (3). Note that the BGM distributions at the intermediate boosting rounds can be specified up to a normalization constant if samples from the previous BGM distribution are generated via MCMC sampling. If the partition function is required, it can be estimated using techniques such as Annealed Importance Sampling (Neal, 2001).1
The weights 0 ≤ αt ≤ 1 can be interpreted as our confidence in the classifier density estimate. While in practice we use heuristic strategies for assigning weights to the intermediate models, the greedy optimum value of these weights at every round is a critical point for δtKL (defined in Theorem 1). We consider a few special cases below.
• If dt is uninformative, i.e., dt ≡ 0.5, then δtKL(ht, αt) = 0 for all 0 ≤ αt ≤ 1. • If dt is Bayes optimal, then δtKL attains a maxima at α?t = 1. (Theorem 2). • For a completely adversarial classifier w.r.t. the Bayes optimality criteria, i.e., dt(x) = u(y = −1|x), we have the following result.
Corollary 1. If dt is completely adversarial, then δtKL attains a maxima of zero at α?t = 0.
Proof. See Appendix A.1.3.
2.3 GENERATIVE APPROACH FOR BOOSTING GENERATIVE MODELS
In the greedy optimization framework of unsupervised boosting, we want to learn an intermediate model distribution at every round that maximizes δtKL when factored as a product with the BGM distribution in the previous round. In the generative approach, the intermediate model specifies a ratio of densities and maximizes the log-likelihood of data sampled from a reweighted data distribution,
max ht Ex∼Dt [log ht]
where Dt ∝ p
qt−1 . (5)
The pseudocode for the corresponding unsupervised boosting algorithm, GenBGM is is given in Algorithm 2. Starting with a uniform distribution overX , GenBGM learns an intermediate model at every round that maximizes the log-likelihood of data sampled from a reweighted data distribution.
1For many applications of generative models such as sample generation and feature learning, we can sidestep computing the partition function.
2.4 GENERATIVE-DISCRIMINATIVE APPROACH FOR BOOSTING GENERATIVE MODELS
Intermediate models need not be exclusively generators or discriminators as in Algorithm 1 and Algorithm 2; we can design a boosting algorithm that uses any ensemble of generators and discriminators as intermediate models. If an intermediate distribution is required to be a generator, we train a generative model by appropriately reweighting our training set. If a discriminator odds ratio is used to specify an intermediate distribution, we set up the corresponding binary classification problem.
In practice, we want BGMs to generalize to data points outside the training set X. Regularization in BGMs is imposed primarily in two ways. First, every intermediate model can be independently regularized by incorporating explicit terms in the learning objective, early stopping of training based on validation error, specialized techniques such as dropout, etc. Moreover, regularization in BGMs is also imposed by restricting the number of rounds of boosting. If the intermediate models are exseveral applications of pressive enough, then very few rounds of boosting are required. We now do an empirical study of BGMs for several applications of generative modeling.
3 EMPIRICAL EVALUATION
We evaluated the performance of BGMs as a general-purpose meta-algorithm for generative modeling applications on real and synthetic datasets for three tasks: density estimation, sample generation, and unsupervised feature learning for downstream semi-supervised classification.
3.1 DENSITY ESTIMATION
A common pitfall with training generative models is model misspecification w.r.t. the true underlying data distribution. To illustrate how BGMs can effectively correct for model misspecification, we consider density estimation on synthetic data. The true data distribution in Figure 1 (a) is a equi-weighted mixture of four Gaussians centered symmetrically around the origin, each having an identity covariance matrix. We only observe 1,000 samples drawn i.i.d. from the data distribution (shown as black dots), and the task is to learn this distribution.
Experimental setup. As a baseline (misspecified) model, we fit a mixture of two Gaussians to the data shown in Figure 1 (b). We evaluate the following ensembling techniques.
1. Bagging. Bagging works just like GenBGM but without any reweighting at every round of ensembling. The intermediate generative models are mixtures of two Gaussians.
2. GenBGM (Algorithm 2). The intermediate models are mixtures of two Gaussians.
3. DiscBGM (Algorithm 1). The binary classifiers used to specify the intermediate models are support vector machines (SVMs) with a radial basis function kernel.
In all ensembles, equal weights are heuristically assigned to every model such that ∑Ti=1αi = 1. For the bagging and GenBGM approaches, ensembling is stopped after T = 3 rounds when the addition of a new model does not result in any significant change in the density estimate. For the DiscBGM approach, ensembling is stopped after T = 15 rounds to prevent overfitting.
Results and discussion. The contour plots for the density estimated by the three approaches are shown in Figure 1. While the bagging approach is not very effective, GenBGM and DiscBGM steer the initial misspecified distribution towards the true distribution. DiscBGM is more conservative in assigning density mass to outliers and requires more rounds of boosting as opposed to GenBGM.
3.2 SAMPLE GENERATION
In this task, we generate samples from the learned BGM models and visually inspect their quality. We consider sample generation for the binarized MNIST handwritten digits dataset (LeCun et al., 2010), which contains 50,000 train, 10,000 validation, and 10,000 test images of dimensions 28×28. Experimental setup. Boosting is a particularly attractive framework for improving weak learners. For a baseline generative model, we consider the following two latent variable models.
Variational Autoencoder (Kingma & Welling, 2014). VAEs are directed models with continuous latent units where the posterior over the latent units is specified using a neural network. We use the evidence lower bound as a proxy for approximately evaluating the log-likelihood during learning.
Restricted Boltzmann Machine (Smolensky, 1986). RBMs are undirected 2-layer models with discrete latent units such that the latent and visible layer form a fully-connected bipartite graph. In RBMs, the log-likelihood can be tractably computed up to a normalization constant and learning is done using contrastive divergence (Hinton, 2002).
We compare the baseline models with several BGMs. The starting distribution h0 for a BGM is specified using a VAE or an RBM with the same model specification and learning procedure as the baseline models. However, when baseline models are used on their own (T = 0, i.e., no boosting) the training is run for 50 epochs. It is reduced to 10 epochs when the base model is used to specify
a BGM with T > 0, in an attempt to ensure fairness in terms of computation. The intermediate distributions are specified either using a VAE or RBM, or through a Convolutional Neural Network (CNN) (LeCun & Bengio, 1995) that performs binary classification.
The model architectures, learning procedure, and hyperparameters for the VAE, RBM, and CNN are described in Appendix A.2. The boosting sequences we consider are as follows.
1. T = 0: Baseline VAE, Baseline RBM.
2. T = 1: VAE→ VAE, RBM→ RBM, VAE→ RBM, VAE→ CNN, RBM→ CNN.
3. T = 2: VAE→ CNN→ RBM.
The weights, α’s, at every round are set to unity for all BGMs, with exceptions made in the cases of RBM→ RBM (α0 = 0.1, α1 = 0.9) and VAE→ RBM (α0 = 0.3, α1 = 0.7) where the tuned weights offered significant performance enhancements over the default setting. For the baseline models, we use the respective customary sampling technique, i.e., forward sampling for VAEs and blocked Gibbs sampling for RBMs. Samples from the BGMs are generated by running a Markov chain using the Metropolis-Hastings algorithm with a discrete, uniformly random proposal and the BGM distribution as the stationary distribution for the chain.
Results and discussion. The samples generated by the baseline models and BGM models are shown in Figure 2. While BGMs significantly improve over baselines models in all cases, evaluating the relative performance of the intermediate models purely based on the samples is hard since these models have different architectures and parameter settings.
Learning in BGMs is also computationally efficient. We show the wall-clock time taken to learn these models in Table 1. In many cases, BGMs generate significantly better looking samples with a lower training time compared to the baseline models. A key observation that emerges from the results is that discriminators are more computationally expensive since they require MCMC sampling from the previous generative model distribution as opposed to using intermediate generative models, which only require reweighting of the training set.
3.3 UNSUPERVISED FEATURE LEARNING
Latent variable models are particularly attractive for unsupervised feature learning since they directly learn hidden representations that model interdependencies between the data variables. In this task, we evaluate the latent representations learned by BGMs for semi-supervised classification on the MNIST dataset consisting of 10 classes.
Setup. We consider the same baselines as before and compare against BGM sequences that have more than a single generator. For the BGM sequences, we concatenate the parameters specifying the posterior over the latent variables (conditioned on the observed variables) in the intermediate models to form a feature representation which we then feed into a transductive-SVM. For the transductiveSVM, we use a publicly available implemention (Joachims, 1999) with a linear kernel and all other parameters set to their default values. Due to computational constraints on the classification procedure, we experiment with subsets of the training dataset and perform semi-supervised classification on a class-balanced sampling of 1,000 and 2,000 training data points varying the number of labelled instances per class from 5 to 80. The procedure is repeated 10 times for statistical significance.
Results and discussion. The classification accuracies are show in Figure 3. We observe that BGMs closely match, and in some cases outperform the representations learned by baseline models in spite of making fewer passes over the data. A likely explanation of this phenomena is due to the fact that the learning objective for intermediate models is aware of the BGM distribution at the previous
round, and hence, is more computationally efficient in modeling specific regions of the underlying distribution that are not covered by the BGM distribution in the previous round.
While it is difficult to make general statements about the intermediate models (which are likely to be dataset specific), a surprising observation is that representations learned by the sequence VAE→ CNN → RBM are weaker for classification purposes than the representations learned by a similar VAE→ RBM sequence. The likely explanation for this observation is that having a generator later in the sequence offers diminishing advantage from the perspective of feature learning, assuming the previous intermediate models are making progress in modeling the underlying true distribution.
4 DISCUSSION AND RELATED WORK
In this work, we revisited boosting, a meta-algorithmic framework developed in response to a seminal question posed by Kearns & Valiant (1994): can a set of weak learners create a strong learner? For the supervised learning problem, boosting has offered interesting theoretical insights into the fundamental limits of learning and led to the development of practical algorithms that work well in practice (Schapire, 1990; Freund et al., 1999; Friedman, 2002; Caruana & Niculescu-Mizil, 2006).
The algorithmic framework we propose in this work builds on the insights offered by prior work in boosting, yet is significantly different as the motivation is to learn generative models in unsupervised settings. In order to do so, we first defined an appropriate objective function for the generative model. We considered the standard log-loss because of its tight connections with the maximum likelihood principle. In the supervised setting, Lebanon & Lafferty (2002) have shown theoretical results connecting the log-loss for exponential families and the exponential loss minimized by AdaBoost (Freund et al., 1999). Subsequently, we showed how we can greedily optimize a factored generative model as a sequence of intermediate models. Finally, we incorporated the boosting intuition to develop an algorithmic framework where the intermediate models are tightly coupled with the previous models in the sequence and yet can be efficiently learned in practice.
In the context of unsupervised learning, recent advancements in deep generative models have significantly improved our ability to model high-dimensional distributions. For example, highly expressive models such as pixel-RNNs (Oord et al., 2016) and ladder networks (Rasmus et al., 2015) exhibit state-of-the-art performance in generating natural images and semi-supervised learning respectively. The flexibility in choosing intermediate models in BGMs allows for potential integration of these models in our proposed framework.
There has also been a renewed interest in the use of density ratios to distinguish data samples from the model samples. This unsupervised-as-supervised learning approach was first proposed by Friedman et al. (2001) and forms the basis for using binary classifiers for specifying intermediate models in Algorithm 1. The approach has subsequently been applied elsewhere, including parameter estimation in unnormalized models (Gutmann & Hyvärinen, 2010). Tu (2007)’s approach for generative modeling is closely related to Algorithm 1, but fails to account for imperfections in learning of discriminative models, and the ability to incorporate generative models alongside discriminative models. Hybrid generative-discriminative classifiers have been applied to supervised settings (Truyen et al., 2006; Grabner et al., 2007; Negri et al., 2008; Ratner et al., 2016).
Recently, the unsupervised-as-supervised learning approach has been successfully applied for sample generation in generative adversarial networks (GAN) (Goodfellow et al., 2014). GANs consist of a pair of generative-discriminative networks. While the discriminator maximizes the conditional entropy as in Eq. (3), the generator minimizes the same objective. For a parametric class of generative and discriminative networks, the stationary point is a saddle point i.e., a local minima for the generator and a local maxima for the discriminator. Accordingly, the GAN objective is not guaranteed to converge, and stable training of GANs is difficult in practice (Goodfellow, 2014). Additionally, GANs do not explicitly represent the likelihood of the generative model limiting their applicability, and Parzen window estimates of the model’s log-likelihood (Breuleux et al., 2011) can be misleading (Theis et al., 2016).
Borrowing terminology from Mohamed & Lakshminarayanan (2016), our work fits into the framework of prescribed probabilistic models that provide an explicit characterization of the loglikelihood and can still benefit from the unsupervised-as-supervised learning approach by incorporating intermediate models specified using discriminative approaches.
5 CONCLUSION
We presented a general-purpose framework for boosting generative models by explicit factorization of the model likelihood as a product of simpler intermediate model distributions. These intermediate model distributions are learned greedily using discriminative or generative approaches, gradually increasing the overall model’s capacity. We demonstrated the effectiveness of boosted generative models by designing ensembles of weakly trained variational autoencoders, restricted Boltzmann machines, and convolutional neural networks. Our ensembles improve upon baseline generative models on density estimation, sample generation, and unsupervised feature learning without incurring any significant computational overhead.
As a future work, we would like to apply our framework to more sophisticated models on complex datasets such as natural images. The optimal weighting for intermediate models also remains an open question to explore in future work. Finally, in the proposed framework, an intermediate model specified using a discriminator requires MCMC sampling from the BGM distribution at the previous round. This can be expensive, and future work could explore the design of more efficient strategies.
A APPENDICES
A.1 PROOFS
A.1.1 THEOREM 1
Proof. We first derive the sufficient condition,
δtKL(ht, αt) = ∫ p log qt dx− ∫ p log qt−1 dx
= ∫ p log
hαtt · qt−1 Zt
− ∫ p log qt−1 (using Eq. (2))
= αt · Ep[log ht]− logEqt−1 [hαtt ] (6) ≥ αt · Ep[log ht]− logEqt−1 [ht]αt (Jensen’s inequality) = αt · [ Ep[log ht]− logEqt−1 [ht]
] ≥ 0 (by assumption).
Note that if αt = 1, the sufficient condition is also necessary. For the necessary condition, 0 ≤ δtKL(ht, αt) = αt · Ep[log ht]− logEqt−1 [hαtt ]
≤ αt · Ep[log ht]− Eqt−1 [log hαtt ] (Jensen’s inequality) = αt · [Ep[log ht]− Eqt−1 [log ht]] (Linearity of expectation) ≤ Ep[log ht]− Eqt−1 [log ht] (since 0 ≤ αt ≤ 1).
A.1.2 THEOREM 2
Proof. For the proposed binary classification problem, since the m positive training examples are sampled from p and the k negative training examples are sampled from qt−1,
p = u(x|y = +1) u(y = +1) = m m+ k
(7)
qt−1 = u(x|y = −1) u(y = −1) = k
m+ k . (8)
The Bayes optimal density d can be expressed as, dt = u(y = +1 | x) (from Definition 1) = u(x | y = +1)u(y = +1)/u(x). (9) Similarly,
1− dt = u(x | y = −1)u(y = −1)/u(x). (10) From Eqs. (7- 10), we have,
γ · dt 1− dt = p qt−1 .
Finally from Eq. (2), qt = qt−1 · hαtt
= p
finishing the proof.
A.1.3 COROLLARY 1
Proof. For a completely adversarial classifier w.r.t. Bayes optimality, dt = u(x | y = −1)u(y = −1)/u(x) (11)
1− dt = u(x | y = +1)u(y = +1)/u(x). (12) From Eqs. (7,8, 11,12),
ht = γ · dt
1− dt = qt−1 p .
Substituting the above intermediate model distribution in Eq. (6), δtKL(ht, αt) = αt · Ep [ log
qt−1 p
] − logEqt−1 [ qt−1 p ]αt ≤ αt · Ep [ log
qt−1 p
] − Eqt−1 [ αt · log
qt−1 p
] (Jensen’s inequality)
= αt · [ Ep [ log
qt−1 p
] − Eqt−1 [ log
qt−1 p
]] (Linearity of expectation)
= −αt [DKL(p ‖ qt−1) +DKL(qt−1 ‖ p)] ≤ 0 (DKL is non-negative).
By inspection, the equality holds when αt = 0 finishing the proof.
A.2 MODEL ARCHITECTURES AND PARAMETER SETTINGS
The baseline VAE model consists of a deterministic hidden layer with 500 units between the visible layer and stochastic hidden layer with 50 latent variables. The inference network specifying the posterior also contains a single deterministic layer with 500 units. The prior over the latent variables is standard Gaussian, the hidden layer activations are tanh, and learning is done using Adam (Kingma & Ba, 2015) with a learning rate of 10−3 and mini-batches of size 100.
The baseline RBM model consists of 250 hidden units trained for 50 epochs using Stochastic Gradient Descent with a learning rate of 5× 10−2, mini-batches of size 100, and 15 steps of contrastive divergence.
The CNN contains two convolutional layers and a single full connected layer with 1024 units. Convolution layers have kernel size 5×5, and 32 and 64 output channels, respectively. We apply ReLUs and 2× 2 max pooling after each convolution. The net is randomly initialized prior to training, and learning is done for 2 epochs using Adam (Kingma & Ba, 2015) with a learning rate of 10−3 and mini-batches of size 100.

This paper extends boosting to the task of learning generative models of data. The strong learner is obtained as a geometric average of “weak learners”, which can themselves be normalized (e.g. VAE) or un-normalized (e.g. RBMs) generative models (genBGM), or a classifier trained to discriminate between the strong learner at iteration T-1 and the true data distribution (discBGM). This latter method is closely related to Noise Contrastive Estimation, GANs, etc.
The approach benefits from strong theoretical guarantees, with strict conditions under which each boosting iteration is guaranteed to improve the log-likelihood. The downside of the method appears to be the lack of normalization constant for the resulting strong learner and the use of heuristics to weight each weak learner (which seems to matter in practice, from Sec. 3.2). The discriminative approach further suffers from an expensive training procedure: each round of boosting first requires generating a “training set” worth of samples from the previous strong learner, where samples are obtained via MCMC.
The experimental section is clearly the weak point of the paper. The method is evaluated on a synthetic dataset, and a single real-world dataset, MNIST: both for generation and as a feature extraction mechanism for classification. Of these, the synthetic experiments were the clearest in showcasing the method. On MNIST, the baseline models are much too weak for the results to be convincing. A modestly sized VAE can obtain 90 nats within hours on a single GPU, clearly an achievable goal. Furthermore, despite arguments to the contrary, I firmly believe that mixing base learners is an academic exercise, if only because of the burden of implementing K different models & training algorithms. This section fails to answer a more fundamental question: is it better to train a large VAE by maximizing the elbow, or e.g. train 10 iterations of boosting, using VAEs 1/10th the size of the baseline model ? Experimental details are also lacking, especially with respect to the sampling procedure used to draw samples from the BGM. The paper would also benefit from likelihood estimates obtained via AIS.
With regards to novelty and prior work, there is also a missing reference to “Self Supervised Boosting” by Welling et al [R1]. After a cursory read through, there seems to be strong similarities to the GenBGM approach which ought to be discussed.
Overall, I am on the fence. The idea of boosting generative models is intriguing, seems well motivated and has potential for impact. For this reason, and given the theoretical contributions, I am willing to overlook some of the issues highlighted above, and hope the authors can address some of them in time for the rebuttal.
[R1]