Expectation learning is a continuous unsupervised learning process which uses multisensory bindings to modulate unisensory perception. As humans, we learn to associate a barking sound with the visual appearance of a dog, and we continuously fine-tune this association over time, as we learn, e.g., to associate highpitched barking with small dogs. In this work, we address the problem of building a computational model that captures two important properties of expectation learning, namely continuity and the lack of any external supervision other than temporal co-occurrence. To this end, we present a novel hybrid neural model based on audio/visual autoencoders and a recurrent self-organizing network for stimulus reconstruction and multisensory binding. We demonstrate that the proposed model is capable of learning concept bindings, i.e. dog barking with dogs, by evaluating it on unisensory classification tasks for audi-visual stimuli using the 43,500 Youtube videos in the animal subset of the AudioSet corpus. In addition, our analysis and discussion explain how the expectation learning mechanism enforces the generation of high-level bindings and how they contribute to audiovisual recognition.
The authors proposed an unsupervised learning framework to learn multisensory binding, using visual and auditory domain from animal videos as example. First, the visual and auditory inputs are autoencoded, and these latent codes are binding using a recurrent self-organizing network (Gamma-GWR). Furthermore, the authors proposed the expectation learning idea, which is inspire by psychology literature. In short, after the first pass of training using the real data. The authors fine tuned the model to bind the real data from one domain and the reconstructed data from another domain. This could be a good idea, as the authors pointed out, human usually bind all kinds of yellow bird to a same mental 'chirping' sounds. So, this expectation learning could potentially group the representation to a canonical one. Also, the authors showed in Table 1 that with the expectation learning, the model's recognition accuracy is improved a bit. I think it would be interesting to show the reconstruction output example (as in Fig. 3) for both model with and without expectation learning. To see if it is as the authors claim, that the model with expectation learning is reconstructing the missing modality with more canonical images/sounds. (This may not be the goal in other practice, though I'm convinced it is a potentially good psychological model as it explain well the multisensory imagery effect (Spence & Deroy, 2013). 
I found this manuscript quite hard to follow though. The description seems sometime not flowing very smoothly. And there are some clear typos and mess up of math notations make the reading unpleasant. I have noted down several points below, and hope the authors could improve in the next iteration.
1. The description of variational autoencoder is not well written. The citation (Chen, 2016) is not the standard VAE paper people usually cite (unless the author is adopting something specific from the Chen's paper.). For example, the authors wrote "the KL divergence between the encoded representation and a sample from the Gaussian distribution" which sounds incorrect to me.
2. Why a Variational autoencoder is necessary for visual domain, but a regular autoencoder is used in auditory domain?
Typos:
1. page 2, 2nd line: a online --> an online
2. Use subscript I-1 to mean the winner neuron at t-1, I think this is not quite clear. I suggest to follow the notation in (Parisi & Wermter 2017), use I(t-1), which is easier to follow.
3. page 7, 2nd line: more than 17% for audio.  -> for vision.
4. page 8, 3rd line: not on the original high-abstraction data. Do the authors mean highly specific data? That seems make more sense.
5. Several notation mismatch here and there. for example, in formula 6 it is w_j^s, but in the text below it become w_{j,s}.