No sweat! This makes me think that the implementation of virtual batch normalization on the SEGAN github is wrong although it trains the network properly. Can you confirm? I don’t see anywhere multiple forward-passes on the reference batch!
On 1 they define the VBN class. On 2 they define the graph through which the hidden state of the layer goes through the VBN layer before the non-linearity. On 3 they instantiate the VBN class if it’s the first batch seen otherwise they call VBN on the current hidden state.


https://github.com/santi-pdp/segan/blob/master/bnorm.py 12


https://github.com/santi-pdp/segan/blob/1774069184e3e9666170503a201f46969754459e/discriminator.py#L47 2


https://github.com/santi-pdp/segan/blob/b2463fc5172f6ac1fcb0e89313be953c2dc0138e/model.py#L297 6