michaelklachko:

I’ve never actually said anything about weight sharing. I was merely pointing out that unsupervised pre-training for classification is unnecessary, and that AEs can be trained end to end, rather than one layer at a time. This has nothing to do with weight sharing.


Got it! Makes sense. Any chance you are aware of any experiments that show this? Would be curious to read, no worries if you don’t have these handy.



 michaelklachko:

More importantly, I’m not sure if you can share weights when you use convolutional layers in encoder and decoder: could the same filters be used for both convolution and “deconvolution” purposes? Do you know any papers that did it?


Weight sharing is not possible at all in convolution-deconvolution scenario, as far as I understand it. Which is probably the reason why weight sharing has fallen out of favor in feedforward AEs as well. Even though I could not find any single references for this, I’ve noticed that at some point the feedforward AEs implementations moved away from weight sharing.