Isn’t that odd? I see two scenarios: 1)encoder throws away information needed for classification 2) your two tasks are conflicting, wanting different rerpesentations. Basically, you should check if that classifier trains on its own (with c_loss as the only loss).



 Anuj_Daga:

if possible can you tell me how to implement ELBO loss you mentioned above, it will really helpful if you can show where to implement in my file?


VAEs are already using ELBO loss, here it appears as a variant with closed form KL. If you’d transform z’s distribution with normalizing flows, then you’d need another form. But that’s totally different decoder design, requiring different layers. And I’m not sure this will help you, if classifier doesn’t even work in non-variational mode.



 Anuj_Daga:

https://github.com/wohlert/semi-supervised-pytorch/tree/master/examples/notebooks


Note that they start from a simpler model, where aux classifier is applied to non-encoded inputs. Second model uses “learned encoder as a feature extractor” - but note that they pretrain and freeze the auto-encoder!
And the code looks more sophisticated, for example I see the use of importance sampling. There is also some nice illustrative code in that repo for elbo, flows, etc.