I don’t quite understand, why your batch size is larger in the target than your input.
From the shape calculation in the encoder code, it looks like you are passing a 64x64 tensor.
What kind of criterion are you using? BCELoss?