Yup, the difference between the number of dimensions in the output layer was something that I found out as well. However, I have a relatively smaller network and I can live with “doubling number of params” in the final layer. I will check the network for data leak.