Solved by SimonW in post #2 
              
              
                One hot in MNIST? Do you mean the target labels? Some standard loss functions for this task (e.g. cross entropy http://pytorch.org/docs/master/nn.html#crossentropyloss) has native support so you only need to pass in the class index rather than the one hot vector. If you really want one hot vectors, …