One hot in MNIST? Do you mean the target labels? Some standard loss functions for this task (e.g. cross entropy http://pytorch.org/docs/master/nn.html#crossentropyloss 2) has native support so you only need to pass in the class index rather than the one hot vector. If you really want one hot vectors, just make one on the fly when you train on it, e.g. (if you model is float)
target = torch.FloatTensor(1, 10).zero_()
target[idx] = 1