You are using CPU kernel to generate random numbers. For large tensors, GPU is preferred. Try this instead: torch.cuda.FloatTensor(500, 128, mult).normal_().
Also, it might be the .cuda call taking majority of time.