Thanks, that solved the problem. But I was wondering if there is a more elegant solution. Also, why do you have to zero the grad data every time in a loop? I assume that is something that should take care of itself.