ah damn it, I was completely stuck in the TensorFlow mindset and thinking in terms of computation graphs that are being passed around. Basically, what happens if one passes the pre-constucted tensor to a pytorch module, this tensor is never re-evaluated with the updated parameter. Obviously, in pytorch one has to re-construct the parameter tensor in every forward pass, with the updated inputs, otherwise you will always have the same one. Kind of obvious, now.
Anyways, thanks for the help @googlebot !
oh and what does CLT  mean?