@fma makes sense. What intrigues me right now is why
W = W - eta*W.grad

does not update my parameters. I understand the issue you mentioned about new variable allocation etc but despite of that gradient descent should still work (just memory inefficiently)…this is puzzling me.