albanD:

I am not sure why you need this to be a generator though as you can directly update all the p.grad inplace from this function no?


Well, it allowed me to modify SGD by adding 3 lines of code, instead of recoding everything for vector gradient updates (i.e. momentum, nesterov etc).