Thanks a lot for the explenation! I think i understood it now.
But i have one question to be more certain. Why do you have to multiply the gradient with the current action?
out = -torch.mul(delta_a,current_a)

if delta_a is dQ/da or c then you would have to multiply with da/dw and not the action. Well i think its because you first do -c*a and then you differenciate that which leads to:
d(-c*a)/dw
now you can take out the c from the equation and get:
-c*(da/dw)
am i right with that?