Your welcome! For this one, I probably don’t understand enough about your use case. On the surface it doesn’t make a lot of sense to update A with model.grads. Semantically what is the end result supposed to look like?
For example, if you do .grad(delL_delWo, A, grad_out=(torch.ones_like(dL_dp) for dL_dp in delL_delWO)) you are computing the direction to perturb A which will cause each of the parameter grads delL_delWo to change in the (1, 1, ..., 1) direction.
Not sure what multiplying would accomplish though…