That’s a lot of code… that would really help if you had a small code sample that we can run and see what it is doing easily.
Are you sure the accuracy drop comes from the autograd? Can you try temporarily adding a dy.clone() at the beginning of your function and see if you still have the same behavior?
Such behavior in the loss can also come from instability of your new operation.