I guess you are using this repo 8.
In your code you are filtering out all parameters which do not require gradients, thus you are probably breaking the intended usage of the code.
Have a look at these lines of code 7.
I think if you create your own policies dict using this code for your filtered parameters, it should work again.