I’m just not sure why I’m getting Inf gradients. I suspect what might be happening is since I’m using non-differentiable functions inside my custom autograd function, autograd is turning requires_grad off. I don’t entirely understand how custom autograd functions work, though, so that may be a misunderstanding. What I mean by “take a break” is for autograd to not worry about the details of what’s going on inside my custom function; I want it to just trust me and those computations. That is, in the module following it, I want it to just send over values for me to use as grad_output, and for the module preceding it, I want it to just receive the grad_input I send over and assume those are right.
TL;DR, if autograd doesn’t do anything for custom functions then it’s probably a problem on my side; I’m just trying to figure out what’s going on.