Oh, with this approach, you’re likely to have an expanding history, because self.out will contain a pointer to part of the graph for each iteration. As far as I see, you’re overwriting completely at every forward, so you should recreate it every time like that:
def __init__(self):
    self.out = get_new_out()

def forward(self, input):
    ...
    next_out = get_new_out()
    for step in range(inp.size(0)):
        next_out[step] = ... # an expression containing self.out
    self.out = next_out
    return next_out

You don’t need to worry about zeroing any gradients then. Also, keep in mind that self.out doesn’t require gradient, so it will be always 0.