But isn’t there a way to set retain_graph=False from time to time to save memory ? I wanted to do something like this but every time I do this at a given step I get
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.