@ptrblck
sorry for the late reply,
so I profiled the memory allocation at the start of the training loop and reporting values at the start of 3rd iteration after which these values remain constant for each setting.
If you could please explain this behavior, it would be really helpful.
p.s just for validation, I have tried gc.collect() , deleting input tensors, outputs, clearing cache but these strategies don’t change anything.
Screen Shot 2020-08-15 at 7.23.10 PM987×342 29.4 KB