Hi,
Doing
del input
torch.cuda.empty_cache()
gc.collect()

in your code has very little impact as the Tensor pointed by input is still referenced in the forward call of the nn Module and so has not gone out of scope yet. So nothing can really be cleaned up.
Then the issue is most likely that other things hold onto input. For example, I think the view will keep the input of fc1 alive. So for this one, you won’t use less memory, only the extra stuff to store your new Tensors.
For fc2 it seems that the out of place relu does not hold onto its output. But I am not 100% sure this is the cpp function you call here or if it’s threshold.