Hi, this is a very simple case, though… Here, relu_() is the replace version of relu()
In [5]: a = torch.randn(128, 3, 512, 512).cuda()

In [6]: torch.cuda.max_memory_allocated()
Out[6]: 402653184

In [7]: b = a.relu()

In [8]: torch.cuda.max_memory_allocated()
Out[8]: 805306368

In [9]: c = b.relu_()

In [10]: torch.cuda.max_memory_allocated()
Out[10]: 805306368