Hi,
For the following function,
the shape of l and r are both (1, 64, 640, 360), and the device are both ‘cuda:0’ and the datatype are both torch.float32.
the problem is: once the function is executed, the used memory of GPU increased by 58MB.
why does torch.add function increase the memory usage of GPU?
I tested it many times, and 58MB more memory was increased every time.
even if I use l+r instead of torch.add(l, r), the memory used increased 58MB. 
def _add(self, *inputs):
        l, r = inputs
        return torch.add(l, r)