Hello,everyone.
When I tried to freeze part of my model in the process, I set require_grad to False, but I found that memory increased by more than two times at this point. I wonder if PyTorch will save the weight at once and copy it to continue execution, and if so, is there a better way to save my memory since GPU cannot work now.
I tracked my memory usage and it worked until it was frozen.
At Backbone_idsub <module>: line 202                 Total Used Memory:8267.4 Mb

+ | 52 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
+ | 44 * Size:(1,)                 | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32
- | 42 * Size:(1,)                 | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32
- | 50 * Size:()                   | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:8267.4 Mb

+ | 54 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
+ | 46 * Size:(1,)                 | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32
- | 52 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 44 * Size:(1,)                 | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:8267.4 Mb

+ | 48 * Size:(1,)                 | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32
+ | 56 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 54 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 46 * Size:(1,)                 | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:8269.6 Mb

+ | 50 * Size:(1,)                 | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32
+ | 58 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 48 * Size:(1,)                 | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32
- | 56 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:8267.4 Mb

+ | 60 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
+ | 52 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 50 * Size:(1,)                 | Memory: 0.0001 M | <class 'torch.Tensor'> | torch.float32
- | 58 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:8267.4 Mb

+ | 62 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
+ | 54 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 60 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 52 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:8267.4 Mb

+ | 56 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
+ | 64 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 62 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 54 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:8267.4 Mb

+ | 58 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
+ | 66 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 56 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 64 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:8267.4 Mb

+ | 68 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
+ | 60 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 58 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 66 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:8267.4 Mb

+ | 70 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
+ | 62 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 68 * Size:()                   | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32
- | 60 * Size:(1,)                 | Memory: 0.0002 M | <class 'torch.Tensor'> | torch.float32

At Backbone_idsub <module>: line 202                 Total Used Memory:20112.1Mb

Looking forward to your reply :).