Two questions about your snippet:


why usage grows to ~500 MB though first inference took 202 MB and every next inference does the same job?
shouldn’t the allocated memory to be deallocated at each new iteration?


why I get less and stable memory usage when move network run in another method:


model = PSPNet()
x = torch.randn(2, 3, 224, 224)
process = psutil.Process(os.getpid())

def run_network():
    out = model(x)

for idx in range(100):
    print(idx, process.memory_full_info().rss / 1024**2)
    run_network()

result:
1 328.0234375
2 344.46484375
3 354.515625
4 344.19921875
5 344.16015625
6 343.96484375
7 344.02734375
8 344.08984375
9 344.17578125
10 343.98828125
11 344.05078125
12 344.1171875
13 344.1796875
14 343.984375
15 344.046875
16 344.1328125
17 344.19140625
18 344.25390625
19 344.05859375
20 344.12109375
21 344.18359375
22 344.24609375
23 344.05078125
24 344.11328125
25 344.17578125
26 344.23828125
27 344.04296875
28 344.10546875
29 344.16796875
30 344.23046875
31 344.03515625
32 344.12890625
33 344.19140625
34 344.25390625
35 344.05859375
36 344.1484375
37 344.22265625
38 344.2890625
39 344.1171875
40 344.1796875
41 344.2421875
42 344.3046875
43 344.109375
44 344.171875
45 344.234375
46 344.296875
47 344.359375
48 344.16015625
49 344.21875
50 344.28125
51 344.34375
52 344.1484375
53 344.2109375
54 344.2734375
55 344.3359375
56 344.140625
57 344.203125
58 344.265625
59 344.328125
60 344.1328125
61 344.1953125
62 344.2578125
63 349.5078125
64 344.1875
65 344.1875
66 344.25
67 349.5
68 344.1796875
69 344.1796875
70 344.2421875
71 344.3046875
72 344.109375
73 344.171875
74 344.234375
75 344.296875
76 344.359375
77 344.1640625
78 344.2265625
79 349.4765625
80 344.4140625
81 344.15625
82 344.21875
83 344.28125
84 344.34375
85 344.171875
86 344.234375
87 344.296875
88 344.359375
89 344.1640625
90 344.2265625
91 344.2890625
92 344.34765625
93 344.15234375
94 344.21484375
95 344.27734375
96 344.33984375
97 344.14453125
98 344.20703125
99 349.45703125

also I noticed that the following code:
def run_net(self):
    x = torch.randn(2, 3, 480, 480)
    logit = self.net(x)

in the first iterations allocated memory both for the x and when run model:
 Line #    Mem usage    Increment   Line Contents
 ================================================
     50    252.8 MiB    252.8 MiB       @profile
     51                                 def run_net(self):
     52    258.3 MiB      5.5 MiB           x = torch.randn(2, 3, 480, 480)
     53    668.6 MiB    410.3 MiB           logit = self.net(x)

total memory is 252MB, 5.5 for x and 410 for inference
while after n iterations the following scene appears:
Line #    Mem usage    Increment   Line Contents
================================================
    50   1001.2 MiB   1001.2 MiB       @profile
    51                                 def run_net(self):
    52   1001.2 MiB      0.0 MiB           x = torch.randn(2, 3, 480, 480)
    53   1217.6 MiB    216.5 MiB           logit = self.net(x)

so for x memory is not allocated at all and in inference time it allocated only half of initial volume but total memory is now 1001 MB.
I suppose that’s the same nature as if some kind of caching is happening