I have created a small dataset which only contains 30 samples.However the memory still decreased rapidly.(batch_size=1)

So I think this issue is not caused by dataloader and I am sure reason is the node expansion.
When I do DDP training in one node,the memory problem is not serious.When I expand the nodes,the memory problem is serious.I want to know if this is pytorch’s problem?
My model like this
image1139×413 72.2 KB