Hi,
I noticed that there is a large (~500MB) cuda context created per process on GPU.
can see it simply by doing:
import torch
torch.randn(1,device=0)

it takes 500MB (used to take 750MB in previous versions).
when multiprocessing on same GPU this is a lot of unneeded memory.
How can we work around this?