When I run my experiments on GPU, it occupies large amount of cpu memory (~2.3GB). However, when I run my exps on cpu, it occupies very small amount of cpu memory (<500MB). This memory overhead restricts me on training multiple models.
Can someone please help me on debug whuch component is causing this memory overhead?
I have added sample code at GitHub - divyeshrajpura4114/asv-sample 2