P.S.
If I profile the MNIST learning with cProfile i.e. without autograd profile, the process memory consumption keeps around 2GB of RAM. (for epochs 10 (default value)).
It seems autograd profiler’s Event memory consumption is high.