My mistake, previously I read the doc of 0.1.12, now in the doc of 0.2.0 it confirms my above thinking. I was misguided by this old non-official doc:  http://pytorch-zh.readthedocs.io/en/latest/notes/extending.html 14