Thanks for your reply. I think torch.utils.checkpoint is useful for me.
But I have some questions of meomory use of torch. For monitoring memory useage of torch, I run my custom model which contains code below in both way model->eval() and model->train().
printShape(x);  [4, 3, 40695]
printMemory();
x = conv1x1(x);
printShape(x);  //[4, 64, 40695]
printMemory(); //increase about 41mb
x = bn1(x);
printShape(x);  //[4, 64, 40695]
printMemory(); //increase about 41mb
x = func::relu(x);
printShape(x);  //[4, 64, 40695]
printMemory(); //increase about 41mb

But there is no difference between two way and a non-paramters module relu also takes some memory. It looks like all intermediate tensors always not be freed for some reason, until call backward() (and I run with a CPU, not GPU). Now, I am tring to know why tensor not released even if not used. can you help me?