Leslie_Wang:

just want to figure out if pytorch will set inplace=True in train mode automatically?


No, the inplace argument won’t be changed and will keep the value you’ve set during the instantiation.
If you are not using inplace ReLUs, some other operation might manipulate the data inplace, such as assignments using indexing etc.