Hi,
Based on my own understanding of graphic mode, the ReLU will introduce extra memory when training even the inplace flag is set.
The inplace flag only indicates the output resue the input memory in the forward round. However, it need to save extra information for the latter backward propogation.  It either saves the input tensor X or positions where X>=0.  I do not know where it happens, but I believe the saving for backward is indeed required.
If only in inference mode, I think ReLU(True) will not introduce extra memory.