Inplace operations can overwrite values required to compute gradients and might thus yield an error message such as:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1024]], which is output 0 of LeakyReluBackward1, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

These limitation is also described here and you should thus be careful with its usage.
The default behavior of setting inplace=False would work without raising any errors and you are of course free to check, if inplace=True would work for your model. If PyTorch doesn’t raise any errors, the gradient calculation will be right.