Multiple operations can be fused if you are scripting the model via torch.jit.script. If an operation can be fused with their neighbors depends on the actual operations as well as the used fuser backend (e.g. nvfuser should be able to fuse dropout-add-relu etc. or other pointwise operations).
I’m not sure if you are referring to the training and validation loop, but inplace ops would work there in the same way with the benefit that Autograd would not complain of disallowed inplace manipulations during the validation run if the forward pass is wrapped in a torch.no_grad() context.