Thanks, I thought I’d read somewhere about the memory being shared but last time I did a quick test to try that, it looked to be different. Looking back I just realized that I didn’t use an in-place op in my quick test, fixed and confirmed they are in fact sharing memory and all is well with the universe 
I’ll keep the second case in mind, with models working on large images where targets are also masks/images, a few hundred MB could tip the balance, but still small in comparison to the model parameters and gradient state.