Each conda env should use their own set of installed libraries.
I’m not sure, if some libs are reused from the base environment, but you can definitely install different PyTorch builds in separate environments.