Pandas shouldn’t interfere with PyTorch, so that’s a bit strange.
Did you make sure to activate the same conda environment in case you were using one?
EDIT: Did you see any message of uninstalling PyTorch during the Pandas installation?