Got it working! Had to abandon the old environment and build from scratch. This ordering (followed by all the other packages)  led to success. Thank you so much for your help!
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch

conda list cuda

conda install -c pytorch ignite --no-deps