It seems that ignite is indeed pointing towards an older PyTorch release with CUDA10.
Note that pip install torch would also install the default CUDA 10.2 runtime, so stick to the CUDA 11.3 pip install command.
In any case, once you’ve properly installed a PyTorch binaries with CUDA 11.3, try to skip the dependencies in ignite via conda install -c pytorch --no-deps.
CC @vfdev-5 for visibility in case this is a known issue in ignite