The problem is with your docker file, you are supposed to do
conda install -y pytorch cuda80 -c soumith
if you want to run on P100. FWIW, you would not have this problem if you used either devel Dockerfile (in pytorch root directory) or runtime Dockerfile (in tools/docker). You’d have to modify either of them to install torchtext and its dependencies, or, alternatively, build a base image with the provided Dockerfile and have a separate Dockerfile that would reference this base image in the FROM line and install whatever add-ons you need.