Running into a similar bug installing 0.4 on V100s:
pip3 install http://download.pytorch.org/whl/cu90/torch-0.4.0-cp35-cp35m-linux_x86_64.whl 

When I place the model on cuda I get:
Found GPU0 Tesla V100-DGXS-16GB which requires CUDA_VERSION >= 9000 for
     optimal performance and fast startup time, but your PyTorch was compiled
     with CUDA_VERSION 8000. Please install the correct PyTorch binary
     using instructions from http://pytorch.org

It appears there is some issue with the compiled version not playing nicely with the V100 architecture.
Temporary workaround I’m using is to downgrade to 0.3.1:
RUN pip3 install http://download.pytorch.org/whl/cu90/torch-0.3.1-cp35-cp35m-linux_x86_64.whl