I changed the directory but I am still getting an import error. Maybe sys.path is incomplete:
pytorch_cuda9:latest:
# python
Python 3.6.4 |Anaconda, Inc.| (default, Dec 21 2017, 21:42:08) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pytorch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pytorch'
>>> import sys
>>> print (sys.path)
['', '/opt/conda/envs/pytorch-py3.6/lib/python36.zip', '/opt/conda/envs/pytorch-py3.6/lib/python3.6', '/opt/conda/envs/pytorch-py3.6/lib/python3.6/lib-dynload', '/opt/conda/envs/pytorch-py3.6/lib/python3.6/site-packages']

pytorch/pytorch:latest:
# python
imporPython 3.5.2 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:53:06) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys    
>>> print (sys.path)
['', '/opt/conda/envs/pytorch-py35/lib/python35.zip', '/opt/conda/envs/pytorch-py35/lib/python3.5', '/opt/conda/envs/pytorch-py35/lib/python3.5/plat-linux', '/opt/conda/envs/pytorch-py35/lib/python3.5/lib-dynload', '/opt/conda/envs/pytorch-py35/lib/python3.5/site-packages', '/opt/conda/envs/pytorch-py35/lib/python3.5/site-packages/setuptools-27.2.0-py3.5.egg', '/opt/conda/envs/pytorch-py35/lib/python3.5/site-packages/torchvision-0.1.9-py3.5.egg']