This is the issue and how to fix it. If you don’t want to read the thread the issue is caused by pyglet not playing nice with jupyternotebook. You either have to downgrade to 1.2.4 or do this:
after some checking.
python3.6/site-packages/pyglet/__init__.py

change

 if 'sphinx' in sys.modules:
    setattr(sys, 'is_epydoc', True)
to

if 'sphinx' in sys.modules:
    setattr(sys, 'is_epydoc', False)

pyglet has problem with jupyter, jupyter imports sphinx be default
but sphinx imported will lead pyglet thinks it is generating document? so cannot find display correctly.

https://github.com/openai/gym/issues/775 32