Created on 2019-04-19 01:01 by ncoghlan, last changed 2019-04-20 02:26 by terry.reedy.
While trying to create an example for a pickle bug discussion, I deliberately dropped `__main__` out of sys.modules, and the REPL session lost all of its runtime state.
Simplified reproducer:
```
>>> import sys
>>> mod = sys.modules[__name__]
>>> sys.modules[__name__] = object()
>>> dir()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'dir' is not defined
```
(Initially encountered on Python 2.7, reproduced on Python 3.7)
If I'd just dropped the reference to `__main__` entirely, that would make sense (since modules clear their namespaces when they go away), but I didn't: I saved a reference in a local variable first.
So it appears the CPython REPL isn't keeping a strong reference to either `__main__` or `__main__.__dict__` between statements, so the cyclic GC kicked in and decided the module could be destroyed.
Additional info showing the module getting reset back to the state of a freshly created module namespace:
```
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']
>>> __builtins__
<module 'builtins' (built-in)>
>>> import sys
>>> mod = sys.modules[__name__]
>>> sys.modules[__name__] = object()
>>> __builtins__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '__builtins__' is not defined
>>> __annotations__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '__annotations__' is not defined
>>> __doc__
>>> __loader__
>>> __name__
'__main__'
>>> __package__
>>> __spec__
>>> mod
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'mod' is not defined
```
The ``sys`` import gets cleared as well (accidentally omitted from the previous comment):
```
>>> sys
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'sys' is not defined
```
The relevant functions:
* PyRun_InteractiveLoopFlags:
* PyRun_InteractiveOneObjectEx:
So it turns out I was wrong: nothing is getting cleared anywhere, but instead each statement in the REPL is *importing* `__main__` again in order to find the namespace to use for the statement execution.
Because of the specific API it uses to do that, a non-module object like the one I injected gets replaced with a regular (empty) module object:
However, it *doesn't* have the extra code needed to make the `builtins` available:
So I now think the only actual *bug* here is the fact that the REPL isn't making sure that `__builtins__` is set appropriately - the rest can be chalked up to implementation defined behaviour around what happens if `__main__` gets replaced or removed in sys.modules while the REPL is running.
To me, the failure of dir() in message 1 is surprising and possibly a bug. I always though of a module globals = locals = dict() instance as continuous across statements, whether in batch or interactive move. In batch mode
import sys
mod = sys.modules[__name__]
sys.modules[__name__]
print(dir())
works. Adding '-i' to the command line is supposed to allow one to enter interactive statements to be executed in the same namespace.
In IDLE's Shell, dir() in msg 1 executes normally. This is because idlelib.run.Executive() initializes the instance by caching globals().
self.locals = __main__.__dict__
Then self.runcode(self, code) executes user statements with
exec(code, self.locals)
With exec in the old statement form of 'exec code in self.locals', this pair predates the first patch git has access to, on 5/26/2002 (GvR, committed by Chui Tey).
Could and should, python do similarly, and keep a reference to the module namespace? What did Python do in 2002? What do other implementations and simulated Shells do now?