On 6 June 2017 at 20:08, Nathaniel Smith <njs at pobox.com> wrote: > On Mon, Jun 5, 2017 at 8:49 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: >> The reason this kind of approach is really attractive to >> redistributors from a customer risk management perspective is that >> like gevent's monkeypatching of synchronous networking APIs, it's >> *opt-in at runtime*, so the risk of our accidentally inflicting it on >> a customer that doesn't want it and doesn't need it is almost exactly >> zero - if none of their own code includes the "import _tls_bootstrap; >> _tls_bootstrap.monkeypatch_ssl()" invocation and none of their >> dependencies start enabling it as an implicit side effect of some >> other operation, they'll never even know the enhancement is there. >> Instead, the compatibility risks get concentrated in the applications >> relying on the bootstrapping API, since the monkeypatching process is >> a potentially new source of bugs that don't exist in the more >> conventional execution models. > > OK. To unpack that, I think it would mean: > > 2.7's ssl.py and _ssl.c remain exactly as they are. > > We create a new _tls_bootstrap.py and __tls_bootstrap.c, which start > out as ~copies of the current ssl.py and _ssl.c code in master. > (Unfortunately the SSLObject code can't just be separated out from > _ssl.c into a new file, because the implementations of SSLObject and > SSLSocket are heavily intertwined.) Then the copied files are hacked > up to work in this monkeypatching context (so that e.g. instead of > defining the SSLError hierarchy in C code, it gets imported from the > main ssl.py). > > So: we end up with two copies of the ssl code in the py27 tree, both > diverged from the copy that in the py3 tree, in different ways. If we went down this path, I think the only way to keep it maintainable would be to do one of two things: 1. Do the refactoring in Py3 first, but with the namespace pairing being "ssl/_enable_ssl_backport" rather than "_tls_bootstrap/ssl" 2. Don't try to share any implementation details with Py2, and instead provide a full "_ssl_backport" module more along the lines of importlib2 With the second option, we'd just do a wholesale backport of the 3.6 SSL module under a different name, and projects like pip would use it as follows: # See if there is an updated SSL module backport available try: import _ssl_backport except ImportError: pass else: _ssl_backport.install() # Fails if ssl or _ssl were already imported # At this point, sys.modules["ssl"] and sys.modules["_ssl"] would match the # API of the Python 3.6 versions, *not* the Python 2.7 versions. # At the first 2.7.x maintenance release after 3.7, they'd be updated # to match 3.7, and then the same again for 3.8. # (3.9 is likely to be after the end of 2.7 maintenance) # After that, proceed as usual with feature-based checks try: from ssl import MemoryBIO, SSLObject expect ImportError: # Otherwise fall back to using PyOpenSSL try: from OpenSSL.SSL import Connection except ImportError: raise ImportError("Failed to bootstrap asynchronous SSL/TLS support: <details>") Since there's no chance of breaking applications that don't opt in, while still making the newer network security features available to customers that want them, I think I could make a reasonable case for backporting such an "_ssl_backport.install()" implementation into the RHEL 7 system Python, but even if I wasn't able to do that, this kind of runtime injection into sys.modules could still be shipped as an add-on library in a way that an updated standard library SSL module can't. Such an approach would make the 2.7 regression tests noticeably slower (since we'd need to re-run a subset of them with the backport being injected by regrtest before anything loads the ssl module), and means backporting SSL fixes to 2.7 would become a two step process (once to resync the _ssl_backport module, and the second to update the default implementation), but I think there would be enough benefits to make that worth the pain: - we'd have a baseline 2.7 SSL/TLS implementation that we can readily keep in line with its 3.x counterparts - this approach has a reasonable chance of making it through the review processes for long term support distributions - system and application integrators can more readily do their own backports independently of redistributors Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia