If I were designing Python's import from scratch
Very interesting write-up, I&#x27;m glad to see a core developer thinking deeply on this topic because Python has a lot of room for improvement.<p>This touches briefly on 1 of 3 major issues I have with python. I&#x27;ll try my best to articulate them from smallest-to-greatest impact.<p><i></i>1. <i>import</i> semantics<i></i><p>It would be amazing to see the <i>import</i> module be abstracted out of the <i>sys</i> module. Not just for the reasons mentioned in the article. Ideally, it would allow developers to override&#x2F;extend the implementation of <i>import</i> and experiment with new ways to handle.<p>For instance, why can&#x27;t a developer write a module to experiment with using es6-module style imports.<p>```
import {attr1, attr2} from module@[version]&#x2F;submodule
import * from module2
```<p>When I wrote the pypreprocessor lib, I simply wanted to add the ability to import and conditionally-compile code using inline c-style preprocessor directives. It works by blocking the import, preprocessing the code, then importing the postprocessed version of the code.<p>Implementing it is ugly because there&#x27;s no way to inline the preprocessor step. If I had the ability to extend and inline the preprocessor step as a custom import module, the specifics would be transparent to the user.<p>With a preprocessor it would be possible to write python2&#x2F;python3 code side-by-side making the transition much easier for library developers. Implementing it as a custom import add-on would allow developers to extend the platform without polluting core (ie since GvR is vehemently against adding preprocessor capability to core).<p>Why does python default to the mentality of &#x27;one true way&#x27;? There is a huge ecosystem of developers willing to build&#x2F;extend and experiment with the language in ways that aren&#x27;t immediately apparent to core design committee.<p><i></i>2. Packages violates &#x27;Explicit is better than implicit&#x27;<i></i><p>Package management as a whole is a terrible, broken experience for library developers and consumers of those libraries alike. Global-only package installations, dependency hell, lack of version configuration, inability to automate dependency loading, etc.<p>&quot;Vendoring is not hard thanks to relative imports and most projects don&#x27;t seem to need it.&quot;<p>I highly disagree. Expecting users to either install dependencies globally or manually copy them to the package contents sucks. The former introduces potential environmental-level side effects such as dependency conflicts and requires virtualenv to isolate packages. The latter is prone to error, adds maintenance overhead as dependencies are updated, and makes it unclear to developers who want to add dependencies without polluting source control.<p>Just look at the NPM ecosystem for inspiration. Vendoring, incl versioned dependency management is done on a per-project level by default and the benefits are clear. I can execute `npm install` from the CLI and the package manager will download+install all of a projects dependencies (ie and dependencies of dependencies) automagically. There&#x27;s a reason the NPM eclipsed every other packaging ecosystem in such a short period. It &#x27;just works&#x27;.<p>Virtualenv is a hack. Instead of supporting package-level dependencies, it forces developers to limit the environment to the package-level. It introduces all of the same problems inherent to using globally installed dependencies, except at a local level.<p>This violates II of &#x27;The Twelve Factor App&#x27;
<a href="http:&#x2F;&#x2F;12factor.net&#x2F;dependencies" rel="nofollow">http:&#x2F;&#x2F;12factor.net&#x2F;dependencies</a><p><i></i>Pets vs Cattle<i></i><p>While the current model works well with &#x27;pets&#x27;, ie long-running processes and persistent environments. It causes a lot of problems when it comes to &#x27;cattle&#x27;, ie transient&#x2F;disposable environments because it requires developers to re-create an identical copy of the development environment on the target for every deployment. Not only does this add a lot of provisioning overhead&#x2F;complexity but is highly prone to error if&#x2F;when the environment changes (ex modules added, configuration changes).<p>DevOps is hard enough, even without the overhead of cloning environment-specific details of the language platform.<p><i></i>3. Religious adherence to &#x27;the one true way&#x27;<i></i><p>Specifically, python is a platform implementation when it &#x27;should&#x27; be a language.<p>Don&#x27;t get me wrong, Python is a significant improvement over the everything-and-the-kitchen-sink approach used by platforms like .NET&#x2F;Java. I understand the need for including enough core functionality to get users up and running quickly. What I don&#x27;t agree with is the difficulty of extending&#x2F;overriding the default functionality.<p>Along those lines, I&#x27;d include a requirements:
- expose the core modules as classes (ie not just <i>import</i>) that can be extended&#x2F;overridden
- take some time to engineer a really good package management tool
- setup.py shouldn&#x27;t require hand-coding, pip should be capable of generating it
- unify the configuration at the package-level
- additional package install steps can be added as pre&#x2F;post hooks in the configuration<p>-------<p>I have some obvious biases that come from the JS&#x2F;Node.js development community. Don&#x27;t get me wrong, I love python as a language and will continue to use it in the distant future. Without good package&#x2F;version management I don&#x27;t see it as a good alternative for building anything but trivial applications.