Introduction
This tutorial will cover
setuptools and how to package libraries and applications
for
pip,, and distribution to others.
These examples will show you how to create distributable packages. We'll look at how import works, how the setup.py file and how to use it to build and install packages. I recommend creating a new virtual environment and activating it before testing out any of these examples. If you aren't familiar, read up on virtual environments. That will ensure you have an isolated environment do not contaminate your system install or any project. You can also get started with a simple distutils example from the official documentation. See my Python Virtual Environments Tutorial if you need to learn more about virtual environments.
Also see Python import, sys.path, and PYTHONPATH Tutorial and my Python Virtual Environment Tutorial.
If you need Python, check out my tutorials:
Terminology
Before going any further, let's look at some terms that might be confusing:
- Module - A Python file (.py) with code, functions, classes, etc.
- Package - A directory with Python modules files. Might have an init.py and/or main.py along with Python modules (.py files). The word package also commonly refers to a distributable package.
- Distributable package - A Python library or application that was pacakged with a setup.py file that is built in to a distributable file that can be published on PyPI.org. This term may be shortened to just "package" which can cause confusion between the other use of the word package which refers to a directory with Python modules. Even in this tutorial, I may refer to a distribution as a "package" and when dicussing "packaging" it refers to creating a distributable.
- setuptools - The python package used to create and manage distributable Python packages.
- distutils - The older version of setuptools, which is now legacy. Use setuptools instead.
- setup.py - The primary file of a distributable package that defines author, version information, files to include, etc. Used to build, install, and upload the package it belongs to. Builds packages that can be installed using pip. Functions provided by distutils package.
- pip - Tool for downloading, installing, uninstalling, upgrading distirubted packages in a Python instance. Works with packages built using setuptools and setup.py.
- PyPI - Python Package Index, pypi.org - the public repository for Python packages.
Minimal package
This is the simplest possible package that you can build and install. I would never recommend making a package like this, but it demonstrates the core element of a distributable package, the setup.py file.
This example will default to version 0.0.0, have no author information, and in general, is not fit for sharing. Technically it will work and you can install modules to your local site-packages for re-use but again, this is only an example for learning. Read the next section for how to make a better package.
All you really need to do to turn your regular package in to a package ready for distribution is add a setup.py file. The setup.py file defines the package name, version number, author information, files, and any additional setup. All of the magic comes with this setup.py file. This is a really simple setup.py example, taken from,. Other useful resources include an example pypi project or the setup script documentation.
Note that if you just want to create a package for use locally in a single project, you do not need a setup.py and do not need to create a distributable package. This describes the process for building a package that is intended to be installed to your Python site-packages for use among multiple projects or shared with other people.
# Directory structure \ # Project directory root \setup.py # Describes distributable package info \mylib.py # Contains the python code for sharing
# setup.py from setuptools import setup setup( name='my_module', py_modules=['my_module'], # Include my_module.py )
The example above shows the simples possible distributable package you can make. It may not make it up to pypi.org but it will technically install locally. All you need is a single .py file (your module) and a setup.py to provide the name and list of modules to include. With this, you can run python setup.py install and it will install the module to your site-packages directory.
python setup.py install
When you install it, the package will be copied in to your site packages
directory making your package available for import. To learn more
about
import,
sys.path,
PYTHONPATH and how Python searches
for modules to import, check out my Python import, sys.path, and PYTHONPATH Tutorial
.
Then you would import it like this in a Python script:
import my_module
Install versus develop
python setup.py install # or python setup.py develop # develop will create symlinks so it doesn't require reinstall after every change # This works in Windows too!
See other setup.py commands with:
python setup.py --help-commands
Create a distribution package
There are different distribution options. The simplest option is the source
distribution (
sdist) where it simply packages up all the source code.
Other packages can be built using a build distribution (
bdist) which
are good for packages with binary blobs or creating a special format.
Some options include
rpm and
msi for Linux packages and Windows
installers!
pip install twine # tool to push to pypi python setup.py sdist # Package the release file twine upload dist/<package_file> # Other package options python setup.py bdist --format=gztar # .tar.gz python setup.py bdist --format=zip # .zip python setup.py bdist --format=rpm # Yum package (Fedora/CentOS/RHEL) python setup.py bdist --format=msi # Windows installer python setup.py bdist --format=wininst # self extracting zip
Push package to PyPI.org
To update your package and push it to pypi, update your
setup.py to
make the version number higher than the current version, and repackage
with
python setup.py sdist. Follow that up with a
twine upload.
# Build the package python setup.py sdist # Upload the package twine upload dist/mypackage.tar.gz
Set up \~/.pypirc
When using
twine to upload, for example,
twine upload dist/mypackage-1.0.0.tar.gz,
you can avoid entering your password
each time by setting up a
~/.pypirc file.
This step is optional, but if you upload packages to pypi.org regularly, this will save a lot of time.
Here is an example:
# ~/.pypirc [distutils] index-servers=pypi [pypi] # repository = username = myusername password = mysecretpassword
Name the file
.pypirc and place it in your
$HOME directory.
Create a distributable package
This will explain how to package a Python application for sharing on pypi.org
that can be installed with
pip. Another method of packaging an application
is to create a standalone .exe, .app, or Linux executable. You can use PyInstaller
to do this. If you are interested in that, check out my PyInstaller tutorial.
The previous example is an absolute minimum. It didn't have any author or version information about the module. You can provide a lot more useful information in the setup.py file. Not only can you provide links to a website or documentation, you can specify executable scripts that should be installed with your module and dependencies that it relies on. This example is a better template for a new package.
# Directory structure \ \README.rst # Simple documentation \setup.py # Used to build package and install \mypackage\__init__.py # Executed when package is imported (optional) \mypackage\__main__.py # Executed with python -m mypackage (optional) \mypackage\mymodule1.py # Modules the package provides/uses \mypackage\mymodule2.py # Modules the package provides/uses
That's the core of a basic package. You do not have to create a package and can simply provide modules as demonstarted in the first example. In the setup.py file you have options for py_modules to include and packages to include. For more information about what you can provide in the setup.py file check out the official documentation on the setup script.
Add executable scripts
Option 1) Add
scripts
Option 2) Use entrypoint
When addding executable scripts to a package, I recommend placing them in a
bin/
directory. Then, in the
setup.py file, provide the list of scripts to the
scripts parameter.
. Create a .bat file for Windows and a shell script with a shebang for Linux/Mac. Refer to the examples provided for how to build the scripts.
Entrypoints (scripts)
From', ] } )
PyInstaller is another option for packaging applications for distribution.
PyInstaller lets you create Windows
.exe, Mac
.app, and Linux
executable files. You can even have it package as a single file.
You can follow my PyInstaller tutorial here:.
scripts vs entry_points
Example setup.py template
Here is an example
setup.py that you can use a starting place for your own package.
Remove any pieces you don't need and modify everything else.
# setup.py template # import os from setuptools import setup from sys import platform requirements = [ 'docopt', 'discord.py==0.16.12' ] if platform == "win32": requirements.append('windows-curses') setup( name = "an_example_pypi_project", version = "0.0.4", author = "Andrew Carter", author_email = "andrewjcarter@gmail.com", description = ("An demonstration of how to create, document, and publish " "to the cheese shop a5 pypi.org."), license = "MIT", keywords = "example documentation tutorial", url = "", # Individual Python modules (.py files) py_modules=['mymodule'], # Directories (package) packages=[ 'mypackage', 'tests' ], long_description_content_type='text/markdown', long_description=open('README.md').read(), scripts=scripts, package_data={ # Other misc files that should be installed 'mypackage': [ 'data/defaults.xml', # Relative to setup.py dir 'data/*.dat', ], }, python_requires='<3.7', # Optional install_requires=requirements, platforms='any', classifiers=[ # Options at 'Development Status :: 4 - Beta', 'Environment :: Console :: Curses', 'Environment :: Web Environment', 'Environment :: MacOS X', 'Environment :: X11 Applications :: Qt', 'Framework :: Django :: 2.1', 'Framework :: Flask', 'Intended Audience :: End Users/Desktop', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'Natural Language :: English', 'License :: OSI Approved :: MIT License', 'Operating System :: Android', 'Operating System :: MacOS :: MacOS X', 'Operating System :: Microsoft :: Windows', 'Operating System :: POSIX', 'Programming Language :: Python', 'Topic :: Utilities', 'Topic :: Software Development', 'Topic :: Security', ], )
You can also add dependencies from GitHub.
Create a standalone .exe, .app, or Linux Executable
If you want to package your application as a standalone executable, I recommend using PyInstaller. This allows you to easily distribute your app without the user needing to install or even know anything about Python. Check out my PyInstaller Tutorial for more information.
Conclusion
After reading this, you should understand how to create distributable
packages with
distutils using
setup.py. You should be able to create
your own package, add dependencies, include executable scripts, build it,
and upload it to pypi.org for others to download.
References
-
-
-
-
-
-
-
- Python import, sys.path, and PYTHONPATH Tutorial
-
-
-
-
-
-
-