In another article, I outlined how to handle Python imports and dependencies in Poetry. Poetry has a lot to recommend it:
pyproject.toml
file for youpyproject.toml
(nice if that’s what you want)But there are a few minor cons:
So, here is an alternative (with its own minor pros and cons). I intend this as a simple reference you can paste commands out of. It will get you through the usual workflow of creating a Python library and hosting it on PyPI.
Put your Python modules in the red folder. Don’t create the venv
folder. We’ll do that in a later step. It’s here in in the image just to show you where it will go.
pip install pip-tools build twine
python -m venv venv
echo " if you are using Windows"
.\venv\Scripts\activate
echo " if you are using Linux"
source venv/bin/activate
A tool might ask for your project interpreter. If you are using Windows, this will be .\venv\Scripts\python.exe
. If you are using Linux, this will be venv/bin/python
.
pyproject.toml
[project]
name = "no_spaces_in_your_project_name"
version = "0.1.0"
description = "describe your project here"
authors = [{ name = "Shay Hill", email = "shay_public@hotmail.com" }]
license = {text = "MIT"}
readme = "README.md"
requires-python = ">=3.9"
dependencies = ["numpy", "requests"]
[project.optional-dependencies]
dev = ["pytest", "pre-commit", "commitizen"]
[build-system]
requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"
pip-compile -o requirements.txt pyproject.toml --upgrade --strip-extras
pip-compile --extra=dev -o dev-requirements.txt pyproject.toml --upgrade --strip-extras
echo " if you are using Windows"
pip-sync --python-executable .\venv\Scripts\python.exe requirements.txt dev-requirements.txt
echo " if you are using Linux"
pip-sync --python-executable venv/bin/python requirements.txt dev-requirements.txt
For right now, you will only need the dev-requirements.txt
file. It has all your dependencies, including your dev dependencies, but I wanted to show you how to pip-sync multiple files.
python -m pip install -e .
Don’t miss that trailing period. If you update your requirements in pyproject.toml
, remember to repeat steps 5, 6, and 7.
python -m build
twine upload --repository testpypi dist/*
echo " or"
twine upload dist/*
Comment: You may want to start by reserving your project name on PyPI and testPyPI.
Do I commit my venv folder? No. It is common practice to put venv
in your global .gitignore
file.
What should I do with my requirements.txt and dev-requirements.txt files? Unless you know you need pinned dependencies, assume you don’t need your requirements files. Don’t commit them. The same goes for the dist
and src\*.egg-info
folders the above instructions will generate.
What if I really need to pin a top-level requirement? If you need to pin a top-level requirement in pyproject.toml
(like Poetry does), see PEP 508. There is a lot of information there you won’t need, but the short version is that dependencies = ["numpy>=1,<1.25", "requests<=2.26.0", "pydantic==1.8"]
and similar work.
What if I want to publish a script? Find some more use cases here. This video walks through a lot of the same steps, nice and slowly.