setup.py vs setup.cfg in Python | Towards Data Science

Introduction

In one of my recent articles, I discussed about the difference between requirements.txt and setup.py files that can eventually help developers manage the dependencies of their packages in a way that it is also easy for them to redistribute them.

In today’s article I am going to focus on the setuptools package and discuss about the difference between the setup.py and setup.cfg files. Additionally, we will also discuss how these files can comprehend requirements.txt file that is supposed to list dependencies.

Finally, we will also discuss about the purpose of pyproject.toml file and how the package distribution landscape was changed after the acceptance of PEP-517. We will also demonstrate how to transition to the use of pyproject.toml with setuptools in a way that you are PEP-517 and PEP-518 compliant.

setuptools in Python

[setuptools](https://setuptools.pypa.io/en/latest/) is a library which is built on top of [distutils](https://docs.python.org/3/library/distutils.html) that has been deprecated (and up for removal as of Python 3.12). The package offers a wide range of functionality that facilitates packaging of Python projects which is a process that constantly evolving and sometimes can get quite challenging.

By packaging we usually mean both dependency management and package distribution. In other words, packaging is all about handling the dependencies of your projects (and even the dependencies of dependencies and so on) as well as how to distribute your package in order to make it widely available and usable by other projects.

The setup.py file

The setup.py file is probably the most important file that is supposed to be included at the root your Python project directory and it mostly serves two primary purposes:

It contains various information relevant to your package including options and metadata, such as package name, version, author, license, minimal dependencies, entry points, data files and many more.
Serves as the command line interface that enables the execution of packaging commands.

Example setup.py file

# setup.py placed at root directory

from setuptools import setup

setup(
    name='examplepackage'
    version='1.0.1',
    author='Giorgos Myrianthous',
    description='This is an example project',
    long_description='This is a longer description for the project',
    url='https://medium.com/@gmyrianthous',
    keywords='sample, example, setuptools',
    python_requires='>=3.7, <4',
    install_requires=['pandas'],
    extras_require={
        'test': ['pytest', 'coverage'],
    },
    package_data={
        'sample': ['example_data.csv'],
    },
    entry_points={
        'runners': [
            'sample=sample:main',
        ]
    }
)

And here’s the explanation for the keywords that were used when calling the setup() method:

name: This is a string corresponding to the name of the package
version: A string corresponding to the version number of the package
author: Indicates the author(s) of the package
description: This is a string for a short description of the package (usually a single-line description)
long_description: A string for a longer description for the package
url: A string indicating the URL to the package (usually the GitHub repository or the PyPI page).
keywords: This is a comma-separated string (can also be a list of strings) that contains some keywords relevant to the package
python_requires: This is a comma-separate string that contains version specifiers for the Python version supported by the package.
install_requires: A list of strings containing the minimal dependencies required in order for the package to run successfully.
extras_require: A dictionary where the keys correspond to names of extra modes and values are lists containing the minimal dependencies required. For example, an extra mode could be test where the list of dependencies should include all the additional packages required in order to execute the tests of defined in the package.
package_data: This is a dictionary where keys are package names and values are lists of glob patterns.
entry_points: This is a dictionary where keys correspond to entry point names and values to the actual entry points defined in the source code.

For a more comprehensive list on the available keywords for metadata and options, you can refer to relevant section in the official documentation.

The setup.cfg file

Traditionally, the setup.py file was used in order to build packages, e.g. using the build command through the command line interface.

$ python setup.py build

In the example setup.py file demonstrated above, we’ve seen that most of the code is just about listing some options and metadata about the Python project. In fact, this may not be the best approach in terms of code quality and design. Therefore, specifying these package details in the setup.cfg file may be more appropriate.

The setup.cfg is an ini file, containing option defaults for setup.py commands. You can pretty much specify every keyword we used in the setup.py file in the new setup.cfg file and simply use the setup.py file as the command line interface.

Example setup.cfg file

# setup.cfg file at the root directory

[metadata]
name = examplepackage
version = 1.0.1
author = Giorgos Myrianthous
description = This is an example project
long_description = This is a longer description for the project
url = https://medium.com/@gmyrianthous
keywords = sample, example, setuptools

[options]
python_requires = >=3.7, <4
install_requires = 
    pandas

[options.extras_require]
test = 
    pytest
    coverage

[options.package_data]
sample = 
    example_data.csv'

setup.py with setup.cfg

Now assuming that you have moved all the options into a setup.cfg file as described in the previous section, you can now create a dummy setup.py that will simply call the setup() method:

from setuptools import setup

if __name__ == '__main__':
    setup()

For more comprehensive examples on how to use setup.py and setup.cfg files you can refer the PyPA sample project on GitHub.

The pyproject.toml file and the transition to PEP-517

As of PEP-517 and PEP-518 setuptools is no longer the de-facto tool that is supposed to be used when packaging Python projects. According to the specification of PEP-518, the build system dependencies for a Python project should be included in a file named pyproject.toml that follows the TOML format.

Over time, setup.py was gaining popularity among the Python community but one of the biggest problems with setuptools is that the use of an executable file (i.e. the setup.py) cannot be executed without knowing its dependencies. And there is really no way to know what these dependencies are unless you actually execute the file that contains the information related to package dependencies.

The pyproject.toml file is supposed to solve the build-tool dependency chicken and egg problem since pip itself can read pyproject.yoml along with the version of setuptools or wheel the project requires. The file will expect a [build-system] table that is used to store build-related information.

Now let’s put the pieces of the puzzles together. If you want to use setuptools you now have to specify a pyproject.toml file with the content shown below:

# pyproject.toml file specified at the root of the directory

[build-system]
requires = ["setuptools>=42", "wheel"]   # PEP 508 specifications.
build-backend = "setuptools.build_meta"

Then you also need to specify a setup.py or setup.cfg file as we showcased in the previous sections of the tutorial. Note that I personally prefer the latter notation.

Finally, you can build your project using a builder, such as PyPA build that you can retrieve through pip (pip install build) and finally invoke the builder

$ python3 -m build

And voila! Your distribution will now be ready to be uploaded on PyPI so that it can be widely accessible.

Note that if you want to install packages in editable mode (i.e. by running pip install -e .) you must have a valid setup.py file apart from setup.cfg and pyproject.toml. You can use the same dummy setup file I shared in the previous section that makes just a single call to the setup() method.

So do I still need a requirements.txt file?

The short answer is, most probably yes. setuptools files are supposed to hold abstract dependencies while requirements.txt is supposed to list concrete dependencies.

For a more comprehensive read with respect to the purpose of requirements.txt and setuptools you can read one of my recent articles shared below.

requirements.txt vs setup.py in Python

Final Thoughts

In today’s article we discussed about the setuptools package and how to make use of setup.py and setup.cfg files in order to manage package dependencies and make package distribution easier in Python.

Additionally, we discussed about the various options and metadata that can be specified as well as the differences between setup.py and setup.cfg files. We also discussed about requirements.txt file and how it can be used to complete the puzzle of dependency management in Python packages. Finally, we introduced PEP517 and the purpose of a file called pyproject.toml.

As a final note, remember that the package distribution guidelines in Python are constantly evolving – especially in the last few years – so make sure to keep yourself updated and follow the best practises from time to time.

Become a member and read every story on Medium. Your membership fee directly supports me and other writers you read. You’ll also get full access to every story on Medium.