Page MenuHomePhabricator

tbs: user-story 12: use pre-commit for the toolforge-cli repo
Closed, ResolvedPublic2 Estimated Story Points


We currently have two bash scripts for linting & formatting:


Linting & formatting could be moved to use pre-commit, that is then run via tox. In my experience, it's cleaner to handle all linting & formatting config in one single pre-commit-config.yaml file and orchestrate everything with tox instead of relying on bash scripts.

Pre-commit is...

  • an open-source framework for managing and maintaining multi-language pre-commit hooks
  • It has been around for a long time and has over 6 million monthly downloads on PyPI
  • Because new hooks are constantly added (by the project's maintainers and the community), there is rarely a need to reinvent the wheel
  • It does not require root access

Event Timeline

Can you move this to a decision request? (

Some things to keep in mind:

  • Offer more than one option (ex. using pre-commit + tox, using scripts + tox, using only tox, using only scripts, ...)
  • Try to describe the same aspects for every option (ex. usability (installation, usage, maintenance), open-source, effort required, ...)
  • One of the options should be "staying as we are" kind of thing
Slst2020 changed the task status from Open to In Progress.Dec 14 2022, 10:45 AM
Slst2020 claimed this task.
dcaro renamed this task from [] Proposal: use pre-commit instead of custom bash scripts to tbs: user-story 12: use pre-commit for the toolforge-cli repo.Dec 14 2022, 3:43 PM

To avoid writing too long a commit message, I'm detailing the changes here for reference:

  • The specific configuration of each tool can be found in either pyproject.toml, or in tox.ini for flake8, which still (!) has no compatibility with pyproject.toml. This is just as before – no changes here.
  • The main difference is that all the commands are now run using pre-commit. Pre-commit creates its own environment for each tool, installs the project dependencies with poetry, and runs the tool. The way and order in which the tools are run is configured inside .pre-commit-config.yaml.
  • A few common checks/lints have been added:
    • removal of trailing whitespace
    • making sure each file ends with an empty line
    • a toml checker
    • a yaml checker
  • All the tools pre-commit runs are the latest version. This needs to be updated from time to time by running poetry run pre-commit autoupdate locally.
  • I've set the Python version to 3.10 for all checks. This was a somewhat arbitrary choice, and can easily be changed. Or, kind of – there's a bug when running the latest version of flake8 with Python 3.7, for which the fix is to either upgrade the Python version, or downgrade flake8.
  • As mentioned above, pre-commit takes care of all its dev dependencies, so these have been removed from pyproject.toml. Tools that need additional dependencies, such as mypy (types-requests, types-PyYAML...), have these defined inside .pre-commit-config.yaml.

Unrelated to pre-commit, all dev dependencies inside pyproject.toml have been moved to [], as the old section naming is about to become deprecated.

Change 868332 had a related patch set uploaded (by Slavina Stefanova; author: Slavina Stefanova):

[cloud/toolforge/toolforge-cli@main] dev: Use pre-commit for code-quality checks

Change 868332 merged by jenkins-bot:

[cloud/toolforge/toolforge-cli@main] dev: Use pre-commit for code-quality checks