Maniphest T420475

Rework linters and checkers of I/F Python repositories for automation
Open, MediumPublic
Actions

Assigned To

None

Authored By

	• elukey
	Mar 18 2026, 2:49 PM

Project Tags

Referenced Files

None

Subscribers

Description

New linters for Python like ruff are nowadays reliable and way faster than their "traditional" counterparts in pure Python.

In https://gerrit.wikimedia.org/r/c/operations/software/pywmflib/+/1152016 me and Riccardo took a stab in refactoring tox settings for pywmflib, selecting a series of options for ruff that made sense for the repo.

Federico has been working on https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/1240635 to do a similar work for the cookbooks repo, It would be great if we could align the configurations and apply them to:

spicerack
cookbook
pywmflib

And possibly many more after that, if needed. It would really reduce the current pain in waiting a long time for CI to complete when a code review is filed, and/or to run local linters/tests before sending it.

I propose to work on the following:

Figure out a common set of configuration for tox to introduce ruff to the above repositories.
Work with releng on a new solution to test Python projects in CI, similar to the current python-tox. It would be nice to use uv instead of pip as package manager for example, to speed up the venv creation (that is currently a slow process as well).

Details

Related Changes in Gerrit:

Subject	Author	Repo	Branch	Lines +/-
_cookbook: fix parallel test failures with pytest-xdist (-n auto)	Elukey	operations/software/spicerack	master	+13 -5
Improve tox and setup's configuration	Elukey	operations/cookbooks	master	+13 -17
tox: rework venvs to speed up local and CI timings	Elukey	operations/software/spicerack	master	+16 -24
Move linting to Ruff and apply code fixes	Elukey	operations/software/spicerack	master	+278 -278
First pass of ruff check --fix	Elukey	operations/software/spicerack	master	+99 -123

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T420475 Rework linters and checkers of I/F Python repositories for automation
		Open		None	T421348 Add tox-uv support to the tox-v{3,4} Docker images

Event Timeline

• elukey created this task.Mar 18 2026, 2:49 PM

• elukey triaged this task as Medium priority.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 18 2026, 2:49 PM

I tried to compare what we did with wmflib with Federico's proposal, to try to summarize a possible standard approach. The differences are:

prospector.yaml is deleted in the cookbook's change, while we kept a basic functionality for wmflib (pyroma, vulture mostly).
pyproject.toml in the cookbook's change contain ruff and non ruff config, meanwhile in the wmflib version there are two files (pyproject.toml and ruff.toml).
The ruff config is mostly the same, there are some differences like convention = "google" for the pydoc but nothing substantial afaics.

Overall I think the two configs can co-exists, we could standardize the name of the files etc.. but it can be done on a later stage. Probably the question is if we want to keep pyroma/vulture/prospector or not. @FCeratto-WMF what is your opinion since you removed it? So we can take a decision and move on.

Thanks a lot for the work!

After a chat with Riccardo prospector is used in wmflib only for pyroma and vulture, and the latter is disabled for cookbooks, so now I better understand Federico's intent.

• elukey created subtask T421348: Add tox-uv support to the tox-v{3,4} Docker images.Mar 26 2026, 10:54 AM

@elukey indeed I noticed that prospector was not that useful anymore as vulture was not used, many checks are done by ruff, and the only remaining part is pyroma. If desired I can add back pyroma by itself.

Change #1265476 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/software/spicerack@master] First pass of ruff check --fix

https://gerrit.wikimedia.org/r/1265476

gerritbot added a project: Patch-For-Review.Mar 31 2026, 4:14 PM

Change #1265476 abandoned by Elukey:

[operations/software/spicerack@master] First pass of ruff check --fix

Reason:

We'll probably need to go in one pass, abandoning for the moment.

https://gerrit.wikimedia.org/r/1265476

Maintenance_bot removed a project: Patch-For-Review.Apr 1 2026, 7:30 AM

Change #1267058 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/software/spicerack@master] [WIP] Move linting to Ruff and apply code fixes

https://gerrit.wikimedia.org/r/1267058

gerritbot added a project: Patch-For-Review.Apr 2 2026, 1:58 PM

Change #1267678 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/software/spicerack@master] tox: rework venvs to speed up local and CI timings

https://gerrit.wikimedia.org/r/1267678

Filed a couple of changes to introduce ruff to spicerack and rework how linting/testing/docs run. I got down to ~60s local time and ~3m CI time (with tox creating venvs and installing deps), and ~19s while running locally with venvs already installed.

Change #1267058 merged by jenkins-bot:

[operations/software/spicerack@master] Move linting to Ruff and apply code fixes

https://gerrit.wikimedia.org/r/1267058

Change #1270380 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/software/spicerack@master] _cookbook: fix parallel test failures with pytest-xdist (-n auto)

https://gerrit.wikimedia.org/r/1270380

Change #1267678 merged by Elukey:

[operations/software/spicerack@master] tox: rework venvs to speed up local and CI timings

https://gerrit.wikimedia.org/r/1267678

Change #1271594 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/cookbooks@master] Improve tox and setup's configuration

https://gerrit.wikimedia.org/r/1271594

Change #1271594 merged by jenkins-bot: