Page MenuHomePhabricator

New upstream release for Pygments (2.19.2)
Closed, ResolvedPublicFeature

Description

To whom it may concern:

A new upstream version of Pygments is now available: 2.19.2.

Previously: T364249: New upstream release for Pygments (2.18.0)

Event Timeline

bd808 renamed this task from New upstream release for Pygments to New upstream release for Pygments (2.19.0).Jan 6 2025, 4:50 PM
bd808 changed the task status from Open to Stalled.Jan 6 2025, 4:53 PM
bd808 subscribed.

Probably easiest to roll into T364249: New upstream release for Pygments (2.18.0), but both are still blocked on getting a Python 3.8+ runtime into the shellbox container used for the Wikimedia production wiki deploment.

bd808 changed the subtype of this task from "Task" to "Feature Request".
bd808 renamed this task from New upstream release for Pygments (2.19.0) to New upstream release for Pygments (2.19.1).Jan 6 2025, 5:57 PM
bd808 updated the task description. (Show Details)
bd808 updated the task description. (Show Details)
bd808 changed the task status from Stalled to Open.Jun 5 2025, 10:11 PM
bd808 changed the task status from Open to In Progress.Jun 6 2025, 3:22 PM
bd808 claimed this task.
bd808 added a subscriber: taavi.

While reviewing the upstream changes here I noticed there are some bug fixes from @taavi that are included. :)

bd808 renamed this task from New upstream release for Pygments (2.19.1) to New upstream release for Pygments (2.19.2).Jun 24 2025, 5:39 PM
bd808 updated the task description. (Show Details)

2.19 also has support for the Vue mode, which would be useful for the newly created content model (T340460).

(@bd808 I see this has been marked as in-progress for a while. Any update?)

(@bd808 I see this has been marked as in-progress for a while. Any update?)

I got busy with other things, but I will try to get back on track soon. I think I was having some weird problem with my MediaWiki dev environment when I last touched this... hopefully I will figure that out.

$ ./create_pygmentize_bundle
/usr/bin/env: ‘python3’: No such file or directory

Now I remember where I got stuck. :/

In T351744#9376409 I described an update process using a Toolforge container. Today I worked out how to do the needful using https://www.mediawiki.org/wiki/MediaWiki-Docker. Today that requires a bit of custom setup as the base image that we are using for MediaWiki there (docker-registry.wikimedia.org/dev/bookworm-php83-fpm:1.0.0) does not currently include a python3 runtime.

I started with a variation on https://www.mediawiki.org/wiki/MediaWiki-Docker/Configuration_recipes/Customize_base_image to get a base image with the needed software. I decided to use Blubber as my Dockerfile solution due to personal familiarity and the potential to upstream this at some point. I also wanted to avoid having a bunch of untracked files in my git clone so after looking at .gitignore I decided to make a .settings directory to tuck things away in.

.settings/blubber.yaml
# syntax=docker-registry.wikimedia.org/repos/releng/blubber/buildkit:v1.3.0
version: v4

variants:
  mediawiki:
    base: docker-registry.wikimedia.org/dev/bookworm-php83-fpm:1.0.0
    runs:
      insecurely: true
    lives:
      in: /var/www/html/w
    apt:
      packages:
        - python3
docker-compose.override.yml
services:
  mediawiki:
    build:
      context: .
      dockerfile: .settings/blubber.yaml
      target: mediawiki
      args:
        LIVES_UID: $MW_DOCKER_UID
        LIVES_GID: $MW_DOCKER_GID
    image: "mediawiki:custom"
$ docker compose build
[+] Building 14.8s (10/10) FINISHED
 => [internal] load local bake definitions                                 0.0s
 => => reading from stdin 515B                                             0.0s
 => [internal] load build definition from blubber.yaml                     0.0s
 => => transferring dockerfile: 335B                                       0.0s
 => resolve image config for docker-image://docker-registry.wikimedia.org  0.5s
 => CACHED docker-image://docker-registry.wikimedia.org/repos/releng/blub  0.0s
 => CACHED [mediawiki] 🌐 docker-registry.wikimedia.org/dev/bookworm-ph     1.0s
 => => resolve docker-registry.wikimedia.org/dev/bookworm-php83-fpm:1.0.0  1.0s
 => [mediawiki] 🖥️ # apt-get update && apt-get install -y "python3"       11.6s
 => [mediawiki] 🖥️ # (getent group "20" || groupadd -o -g "20" -r "s       0.3s
 => [mediawiki] 🖥️ # (getent group "900" || groupadd -o -g "900" -r        0.2s
 => exporting to image                                                     0.1s
 => => exporting layers                                                    0.1s
 => => writing image sha256:00021589d13bb5698ebdb7124f596560d5ac2c07cd8d3  0.0s
 => => naming to docker.io/library/mediawiki:custom                        0.0s
 => resolving provenance for metadata file                                 0.0s
[+] Building 1/1
 ✔ mediawiki  Built                                                        0.0s
$ docker compose down
$ docker compose up -d

Now that my local dev container has a python3 runtime I can use it to update and test the extension:

$ docker compose exec mediawiki extensions/SyntaxHighlight_GeSHi/pygments/create_pygmentize_bundle
Querying PyPI for the latest Pygments release...
Retrieving version 2.19.2 (https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl)...
Verifying...
Creating executable ZIP bundle...
Done. Wrote 1225467 bytes to /var/www/html/w/extensions/SyntaxHighlight_GeSHi/pygments/pygmentize
Don't forget to run updateCSS.php and updateLexerList.php (in ../maintenance).
$ docker compose exec mediawiki php maintenance/run.php ./extensions/SyntaxHighlight_GeSHi/maintenance/updateCSS.php
CSS written to /var/www/html/w/extensions/SyntaxHighlight_GeSHi/modules/pygments.generated.css
$ docker compose exec mediawiki php maintenance/run.php ./extensions/SyntaxHighlight_GeSHi/maintenance/updateLexerList.php
Updated language list written to SyntaxHighlight.lexers.php
$ cd extensions/SyntaxHighlight_GeSHi/
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   SyntaxHighlight.lexers.php
        modified:   modules/pygments.generated.css
        modified:   pygments/VERSION
        modified:   pygments/pygmentize

no changes added to commit (use "git add" and/or "git commit -a")

Change #1171738 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[mediawiki/extensions/SyntaxHighlight_GeSHi@master] Update Pygments to 2.19.2

https://gerrit.wikimedia.org/r/1171738

Change #1171742 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[mediawiki/libs/Shellbox@master] syntaxhighlight: Update to 2.19.2

https://gerrit.wikimedia.org/r/1171742

bd808 triaged this task as Medium priority.Jul 23 2025, 12:13 AM

Change #1171738 merged by jenkins-bot:

[mediawiki/extensions/SyntaxHighlight_GeSHi@master] Update Pygments to 2.19.2

https://gerrit.wikimedia.org/r/1171738

Change #1172034 had a related patch set uploaded (by Reedy; author: Bryan Davis):

[mediawiki/extensions/SyntaxHighlight_GeSHi@REL1_43] Update Pygments to 2.19.2

https://gerrit.wikimedia.org/r/1172034

Change #1172035 had a related patch set uploaded (by Reedy; author: Bryan Davis):

[mediawiki/extensions/SyntaxHighlight_GeSHi@REL1_44] Update Pygments to 2.19.2

https://gerrit.wikimedia.org/r/1172035

Change #1172034 merged by jenkins-bot:

[mediawiki/extensions/SyntaxHighlight_GeSHi@REL1_43] Update Pygments to 2.19.2

https://gerrit.wikimedia.org/r/1172034

Change #1172035 merged by jenkins-bot:

[mediawiki/extensions/SyntaxHighlight_GeSHi@REL1_44] Update Pygments to 2.19.2

https://gerrit.wikimedia.org/r/1172035

Change #1171742 merged by jenkins-bot:

[mediawiki/libs/Shellbox@master] syntaxhighlight: Update to 2.19.2

https://gerrit.wikimedia.org/r/1171742

Change #1173411 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/deployment-charts@master] shellbox: Bump image versions to 025-07-28-151806

https://gerrit.wikimedia.org/r/1173411

Change #1173411 merged by jenkins-bot:

[operations/deployment-charts@master] shellbox: Bump image versions to 025-07-28-151806

https://gerrit.wikimedia.org/r/1173411

I used the process from https://wikitech.wikimedia.org/wiki/Shellbox#Deploying_a_new_version to deploy the new containers to the "staging", "codfw", and "eqiad" clusters.