Page MenuHomePhabricator

Reduce the number of CI images
Open, Needs TriagePublic

Description

I count 98 images defined in integration/config. With the new Zuul and Nodepool, we will have the jobs tied to labels which themselves are associated to one, or several, image (s). Example:

/etc/nodepool/nodepool.yaml
labels:
  - name: quibble-bullseye-php81
  - name: quibble-bullseye-php82
  - name: quibble-bullseye-php83
  - name: quibble-bullseye-php84

providers:
      - name: mediawiki
        max-servers: 8
        labels: 
          - !!merge : *default_label
            name: quibble-bullseye-php81
            image: docker-registry.wikimedia.org/releng/quibble-bullseye-php81:1.14.1-s2
          - !!merge : *default_label
            name: quibble-bullseye-php82
            image: docker-registry.wikimedia.org/releng/quibble-bullseye-php82:1.14.1-s2
          - !!merge : *default_label
            name: quibble-bullseye-php83
            image: docker-registry.wikimedia.org/releng/quibble-bullseye-php83:1.14.1-s2
          - !!merge : *default_label
            name: quibble-bullseye-php84
            image: docker-registry.wikimedia.org/releng/quibble-bullseye-php84:1.14.1-s2

Having 98 such labels is a lot, we can certainly find a way to reduce them by aggregating images.

For example the php-ast image has been removed by embedding the build in the existing php images T396312. As part of doing that task, we have php??-compile images which I guess can be merged in?

Same for composer for which we have three variants per PHP version:

composer-package-php81
composer-package-php82
composer-package-php83
composer-package-php84
composer-php81
composer-php82
composer-php83
composer-php84
composer-scratch
composer-test-php81
composer-test-php82
composer-test-php83
composer-test-php84

We can surely provide a single php81 image that has composer, can do the package work and compile an extension.

Same for node:

node24
node24-test
node24-test-browser
node24-test-browser-php81-composer
node24-test-browser-php83-composer

Though the browser ones are heavy.

Event Timeline

We also still have a bunch of Buster based images: T335765

Change #1167650 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: remove tox-poolcounter

https://gerrit.wikimedia.org/r/1167650

Change #1167650 merged by jenkins-bot:

[integration/config@master] dockerfiles: remove tox-poolcounter

https://gerrit.wikimedia.org/r/1167650

Change #1170585 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: add test script to composer images

https://gerrit.wikimedia.org/r/1170585

Change #1170587 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: update for composer-php image with new entrypoints

https://gerrit.wikimedia.org/r/1170587

Change #1170592 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: switch composer-package jobs to base composer image

https://gerrit.wikimedia.org/r/1170592

Change #1170593 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: remove composer-package-php* images

https://gerrit.wikimedia.org/r/1170593

Change #1170594 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: switch composer-test jobs to base composer image

https://gerrit.wikimedia.org/r/1170594

Change #1170595 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: remove composer-test-php* images

https://gerrit.wikimedia.org/r/1170595

Change #1170585 merged by jenkins-bot:

[integration/config@master] dockerfiles: add test script to composer images

https://gerrit.wikimedia.org/r/1170585

Mentioned in SAL (#wikimedia-releng) [2025-07-21T14:30:48Z] <James_F> Docker: [composer] Add test scripts to composer images, for T397429

Change #1170587 merged by jenkins-bot:

[integration/config@master] jjb: update for composer-php image with new entrypoints

https://gerrit.wikimedia.org/r/1170587

Change #1170592 merged by jenkins-bot:

[integration/config@master] jjb: switch composer-package jobs to base composer image

https://gerrit.wikimedia.org/r/1170592

Change #1170594 merged by jenkins-bot:

[integration/config@master] jjb: switch composer-test jobs to base composer image

https://gerrit.wikimedia.org/r/1170594

Change #1170595 merged by jenkins-bot:

[integration/config@master] Docker: Drop composer-test-php* images, now unused

https://gerrit.wikimedia.org/r/1170595

Change #1171235 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] [WIP] Docker: [mediawiki-phan*] Migrate to Quibble hierarchy

https://gerrit.wikimedia.org/r/1171235

Change #1172087 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] [WIP] jjb: Switch phan jobs over to using Quibble image

https://gerrit.wikimedia.org/r/1172087

Change #1171235 merged by jenkins-bot:

[integration/config@master] Docker: [quibble-bullseye] Copy Phan running scripts so we can use these images

https://gerrit.wikimedia.org/r/1171235

Mentioned in SAL (#wikimedia-releng) [2025-07-24T13:09:00Z] <James_F> Docker: [quibble-bullseye] Copy Phan running scripts so we can use these images, for T397429

Change #1172327 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Docker: Drop mediawiki-phan-php*, no longer used

https://gerrit.wikimedia.org/r/1172327

Change #1172087 merged by jenkins-bot:

[integration/config@master] jjb: Switch phan jobs over to using Quibble image

https://gerrit.wikimedia.org/r/1172087

Change #1172327 merged by jenkins-bot:

[integration/config@master] Docker: Drop mediawiki-phan-php*, no longer used

https://gerrit.wikimedia.org/r/1172327

Change #1170593 merged by jenkins-bot:

[integration/config@master] Docker: Drop composer-package-php* images, now unused

https://gerrit.wikimedia.org/r/1170593

There is a new Phan warning in MediaWiki, I suspect it's caused by these config changes (which I think were deployed here: https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL#2025-07-25 13:40 James_F: Docker: [quibble-bullseye] Update Quibble to 1.14.2):

includes/diff/TextDiffer/Wikidiff2TextDiffer.php:159 PhanParamTooManyInternal Call with 4 arg(s) to \wikidiff2_do_diff(string $text1, string $text2, int $numContextLines) which only takes 3 arg(s). This is an ArgumentCountError for internal functions in PHP 8.0+.

Fixing in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1172658.

There is a new Phan warning in MediaWiki, I suspect it's caused by these config changes (which I think were deployed here: https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL#2025-07-25 13:40 James_F: Docker: [quibble-bullseye] Update Quibble to 1.14.2):

includes/diff/TextDiffer/Wikidiff2TextDiffer.php:159 PhanParamTooManyInternal Call with 4 arg(s) to \wikidiff2_do_diff(string $text1, string $text2, int $numContextLines) which only takes 3 arg(s). This is an ArgumentCountError for internal functions in PHP 8.0+.

Fixing in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1172658.

No, the phan changes were deployed last night in https://gerrit.wikimedia.org/r/c/integration/config/+/1172087 — did it break with that change, or only with the new version of Quibble? Either way, it shouldn't have happened, but Phan is sadly variable in its run paths.

There is a new Phan warning in MediaWiki, I suspect it's caused by these config changes (which I think were deployed here: https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL#2025-07-25 13:40 James_F: Docker: [quibble-bullseye] Update Quibble to 1.14.2):

includes/diff/TextDiffer/Wikidiff2TextDiffer.php:159 PhanParamTooManyInternal Call with 4 arg(s) to \wikidiff2_do_diff(string $text1, string $text2, int $numContextLines) which only takes 3 arg(s). This is an ArgumentCountError for internal functions in PHP 8.0+.

Fixing in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1172658.

No, the phan changes were deployed last night in https://gerrit.wikimedia.org/r/c/integration/config/+/1172087 — did it break with that change, or only with the new version of Quibble? Either way, it shouldn't have happened, but Phan is sadly variable in its run paths.

Looks like the phan stub for wikidiff is no longer used as the real functions are part of the new image. Phan is now using the real function declaration and fails on the compact code.

Change #1174399 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: remove java8-scala-spark

https://gerrit.wikimedia.org/r/1174399

Change #1174399 merged by jenkins-bot:

[integration/config@master] dockerfiles: remove java8-scala-spark

https://gerrit.wikimedia.org/r/1174399