I'm adding Performance-Team for guidance too, I'm wondering how much work would it be to adapt xhgui to twig 3 ? If that's a lot of work we can consider having compatible twig in bookworm (cc @Krinkle)

I had a closer look at this: Currently xhgui upstream only supports Twig 1.x. If someone from Perf team can submit patches upstream to move to current API that would be fantastic, but this seems unlikely available as a short term fix for the migration. At the minimum we should file an upstream issue to ask for support for Twig 3.

I did some investigation on the history of the current Twig package. One notable source of confusion is that the Debian source package name changed over time: Currently it's "php-twig", but older versions of Debian used "twig". Digging in Phab history when xhgui was packaged (https://phabricator.wikimedia.org/T254310) Dave imported the last version of Twig 1.x which was in Debian Stretch,

Since upstream xhgui doesn't yet support Twig 3 and Twig 1.x is still supported (1.44.7 came out some months ago), one other option would be to simply take the last package in use and update it to 1.44.7. I did a quick PoC build (I needed to disable the docs building due to some changes in Sphinx due to some changes between Debian 9 and 12, but not important for us) and the result can be found on build2001 in /var/cache/pbuilder/result/bookworm-amd64/php-twig_1.44.7-1_all.deb

Could you test it on the new Bookworm hosts, please? If that works then we can create a repository component component/twig1 for bookworm-wikimedia and update Puppet to use it for Bookworm.

Thanks for you assistance and support, Mortiz. I'm currently testing the changes in the Bookworm hosts and I'll keep you posted.
In the meantime I filed an upstream issue kindly asking the XHGui developers for their assistance in supporting Twig 3.

Warm regards.

As a short-term measure, I recommend we update to Twig 1.44.7 which upstream already supports.

Upstream does not support managing Composer dependencies via Debian packages, and indeed most of its dependencies have no equivalent Debian packages (only php-twig exists, the rest is custom to WMF). As such, the fact that the php-twig package was out of date for a long time, or that XHGui does not support the somewhat arbitrary version Debian happens to package, is not surprising. Indeed, most of the PHP ecosystem does not support managing depencies via Debian. This includes, perhaps most notably MediaWiki, and also WordPress.

I suggest we instead deploy XHGui from Git (initially using Scap, but possibly later using Blubber/Kubernetes like we do with Shellbox and other PHP services). This will effectively undo T254310, as we already deployed XHGui this way before that change.

Record of changes (given that gerritbot can't tag private tasks)

• I've un-archived the https://gerrit.wikimedia.org/g/operations/software/xhgui repository.
• Updated wmf_deploy to match upstream 0.12.0 which is the version we currently have in production (no-op).
• Create https://gerrit.wikimedia.org/g/performance/xhgui-vendor and populate with current version of dependencies, except with the updated twig 1.44.7 and xhgui-collector 1.8.1. For details, check the commit message at https://gerrit.wikimedia.org/g/performance/xhgui-vendor/+/bfce9a5d54cf2b284ce7c0d430984fb631881b87.
• Added the repo as a submodule at https://gerrit.wikimedia.org/r/c/operations/software/xhgui/+/935168

What remains is:

• write a Puppet patch that exposes xhgui as a simple docroot git clone
• try it out in beta and confirm that https://performance.wikimedia.beta.wmflabs.org/xhgui/ still works
• deploy to prod, and verify at https://performance.wikimedia.org/xhgui/
• decom deployment-xhgui03 from beta cluster
• decom xhgui1001.eqiad/xhgui2001.codfw from prod
• decom xhgui Puppet provisioning.

What remains is:

• write a Puppet patch that exposes xhgui as a simple docroot git clone
• deploy to prod, and verify at https://performance.wikimedia.org/xhgui/

Ready for review at https://gerrit.wikimedia.org/r/c/operations/puppet/+/935512/.

• try it out in beta and confirm that https://performance.wikimedia.beta.wmflabs.org/xhgui/ still works

Done. I've cherry-picked the above to the the local puppetmaster in Beta Cluster (deployment-prep).

• decom deployment-xhgui03 from beta cluster

Done. Removed via Horizon.

• decom xhgui1001.eqiad/xhgui2001.codfw from prod

Once the above is deployed and I've verified that we don't need to switch back, the xhgui* Debian 10 Buster hosts can be deleted from Ganeti.

Note that the webperf* hosts already run Debian 11 Bullseye.

• decom xhgui Puppet provisioning.

Ready for review at https://gerrit.wikimedia.org/r/c/operations/puppet/+/935522/.

Decommission tasks for the Bookworm hosts: T341160, T341161