Just general advice from a "eureka" moment I had at the recent hackathon... we should probably avoid any file system operations on Toolforge. The actual issue boils down to the fragility of NFS. So use APCu instead of file caching where possible, etc.

That said,

Cache the version number, for maybe 10 minutes. That would reduce the number of times the command has to be run. During times of quick development it could be slightly annoying to sometimes have the displayed version number be out of sync with the actual version number, but for most cases it probably wouldn't bother anyone.

Why not longer, say a full week? It's standard practice to clear the cache on deploy, and it's in the deploy script. Locally, sure, it might be a little misleading. Maybe we could check the environment and conditionally cache?

With such a brief cache expiry, I'm guessing we will still get emails about whatever tool being down since UptimeRobot runs at 5 minute cycles. Inevitably it will hit the NFS issue at some point.

The actual issue boils down to the fragility of NFS.

Though that doesn't explain why WS Export has the git commands timing out (T335553#9014538), as VPS does not use NFS. That is mysterious to me, as other VPS tools like XTools and Wikimedia OCR that do the same git operations do not seem to time out.

Regardless, I do believe the issue with NFS still stands. I was chatting with Cloud Services at the Hackathon and they confirmed that is the likely culprit as to why file system-related things often fail on Toolforge.

Good point about avoiding file operations on NFS. We're using the filesystem cache adapter for WS Export, and the files are stored on the volume mounted at /ws-export. So I also don't know why WS Export is failing there. But SVG Translate is probably failing at least sometimes due to NSF slowness. I've noticed sometimes that doing git status in the svgtranslate directory can take a long time (although never minutes, I don't think).

It's probably a good idea to move to a faster caching system, on Toolforge and VPS, regardless. The cache directories are not currently large:

• SVG Translate: /data/project/svgtranslate/app/var/cache/prod/pools/ 1.8M
• WS Export: /ws-export/var/cache/prod/pools/ 880K

So perhaps moving to Redis would be good?

Anyway, I've started a pull request for updating SVG Translate to use the new version of the bundle, with Git caching, but it's failing on unrelated stuff that I can't currently replicate on my local! So that's annoying. I'll get that fixed, and the caching deployed, and then think about further work to improve things.

In T334454#9016276, @MusikAnimal wrote:

It's standard practice to clear the cache on deploy, and it's in the deploy script.

Actually, it's only the application cache that's cleared when we run composer install, the cache pools are not touched unless you run ./bin/console cache:pool:clear <pools>....

I converted another Symfony tool recently to use Redis and it was pretty easy and now works much faster.

In T334454#9151444, @Samwilson wrote:

Actually, it's only the application cache that's cleared when we run composer install, the cache pools are not touched unless you run ./bin/console cache:pool:clear <pools>....

I've worked with Symfony for however many years now and never knew this... thanks! Ref: https://symfony.com/doc/current/cache.html#clearing-the-cache

I converted another Symfony tool recently to use Redis and it was pretty easy and now works much faster.

Sounds good to me!

The patches have been failing due to a peculiar issue with PHP changing how it handled XML prefixes. For example, the following code produces different output only for 8.1.21–8.1.24 and 8.2.8–8.2.11:

<?php
$dom = new DOMDocument();
$dom->loadXML('<?xml version="1.0" encoding="UTF-8"?><svg xmlns:svg="http://www.w3.org/2000/svg" xmlns="http://www.w3.org/2000/svg"><text><svg:tspan>T</svg:tspan></text></svg>');
$text = $dom->getElementsByTagName('text')[0];
$switch = $dom->createElementNS('http://www.w3.org/2000/svg', 'switch');
$text->parentNode->insertBefore($switch, $text);
$switch->appendChild($text);
echo $dom->saveXML($dom->getElementsByTagName('switch')[0]);

Output for 8.0.1 - 8.0.30, 8.1.0 - 8.1.20, 8.1.25 - 8.1.27, 8.2.0 - 8.2.7, 8.2.12 - 8.2.16, 8.3.0 - 8.3.3

<svg:switch><svg:text><svg:tspan>T</svg:tspan></svg:text></svg:switch>

Output for 8.1.21 - 8.1.24, 8.2.8 - 8.2.11

<switch><text><tspan>T</tspan></text></switch>

See it working here: https://3v4l.org/YMY6j

So I'm updating the tests to make the prefix optional, for greatest flexibility for developers working on it (and because as far as actually rendering the SVG goes either form seems to work fine). There are various other open tasks about how we handle XML prefixes in SVG Translate, so perhaps this will need revisiting at some point, but for now I just want to get things working again.

We can also update the Toolforge webservice to PHP 8.2 which will help avoid some other issues we've got with the current tests.

I've made a minimal PR that gets things running again: https://github.com/wikimedia/svgtranslate/pull/725 (ready for review)

The PHP 8.2 upgrade I think can wait; I've created T357844 for that.

I've tagged and released version 1.2.4… and something seems to be broken! The preview doesn't work now. I'm investigating.