The main workflow here is using the grid jobs framework to compile something with && webservice restart

Mostly to remind myself, the source code is at https://github.com/toollabs/daily-image-l but there are uncommitted changes on the tool itself. It's still in Python 2, but porting it should be trivial. All of the use of mwparserfromhell can be improved by using Parsoid HTML, so it's probably worth porting to Rust too.

I've stopped the updating job and marked the tool as no longer updating.

I will probably need past the December 14th initial deadline to migrate this tool since most of the code is pretty legacy.

Is there any data in chat that needs to be archived/saved somewhere?

Should be done now: https://github.com/mzmcbride/database-reports/commit/e78577b7f6bd19c584d78748befaf091c5a50071

In T320062#9355901, @Izno wrote:

https://steve-adder.toolforge.org/ seems to have gone down, I'd guess as a result of this task. :)

The main dbreps job is running k8s now, it's just the build-rust.sh script that needs to be ported.

I've disabled the cron job for now, it was broken for other reasons. This tool did useful stuff in the past so I would like to revive it eventually but when that happens it will be grid-free.

I've disabled this tool, source code is at https://gerrit.wikimedia.org/g/labs/tools/ninthcircuit if anyone wants to revive it.

I've disabled this tool, I don't think anyone was using it. The code is available at https://gitlab.wikimedia.org/toolforge-repos/vendor.

I've added a note to the docs that the default quotas are changing soon: https://wikitech.wikimedia.org/w/index.php?diff=2128934&oldid=2128314&title=Help:Toolforge/Kubernetes (which would've stopped me from filing this request in the first place :))

Ah cool, I'm happy to wait a few weeks for the new quotas!

In T350219#9340029, @daniel wrote:

@Legoktm This works now:

curl -X POST -F "html=test" 'https://en.wikipedia.org/w/rest.php/v1/transform/html/to/wikitext/Mwbot-rs%2FDISPLAYTITLE'

Probably me - I had merged the changes in Git but not yet deployed them. But now it's resolved!

Nope, it's back :( https://gitlab.wikimedia.org/repos/mwbot-rs/toolforge/-/jobs/165687

Switching to path-based input is something that tool developers would need to support in each tool

The far easier solution would be to just have the tool in question not use a query string for parameters, aka [[toolforge:scholia/Q1513315]], which is, AFAICT, what was implemented? Visiting https://iw.toolforge.org/scholia/Q1513315 works for me.

So I've posted a patch that is basically what I outlined in T35186#7487506. Rough next steps:

In T350219#9321723, @daniel wrote:

The new endpoints are:

• https://test.wikipedia.org/w/rest.php/v1/page/Mwbot-rs%2FDISPLAYTITLE/html (note that the "/html" is at the end now). This also supports the stash=true parameter.
• https://test.wikipedia.org/w/rest.php/v1/transform/html/to/wikitext/Mwbot-rs%2FDISPLAYTITLE/554795

Harry (of WikiTrackBot) was telling me at Wikidata Day that there's cool stuff happening in the TypeScript world in which there are constants for each property, and then when getting a property's claims, it is automatically typed to the correct property type.

In T350219#9321723, @daniel wrote:

In T350219#9321406, @Legoktm wrote:

My understanding is that transformation relies on the current content for selser to work (@ssastry can correct me if I'm wrong!)

It can, it depends on what you post. And it only works if you had originally requested the HTML with the "stash" flag set.

Following the discussion at https://commons.wikimedia.org/w/index.php?title=Commons:Village_pump&oldid=820424623#User_talk:CommonsDelinker/commands, I've switched the "renamecats.py" script to run under Python 3 and it seems to work now.

In T350219#9319871, @daniel wrote:

Unless perhaps you are relying on stashing, or on loading the HTML from the backend...

I'm seeing less 404s now (yay), but I'm still getting one when POSTing to https://test.wikipedia.org/api/rest_v1/transform/html/to/wikitext/Mwbot-rs%2FDISPLAYTITLE/554795.

There's no container image that contains wget... guess I can use curl instead.

Builds now happen using the jobs framework instead of on the grid.

This was stabilized in toolforge 5.6.0.

This seems to have resolved itself?

This feature was removed from mwapi, compatibility with WASM will be re-evaluated once it's reimplemented.

@mirror-kt very cool! Looks like it's basically a drop-in replacement too, and lets us upgrade the markup5ever dependency.

Will the fix be backported?

(This is causing test failures for mwbot-rs because our test pages are on testwiki: https://gitlab.wikimedia.org/repos/mwbot-rs/mwbot/-/merge_requests/57#note_58082)

Guess not. Closing as resolved.

Mostly resolved, now onto T349975...

Thanks @dancy, that was the clue I needed! Rust tries to automatically parallelize for the number of CPUs, so reducing the concurrency to 1 seems to keep it under the memory limit.

@mirror-kt shared the rationale for this on GitLab:

I am running a bot to help renaming categories in jawp, based on dewp's project. a command to undo changes made by the bot was requested (it's hard to undo manually), but bot.api().post_with_token() doesn't work with SaveLock, so the rate By incorporating this function into mwbot itself, rate limits can be shared with normal page editing.

The problem is that Parsoid only lets us edit template parameters as wikitext, while Wikicode is HTML based. This is the same when using VisualEditor to edit template invocations, it gives you a blob of wikitext to edit.

Now official: https://github.com/docker-library/official-images/commit/4944058549fbb61025f6226751c566d5155e670a

@ChristianHeusel you should have an invite to join the wikimedia/mediawiki-docker repository in your inbox!

https://github.com/docker-library/official-images/pull/15599 to make this official.

Thanks for volunteering @ChristianHeusel!

It's not 2024 yet.

For reference, this is what it looks like with the patch applied:

There's still some TODOs left in https://gitlab.wikimedia.org/legoktm/rawdog but in my limited local testing, it seems to work. I'll do another pass in a few days (I also want to add some Python CI tools) before uploading to apt.wikimedia.org. I also pushed up a puppet patch that should make all the necessary changes (but that too needs more work).

Should this patch also deprecate the hardcoding of outdent/tracked in DT with the intention that communities add "mw-notalk" to those templates?

Sorry about the delay in reviewing, thanks for the patch!

In T341565#9203219, @jhsoby wrote:

Basically, we only upload XMLs sparingly: when a test wiki is rejected, but someone might want/need the content later (to import it to a different MediaWiki installation somewhere). We could, for instance, set up a simple toolforge project to store those XMLs instead if that's ok.

The functionality of ExtensionDistributor is provide special pages Special:ExtensionDistributor / Special:SkinDistributor (and related APIs) which is merely a frontend of tarballs at https://extdist.wmflabs.org/dist/extensions/ (which are generated by an independent tarball generator).

In T344112#9092517, @Legoktm wrote:

The audit list is arbcom-audit-en@.

The audit list is arbcom-audit-en@.

Nobody knew the answer.

of a Lua wikitext parser that I've been writing