I have reworked the task description, and played around with the wikitext and pages a little, both identifying an initial list, and trying out a parsing query (if that would end up being part of the direction at all).

Many thanks, and I have added a link on the datalake wikitech page https://wikitech.wikimedia.org/w/index.php?title=Data_Platform/Data_Lake&diff=prev&oldid=2463263
And put an initial sample query at https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Dev_Metrics/Git_Interaction
Generally speaking looks good, and here are some initial very crude graphs for just this dataset without any further processing.

Current state overview...

Cloud: In wmde/wbaas-deploy, I found no maxSerializedEntitySize or maxArticleSize setting. In wbstack/mediawiki, the platform’s LocalSettings.php doesn’t override maxSerializedEntitySize. The bundled Wikibase default sets it from $wgMaxArticleSize, as described in the docs. This implies that cloud uses the default wikibase setup currently

If you did want to go down that route, it's probably easier to validate in the databases if anyone has that setting enabled yet.
As trying to take it into account would be slightly trickier!

The Modern skin fade-out is proposed across the API, dashboard, and MediaWiki:

This means removing it both in the dashboard and if possible also on the MW unless it is already explicitly chosen.

https://github.com/wbstack/queryservice-updater/pull/169

Looks like I came across this one at the weekend https://github.com/wbstack/ui/pull/1223

I traced this back to the SQL deployment migration in "e052b53e" (https://github.com/wmde/wbaas-deploy/commit/e052b53e09fd7c86e4ad63f0b9745618a8d9389f) (“SQL setup (local & prod)”, 2021-10-11), which I authored.

Managed to setup gitlab-runner-addshore-1018 on the project using Trixie and all went according to plan.
Updated my own docs for when i create the next one https://gitlab.wikimedia.org/repos/releng/cli/-/commit/8f7a378eabe55345422a9beae3a3daebd8c48558
But I have deleted both old instances now, so this can be considered resolved!

this project will be shut down on Tuesday, September 1st.

I'll be looking at it today :)

Excellent to hear that works, looks like I can get the PR merged and make a new release with it!

Chekcing in with Test Platform @thcipriani to see if this should remain open then, or just head to invalid / declined?

DB access (we're already using Sqoop to export data from the wikis' DBs)

So, SAL is certainly one of the more minor data sets here I believe, however it covers an interesting area of activity.

You should find a zip with some builds in https://gitlab.wikimedia.org/repos/releng/cli/-/jobs/934593/artifacts/download to try!

@Antoine_Quhen If you give me some pointers for schema location and or script location / structure I can try and throw up some sort of commits for review

The linked artifact didn't seem to compile to arm, since I got an exec file format error when trying to run it

If you could try this build that would be great https://gitlab.wikimedia.org/repos/releng/cli/-/jobs/933228/artifacts/download
You should be able to use the update command with a param to switch over to it
I still have never gotten to use the cli on arm yet to try and iron out any issues!

I just did sudo mw update --version v0.31.0 followed by sudo mw update --version v0.32.0 and it seemed to work fine for me?
On what setup were you trying this?

In T433696#12192395, @Antoine_Quhen wrote:

@Ahoelzl how will this data be used? example queries? target dashboards / graphs?

The discussion on the git ticket T433696 has been going well, so I thought I'd also go ahead and created T434047: data lake: phabricator events again to see if this is a useful way to introduce the potential data sources.

No, but we can fire a different event that would invalidate the previous one.

I'd argue that the commit itself is not the event within git, though we could likely get away with modeling it that way.
I suspect that is for example why gitlab has a push hook, not a commit hook https://docs.gitlab.com/administration/system_hooks/#triggered-events
This also ties in with why I wrote the point on git mutability in the description

I expect the hook approach would not work for GitHub, as the scope of the data span multiple orgs, repositories and users, and hooks are either at the org or repo level.
So an alternative approach that can handle git repositories in general would likely still be needed.

I have initially created T433696: data lake: git commit history and already have a similar set of info prepped in markdown for Phabricator too.
Let me know if this is a good format / if you think things are missing and or there is too much there.
Happy to basically write this initial content up for each of the proposed sources to aim to get the context of each one shared as efficiently as possible.
And or, let me know if I should be putting it somewhere else!

Could be an external service, or such a thing could also be built into the kartographer extension

I expect you can just setup a little proxy so that the requests come form your servers rather than from your users.

So, im not sure there is a "mw dev status" command.
Did you find it in the docs somewhere?
And if not, what do you think it should display?

Turns out I can also reproduce this easily in a chrome web browser in mobile view (while logged in as Addshore)

In T357047#11929527, @Anton.Kokh wrote:

Please wait until June 3, there will be a call about OSM where this question will be clarified

Potentially related and potentially useful https://addshore.com/2026/06/wikidata-deletion-request-trends-rfds/

Should work for wikibase.cloud

@Tarrow not sure why it would ultimatly cause mediawiki to take so long serving the page?
At least for me locally, a similar page renders right away and served the error snippet

I don't think I'll put it in directly,
but I have made it nice and easy to make your own images as part of it / build on the defaults

Marking as declined, but also will note the below...

Considered, but I dont really want to maintain another set of images for mwcli.
Instead, inline with T298484: [mwcli docker] Document how to user Dockerfile overrides for mediawiki service there is functioanlity now to easily define custom Dockerfiles, and the mediawiki one at least has an example right now..

And a side note post the Wikimedia-Hackathon-2026

In https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/649 there is a nice little workflow for this tbh!

For now implemented as this

Yes, and I didn't have time to try locally at all yet either.

I believe I heard that the WMF has decided against swagger UI citing security and maintenance concerns?
(I have nothing to point to beyond overhearing this at the hackathon)
So it might be "best" to go for something else?