Current state:

Part 5: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/666724

Part 4: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/666716

Part 3: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/666715

Part 2: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/666628/2

Part 1: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/666611

I just did another suggester update, with the script described above (T272571#6771954) and we encountered no issues this time, thus this should be fine.

This is https://github.com/wmde/wikibase-release-pipeline/pull/59 (which just needs to be accepted and merged).

https://github.com/wmde/wikit/pull/370

Currently the procedure for updating the table involves a hand-written script, which does data clean up (deletes LOTS of rows) in the database table after the dataset is inserted (which is probably what was causing these issues). See PropertySuggester_update.

In T270158#6692094, @Lucas_Werkmeister_WMDE wrote:

The main use of the Travis CI that I can remember is that it uses wikis with non-English content languages, so occasionally it flags up that we’ve unintentionally written a test which relies on $wgLanguageCode = "en". But I don’t see why we couldn’t set different language codes for some of our Jenkins jobs.

I still need to test my changes (sadly there are no automated tests for these scripts and it would be excruciatingly hard to write some for the current bash scripts). Once that is done, this is good to go from my side.

Poking around and git bisect point to 7f430f142d7f19cb575f0e665719619c9312ae57.

In T265899#6560252, @Lydia_Pintscher wrote:

Wild guess: some module is comparing local and WD description? Or checking of one exists on WD if not on WP?

@Physikerwelt Nowadays we pull in some of the (JS) libraries using git submodules. Are your submodules in Wikibase all initialized and up to date?

The relevant configuration has just been removed, meaning that this is disabled on beta again (but might still be in caches).

In T232620#6503844, @Bugreporter wrote:

The scope of this task is a bit wider than the patch. If an item can not be created because of an label/description conflict, the ID is still allocated. Therefore the issue described in T232620#6456047 still exists and there are no way to detect this.

Other circumstances that item creation will fail may include rate limit, abusefilter and malformed entity content.

In T247690#6228627, @Lydia_Pintscher wrote:

Tested more. Looking acceptable now :D Thank you!

Leaving the ticket open because of @matej_suchanek's comment.

I can see @thiemowmde's concerns, but I also see the trouble with using at-ease. As a compromise, it might be worthwhile changing at-ease to follow a more functional style:

@toan Which of the two patches should be reviewed? Are they still WIP (if not, you should publish them for review in Gerrit).

As this never really got anywhere, let's abandon this (at least for the foreseeable future).

Just a note for the future: While the individual dump run recovered and eventually produced a dump, the cron itself (which was supposed to start other RDF dump flavours afterwards) got killed.

I tagged the release now, so we only need to update the submodule in Wikibase now.

Seems this automatically recovered :)

This still needs the release preparation patch to be merged (https://gerrit.wikimedia.org/r/c/wikibase/javascript-api/+/596231), then we can tag it (while doing that, we should also tag the past 3.1.1 release as that was forgotten back then). After that, the subcomponent can be bumped in Wikibase and we should be good here.

I just created a one-off script for getting the items the above items conflict with: https://gist.github.com/mariushoch/ae4538741ec3e5409dfa4e9a90c346dc.

The rebuild items per site run finished. Sadly the following 2,701 items produced errors, where one or more of their sitelinks could not be saved to the table (typically because the sitelink is already recorded for another item):