Page MenuHomePhabricator

Merge Wikidata branch into master
Closed, ResolvedPublic

Description

The wikidata harvesting should be far enough gone that we can merge it back into master. The only open issue remaining being T172691: Ensure wikidata harvest is possible for very large datasets.

Since master has also changed in the meantime the whole branch needs to be rebased and the result carefully compared to our desired results See e.g. T172841: Ensure all output works for Wikidata harvests

When merging into master care has to be taken since output has changed there as well (in https://gerrit.wikimedia.org/r/#/c/309858/ ). A naive merge will fail on FormatHtml and FormatHtmlList.

It also looks as though the merge commit will have to squash all of the underlying (reviewed) commits as gerrit doesn't play nice with branches. At the very least we should probably explicitly link to the squashed reviews in the commit message.

Details

Related Changes in Gerrit:

Event Timeline

I have a slightly outdated rebase on my machine which I can probably use as a template to ensure I get a new rebase right.

We might want to introduce a mechanism by which wikidata harvests are not run as part of the "harvest all" command. This will allow us to do more testing without crashing everything during WLM.

Per T171300: Ensure reports work for Wikidata harvests we would e.g. need to check that reports work (which is not possible in testing)

Change 374899 had a related patch set uploaded (by Lokal Profil; owner: Lokal Profil):
[labs/tools/heritage@master] [WIP]Merge Wikidata branch into master

https://gerrit.wikimedia.org/r/374899

Change 374899 merged by jenkins-bot:
[labs/tools/heritage@master] Merge Wikidata branch into master

https://gerrit.wikimedia.org/r/374899

Lokal_Profil claimed this task.