Don't think it is a good idea to modify the existing RebuildTermSqlIndex code. We can just create a new script. The existing code has things in there we don't need, and still having the wb_terms specific thing around might be useful to various users.

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/503331

https://github.com/wmde/WikibaseDataModelServices/pull/214

In T219894#5115237, @JeroenDeDauw wrote:

https://github.com/wmde/WikibaseDataModelServices/pull/214

Merged

Change 503331 had a related patch set uploaded (by Jforrester; owner: Jeroen De Dauw):
[mediawiki/extensions/Wikibase@master] Add a rebuilder script for the terms store

https://gerrit.wikimedia.org/r/503331

Change 503331 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add a rebuilder script for the terms store

https://gerrit.wikimedia.org/r/503331

https://github.com/wmde/WikibaseDataModelServices/pull/216

AFAIK the script we currently have (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/505670) is sufficient for this task. It has continuation based on page id rather than property id. I figure that won't fly for items but likely is OK for properties. Do we need continuation at all for properties?

With https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/501992/ the rebuilding works using the Doctrine Term Store.

There is some issues though. Some properties result in an error, and on re-run many of them do.

When I throw the exception I get this: https://pastebin.com/ejFn1pnT

*Somehow* a duplicate entry is getting inserted. I added an extra test for this but the test is not catching the issue. https://github.com/wmde/doctrine-term-store/pull/7 Probably this is some kind of evil encoding issue.

I'm calling it a day. Current guess is that the tables are not created right because we are not using this setting in mediawiki/doctrine-connection.

(https://www.doctrine-project.org/projects/doctrine-dbal/en/2.9/reference/configuration.html)

Or maybe something is wrong here https://github.com/wmde/doctrine-term-store/blob/master/src/PackagePrivate/Doctrine/NormalizedStore.php#L100-L105

One thing left in the maintenance script itself: support property id ranges. I will work on this today

Ready for review: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/507954

Change 507954 had a related patch set uploaded (by Jeroen De Dauw; owner: Jeroen De Dauw):
[mediawiki/extensions/Wikibase@master] Use property id based limits in rebuildPropertyTerms.php

https://gerrit.wikimedia.org/r/507954

Change 507954 abandoned by Jeroen De Dauw:
Use property id based limits in rebuildPropertyTerms.php

https://gerrit.wikimedia.org/r/507954