It seems after https://github.com/wikimedia/mediawiki-extensions-Wikibase/commit/6e32ab5d3fde0c368831f53f6e81f5c154bf20f7 was merged... Wikidatawiki on beta is causing the db updates to not finish in 45 minutes
As it has 335,601 items... It's gonna take a while
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Reedy | T167981 beta-update-databases-eqiad times out on wikidatawiki (maintenance update) | |||
| Resolved | aude | T168036 rebuildTermSqlIndex.php causing beta-update-databases-eqiad to timeout and abort | |||
| Resolved | WMDE-leszek | T168708 Populating full entity column in wb_terms table when running MW update.php should be fast and run only once |
Mentioned in SAL (#wikimedia-releng) [2017-06-16T03:10:03Z] <Reedy> running mwscript extensions/Wikibase/repo/maintenance/rebuildEntityPerPage.php --wiki=wikidatawiki in screen as root on deployment-tin for T168036
Mentioned in SAL (#wikimedia-releng) [2017-06-16T03:13:50Z] <Reedy> running mwscript extensions/Wikibase/repo/maintenance/rebuildTermSqlIndex.php --wiki=wikidatawiki in screen as root on deployment-tin for T168036
Ah, seems it's actually rebuildTermSqlIndex.php
And the script doesn't have any sort of resuming functionality... So it gets so far through, and then when the beta job dies, it loses progress, and starts again on the next run...
So I ran the script... And update.php for wikidatawiki on beta...
...have term_full_entity_id field in wb_terms table.
Processed up to page 1026 (Q1014)
Processed up to page 2026 (Q2015)Something is severely broken here
Or is it?
It's hard to tell what's running. That syntax seems to be used by multiple scripts
So after nuking hundreds of thousands of items and properties...
...index term_search already set on wb_terms table.
...have term_full_entity_id field in wb_terms table.
Processed up to page 509402 (Q350016)
Processed up to page 510931 (Q351021)
Processed up to page 512461 (Q352026)
Processed up to page 513986 (Q353031)
Processed up to page 515516 (Q354036)
Processed up to page 517045 (Q355041)
Processed up to page 518567 (Q356046)
Processed up to page 520117 (Q357051)
Processed up to page 521626 (Q358056)
Processed up to page 522645 (Q358707)
Done rebuilding item termsIt then did properties immediately after...
Done in 9 min 47 s.
If I immediately then run update.php again...
...have term_full_entity_id field in wb_terms table.
Processed up to page 509402 (Q350016)There's a major problem of this task not being logged as complete; it shouldn't be being run on EVERY run of update.php. Plus, it shouldn't need to update every row multiple times?
I'm guessing it's pretty much related to https://github.com/wikimedia/mediawiki-extensions-Wikibase/commit/6e32ab5d3fde0c368831f53f6e81f5c154bf20f7
Mentioned in SAL (#wikimedia-releng) [2017-06-18T19:25:18Z] <Reedy> A lot of items on beta wikidatawiki deleted T168036 T167981
Mentioned in SAL (#wikimedia-releng) [2017-06-18T19:26:53Z] <Reedy> Re-enabled beta-update-databases-eqiad as wikidatawiki takes < 10 minutes T168036 T167981
I'm guessing it's pretty much related to https://github.com/wikimedia/mediawiki-extensions-Wikibase/commit/6e32ab5d3fde0c368831f53f6e81f5c154bf20f7
Yes, it must be that.