Once the tables have been backfilled and patches in T304780 got merged and deployed.
Status as of 2022-08-04:
- enwiki: Read old
- Generally: Read new
- testwiki + itwikisource + s5 + s2: Read new, don't write old
Once the tables have been backfilled and patches in T304780 got merged and deployed.
Status as of 2022-08-04:
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Open | None | T300222 Implement normalizing MediaWiki link tables | |||
Resolved | Ladsgroup | T299417 Normalize templatelinks table | |||
Resolved | Ladsgroup | T306673 Turn on read new for templatelinks on beta and production |
Change 785947 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] labs: Set templatelinks migration to read new
Change 785947 merged by jenkins-bot:
[operations/mediawiki-config@master] labs: Set templatelinks migration to read new
Change 787939 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Set testwiki to READ NEW for templatelinks migration
Change 787939 merged by jenkins-bot:
[operations/mediawiki-config@master] Set testwiki to READ NEW for templatelinks migration
Mentioned in SAL (#wikimedia-operations) [2022-05-02T03:41:12Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:787939|Set testwiki to READ NEW for templatelinks migration (T306673)]] (duration: 00m 49s)
Change 788818 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Set mediawikiwiki to READ NEW for templatelinks migration
Change 788818 merged by jenkins-bot:
[operations/mediawiki-config@master] Set mediawikiwiki to READ NEW for templatelinks migration
Mentioned in SAL (#wikimedia-operations) [2022-05-03T22:43:49Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:788818|Set mediawikiwiki to READ NEW for templatelinks migration (T306673)]] (duration: 00m 50s)
SpecialWhatLinksHere::showIndirectLinks [0.03s] db1157: SELECT page_id,page_namespace,page_title,rd_from,rd_fragment,page_is_redirect FROM (SELECT tl_from,rd_from,rd_fragment FROM `templatelinks` LEFT JOIN `redirect` ON ((rd_from = tl_from) AND rd_title = 'Documentation' AND rd_namespace = 10 AND (rd_interwiki = '' OR rd_interwiki IS NULL)) WHERE tl_target_id = 37 AND tl_from_namespace IN (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,90,91,92,93,100,101,102,103,104,105,106,107,486,487,828,829,1198,1199,2300,2301,2302,2303,2600,5500,5501) ORDER BY tl_from_namespace ASC,tl_from ASC LIMIT 102 ) `temp_backlink_range` JOIN `page` ON ((tl_from = page_id)) ORDER BY page_namespace ASC,page_id ASC LIMIT 51
\o/
Change 789562 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Set cebwiki to read new in templatelinks migration
Change 789562 merged by jenkins-bot:
[operations/mediawiki-config@master] Set cebwiki to read new in templatelinks migration
Mentioned in SAL (#wikimedia-operations) [2022-05-05T18:51:54Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:789562|Set cebwiki to read new in templatelinks migration (T306673)]] (duration: 00m 49s)
SELECT tl_target_id,lt_namespace,lt_title,page_id,page_namespace,page_title,page_restrictions,page_is_redirect,page_is_new,page_random,page_touched,page_links_updated,page_latest,page_len,page_content_model FROM `templatelinks` JOIN `linktarget` ON ((tl_target_id=lt_id)) LEFT JOIN `page` ON ((page_namespace=lt_namespace) AND (page_title=lt_title)) WHERE tl_from = 5443966 LIMIT 50
Looks correct \o/ Next week gonna deploy it to much more production wikis.
Change 790021 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Set arwiki to read new in templatelinks migration
Change 790021 merged by jenkins-bot:
[operations/mediawiki-config@master] Set arwiki to read new in templatelinks migration
Mentioned in SAL (#wikimedia-operations) [2022-05-09T04:31:47Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:790021|Set arwiki to read new in templatelinks migration (T306673)]] (duration: 05m 10s)
Mentioned in SAL (#wikimedia-operations) [2022-05-09T04:53:01Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: Revert: [[gerrit:790021|Set arwiki to read new in templatelinks migration (T306673)]] (duration: 05m 03s)
Change 790997 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Set dewiki to read new for templatelinks
Change 790997 merged by jenkins-bot:
[operations/mediawiki-config@master] Set dewiki to read new for templatelinks
Mentioned in SAL (#wikimedia-operations) [2022-05-11T11:56:37Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:790997|Set dewiki to read new for templatelinks (T306673)]] (duration: 00m 49s)
Change 792202 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[mediawiki/core@master] ApiQueryBacklinksprop: Force the correct templatelinks index on read new
Change 792202 merged by jenkins-bot:
[mediawiki/core@master] ApiQueryBacklinksprop: Force the correct templatelinks index on read new
Change 792136 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[mediawiki/core@wmf/1.39.0-wmf.10] ApiQueryBacklinksprop: Force the correct templatelinks index on read new
Change 792136 merged by jenkins-bot:
[mediawiki/core@wmf/1.39.0-wmf.10] ApiQueryBacklinksprop: Force the correct templatelinks index on read new
Mentioned in SAL (#wikimedia-operations) [2022-05-16T14:50:08Z] <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.10/includes/api/ApiQueryBacklinksprop.php: Backport: Revert: [[gerrit:792136|ApiQueryBacklinksprop: Force the correct templatelinks index on read new (T306673)]] (duration: 00m 50s)
Change 792221 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[mediawiki/core@master] ApiQueryBacklinksprop: Make sure the index setting exists
Change 792221 merged by jenkins-bot:
[mediawiki/core@master] ApiQueryBacklinksprop: Make sure the index setting exists
Change 792139 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[mediawiki/core@wmf/1.39.0-wmf.11] ApiQueryBacklinksprop: Make sure the index setting exists
Change 792139 abandoned by Ladsgroup:
[mediawiki/core@wmf/1.39.0-wmf.11] ApiQueryBacklinksprop: Make sure the index setting exists
Reason:
Change 792140 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[mediawiki/core@wmf/1.39.0-wmf.10] ApiQueryBacklinksprop: Make sure the index setting exists
Change 792140 merged by jenkins-bot:
[mediawiki/core@wmf/1.39.0-wmf.10] ApiQueryBacklinksprop: Make sure the index setting exists
Mentioned in SAL (#wikimedia-operations) [2022-05-16T18:42:08Z] <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.10/includes/api/ApiQueryBacklinksprop.php: Backport: [[gerrit:792140|ApiQueryBacklinksprop: Make sure the index setting exists (T306673)]] (duration: 00m 50s)
Change 792565 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Turn on read new for templatelinks on frwiki
Change 792565 merged by jenkins-bot:
[operations/mediawiki-config@master] Turn on read new for templatelinks on frwiki
Mentioned in SAL (#wikimedia-operations) [2022-05-17T08:40:31Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:792565|Turn on read new for templatelinks on frwiki (T306673)]] (duration: 02m 25s)
Before turning it on, I need to fix this issue that I mistakenly posted in the wrong ticket: https://phabricator.wikimedia.org/T299421#7934073
Continuation of debugging the optimizer going haywire from above.
Splitting the page namespaces also didn't help:
SELECT lt_title,tl_from_namespace,tl_from,lt_namespace AS `bl_namespace`,lt_title AS `bl_title`,page_namespace,page_title,page_id,page_content_model,page_is_redirect,page_is_new,page_latest,page_touched,page_len,page_restrictions FROM `page`,`templatelinks` FORCE INDEX (tl_backlinks_namespace_target_id) JOIN `linktarget` ON ((tl_target_id=lt_id)) WHERE tl_from_namespace IN (0,1,2,3,4,5,12,13) AND (tl_from = page_id) AND ( (lt_namespace = 10 AND lt_title = 'SPE') OR (lt_namespace = 10 AND lt_title = 'Semi-protection_étendue') ) AND page_is_redirect = 0 ORDER BY lt_title,tl_from_namespace,tl_from LIMIT 5 ;
Fun observation: Reducing number of pages to one makes the query quite fast:
mysql:root@localhost [frwiki]> explain SELECT lt_title,tl_from_namespace,tl_from,lt_namespace AS `bl_namespace`,lt_title AS `bl_title`,page_namespace,page_title,page_id,page_content_model,page_is_redirect,page_is_new,page_latest,page_touched,page_len,page_restrictions FROM `page`,`templatelinks` FORCE INDEX (tl_backlinks_namespace_target_id) JOIN `linktarget` ON ((tl_target_id=lt_id)) WHERE tl_from_namespace IN -> (0,1,2,3,4,5,12,13) AND (tl_from = page_id) AND (lt_namespace = 10 AND lt_title = 'Semi-protection_étendue') AND page_is_redirect = 0 ORDER BY lt_title,tl_from_namespace,tl_from LIMIT 5 ; +------+-------------+---------------+--------+-------------------------------------+----------------------------------+---------+------------------------------+------+------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +------+-------------+---------------+--------+-------------------------------------+----------------------------------+---------+------------------------------+------+------------------------------------------+ | 1 | SIMPLE | linktarget | const | PRIMARY,lt_namespace_title | lt_namespace_title | 261 | const,const | 1 | Using index | | 1 | SIMPLE | templatelinks | range | tl_backlinks_namespace_target_id | tl_backlinks_namespace_target_id | 13 | NULL | 309 | Using where; Using index; Using filesort | | 1 | SIMPLE | page | eq_ref | PRIMARY,page_redirect_namespace_len | PRIMARY | 4 | frwiki.templatelinks.tl_from | 1 | Using where | +------+-------------+---------------+--------+-------------------------------------+----------------------------------+---------+------------------------------+------+------------------------------------------+ 3 rows in set (0.001 sec) mysql:root@localhost [frwiki]> explain SELECT lt_title,tl_from_namespace,tl_from,lt_namespace AS `bl_namespace`,lt_title AS `bl_title`,page_namespace,page_title,page_id,page_content_model,page_is_redirect,page_is_new,page_latest,page_touched,page_len,page_restrictions FROM `page`,`templatelinks` FORCE INDEX (tl_backlinks_namespace_target_id) JOIN `linktarget` ON ((tl_target_id=lt_id)) WHERE tl_from_namespace IN (0,1,2,3,4,5,12,13) AND (tl_from = page_id) AND (lt_namespace = 10 AND lt_title = 'SPE') AND page_is_redirect = 0 ORDER BY lt_title,tl_from_namespace,tl_from LIMIT 5; +------+-------------+---------------+--------+-------------------------------------+----------------------------------+---------+------------------------------+------+------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +------+-------------+---------------+--------+-------------------------------------+----------------------------------+---------+------------------------------+------+------------------------------------------+ | 1 | SIMPLE | linktarget | const | PRIMARY,lt_namespace_title | lt_namespace_title | 261 | const,const | 1 | Using index | | 1 | SIMPLE | templatelinks | range | tl_backlinks_namespace_target_id | tl_backlinks_namespace_target_id | 13 | NULL | 8 | Using where; Using index; Using filesort | | 1 | SIMPLE | page | eq_ref | PRIMARY,page_redirect_namespace_len | PRIMARY | 4 | frwiki.templatelinks.tl_from | 1 | Using where | +------+-------------+---------------+--------+-------------------------------------+----------------------------------+---------+------------------------------+------+------------------------------------------+ 3 rows in set (0.001 sec)
This is clearly a bug in MariaDb. I tried to debug it a bit to maybe be able to at least hint to the optimizer to do the right thing but no success so far. Somehow MariaDb thinks the index of lt_namespace_title in linktarget have to do select type of range instead of eq_ref or even ref which later leads to join type becoming hash_range which is wrong. I have tried to push it to maybe use range join by setting optimizer switch of join_cache_hashed to off but range join is similar (probably even worse). I'm also quite confused why for looking up two values in linktarget it goes to Using where; Using index; Using filesort. Filesort? WHY? (I optimized linktarget as well, even with alter, same result)
In other words, mariadb tries to read the range of lt_target_id between two requested values instead of just reading the two. The correct way of doing such join is like this:
mysql:root@localhost [frwiki]> explain select * from revision join actor on rev_actor = actor_id where actor_name in ('X', 'Y', 'Z') limit 5; +------+-------------+----------+-------+---------------------+---------------------+---------+-----------------------+------+-----------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +------+-------------+----------+-------+---------------------+---------------------+---------+-----------------------+------+-----------------------+ | 1 | SIMPLE | actor | range | PRIMARY,actor_name | actor_name | 257 | NULL | 3 | Using index condition | | 1 | SIMPLE | revision | ref | rev_actor_timestamp | rev_actor_timestamp | 8 | frwiki.actor.actor_id | 9 | | +------+-------------+----------+-------+---------------------+---------------------+---------+-----------------------+------+-----------------------+ 2 rows in set (0.001 sec)
Basically I think it boils down to not being able to use the correct plan in linktarget table. It picks the correct index but still feels a filesort is needed (it's not)
Okay filesort was red herring, it was due to order by being different (which probably needs fixing) but removing the whole order by clause didn't make a difference:
mysql:root@localhost [frwiki]> explain SELECT lt_title,tl_from_namespace,tl_from,lt_namespace AS `bl_namespace`,lt_title AS `bl_title`,page_namespace,page_title,page_id,page_content_model,page_is_redirect,page_is_new,page_latest,page_touched,page_len,page_restrictions FROM `page`,`templatelinks` FORCE INDEX (tl_backlinks_namespace_target_id) JOIN `linktarget` ON ((tl_target_id=lt_id)) WHERE tl_from_namespace IN (0,1,2,3,4,5,12,13) AND (tl_from = page_id) AND ((lt_namespace = 10 AND lt_title IN ('SPE','Semi-protection_étendue') )) AND page_is_redirect = 0 LIMIT 501; +------+-------------+---------------+------------+-------------------------------------+--------------------------------------------+---------+------------------------------+-----------+---------------------------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +------+-------------+---------------+------------+-------------------------------------+--------------------------------------------+---------+------------------------------+-----------+---------------------------------------------------------------+ | 1 | SIMPLE | linktarget | range | PRIMARY,lt_namespace_title | lt_namespace_title | 261 | NULL | 2 | Using where; Using index | | 1 | SIMPLE | templatelinks | hash_range | tl_backlinks_namespace_target_id | #hash#$hj:tl_backlinks_namespace_target_id | 9:4 | frwiki.linktarget.lt_id | 175855731 | Using where; Using index; Using join buffer (flat, BNLH join) | | 1 | SIMPLE | page | eq_ref | PRIMARY,page_redirect_namespace_len | PRIMARY | 4 | frwiki.templatelinks.tl_from | 1 | Using where | +------+-------------+---------------+------------+-------------------------------------+--------------------------------------------+---------+------------------------------+-----------+---------------------------------------------------------------+ 3 rows in set (0.001 sec)
Found the problem. It's the fact that having an tl_from_namespace forces tl_backlinks_namespace_target_id which is correct and done because of T139056 but if you set a list of namespaces you want and those namespaces basically cover the whole table, you'll force the wrong index.
Strangely in the old way it works:
explain SELECT tl_title,tl_from_namespace,tl_from,tl_namespace AS `bl_namespace`,tl_title AS `bl_title`,page_namespace,page_title,page_id,page_content_model,page_is_redirect,page_is_new,page_latest,page_touched,page_len FROM `page`,`templatelinks` FORCE INDEX (tl_backlinks_namespace) WHERE tl_from_namespace IN (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,100,101,102,103,104,105,828,829,2300,2301,2302,2303,2600) AND (tl_from = page_id) AND ((tl_namespace = 10 AND tl_title IN ('SPE','Semi-protection_étendue') )) ORDER BY tl_title,tl_from_namespace,tl_from LIMIT 11 ; +------+-------------+---------------+--------+------------------------+------------------------+---------+------------------------------+------+------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +------+-------------+---------------+--------+------------------------+------------------------+---------+------------------------------+------+------------------------------------------+ | 1 | SIMPLE | templatelinks | range | tl_backlinks_namespace | tl_backlinks_namespace | 265 | NULL | 446 | Using where; Using index; Using filesort | | 1 | SIMPLE | page | eq_ref | PRIMARY | PRIMARY | 4 | frwiki.templatelinks.tl_from | 1 | | +------+-------------+---------------+--------+------------------------+------------------------+---------+------------------------------+------+------------------------------------------+
I hereby suggest just dropping the index hint, it has been in mariadb 10.1 and knows much better how actually should behave than the current mess.
Change 797477 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[mediawiki/core@master] ApiQueryBacklinksprop: Completely remove index hints
Change 797477 merged by jenkins-bot:
[mediawiki/core@master] ApiQueryBacklinksprop: Completely remove index hints
Change 797220 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[mediawiki/core@wmf/1.39.0-wmf.12] ApiQueryBacklinksprop: Completely remove index hints
Change 797220 merged by jenkins-bot:
[mediawiki/core@wmf/1.39.0-wmf.12] ApiQueryBacklinksprop: Completely remove index hints
Mentioned in SAL (#wikimedia-operations) [2022-05-24T13:50:29Z] <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.12/includes/api/ApiQueryBacklinksprop.php: Backport: [[gerrit:797220|ApiQueryBacklinksprop: Completely remove index hints (T306673)]] (duration: 00m 55s)
Change 799265 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Set templatelinks migration to read new everywhere except three wikis
Change 799265 merged by jenkins-bot:
[operations/mediawiki-config@master] Set templatelinks migration to read new everywhere except three wikis
Mentioned in SAL (#wikimedia-operations) [2022-05-25T08:32:01Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:799265|Set templatelinks migration to read new everywhere except three wikis (T306673)]] (duration: 00m 50s)
Change 801661 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Migrate zhwiki to read new for templatelinks
Change 801661 merged by jenkins-bot:
[operations/mediawiki-config@master] Migrate zhwiki to read new for templatelinks
Mentioned in SAL (#wikimedia-operations) [2022-05-31T12:42:37Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:801661|Migrate zhwiki to read new for templatelinks (T306673)]] (duration: 03m 10s)
Change 813892 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] labs: Make sure templatelinks config overrides production
Change 813892 merged by jenkins-bot:
[operations/mediawiki-config@master] labs: Make sure templatelinks config overrides production
Change 813897 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[mediawiki/extensions/FlaggedRevs@master] Add support for read new of templatelinks
Change 813897 merged by jenkins-bot:
[mediawiki/extensions/FlaggedRevs@master] Add support for read new of templatelinks
Change 820376 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Start reading from new templatelinks columns in commons
Change 820376 merged by jenkins-bot:
[operations/mediawiki-config@master] Start reading from new templatelinks columns in commons
Mentioned in SAL (#wikimedia-operations) [2022-08-04T16:28:39Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:820376|Start reading from new templatelinks columns in commons (T306673)]] (duration: 03m 00s)
Change 829556 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):
[operations/mediawiki-config@master] Make English Wikipedia read new on templatelinks migration
Change 829556 merged by jenkins-bot:
[operations/mediawiki-config@master] Make English Wikipedia read new on templatelinks migration
Mentioned in SAL (#wikimedia-operations) [2022-09-05T07:32:52Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:829556|Make English Wikipedia read new on templatelinks migration (T306673)]] (duration: 03m 31s)
Read-wise, we are read new everywhere now. For writes, check T312865: Turn off writing to the old columns of templatelinks in beta and production