Page MenuHomePhabricator

SpecialUncategorizedPages slow query
Open, Needs TriagePublic

Description

I just caught this slow query on commons - it is using wikiadmin and coming from mwmaint host, so it is from a maintenance script

| 294402967 | wikiadmin2023   | 10.64.16.77:52520    | commonswiki | Query     |   11314 | Sending data                                   | SELECT /* MediaWiki\Specials\SpecialUncategorizedPages::reallyDoQuery  */  page_namespace AS `namespace`,page_title AS `title`  FROM `page` LEFT JOIN `categorylinks` ON ((cl_from = page_id))   WHERE cl_from IS NULL AND page_namespace IN (6,0)  AND page_is_redirect = 0  ORDER BY page_namespace,page_title LIMIT 5000


cumin2024@db1248.eqiad.wmnet[(none)]> show explain for 294402967;
+------+-------------+---------------+-------+---------------------------------------------+-----------------+---------+--------------------------+----------+--------------------------------------+
| id   | select_type | table         | type  | possible_keys                               | key             | key_len | ref                      | rows     | Extra                                |
+------+-------------+---------------+-------+---------------------------------------------+-----------------+---------+--------------------------+----------+--------------------------------------+
|    1 | SIMPLE      | page          | range | page_name_title,page_redirect_namespace_len | page_name_title | 4       | NULL                     | 74007461 | Using where                          |
|    1 | SIMPLE      | categorylinks | ref   | PRIMARY                                     | PRIMARY         | 4       | commonswiki.page.page_id |        3 | Using where; Using index; Not exists |
+------+-------------+---------------+-------+---------------------------------------------+-----------------+---------+--------------------------+----------+--------------------------------------+
2 rows in set, 1 warning (0.071 sec)

This seem to be it:

www-data 10323  0.0  0.0  19944 11620 ?        Ss   Jul01   0:00 /usr/bin/python3 /usr/local/bin/mw-cli-wrapper /usr/local/bin/mwscriptwikiset updateSpecialPages.php s4.dblist
www-data 10412  0.0  0.0   2384   760 ?        S    Jul01   0:00 /bin/sh -c /usr/local/bin/mwscriptwikiset updateSpecialPages.php s4.dblist
www-data 10414  0.0  0.0   6724  3308 ?        S    Jul01   0:00 /bin/bash /usr/local/bin/mwscriptwikiset updateSpecialPages.php s4.dblist
www-data 10440  0.0  0.0   6724  3172 ?        S    Jul01   0:00 /bin/bash /usr/local/bin/mwscript updateSpecialPages.php commonswiki
www-data 10467  0.0  0.1 240056 87572 ?        S    Jul01   0:00 php /srv/mediawiki-staging/multiversion/MWScript.php updateSpecialPages.php commonswiki

Any chances this can be done in a different way?

Event Timeline

That's part of updating query page entries and by design it's slow (it'll fill a cache). The proper fix is to move it to hadoop T309738: Move Mediawiki QueryPages computation to Hadoop (which I've been slowly working on it) but in the mean time, we can reduce its frequency.

That's part of updating query page entries and by design it's slow (it'll fill a cache). The proper fix is to move it to hadoop T309738: Move Mediawiki QueryPages computation to Hadoop (which I've been slowly working on it) but in the mean time, we can reduce its frequency.

Reducing it would work for me :)
Thanks!

Change #1052058 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Reduce frequency of two query pages in commonswiki

https://gerrit.wikimedia.org/r/1052058