Page MenuHomePhabricator

allpages filterlanglinks DBQueryError
Closed, ResolvedPublic

Event Timeline

jayvdb created this task.Dec 11 2014, 2:06 PM
jayvdb updated the task description. (Show Details)
jayvdb raised the priority of this task from to Needs Triage.
jayvdb added a project: MediaWiki-API.
jayvdb changed Security from none to None.
jayvdb added a subscriber: jayvdb.
jayvdb moved this task from Backlog to Test failures on the Pywikibot-tests board.

Change 179146 had a related patch set uploaded (by Anomie):
API: Avoid MySQL filesort with list=allpages&apfilterlanglinks=withlanglinks

https://gerrit.wikimedia.org/r/179146

Patch-For-Review

Anomie claimed this task.Dec 11 2014, 4:14 PM
Anomie added a project: MediaWiki-Core-Team.

There is a very similar, but less frequently occurring case in T73971 : generator imageusage iufilterredir=redirects

There is a very similar, but less frequently occurring case in T73971 : generator imageusage iufilterredir=redirects

Only similar in that it's another query that's slow. The issue there seems to be T78403.

Change 179146 merged by jenkins-bot:
API: Avoid MySQL filesort with list=allpages&apfilterlanglinks=withlanglinks

https://gerrit.wikimedia.org/r/179146

Anomie closed this task as Resolved.Dec 19 2014, 3:17 PM

The fix should be deployed to WMF wikis with 1.25wmf14, see https://www.mediawiki.org/wiki/MediaWiki_1.25/Roadmap for the schedule.

bd808 moved this task from Done to Archive on the MediaWiki-Core-Team board.Dec 22 2014, 10:45 PM

"jayvdb mentioned this in rPWBC33de47f4f43e: Re-enable allpages langlinks tests.Via Daemons · Mar 4 2015, 8:46 AM"

That is wrong; rPWBC33de47f4f43e shows "jayvdb committed rPWBC33de47f4f43e: Re-enable allpages langlinks tests (authored by jayvdb).Via Daemons · Dec 11 2014, 3:08 PM"

Wikidata is T97797.

As for the rest, I'm seeing some weirdness in the database.

The query here is

SELECT  /*! STRAIGHT_JOIN */ page_namespace,page_title,page_id,page_content_model  FROM `page`,`langlinks`   WHERE page_namespace = '0' AND (page_id=ll_from)  GROUP BY page_title, page_id ORDER BY page_title LIMIT 6

For both arwiki and frwiktionary, it seems to work fine on db1033:

+------+-------------+-----------+------+--------------------+------------+---------+---------------------+---------+-------------+
| id   | select_type | table     | type | possible_keys      | key        | key_len | ref                 | rows    | Extra       |
+------+-------------+-----------+------+--------------------+------------+---------+---------------------+---------+-------------+
|    1 | SIMPLE      | page      | ref  | PRIMARY,name_title | name_title | 4       | const               | 1245075 | Using where |
|    1 | SIMPLE      | langlinks | ref  | ll_from            | ll_from    | 4       | arwiki.page.page_id |      12 | Using index |
+------+-------------+-----------+------+--------------------+------------+---------+---------------------+---------+-------------+

On the rest of the s7 DBs (db1028, db1034, db1041, db1039, db1062), the same query has a different plan:

+------+-------------+-----------+------+--------------------+------------+---------+---------------------+---------+----------------------------------------------+
| id   | select_type | table     | type | possible_keys      | key        | key_len | ref                 | rows    | Extra                                        |
+------+-------------+-----------+------+--------------------+------------+---------+---------------------+---------+----------------------------------------------+
|    1 | SIMPLE      | page      | ref  | PRIMARY,name_title | name_title | 4       | const               | 1161329 | Using where; Using temporary; Using filesort |
|    1 | SIMPLE      | langlinks | ref  | ll_from            | ll_from    | 4       | arwiki.page.page_id |       8 | Using index                                  |
+------+-------------+-----------+------+--------------------+------------+---------+---------------------+---------+----------------------------------------------+

Any ideas, database gurus?

I also note that changing the query to just GROUP BY page_title makes all the DBs here happy, so that's an option to fix it.

Change 233427 had a related patch set uploaded (by Anomie):
API: More avoiding MySQL filesorts in list=allpages

https://gerrit.wikimedia.org/r/233427

Anomie moved this task from Unsorted to Needs Review on the MediaWiki-API board.Aug 24 2015, 4:15 PM

Change 233427 merged by jenkins-bot:
API: More avoiding MySQL filesorts in list=allpages

https://gerrit.wikimedia.org/r/233427

Anomie closed this task as Resolved.Aug 26 2015, 4:28 PM

The part of this bug that isn't T97797 should be fixed now, although if the DB people have anything to add it would still be welcome.