Page MenuHomePhabricator

api-testing: "should ignore duplicate redirect source and target if both pages are a match" test fails on SQLite
Closed, ResolvedPublic

Description

See for example https://integration.wikimedia.org/ci/job/integration-quibble-fullrun-sqlite/11/console

This happens deterministically. It seems that we don't run API tests in regular CI for MediaWiki core/extension/skin patches, so that's why it hasn't been noticed before.

00:05:22.844 
00:05:22.844   1) Search
00:05:22.845        GET /search/page?q={term}
00:05:22.845          should ignore duplicate redirect source and target if both pages are a match:
00:05:22.845      AssertionError: expected { id: 22, …(6) } to have nested property 'matched_title' of null, but got 'Redirect source KzFubL4Jfq'
00:05:22.845       at Context.<anonymous> (tests/api-testing/REST/Search.js:115:11)
00:05:22.845       at runMicrotasks (<anonymous>)
00:05:22.845       at processTicksAndRejections (internal/process/task_queues.js:95:5)
00:05:22.845

Event Timeline

Change 766593 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] jjb: Skip api-testing for Quibble fullrun with SQLite

https://gerrit.wikimedia.org/r/766593

Fixing the test would be better, but for now we could just disable API tests on SQLite so we can unblock Quibble's CI.

Change 766593 merged by jenkins-bot:

[integration/config@master] jjb: Skip api-testing for Quibble fullrun with SQLite

https://gerrit.wikimedia.org/r/766593

The difference between running this in SQLite or MySQL is that in MySQL, this code in WikiPage.php returns null:

public function getRedirectTarget() {
		if ( $this->mRedirectTarget !== null ) {
			return $this->mRedirectTarget;
		}

		if ( $this->mHasRedirectTarget === false || !$this->getPageIsRedirectField() ) {
			return null;
		}

Looking at $this->mHasRedirectTarget I see that it's null, and $this->getPageIsRedirectField() returns false.

So I guess it is some broader issue with SQLite + redirects.

Adding some people who worked on the relevant code/tests.

I think what is happening is that the results are coming back in different orders from Sqlite vs MySQL which is affecting how the code deals with duplicates:

In MySQL:

  • $searchResponse includes both Redirect Source and Redirect Target results
  • Redirect Target gets processed first (and therefore returns null on RedirectStore:: getRedirectTarget())
  • Redirect Source gets processed next and is not added to the results because its Redirect Target has already been added, and therefore matched_title is null because Redirect Target is not a redirect page

In Sqlite:

  • $searchResponse includes both Redirect Source and Redirect Target results
  • Redirect Source gets processed first (and therefore returns with a Title on RedirectStore:: getRedirectTarget()) and also has its matched_title field populated because this is a redirect.
  • Redirect Target gets processed next and is not added to the results because its Redirect Source has already been added

Short term I will upload a patch to remove the matched_title check. However the code should be changed to be consistent when dealing with this scenario, even though I don't think it makes a huge difference for the end use case, it may cause future headaches in testing. Will create a ticket for that.

Change 766819 had a related patch set uploaded (by Nikki Nikkhoui; author: Nikki Nikkhoui):

[mediawiki/core@master] Remove 'matched_title' check

https://gerrit.wikimedia.org/r/766819

kostajh claimed this task.

I think what is happening is that the results are coming back in different orders from Sqlite vs MySQL which is affecting how the code deals with duplicates:

In MySQL:

  • $searchResponse includes both Redirect Source and Redirect Target results
  • Redirect Target gets processed first (and therefore returns null on RedirectStore:: getRedirectTarget())
  • Redirect Source gets processed next and is not added to the results because its Redirect Target has already been added, and therefore matched_title is null because Redirect Target is not a redirect page

In Sqlite:

  • $searchResponse includes both Redirect Source and Redirect Target results
  • Redirect Source gets processed first (and therefore returns with a Title on RedirectStore:: getRedirectTarget()) and also has its matched_title field populated because this is a redirect.
  • Redirect Target gets processed next and is not added to the results because its Redirect Source has already been added

Short term I will upload a patch to remove the matched_title check. However the code should be changed to be consistent when dealing with this scenario, even though I don't think it makes a huge difference for the end use case, it may cause future headaches in testing. Will create a ticket for that.

Thanks for digging into this and making the follow-up task. I'll mark this one as resolved.

Change 766798 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] Revert \"jjb: Skip api-testing for Quibble fullrun with SQLite\"

https://gerrit.wikimedia.org/r/766798

Change 766819 merged by jenkins-bot:

[mediawiki/core@master] api-testing: Remove 'matched_title' check

https://gerrit.wikimedia.org/r/766819

Change 766798 merged by jenkins-bot:

[integration/config@master] Revert \"jjb: Skip api-testing for Quibble fullrun with SQLite\"

https://gerrit.wikimedia.org/r/766798