Page MenuHomePhabricator

"should mark the revert as a bot edit" test flaky with Apache + php-fpm
Open, Needs TriagePublic

Description

see https://integration.wikimedia.org/ci/job/integration-quibble-apache-fullrun/20/console

00:03:29.024   The rollback action
00:03:29.024     ✔ rolls back consecutive edits (541ms)
00:03:29.565     ✔ doesn't roll back edits by another user (439ms)
00:03:30.005     ✔ doesn't allow a regular user to roll back edits (268ms)
00:03:30.273     ✔ should undo the last edit on the page using the pageid parameter (324ms)
00:03:30.598     ✔ should throw an error with code missingparam when neither pageid nor title is provided for a rollback action (299ms)
00:03:30.897     ✔ should throw an error with code notoken when the token parameter is not provided for a rollback action (257ms)
00:03:31.154     ✔ should throw an error with code nouser when a user is not provided for a rollback action (236ms)
00:03:31.391     ✔ should throw an error with code onlyauthor if the page has only one author (146ms)
00:03:31.538     ✔ should throw an error with code mustpostparams when the request to the API is not a post request (184ms)
00:03:31.723     1) should mark the revert as a bot edit

...

0:03:39.831   1 failing
00:03:39.831 
00:03:39.831   1) The rollback action
00:03:39.831        should mark the revert as a bot edit:
00:03:39.831      AssertionError: expected undefined to exist
00:03:39.831       at Context.<anonymous> (tests/api-testing/action/Rollback.js:188:10)
00:03:39.831       at runMicrotasks (<anonymous>)
00:03:39.831       at processTicksAndRejections (internal/process/task_queues.js:95:5)

Looking at the test code:

it( 'should mark the revert as a bot edit', async () => {
		const title = utils.title( 'Rollback_' );

		const firstEdit = await alice.edit( title, { text: 'One', summary: 'first' } );
		const secondEdit = await bob.edit( title, { text: 'Two', summary: 'second' } );

		const result = await mindy.action( 'rollback', {
			pageid: firstEdit.pageid,
			user: bob.username,
			summary: 'revert',
			markbot: true,
			token: await mindy.token( 'rollback' )
		}, 'POST' );

		const recentChanges = await mindy.getChangeEntry( { rctitle: title } );

		assert.exists( recentChanges.bot );
		assert.sameTitle( result.rollback.title, title );
		assert.equal( result.rollback.old_revid, secondEdit.newrevid );
		assert.equal( recentChanges.revid, result.rollback.revid );
	} );

I suppose what is happening is that we're fetching the recent change entry before the rollback is stored in the database, since we're now processing requests asynchronously – there's no guarantee that the rollback action is fully processed (there's probably code happening in a deferred update) before the getChangeEntry call happens.

In webdriver we have browser.waitUntil which would allow retrying the mindy.getChangeEntry() call until recentChanges.bot is not undefined, but I'm not sure what helpers exist for supertest/mocha.

Event Timeline

Change 751415 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] api-testing: Mark bot edit revert test as flaky

https://gerrit.wikimedia.org/r/751415

Change 751428 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] api-testing: switch Quibble jobs to Apache

https://gerrit.wikimedia.org/r/751428

Change 751431 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/tools/api-testing@master] (DO NOT SUBMIT) test CI with Apache

https://gerrit.wikimedia.org/r/751431

Change 751415 merged by jenkins-bot:

[mediawiki/core@master] api-testing: Skip flaky bot edit revert test

https://gerrit.wikimedia.org/r/751415

Change 751428 merged by jenkins-bot:

[integration/config@master] api-testing: switch Quibble jobs to Apache

https://gerrit.wikimedia.org/r/751428

Change 751431 abandoned by Hashar:

[mediawiki/tools/api-testing@master] (DO NOT SUBMIT) test CI with Apache

Reason:

https://gerrit.wikimedia.org/r/751431

Tagging #platform-engineering as owners (I think?) of the api-testing tests.

Change 751539 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/core@master] Revert \"api-testing: Skip flaky bot edit revert test\"

https://gerrit.wikimedia.org/r/751539

I have send https://gerrit.wikimedia.org/r/c/mediawiki/core/+/751539 which enable the test again, it should fail on CI since we have switched to Apache earlier today.

Change 751539 merged by jenkins-bot:

[mediawiki/core@master] Revert \"api-testing: Skip flaky bot edit revert test\"

https://gerrit.wikimedia.org/r/751539

Change 751431 restored by Hashar:

[mediawiki/tools/api-testing@master] (DO NOT SUBMIT) test CI with Apache

https://gerrit.wikimedia.org/r/751431

Aklapper edited subscribers, added: Aklapper; removed: Clarakosi.

Adding missing MediaWiki-Core-Tests code project tag as Platform Team Initiatives (API Integration Tests) team tag is archived and its parent Platform Engineering team does not exist anymore