MW's new API tests are a breaking change which cause OpenStackManager and other extensions to fail to pass CI
Closed, ResolvedPublic
Actions

Description

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/3027/console

20:14:41     1) "before all" hook for "should edit page as a user in autopatrol group"
20:14:41 
20:14:41 
20:14:41   3 passing (1s)
20:14:41   1 failing
20:14:41 
20:14:41   1) Testing default autopatrolling rights
20:14:41        "before all" hook for "should edit page as a user in autopatrol group":
20:14:41 
20:14:41       AssertionError: expected 'FAIL' to equal 'PASS'
20:14:41       + expected - actual
20:14:41 
20:14:41       -FAIL
20:14:41       +PASS
20:14:41       
20:14:41       at Client.createAccount (node_modules/api-testing/lib/actionapi.js:413:16)
20:14:41       at process._tickCallback (internal/process/next_tick.js:68:7)

Details

Related Changes in Gerrit:

	Subject	Author	Repo	Branch	Lines +/-
	Follow-up 1d72667: Also drop 'api-testing' generally	Jforrester	integration/config	master	+4 -4
	jjb: Provide mediawiki-quibble-apitests-*-docker; drop from main	Jforrester	integration/config	master	+15 -2

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Stalled	None	T219873 Create a suite of end-to-end API test for MediaWiki core
Stalled	None	T227999 Full implementation of test runner
Open	None	T231893 Run tests automatically when tests changed
Resolved	daniel	T236680 Enable API integration tests in CI for MediaWiki core
Resolved	Jdforrester-WMF	T242913 MW's new API tests are a breaking change which cause OpenStackManager and other extensions to fail to pass CI

Event Timeline

Reedy created this task.Jan 15 2020, 8:18 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 15 2020, 8:18 PM

Yup, this is the new api tests which bake in expectations. Several extensions don't meet them.

Reedy renamed this task from quibble-vendor-mysql-php72-noselenium-docker failing on extension to quibble-vendor-mysql-php72-noselenium-docker failing on OpenStackManager extension.Jan 15 2020, 8:22 PM

Reedy added a parent task: T236680: Enable API integration tests in CI for MediaWiki core.

Reedy added projects: MediaWiki-extensions-OpenStackManager, Core Platform Team Initiatives (API Integration Tests).

Jdforrester-WMF renamed this task from quibble-vendor-mysql-php72-noselenium-docker failing on OpenStackManager extension to MW's new API tests are a breaking change which cause OpenStackManager and other extensions to fail to pass CI.Jan 15 2020, 8:25 PM

DannyS712 subscribed.Jan 24 2020, 10:50 PM

Also breaks WikimediaIncubator:
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/3828/console

03:26:59   Page HTML Bare
03:26:59 INFO:backend.DevWebServer:[Tue Jan 28 11:26:59 2020] 127.0.0.1:39306 [200]: /api.php?format=json&action=query&meta=tokens&type=csrf
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39308 [200]: /api.php
03:27:00     1) "before all" hook in "Page HTML Bare"
03:27:00 
03:27:00   Page Source
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39312 [200]: /api.php?format=json&action=query&meta=tokens&type=csrf
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39314 [200]: /api.php
03:27:00     2) "before all" hook in "Page Source"
03:27:00 
03:27:00   Testing default autopatrolling rights
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39316 [200]: /api.php?format=json&action=query&meta=tokens&type=login
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39320 [200]: /api.php
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39322 [200]: /api.php?format=json&action=query&meta=tokens&type=createaccount%7Cuserrights%7Ccsrf
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39324 [200]: /api.php?format=json&action=query&meta=tokens&type=createaccount
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39326 [200]: /api.php
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39328 [200]: /index.php
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39330 [200]: /api.php
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39332 [200]: /api.php?format=json&action=query&meta=tokens&type=login
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39334 [200]: /api.php
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39336 [200]: /api.php?format=json&action=query&meta=tokens&type=userrights%7Ccsrf
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39338 [200]: /api.php
03:27:01     3) should edit page as a user in autopatrol group
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39340 [200]: /api.php?format=json&action=query&meta=tokens&type=csrf
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39342 [200]: /api.php
03:27:01     4) should edit page as an anonymous user
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39344 [200]: /api.php?format=json&action=query&meta=userinfo
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39346 [200]: /api.php?format=json&action=query&list=recentchanges&rctitle=Autopatrol_MXWuumkfPI&rcprop=patrolled%7Cuser
03:27:01     5) should confirm autopatrolling rights
03:27:01 
03:27:01 
03:27:01   0 passing (2s)
03:27:01   5 failing
03:27:01 
03:27:01   1) Page HTML Bare
03:27:01        "before all" hook in "Page HTML Bare":
03:27:01      AssertionError: User "<anon>": Action "edit" returned error code "wminc-error-unprefixed": Error: This page is [[Help:Contents|unprefixed]].!
03:27:01       at Client.action (node_modules/api-testing/lib/actionapi.js:114:20)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)
03:27:01 
03:27:01   2) Page Source
03:27:01        "before all" hook in "Page Source":
03:27:01      AssertionError: User "<anon>": Action "edit" returned error code "wminc-error-unprefixed": Error: This page is [[Help:Contents|unprefixed]].!
03:27:01       at Client.action (node_modules/api-testing/lib/actionapi.js:114:20)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)
03:27:01 
03:27:01   3) Testing default autopatrolling rights
03:27:01        should edit page as a user in autopatrol group:
03:27:01      AssertionError: User "MindyvPufLUnYVi": Action "edit" returned error code "wminc-error-unprefixed": Error: This page is [[Help:Contents|unprefixed]].!
03:27:01       at Client.action (node_modules/api-testing/lib/actionapi.js:114:20)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)
03:27:01 
03:27:01   4) Testing default autopatrolling rights
03:27:01        should edit page as an anonymous user:
03:27:01      AssertionError: User "<anon>": Action "edit" returned error code "wminc-error-unprefixed": Error: This page is [[Help:Contents|unprefixed]].!
03:27:01       at Client.action (node_modules/api-testing/lib/actionapi.js:114:20)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)
03:27:01 
03:27:01   5) Testing default autopatrolling rights
03:27:01        should confirm autopatrolling rights:
03:27:01 
03:27:01       AssertionError: expected [] to have a length of 2 but got 0
03:27:01       + expected - actual
03:27:01 
03:27:01       -0
03:27:01       +2
03:27:01       
03:27:01       at Context.it (tests/api-testing/action/Autopatrolling.js:24:10)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)

DannyS712 mentioned this in T241630: WikimediaIncubator needs uses of global $wgUser removed.Jan 28 2020, 11:31 AM

Umherirrender added a project: ci-test-error.Jan 28 2020, 7:42 PM

Other extensions:

GoogleAppEngine
EditSubpages

Change 569633 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] jjb: Provide mediawiki-quibble-apitests-*-docker; drop from main

https://gerrit.wikimedia.org/r/569633

gerritbot added a project: Patch-For-Review.Feb 3 2020, 9:34 PM

OK, we're now not running the API tests on extensions, just in a special job.

Change 569633 merged by jenkins-bot:
[integration/config@master] jjb: Provide mediawiki-quibble-apitests-*-docker; drop from main

https://gerrit.wikimedia.org/r/569633

Maintenance_bot removed a project: Patch-For-Review.Feb 3 2020, 10:10 PM

This seems to be failing on OSM still - https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/OpenStackManager/+/565101/

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/4293/console

Also https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/4294/console

Eurgh, of course the special snowflake 'noselenium' jobs need special measures.

Change 570474 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] Follow-up 1d72667: Also drop 'api-testing' generally

https://gerrit.wikimedia.org/r/570474

gerritbot added a project: Patch-For-Review.Feb 5 2020, 11:40 PM

Change 570474 merged by jenkins-bot:
[integration/config@master] Follow-up 1d72667: Also drop 'api-testing' generally

https://gerrit.wikimedia.org/r/570474

Jdforrester-WMF closed this task as Resolved.Feb 5 2020, 11:54 PM

Jdforrester-WMF added a project: Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)).

Jdforrester-WMF moved this task from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)) board.

Maintenance_bot removed a project: Patch-For-Review.Feb 6 2020, 12:10 AM