Page MenuHomePhabricator

MW's new API tests are a breaking change which cause OpenStackManager and other extensions to fail to pass CI
Closed, ResolvedPublic

Description

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/3027/console

20:14:41     1) "before all" hook for "should edit page as a user in autopatrol group"
20:14:41 
20:14:41 
20:14:41   3 passing (1s)
20:14:41   1 failing
20:14:41 
20:14:41   1) Testing default autopatrolling rights
20:14:41        "before all" hook for "should edit page as a user in autopatrol group":
20:14:41 
20:14:41       AssertionError: expected 'FAIL' to equal 'PASS'
20:14:41       + expected - actual
20:14:41 
20:14:41       -FAIL
20:14:41       +PASS
20:14:41       
20:14:41       at Client.createAccount (node_modules/api-testing/lib/actionapi.js:413:16)
20:14:41       at process._tickCallback (internal/process/next_tick.js:68:7)

Event Timeline

Reedy created this task.Jan 15 2020, 8:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 15 2020, 8:18 PM

Yup, this is the new api tests which bake in expectations. Several extensions don't meet them.

Reedy renamed this task from quibble-vendor-mysql-php72-noselenium-docker failing on extension to quibble-vendor-mysql-php72-noselenium-docker failing on OpenStackManager extension.Jan 15 2020, 8:22 PM
Jdforrester-WMF renamed this task from quibble-vendor-mysql-php72-noselenium-docker failing on OpenStackManager extension to MW's new API tests are a breaking change which cause OpenStackManager and other extensions to fail to pass CI.Jan 15 2020, 8:25 PM

Also breaks WikimediaIncubator:
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-noselenium-docker/3828/console

03:26:59   Page HTML Bare
03:26:59 INFO:backend.DevWebServer:[Tue Jan 28 11:26:59 2020] 127.0.0.1:39306 [200]: /api.php?format=json&action=query&meta=tokens&type=csrf
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39308 [200]: /api.php
03:27:00     1) "before all" hook in "Page HTML Bare"
03:27:00 
03:27:00   Page Source
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39312 [200]: /api.php?format=json&action=query&meta=tokens&type=csrf
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39314 [200]: /api.php
03:27:00     2) "before all" hook in "Page Source"
03:27:00 
03:27:00   Testing default autopatrolling rights
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39316 [200]: /api.php?format=json&action=query&meta=tokens&type=login
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39320 [200]: /api.php
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39322 [200]: /api.php?format=json&action=query&meta=tokens&type=createaccount%7Cuserrights%7Ccsrf
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39324 [200]: /api.php?format=json&action=query&meta=tokens&type=createaccount
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39326 [200]: /api.php
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39328 [200]: /index.php
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39330 [200]: /api.php
03:27:00 INFO:backend.DevWebServer:[Tue Jan 28 11:27:00 2020] 127.0.0.1:39332 [200]: /api.php?format=json&action=query&meta=tokens&type=login
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39334 [200]: /api.php
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39336 [200]: /api.php?format=json&action=query&meta=tokens&type=userrights%7Ccsrf
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39338 [200]: /api.php
03:27:01     3) should edit page as a user in autopatrol group
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39340 [200]: /api.php?format=json&action=query&meta=tokens&type=csrf
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39342 [200]: /api.php
03:27:01     4) should edit page as an anonymous user
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39344 [200]: /api.php?format=json&action=query&meta=userinfo
03:27:01 INFO:backend.DevWebServer:[Tue Jan 28 11:27:01 2020] 127.0.0.1:39346 [200]: /api.php?format=json&action=query&list=recentchanges&rctitle=Autopatrol_MXWuumkfPI&rcprop=patrolled%7Cuser
03:27:01     5) should confirm autopatrolling rights
03:27:01 
03:27:01 
03:27:01   0 passing (2s)
03:27:01   5 failing
03:27:01 
03:27:01   1) Page HTML Bare
03:27:01        "before all" hook in "Page HTML Bare":
03:27:01      AssertionError: User "<anon>": Action "edit" returned error code "wminc-error-unprefixed": Error: This page is [[Help:Contents|unprefixed]].!
03:27:01       at Client.action (node_modules/api-testing/lib/actionapi.js:114:20)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)
03:27:01 
03:27:01   2) Page Source
03:27:01        "before all" hook in "Page Source":
03:27:01      AssertionError: User "<anon>": Action "edit" returned error code "wminc-error-unprefixed": Error: This page is [[Help:Contents|unprefixed]].!
03:27:01       at Client.action (node_modules/api-testing/lib/actionapi.js:114:20)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)
03:27:01 
03:27:01   3) Testing default autopatrolling rights
03:27:01        should edit page as a user in autopatrol group:
03:27:01      AssertionError: User "MindyvPufLUnYVi": Action "edit" returned error code "wminc-error-unprefixed": Error: This page is [[Help:Contents|unprefixed]].!
03:27:01       at Client.action (node_modules/api-testing/lib/actionapi.js:114:20)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)
03:27:01 
03:27:01   4) Testing default autopatrolling rights
03:27:01        should edit page as an anonymous user:
03:27:01      AssertionError: User "<anon>": Action "edit" returned error code "wminc-error-unprefixed": Error: This page is [[Help:Contents|unprefixed]].!
03:27:01       at Client.action (node_modules/api-testing/lib/actionapi.js:114:20)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)
03:27:01 
03:27:01   5) Testing default autopatrolling rights
03:27:01        should confirm autopatrolling rights:
03:27:01 
03:27:01       AssertionError: expected [] to have a length of 2 but got 0
03:27:01       + expected - actual
03:27:01 
03:27:01       -0
03:27:01       +2
03:27:01       
03:27:01       at Context.it (tests/api-testing/action/Autopatrolling.js:24:10)
03:27:01       at process._tickCallback (internal/process/next_tick.js:68:7)

Other extensions:

  • GoogleAppEngine
  • EditSubpages

Change 569633 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] jjb: Provide mediawiki-quibble-apitests-*-docker; drop from main

https://gerrit.wikimedia.org/r/569633

Jdforrester-WMF closed this task as Resolved.Feb 3 2020, 9:34 PM
Jdforrester-WMF claimed this task.

OK, we're now not running the API tests on extensions, just in a special job.

Change 569633 merged by jenkins-bot:
[integration/config@master] jjb: Provide mediawiki-quibble-apitests-*-docker; drop from main

https://gerrit.wikimedia.org/r/569633

Eurgh, of course the special snowflake 'noselenium' jobs need special measures.

Change 570474 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] Follow-up 1d72667: Also drop 'api-testing' generally

https://gerrit.wikimedia.org/r/570474

Change 570474 merged by jenkins-bot:
[integration/config@master] Follow-up 1d72667: Also drop 'api-testing' generally

https://gerrit.wikimedia.org/r/570474