Page MenuHomePhabricator
Create Task
Maniphest T403156

Core webdriver.io tests fails when running against Patch Demo
Open, Needs TriagePublicBUG REPORT
Actions

Description

Expected Behavior
You should be able to run core webdriver.io tests against Patch demo.

Actual Behavior
Running webdriver.io tests generates errors so the test do not pass. It's mwbot that cannot login or do it thing. mwbot generates the generic No valid JSON response response and if I add some logging I can see that the user gets 403 Forbidden.

npm run selenium-test
....
[0-1] RUNNING in chrome - file:///tests/selenium/docs/Page_object_pattern/specs/login.js
[0-1] Error: invalidjson: No valid JSON response
[0-1]     at /Users/phedenskog/git/core/node_modules/mwbot/src/index.js:254:31
[0-1]     at tryCatcher (/Users/phedenskog/git/core/node_modules/bluebird/js/release/util.js:16:23)
[0-1]     at Promise._settlePromiseFromHandler (/Users/phedenskog/git/core/node_modules/bluebird/js/release/promise.js:547:31)
[0-1]     at Promise._settlePromise (/Users/phedenskog/git/core/node_modules/bluebird/js/release/promise.js:604:18)
[0-1]     at Promise._settlePromise0 (/Users/phedenskog/git/core/node_modules/bluebird/js/release/promise.js:649:10)
[0-1]     at Promise._settlePromises (/Users/phedenskog/git/core/node_modules/bluebird/js/release/promise.js:729:18)
[0-1]     at _drainQueueStep (/Users/phedenskog/git/core/node_modules/bluebird/js/release/async.js:93:12)
[0-1]     at _drainQueue (/Users/phedenskog/git/core/node_modules/bluebird/js/release/async.js:86:9)
[0-1]     at Async._drainQueues (/Users/phedenskog/git/core/node_modules/bluebird/js/release/async.js:102:5)
[0-1]     at Async.drainQueues (/Users/phedenskog/git/core/node_modules/bluebird/js/release/async.js:15:14)
[0-1]     at process.processImmediate (node:internal/timers:485:21) {
[0-1]   code: 'invalidjson',
[0-1]   info: 'No valid JSON response',
[0-1]   response: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n' +
[0-1]     '<html><head>\n' +
[0-1]     '<title>403 Forbidden</title>\n' +
[0-1]     '</head><body>\n' +
[0-1]     '<h1>Forbidden</h1>\n' +
[0-1]     "<p>You don't have permission to access this resource.</p>\n" +
[0-1]     '<hr>\n' +
[0-1]     '<address>Apache/2.4.38 (Debian) Server at a14d02fca3.catalyst.wmcloud.org Port 8080</address>\n' +
[0-1]     '</body></html>\n'
[0-1] }
[0-1] TypeError in "User.should be able to log in without page object"

I tested multiple users but I guess it should be the "Patch Demo" user that should do the job. I can login as that users through the GUI but maybe it doesn't have correct rights for mwbot or I'm not sure how that works.

Steps to reproduce:
Follow the instructions at https://www.mediawiki.org/wiki/Selenium/Getting_Started/Run_tests_targeting_Patch_demo

Related Objects
Search...

Event Timeline

Peter created this task.Aug 28 2025, 9:28 AM
Restricted Application added a subscriber: • Aklapper. · View Herald TranscriptAug 28 2025, 9:28 AM
• Lucas_Werkmeister_WMDE subscribed.Aug 28 2025, 11:00 AM

Can reproduce:

$ curl -H 'User-Agent: mwbot/2.1.3' 'https://cabcab1a95.catalyst.wmcloud.org/w/api.php?action=query&format=json'
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access this resource.</p>
<hr>
<address>Apache/2.4.38 (Debian) Server at cabcab1a95.catalyst.wmcloud.org Port 8080</address>
</body></html>

Whereas curl’s default user agent works:

$ curl 'https://cabcab1a95.catalyst.wmcloud.org/w/api.php?action=query&format=json'
{"batchcomplete":""}

Is Patch Demo specifically blocking mwbot? (“mwboot” also works, for example.)

• thcipriani subscribed.Aug 28 2025, 4:30 PM

We're blocking anything with "bot" at the moment due to lots of scraping of environments causing load, unfortuantely:

$ curl -H 'User-Agent: mwbot/2.1.3' 'https://51ee652795.catalyst.wmcloud.org/w/api.php?action=query&format=json'
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access this resource.</p>
<hr>
<address>Apache/2.4.38 (Debian) Server at 51ee652795.catalyst.wmcloud.org Port 8080</address>
</body></html>

vs.

$ curl -H 'User-Agent: mwrealhumandontworry/2.1.3' 'https://51
ee652795.catalyst.wmcloud.org/w/api.php?action=query&format=json'
{"batchcomplete":""}operations-puppet (production % u=)
• thcipriani edited projects, added: Catalyst (Radar); removed: Catalyst (PatchDemo).Aug 28 2025, 4:30 PM
zeljkofilipin added a parent task: T405079: Stop using mwbot in Selenium tests.Sep 19 2025, 11:14 AM
Peter added a comment.Oct 21 2025, 4:02 PM

With wdio-mediawiki 6.0.0. we have another user agent. But there are some other tests that fails, let me make a list. We had a discussion today at office hours how we gonna handle that: do we need to setup of tests or should core tests work against patch demo? I think at least we should be able to run automatic tests against Patch demo, so that we can verify that everything works.

Peter added a comment.Oct 22 2025, 6:01 AM

Running tests against patch demo:

Spec Files:	 5 passed, 5 retries, 5 failed, 10 total (100% completed) in 00:08:47

One of the things we run into is the create account limit:

Screenshot 2025-10-22 at 07.43.30.png (1,880×1,394 px, 243 KB)

And then it some of those issues with missing edit link on the main page:

Error: Expect $(`a[title="Edit section: Getting started"]`) to exist

Lets start by taking two decisions:

  1. Do we want to be able to run automatic tests against PatchDemo?
  2. If so, do we want core tests to work or have a special test suite?
zeljkofilipin added a project: Browser Test Platform.Nov 20 2025, 12:06 PM
zeljkofilipin removed a project: Test Platform.Nov 20 2025, 3:39 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits