About once a year on a WP:THURSDAY Page Curation breaks. @Barkeep49

That's actually quite often to break the entire extension despite in theory two people testing each patch (the patch writer, and the +2er). I think proper Selenium tests would fix this, by automated testing all the important code paths. So I'm definitely onboard with this.

I'd suggest starting with the smallest possible useful scenario, then we can build from there. @Novem_Linguae perhaps you could write out a scenario at a high level?

What type of additional setup, if any, does PageTriage need in CI for Special:NewPagesFeed to operate properly?

I envision the list of tests/happy paths being something like...

(big checklist that was here moved to first post)

The cron jobs in cron/updatePageTriageQueue.php could use some tests too, but Selenium probably isn't the best way to do it.

I imagine all these tests would benefit with seeding the test wiki with a couple of test articles in various states of reviewed, unreviewed, and autopatrolled. The setup I use for my localhost wiki that's applicable here and that mirrors (or will mirror) enwiki is something like...

$wgGroupPermissions['autoreviewer']['autopatrol'] = true;
$wgGroupPermissions['patroller']['patrol'] = true;
wfLoadSkin( 'Vector' );
wfLoadExtension( 'PageTriage' );
$wgExtraNamespaces[118] = 'Draft'; // enable AFC
$wgPageTriageDraftNamespaceId = 118; // enable AFC
$wgPageTriageNoIndexUnreviewedNewArticles = true;
$wgPageTriageMaxAge = null;
$wgPageTriageMaxNoIndexAge = 90;

For smallest scenario, maybe tackle the first test case above, Special:NewPagesFeed->should display unreviewed new article. In addition to displaying the new article, we can also have it click the article and make sure the Page Curation toolbar loads and is shown.

Change 845046 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/PageTriage@master] selenium: Initial framework setup and first test

https://gerrit.wikimedia.org/r/845046

Change 845046 merged by jenkins-bot:

[mediawiki/extensions/PageTriage@master] selenium: Initial framework setup and first test

https://gerrit.wikimedia.org/r/845046

Change 845397 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/PageTriage@master] [WIP] selenium: Add basic test for adding article and seeing toolbar

https://gerrit.wikimedia.org/r/845397

@kostajh. Howdy. What's your preferred setup for running and debugging Selenium tests? And what OS and code editor do you use? There's quite a few options to choose from at https://www.mediawiki.org/wiki/Selenium/Reference/Running_tests so I am looking for some direction. Thanks for your insights.

In T320924#8412537, @Novem_Linguae wrote:

@kostajh. Howdy. What's your preferred setup for running and debugging Selenium tests? And what OS and code editor do you use? There's quite a few options to choose from at https://www.mediawiki.org/wiki/Selenium/Reference/Running_tests so I am looking for some direction. Thanks for your insights.

Hi, I'm running MediaWiki locally on macOS via homebrew PHP/Apache/MySQL. So my selenium test running workflow is:

DISPLAY=1 QUIBBLE_APACHE=1 npm run selenium-test

I use PhpStorm as my editor. Highly recommended :) We have access to open-source licenses, see https://www.mediawiki.org/wiki/JetBrains_IDEs

NOINDEX tests

• test various combinations of $wgPageTriageNoIndexUnreviewedNewArticles, $wikitextHasNoIndexMagicWord, $wgPageTriageMaxAge, $wgPageTriageMaxNoIndexAge, articles before the age, articles after the age, reviewed articles, and unreviewed articles

I'd say these should be PHPUnit tests, not Selenium ones.

Never mind. I got it working :) Wrong directory in CLI, I think.

I'm running my localhost Selenium tests with the .env file pointed at a localhost wiki. This used to work, but is not working for me tonight. I'm getting the below error. I tried updating ChromeDriver to 112, to match my major version of Chrome. To experiment, I ran some Selenium tests for a non-mediawiki project, and those worked after I updated the ChromeDriver. But no luck so far with this project.

• Any idea what the problem is?
• Is there a way to get more verbose errors? If the problem is with ChromeDriver, for example, it'd be nice to know, instead of receiving a "timeout" error.
• Any settings I need to tweak to get the browser to be visible instead of hidden? That might provide a good hint.

The CLI command I'm using is npm run selenium-test -- --spec extensions/pagetriage/tests/selenium/specs/*

Change 845397 merged by jenkins-bot:

[mediawiki/extensions/PageTriage@master] selenium: Assert new article appears in feed

https://gerrit.wikimedia.org/r/845397

Change 914720 had a related patch set uploaded (by Novem Linguae; author: Novem Linguae):

[mediawiki/extensions/PageTriage@master] [WIP] selenium test for PageTriageExternalTagsOptions.php

https://gerrit.wikimedia.org/r/914720

The URL in my localhost Selenium tests is messed up. I wonder where the "C:/Program Files/git" part is coming from.

In T320924#8823058, @Novem_Linguae wrote:

The URL in my localhost Selenium tests is messed up. I wonder where the "C:/Program Files/git" part is coming from.

Not sure; make sure you are setting the relevant environment variables.

That's a great hint, thank you. In MINGW64 I ran env | sort and found this environment variable: EXEPATH=C:\Program Files\Git

So I tried it in PowerShell and got it kind of working. Now my errors are 50% of the time it's 60 second timeout due to my localhost being slow (which maybe I can solve by raising the timeout with await browser.setTimeout({ 'script': 500000 }) or something), and 50% of the time Chrome is erroring out with STATUS_ACCESS_VIOLATION. I'll keep working on this, I'll bet I can get it working now :)

@Novem_Linguae do you need help running selenium tests on your machine?

Change 983559 had a related patch set uploaded (by Sohom Datta; author: Sohom Datta):

[mediawiki/extensions/PageTriage@master] [DNM][WIP] Add selenium tests for tags module based on current code

https://gerrit.wikimedia.org/r/983559

Change 983559 merged by jenkins-bot:

[mediawiki/extensions/PageTriage@master] Add selenium tests for tags module based on current code

https://gerrit.wikimedia.org/r/983559

Change #914720 abandoned by Novem Linguae:

[mediawiki/extensions/PageTriage@master] [WIP] selenium test for PageTriageExternalTagsOptions.php

Reason:

not currently working on this

https://gerrit.wikimedia.org/r/914720