- Follow instructions from AbuseFilter (Quickstart) to get AbuseFilter working locally.
- Refactor WebdriverIO tests (in tests/selenium) from sync to async mode.
- Push code to Gerrit.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T225730 Reduce runtime of MW shared gate Jenkins jobs to 5 min | |||
| Open | None | T226869 Run browser tests in parallel | |||
| Open | None | T210726 Selenium framework cleanup | |||
| Open | None | T344754 Browser tests video capture is shared between tests | |||
| Open | None | T324685 Upgrade WebdriverIO to v8 in all repositories | |||
| Resolved | None | T314468 Update vite to Version 3 | |||
| Open | None | T314470 Upgrade all CI jobs for WMF-deployed projects from Node 14 to Node 16 | |||
| Open | None | T256626 Refactor WebdriverIO tests from sync to async mode | |||
| Resolved | pwangai | T300790 Refactor AbuseFilter WebdriverIO tests from sync to async mode |
Event Timeline
Comment Actions
@Osamaahmed17: Please set appropriate project tags (in this case AbuseFilter). Thanks.
Comment Actions
Change 901629 had a related patch set uploaded (by Pwangai; author: Pwangai):
[mediawiki/extensions/AbuseFilter@master] WIP selenium: Refactor WebdriverIO tests from sync to async mode
Comment Actions
Change 901629 merged by jenkins-bot:
[mediawiki/extensions/AbuseFilter@master] selenium: Refactor WebdriverIO tests from sync to async mode
Comment Actions
Hello @zeljkofilipin & @pwangai - For some reason this is making patches randomly fail (either on AbuseFilter or extensions that depend on AbuseFilter e.g. https://gerrit.wikimedia.org/r/c/mediawiki/extensions/OATHAuth/+/903681 - I know nothing about selenium, could you please take a look and see if there's an issue with the tests' code? Thank you.
Comment Actions
Hey @MarcoAurelio , I am working on stablising the few flaky tests that keep randomly failing.
Comment Actions
Change 904447 had a related patch set uploaded (by Pwangai; author: Pwangai):
[mediawiki/extensions/AbuseFilter@master] A recent change of AbuseFilter selenium tests from sync to async mode is making patches randomly fail. This is due to some flaky tests that sometimes fail and other times passes. In this patch I am changing the failing test to use the recommended expect-webdriverio format instead of assert() to see if this will improve the overall stability
Comment Actions
Change 904447 abandoned by Pwangai:
[mediawiki/extensions/AbuseFilter@master] WIP selenium: fix unstable tests
Reason:
Comment Actions
I'm not sure if this task should be closed as resolved. One of the tests that was previously working had to be disabled because it's now flaky. I would like to see that test re-enabled before calling this resolved. I will take a look at the test later.
Comment Actions
@Daimona It would be helpful if you looked at the test to see if we could find a permanent solution. When I run the test multiple times locally on my machine, it doesn't fail, but somehow in CI it keeps failing. I have tried different approaches, but none seem to be stable in CI. Reopening the task.
Comment Actions
Sorry it took me this long, for some reason selenium doesn't work on my local unless I set DISPLAY to :0 and hack the config to disable video recording. That said, the AbuseFilter tests are (very) sequential, I'm wondering if this could be causing the failure. Also, it looks like there is no recording of the failed test in CI, and it's passing for me locally, which makes debugging quite challenging.
Comment Actions
Change 904796 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):
[mediawiki/extensions/AbuseFilter@master] tests: Improve selenium tests
Comment Actions
Change 904796 merged by jenkins-bot:
[mediawiki/extensions/AbuseFilter@master] tests: Improve selenium tests
Comment Actions
Sorry @Daimona but this is making again unrelated changes for extensions that depend on AbuseFilter to fail: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaMessages/+/905659 :-(
Comment Actions
Hmmm this is weird. The selenium suite was run 100 times for the patch above, and it passed 100/100. It could still be a transient issue, but I wonder if it might be caused by something (another extension, config etc.) that isn't there when running tests for AF. The video is also not very helpful, as it only shows the page.
Comment Actions
Change 949566 had a related patch set uploaded (by Paladox; author: Pwangai):
[mediawiki/extensions/AbuseFilter@REL1_40] selenium: Refactor WebdriverIO tests from sync to async mode
Comment Actions
Change 949571 had a related patch set uploaded (by Paladox; author: Daimona Eaytoy):
[mediawiki/extensions/AbuseFilter@REL1_40] tests: Improve selenium tests
Comment Actions
Change 949566 merged by Krinkle:
[mediawiki/extensions/AbuseFilter@REL1_40] selenium: Refactor WebdriverIO tests from sync to async mode
Comment Actions
Change 949571 merged by jenkins-bot:
[mediawiki/extensions/AbuseFilter@REL1_40] tests: Improve selenium tests
Comment Actions
Change 955790 had a related patch set uploaded (by Jforrester; author: Pwangai):
[mediawiki/extensions/AbuseFilter@REL1_39] selenium: Refactor WebdriverIO tests from sync to async mode
Comment Actions
Change 955790 merged by Jforrester:
[mediawiki/extensions/AbuseFilter@REL1_39] selenium: Refactor WebdriverIO tests from sync to async mode