Create script to parse test case names from webdriverio tests
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	SLong-WMF
	Dec 1 2025, 9:28 PM

Description

Create a script that will take a repo URL and return a list of all webdriverIO tests as part of a JSON file.

This would be useful in mapping out test coverage initially, and possibly as part of a timed job that can eventually update documentation.

Event Timeline

SLong-WMF created this task.Dec 1 2025, 9:28 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 1 2025, 9:28 PM

vaughnwalters moved this task from Backlog to Refined on the QS-Test-Automation board.Mar 12 2026, 7:12 PM

vaughnwalters claimed this task.Mar 13 2026, 9:09 PM

vaughnwalters moved this task from Refined to In Progress on the QS-Test-Automation board.

here's the codebase. we can move it where ever:
https://github.com/vaughnwalters/browser-test-scanner

here's an example of the output that the script will create:
https://en.wikipedia.org/wiki/User:VWalters-WMF/sandbox/browsertests

vaughnwalters moved this task from In Progress to In Validation on the QS-Test-Automation board.Mar 16 2026, 4:52 PM

moved the repo from my personal github repo to here: https://gitlab.wikimedia.org/repos/test-platform/misc-scripts/browser-test-scanner

also, when scripts run they now update this section: https://www.mediawiki.org/wiki/Wikimedia_Quality_Services#Automated_tests_available which is transcluded from https://www.mediawiki.org/wiki/Wikimedia_Quality_Services/Automated_tests_available

These files are scheduled in a cron job now and will run and auto update the documentation every monday: https://gitlab.wikimedia.org/repos/test-platform/misc-scripts/browser-test-scanner/-/pipeline_schedules

well.... IP range blocked, will try to figure that out tomorrow:

Running with gitlab-runner 18.9.0 (07e534ba)
  on gitlab-runner-9b6c7c99b-mn9gr -WhWGubPe, system ID: r_QrCeXsApwiOA
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: gitlab-runner
Using Kubernetes executor with image node:20 ...
Using attach strategy to execute scripts...
Using effective pull policy of [] for container build
Using effective pull policy of [] for container helper
Using effective pull policy of [] for container init-permissions
Preparing environment
00:22
Using FF_USE_POD_ACTIVE_DEADLINE_SECONDS, the Pod activeDeadlineSeconds will be set to the job timeout: 1h0m0s...
Waiting for pod gitlab-runner/runner--whwgubpe-project-4248-concurrent-0-xbt9hgit to be running, status is Pending
Waiting for pod gitlab-runner/runner--whwgubpe-project-4248-concurrent-0-xbt9hgit to be running, status is Pending
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
Waiting for pod gitlab-runner/runner--whwgubpe-project-4248-concurrent-0-xbt9hgit to be running, status is Pending
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
Waiting for pod gitlab-runner/runner--whwgubpe-project-4248-concurrent-0-xbt9hgit to be running, status is Pending
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
Waiting for pod gitlab-runner/runner--whwgubpe-project-4248-concurrent-0-xbt9hgit to be running, status is Pending
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
Waiting for pod gitlab-runner/runner--whwgubpe-project-4248-concurrent-0-xbt9hgit to be running, status is Pending
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
Waiting for pod gitlab-runner/runner--whwgubpe-project-4248-concurrent-0-xbt9hgit to be running, status is Pending
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
	ContainersNotReady: "containers with unready status: [build helper istio-proxy]"
Running on runner--whwgubpe-project-4248-concurrent-0-xbt9hgit via gitlab-runner-9b6c7c99b-mn9gr...
Getting source from Git repository
00:01
Gitaly correlation ID: 01KKWFR93T0GGJNAC75QGMTE89
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/repos/test-platform/misc-scripts/browser-test-scanner/.git/
Created fresh repository.
Checking out ef9b655c as detached HEAD (ref is main)...
Skipping Git submodules setup
Executing "step_script" stage of the job script
02:06
$ node scan-repos.js
Scanning 44 repo(s)...
  FOUND https://gerrit.wikimedia.org/r/mediawiki/core (12 files, 23 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/AbuseFilter (4 files, 23 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/AchievementBadges (1 files, 1 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/AdvancedSearch (5 files, 10 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/CampaignEvents (4 files, 12 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/Cargo (2 files, 3 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/CentralNotice (1 files, 1 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/CirrusSearch (1 files, 3 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/Cite (5 files, 10 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/CommunityConfiguration (1 files, 5 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/CodeMirror (4 files, 25 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/CommunityRequests (1 files, 6 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/ContactPage (1 files, 1 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/ContentTranslation (2 files, 5 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/Echo (2 files, 4 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/ElectronPdfService (1 files, 1 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/EntitySchema (6 files, 17 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/examples (2 files, 3 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/FacetedCategory (1 files, 2 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/GlobalWatchlist (1 files, 1 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/GrowthExperiments (9 files, 15 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/GuidedTour (1 files, 1 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/Math (2 files, 5 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/Newsletter (1 files, 1 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/OpenIDConnect (2 files, 2 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/PageTriage (2 files, 5 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/ParserMigration (1 files, 2 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/Popups (1 files, 3 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/ProofreadPage (2 files, 5 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/RelatedArticles (1 files, 1 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/ReportIncident (1 files, 6 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/RevisionSlider (8 files, 21 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/Sanctions (2 files, 9 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/Score (1 files, 1 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/TemplateData (1 files, 3 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/TemplateWizard (1 files, 4 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/TwoColConflict (7 files, 33 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/VisualEditor (3 files, 34 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/Wikibase (12 files, 18 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/WikibaseLexeme (16 files, 17 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/WikibaseQualityConstraints (2 files, 3 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/extensions/WikiLambda (5 files, 15 tests)
  FOUND https://gerrit.wikimedia.org/r/mediawiki/skins/MinervaNeue (2 files, 3 tests)
  FOUND https://gerrit.wikimedia.org/r/wikibase/termbox (4 files, 25 tests)
Results written to /builds/repos/test-platform/misc-scripts/browser-test-scanner/results/
Summary: 44 with tests, 0 without
$ node generate-wikitext.js
Wikitext written to /builds/repos/test-platform/misc-scripts/browser-test-scanner/results/browser-tests.wiki
44 repos, 388 total tests
$ node publish-wikitext.js
Getting login token...
Logging in as [MASKED]...
Login successful.
Updating [MASKED]...
Edit failed: {
  "error": {
    "code": "blocked",
    "info": "'''Your IP address is in a range that has been [[m:Special:MyLanguage/Global blocks|blocked on all Wikimedia Foundation wikis]].''' The block was made by [[User:EPIC|‪EPIC‬]]. The reason given is ''[[m:Special:MyLanguage/NOP|Open proxy/Webhost]]: See the [[m:Special:MyLanguage/WM:OP/H|help page]] if you are affected''. * Start of block: 17:53, 29 December 2025 * Expiry of block: 01:49, 17 August 2030 Your current IP address is 161.35.183.23. The blocked range is 161.35.0.0/16. Please include all above details in any queries you make. If you believe you were blocked by mistake, you can find additional information and instructions in the [[m:Special:MyLanguage/Stewards/Wizard|Stewards Block Wizard]].",
    "blockinfo": {
      "blockid": 3397198,
      "blockedby": "EPIC",
      "blockedbyid": 17195878,
      "blockreason": "[[m:Special:MyLanguage/NOP|Open proxy/Webhost]]: See the [[m:Special:MyLanguage/WM:OP/H|help page]] if you are affected",
      "blockedtimestamp": "2025-12-29T17:53:45Z",
      "blockexpiry": "2030-08-17T01:49:23Z",
      "blocknocreate": "",
      "blockowntalk": "",
      "blockedtimestampformatted": "17:53, 29 December 2025",
      "blockexpiryformatted": "01:49, 17 August 2030",
      "blockexpiryrelative": "in 4 years"
    },
    "*": "See [MASKED] for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/&gt; for notice of API deprecations and breaking changes."
  },
  "servedby": "mw-api-ext.codfw.main-546866d48d-d5zrx"
}
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 1

https://www.mediawiki.org/wiki/Wikimedia_Quality_Services#Automated_tests_available

these docs ☝️ are now auto updated by a weekly toolforge cron job
https://toolsadmin.wikimedia.org/tools/id/browser-test-scanner/info/id/3314

any new repos that need to be scanned for automated tests can be added here
https://gitlab.wikimedia.org/repos/test-platform/misc-scripts/browser-test-scanner/-/blob/main/repos.txt

Keeping this ticket in "in verification" for a week to make sure the cron job runs successfully. But if the cron task works then this can be moved to Done.

This is what currently show on the top of the doc:

Last updated: 2026-03-17 21:18:09 UTC | 44 repos | 388 tests | Updated weekly by an automated cron job on Toolforge. To update which repos appear here, edit repos.txt.

Weekly cron job ran successfully today, moving this to Done

Last updated: 2026-03-23 00:01:50 UTC | 44 repos | 388 tests | Updated weekly by an automated cron job on Toolforge. To update which repos appear here, edit repos.txt.

vaughnwalters closed this task as Resolved.Fri, Apr 3, 8:35 PM

Create script to parse test case names from webdriverio testsClosed, ResolvedPublicActions

Description

Event Timeline

Create script to parse test case names from webdriverio tests
Closed, ResolvedPublic
Actions