Page MenuHomePhabricator

Simplify Wikifunctions e2e test cases
Open, In Progress, MediumPublic

Description

Currently the e2e tests are (a) tightly coupled to Codex implementation and (b) written similar to unit tests (i.e. testing edge cases), we'd like to simplify and make the tests more robust by defining a "happy path" to test and leveraging simpler selectors for codex components.

Happy path:

  1. Create a function
  2. Add an code implementation
  3. Add a tester
  4. Add a composition implementation
  5. Run the function
  6. Enable the function

  • basic.js
  • function.js
  • connect.js
  • implementation.js
  • WIP tester.js
  • type.js — Probably just delete these, not reasonably tested.

Event Timeline

Why did this task get created in a legacy todo column? Should it go through the team's regular triage process? Sorry if I missed something!

Sorry, that was my bad. I wasn't aware that was a legacy column. I'll move it to triage. FWIW this will primarily be worked on by Vaughn with assistance from Zeljko and I.

Jdforrester-WMF moved this task from To triage to Backlog on the Abstract Wikipedia team board.

Change 989464 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/WikiLambda@master] e2e: Run only installation checks in the daily Jenkins job

https://gerrit.wikimedia.org/r/989464

Change 989464 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] e2e: Run only installation checks in the daily Jenkins job

https://gerrit.wikimedia.org/r/989464

Jdforrester-WMF changed the task status from Open to In Progress.Jan 11 2024, 9:05 PM
Jdforrester-WMF assigned this task to vaughnwalters.
Jdforrester-WMF moved this task from Backlog to In Progress on the Abstract Wikipedia team board.

I thought this didn't help, but looks like there were two passing test runs (in a row!). That didn't happen for at least the last 30 days, if not longer.

Screenshot 2024-01-12 at 11.08.44.png (214×504 px, 27 KB)

I still don't know why a lot of the time the job times out after 60 minutes (for example today), but at least there's some progress.

Change #1015458 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] Bug: T349836

https://gerrit.wikimedia.org/r/1015458

Change #1015458 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding some test IDs and fixing local broken tests for function.js

https://gerrit.wikimedia.org/r/1015458

Change #1015464 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] Bug: T349836

https://gerrit.wikimedia.org/r/1015464

Change #1015464 abandoned by Vaughn Walters:

[mediawiki/extensions/WikiLambda@master] Bug: T349836

Reason:

duplicate

https://gerrit.wikimedia.org/r/1015464

Change #1015458 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding some test IDs and fixing local broken tests for function.js

https://gerrit.wikimedia.org/r/1015458

Change #1017083 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding localization strings for more dynamic testing

https://gerrit.wikimedia.org/r/1017083

Change #1017083 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding localization strings for more dynamic testing

https://gerrit.wikimedia.org/r/1017083

Change #1017085 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding function.js back into selenium-test

https://gerrit.wikimedia.org/r/1017085

Change #1017085 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding function.js into selenium-daily

https://gerrit.wikimedia.org/r/1017085

Change #1025909 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Updating README and updating to dynamic selector

https://gerrit.wikimedia.org/r/1025909

Change #1025909 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Updating README and updating to dynamic selector

https://gerrit.wikimedia.org/r/1025909

Change #1028933 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Fixing and refactoring tester.js

https://gerrit.wikimedia.org/r/1028933

Change #1028933 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Fixing and refactoring tester.js

https://gerrit.wikimedia.org/r/1028933

Change #1030562 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding tester.js back into selenium daily tests

https://gerrit.wikimedia.org/r/1030562

Change #1031586 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-testid selectors into tester.js

https://gerrit.wikimedia.org/r/1031586

Change #1030562 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding tester.js back into selenium daily tests

https://gerrit.wikimedia.org/r/1030562

Change #1031586 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-testid selectors into tester.js

https://gerrit.wikimedia.org/r/1031586

Change #1031597 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-testid selectors into tester.js related files

https://gerrit.wikimedia.org/r/1031597

Change #1032739 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Editing tester.js related files to pass in DUCT

https://gerrit.wikimedia.org/r/1032739

Change #1031597 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-testid selectors into tester.js related files

https://gerrit.wikimedia.org/r/1031597