Page MenuHomePhabricator

Flaky edit check tests in GrowthExperiments cypress test ReviseTone.cy.ts
Closed, ResolvedPublicBUG REPORT

Description

See https://integration.wikimedia.org/ci/job/quibble-with-GrowthExperiments-extensions-browser-tests-only-vendor-php83/18451/console
"Shows the Revise Tone Edit Check and tags edits" check seems to be flaky. This is what industry professionals (me) have identified as "an issue".


Derived Requirements

Ensure the Edit Check sidebar opens normally when an Edit Check is triggered, with the triggered check displayed and expanded. The triggered Edit Check must remain visible and interactive after the sidebar opens, and repeated initial sidebar openings must not leave the check hidden, collapsed, or unusable.

Test Case 1: Verify the Edit Check sidebar opens normally

  1. Open the test page in VisualEditor.
  2. Make an edit that triggers an Edit Check.
  3. Allow the Edit Check sidebar to open automatically.
  4. Verify the sidebar opens normally.
  5. Verify the triggered Edit Check is displayed and expanded.
  6. ✅❓❌⬜ AC1: The Edit Check sidebar opens normally and displays the triggered Edit Check.

Test Case 2: Verify the Edit Check is usable after the sidebar opens

  1. Trigger an Edit Check with the sidebar initially closed.
  2. Allow the sidebar to open.
  3. Verify the appropriate Edit Check is displayed.
  4. Select an available action on the check.
  5. Verify the action works normally.
  6. ✅❓❌⬜ AC2: The triggered Edit Check remains visible and interactive after the sidebar initially opens.

Test Case 3: Verify repeated initial sidebar opening

  1. Trigger an Edit Check with the sidebar initially closed.
  2. Verify the sidebar and check open normally.
  3. Close or restart the editing session.
  4. Repeat the same workflow several times.
  5. Verify the sidebar consistently opens with the appropriate Edit Check displayed.
  6. ✅❓❌⬜ AC3: Repeated initial Edit Check sidebar openings complete without the check becoming hidden, collapsed, or unusable.

QA Results - TestWiki

ACStatusDetails
1✅T439748#12409121
2✅T439748#12409121
3✅T439748#12409121

Event Timeline

Restricted Application added a subscriber: • Aklapper. · View Herald Transcript
Umherirrender subscribed.

Also affects other gated extensions - https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/1342744

Also affects "Closes the Editor when declining Edits and suggests a new task"
https://integration.wikimedia.org/ci/job/quibble-with-GrowthExperiments-extensions-browser-tests-only-vendor-php83/18596/console

     CypressError: Timed out retrying after 20050ms: `cy.click()` failed because this element is not visible:

`<a class="oo-ui-buttonElement-button" role="button" tabindex="0" rel="nofollow">...</a>`

This element `<a.oo-ui-buttonElement-button>` is not visible because its parent `<div.ve-ui-editCheckActionWidget-body>` has CSS property: `display: none`

Fix this problem, or use `{force: true}` to disable error checking.

https://on.cypress.io/element-cannot-be-interacted-with
      at runVisibilityCheck (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:153508:58)
      at Object.isStrictlyVisible (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:153522:10)
      at runAllChecks (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:114001:26)
      at retryActionability (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:114070:16)
      at tryCatcher (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:1780:23)
      at Promise.attempt.Promise.try (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:4288:29)
      at whenStable (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:152216:70)
      at <unknown> (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:152116:14)
      at tryCatcher (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:1780:23)
      at Promise._settlePromiseFromHandler (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:1492:31)
      at Promise._settlePromise (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:1549:18)
      at Promise._settlePromise0 (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:1594:10)
      at Promise._settlePromises (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:1674:18)
      at Promise._fulfill (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:1618:18)
      at <unknown> (http://127.0.0.1:9413/__cypress/runner/cypress_runner.js:5423:46)
Umherirrender renamed this task from Flaky edit check tests to Flaky edit check tests in GrowthExperiments cypress test ReviseTone.cy.ts.Oct 1 2026, 4:46 PM
Umherirrender added a project: Browser-Tests.

Copying the image here because it makes what's happening obvious:

image.png (1,280×720 px, 166 KB)

i.e. the check is being added, scrolled to, and then not expanded.

Change #1351387 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] EditCheck: Keep the focus of an action when the sidebar opens late

https://gerrit.wikimedia.org/r/1351387

Change #1351387 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] EditCheck: Keep the focus of an action when the sidebar opens late

https://gerrit.wikimedia.org/r/1351387

DMburugu subscribed.

Thanks for responding to this @DLynch.

@DLynch Validated the Edit Check sidebar opens normally when an Edit Check is triggered, with the triggered check displayed and expanded. The triggered Edit Check must remain visible and interactive after the sidebar opens, and repeated initial sidebar openings must not leave the check hidden, collapsed, or unusable. I will mark this as Resolved. Thanks for all your work!

Also for the video in AC3, just to be aware, it did pass but also wanted to let you know that it flickers very briefly of other suggestions until it goes back to normal.

The specific timing-dependent scenario where ensureActionIsShown occurs during the sidebar opening animation is not reliably reproducible through manual testing and is covered by the Selenium test.

Test Result - TestWiki

Status: ✅ PASS
Environment: TestWiki
OS: macOS Tahoe 26.6.2
Browser: Chrome 154
Device: MBA
Emulated Device: NA
Test Artifact(s): T439748
Test Steps

Test Case 1: Verify the Edit Check sidebar opens normally

  1. Open the test page in VisualEditor.
  2. Make an edit that triggers an Edit Check.
  3. Allow the Edit Check sidebar to open automatically.
  4. Verify the sidebar opens normally.
  5. Verify the triggered Edit Check is displayed and expanded.
  6. ✅ AC1: The Edit Check sidebar opens normally and displays the triggered Edit Check.

Test Case 2: Verify the Edit Check is usable after the sidebar opens

  1. Trigger an Edit Check with the sidebar initially closed.
  2. Allow the sidebar to open.
  3. Verify the appropriate Edit Check is displayed.
  4. Select an available action on the check.
  5. Verify the action works normally.
  6. ✅AC2: The triggered Edit Check remains visible and interactive after the sidebar initially opens.

Test Case 3: Verify repeated initial sidebar opening

  1. Trigger an Edit Check with the sidebar initially closed.
  2. Verify the sidebar and check open normally.
  3. Close or restart the editing session.
  4. Repeat the same workflow several times.
  5. Verify the sidebar consistently opens with the appropriate Edit Check displayed.
  6. ✅AC3: Repeated initial Edit Check sidebar openings complete without the check becoming hidden, collapsed, or unusable.

Also for the video in AC3, it did pass but also wanted to let you know that it flickers very briefly of other suggestions until it goes back to normal.

VideoFlicker
2026-10-07_14-21-37.png (1,560×1,244 px, 540 KB)
GMikesell-WMF edited projects, added: Verified; removed: Editing QA.
GMikesell-WMF updated the task description. (Show Details)