Page MenuHomePhabricator

ReviseTone.cy.ts tests are too flaky
Closed, ResolvedPublicBUG REPORT

Description

The test in cypress/e2e/ReviseTone.cy.ts#116, which was added in rEGRE433531a, has been failing for a bunch of unrelated patches.

From https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1197009:

ReviseTone.cy.ts
00:39:48   Running:  ReviseTone.cy.ts                                                                (6 of 8)
00:39:51 
00:39:51 
00:39:51   Revise Tone
00:39:54     On desktop
00:40:01       βœ“ Closes the Editor when declining Edits and suggests a new task (10188ms)
00:40:11       βœ“ Shows the Revise Tone Edit Check and tags edits (9810ms)
00:40:11     On mobile
00:40:11       - Closes the Editor when declining Edits and suggests a new task
00:40:42       1) Shows the Revise Tone Edit Check and tags edits
00:40:42         cy:command βœ”  visit	/index.php -> 301: http://127.0.0.1:9413/index.php?title=Main_Page
00:40:42         cy:command βœ”  window	
00:40:42         cy:command βœ”  assert	expected **<window>** to have property **mw**
00:40:42         cy:command βœ”  window	
00:40:42         cy:command βœ”  its	.mw.Api
00:40:42         cy:command βœ”  assert	expected **[Function]** to exist
00:40:42         cy:command βœ”  window	
00:40:42             cy:xhr ➟  POST http://127.0.0.1:9413/api.php
00:40:42                       Status: 200
00:40:42             cy:xhr ➟  POST http://127.0.0.1:9413/api.php
00:40:42                       Status: 200
00:40:42         cy:command βœ”  visit	index.php?title=Special:Homepage/suggested-edits&mobileaction=toggle_view_mobile -> 301: http://127.0.0.1:9413/index.php?title=Special:Homepage/suggested-edits
00:40:42         cy:command βœ”  get	.se-card-title
00:40:42         cy:command βœ”  assert	expected **<h3.se-card-title>** to have text **Kristallsee**
00:40:42                       Actual: 	"Kristallsee"
00:40:42                       Expected: 	"Kristallsee"
00:40:42         cy:command βœ”  get	a.se-card-content
00:40:42         cy:command βœ”  assert	expected **<a.se-card-content>** not to have attribute **href** with the value **#**
00:40:42         cy:command βœ”  get	.suggested-edits-next .oo-ui-buttonElement-button
00:40:42         cy:command βœ”  click	
00:40:42             cy:xhr ❖  GET http://127.0.0.1:9413/api/rest_v1/page/summary/Kristallsee
00:40:42                       Status: 404
00:40:42             cy:xhr ❖  GET https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/127.0.0.1/all-access/user/Kristallsee/daily/20250818/20251017
00:40:42                       Status: 404
00:40:42          cons:warn ❖  This page is using the deprecated ResourceLoader module "moment".
00:40:42                       [1.44] Use mediawiki.DateFormatter or native Intl function instead. See https://phabricator.wikimedia.org/T146798
00:40:42             cy:xhr ❖  GET http://127.0.0.1:9413/api/rest_v1/page/summary/Ma'amoul
00:40:42                       Status: 404
00:40:42             cy:xhr ❖  GET https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/127.0.0.1/all-access/user/Ma'amoul/daily/20250818/20251017
00:40:42                       Status: 404
00:40:42             cy:xhr ❖  GET http://127.0.0.1:9413/api/rest_v1/page/summary/Eldfjall
00:40:42                       Status: 404
00:40:42             cy:xhr ❖  GET https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/127.0.0.1/all-access/user/Eldfjall/daily/20250818/20251017
00:40:42                       Status: 404
00:40:42         cy:command βœ”  get	.se-card-title
00:40:42         cy:command βœ”  assert	expected **<h3.se-card-title>** to have text **Eldfjall**
00:40:42                       Actual: 	"Eldfjall"
00:40:42                       Expected: 	"Eldfjall"
00:40:42         cy:command βœ”  get	a.se-card-content
00:40:42         cy:command βœ”  assert	expected **<a.se-card-content>** not to have attribute **href** with the value **#**
00:40:42         cy:command βœ”  get	a.se-card-content
00:40:42         cy:command βœ”  click	
00:40:42         cy:command βœ”  page load	--page loaded--
00:40:42         cy:command βœ”  new url	http://127.0.0.1:9413/index.php?title=Eldfjall&getasktype=revise-tone&gesuggestededit=1&geclickid=2ukhqop6aicq9uvb712agodkid570vfu&genewcomertasktoken=g49u61jduc29tf74fob2h8gfiikce09l&veaction=edit&section=all
00:40:42         cy:command βœ”  get	.ve-ui-editCheckActionWidget
00:40:42         cy:command βœ”  new url	http://127.0.0.1:9413/index.php?title=Eldfjall&getasktype=revise-tone&gesuggestededit=1&genewcomertasktoken=g49u61jduc29tf74fob2h8gfiikce09l&veaction=edit&section=all
00:40:42         cy:command βœ”  new url	http://127.0.0.1:9413/index.php?title=Eldfjall&gesuggestededit=1&genewcomertasktoken=g49u61jduc29tf74fob2h8gfiikce09l&veaction=edit&section=all
00:40:42         cy:command βœ”  new url	http://127.0.0.1:9413/index.php?title=Eldfjall&gesuggestededit=1&veaction=edit&section=all
00:40:42             cy:xhr ➟  POST http://127.0.0.1:9413/api.php
00:40:42                       Status: 200
00:40:42         cy:command βœ”  new url	http://127.0.0.1:9413/index.php?title=Eldfjall&gesuggestededit=1
00:40:42         cy:command βœ”  new url	http://127.0.0.1:9413/index.php?title=Eldfjall&gesuggestededit=1#/editor/all
00:40:42          cons:warn ❖  This page is using the deprecated ResourceLoader module "moment".
00:40:42                       [1.44] Use mediawiki.DateFormatter or native Intl function instead. See https://phabricator.wikimedia.org/T146798
00:40:42             cy:xhr ➟  GET http://127.0.0.1:9413/api.php?action=visualeditor&format=json&paction=parse&page=Eldfjall&uselang=en&editintro=&preload=&preloadparams=&formatversion=2
00:40:42                       Status: 200
00:40:42          cons:info ✱  You are running a development build of Vue.
00:40:42                       Make sure to use the production build (*.prod.js) when deploying for production.
00:40:42             cy:xhr ➟  POST http://127.0.0.1:9413/api.php
00:40:42                       Status: 200
00:40:42         cy:command βœ”  contains	a, Revise
00:40:42         cy:command βœ”  click	
00:40:42         cy:command βœ”  get	#Tourism_and_Hiking + p
00:40:42         cy:command βœ”  type	{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}{backspace}Eldfjall attracts approximately 25,000 visitors annually, primarily during the summer months from June to September.
00:40:42         cy:command βœ”  get	.ve-ui-toolbar-saveButton
00:40:42         cy:command βœ”  assert	expected **<a.oo-ui-tool-link.ve-ui-toolbar-saveButton>** to be **visible**
00:40:42         cy:command βœ”  click	
00:40:42         cy:command ✘  get	.ve-ui-mwSaveDialog .oo-ui-processDialog-actions-primary
00:40:42         cy:command ✘  assert	expected **.ve-ui-mwSaveDialog .oo-ui-processDialog-actions-primary** to be **visible**
00:40:42                       Actual: 	".ve-ui-mwSaveDialog .oo-ui-processDialog-actions-primary"
00:40:42                       Expected: 	"visible"
00:40:42 
00:40:42 
00:40:42 
00:40:42 
00:40:42   2 passing (51s)
00:40:42   1 pending
00:40:42   1 failing
00:40:42 
00:40:42   1) Revise Tone
00:40:42        On mobile
00:40:42          Shows the Revise Tone Edit Check and tags edits:
00:40:42      AssertionError: Timed out retrying after 20000ms: Expected to find element: `.ve-ui-mwSaveDialog .oo-ui-processDialog-actions-primary`, but never found it.
00:40:42       at Context.eval (webpack://GrowthExperiments/./cypress/e2e/ReviseTone.cy.ts:116:72)
00:40:42 
00:40:42 
00:40:42 
00:40:43 
00:40:43   (Results)
00:40:43 
00:40:43   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
00:40:43   β”‚ Tests:        4                                                                                β”‚
00:40:43   β”‚ Passing:      2                                                                                β”‚
00:40:43   β”‚ Failing:      1                                                                                β”‚
00:40:43   β”‚ Pending:      1                                                                                β”‚
00:40:43   β”‚ Skipped:      0                                                                                β”‚
00:40:43   β”‚ Screenshots:  1                                                                                β”‚
00:40:43   β”‚ Video:        true                                                                             β”‚
00:40:43   β”‚ Duration:     51 seconds                                                                       β”‚
00:40:43   β”‚ Spec Ran:     ReviseTone.cy.ts                                                                 β”‚
00:40:43   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Event Timeline

Restricted Application added a subscriber: Aklapper. Β· View Herald Transcript
Dragoniez triaged this task as Unbreak Now! priority.Oct 20 2025, 12:31 AM
Dragoniez lowered the priority of this task from Unbreak Now! to Needs Triage.Oct 20 2025, 12:35 AM

Change #1197223 had a related patch set uploaded (by Michael Große; author: Michael Große):

[mediawiki/extensions/GrowthExperiments@master] tests(Cypress): Skip second Tevise Tone mobile browser test

https://gerrit.wikimedia.org/r/1197223

Change #1197223 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] tests(Cypress): Skip second Revise Tone mobile browser test

https://gerrit.wikimedia.org/r/1197223

Dragoniez claimed this task.

Closing this task as resolved for now in favour of T407152 (sorry I wasn’t aware of the task addressing similar things).

@Michael Thanks for the patch!