Citoid continuous integration broken
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	Novem_Linguae
	May 31 2024, 5:14 PM

Description

https://gerrit.wikimedia.org/r/q/project:mediawiki/extensions/Citoid

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php74-selenium/14592/console

09:02:04   Running:  ve-cite/reuseRefs.cy.js                                                         (3 of 5)
09:02:07 
09:02:07 
09:02:07   Re-using refs in Visual Editor
09:02:17     (Attempt 1 of 3) should display re-used reference in article with correct footnote number and notification in context dialog
09:02:25     (Attempt 2 of 3) should display re-used reference in article with correct footnote number and notification in context dialog
09:02:34     1) should display re-used reference in article with correct footnote number and notification in context dialog
09:02:43     (Attempt 1 of 3) should display correct ref content and name attribute for re-used ref with existing name attribute
09:02:51     (Attempt 2 of 3) should display correct ref content and name attribute for re-used ref with existing name attribute
09:02:59     2) should display correct ref content and name attribute for re-used ref with existing name attribute
09:02:59 
09:02:59 
09:02:59   0 passing (52s)
09:02:59   2 failing
09:02:59 
09:02:59   1) Re-using refs in Visual Editor
09:02:59        should display re-used reference in article with correct footnote number and notification in context dialog:
09:02:59      AssertionError: Timed out retrying after 5000ms: Expected to find element: `.ve-ui-toolbar-group-citoid`, but never found it.
09:02:59       at Object.openVECiteoidReuseDialog (webpack:///./e2e/utils/functions.helper.js:136:4)
09:02:59       at Context.eval (webpack:///./e2e/tests/ve-cite/reuseRefs.cy.js:48:11)
09:02:59 
09:02:59   2) Re-using refs in Visual Editor
09:02:59        should display correct ref content and name attribute for re-used ref with existing name attribute:
09:02:59      AssertionError: Timed out retrying after 5000ms: Expected to find element: `.ve-ui-toolbar-group-citoid`, but never found it.
09:02:59       at Object.openVECiteoidReuseDialog (webpack:///./e2e/utils/functions.helper.js:136:4)
09:02:59       at Context.eval (webpack:///./e2e/tests/ve-cite/reuseRefs.cy.js:89:11)
09:02:59 
09:02:59 
09:02:59 
09:03:00 
09:03:00   (Results)
09:03:00 
09:03:00   ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
09:03:00   │ Tests:        2                                                                                │
09:03:00   │ Passing:      0                                                                                │
09:03:00   │ Failing:      2                                                                                │
09:03:00   │ Pending:      0                                                                                │
09:03:00   │ Skipped:      0                                                                                │
09:03:00   │ Screenshots:  6                                                                                │
09:03:00   │ Video:        true                                                                             │
09:03:00   │ Duration:     52 seconds                                                                       │
09:03:00   │ Spec Ran:     ve-cite/reuseRefs.cy.js                                                          │
09:03:00   └────────────────────────────────────────────────────────────────────────────────────────────────┘
09:03:00 
09:03:00 
09:03:00   (Screenshots)
09:03:00 
09:03:00   -  /workspace/log/Cite/ve-cite/reuseRefs.cy.js/Re-using refs in Visual Editor -- sh     (1280x720)
09:03:00      ould display re-used reference in article with correct footnote number and notif               
09:03:00      ication in context dialog (failed).png                                                         
09:03:00   -  /workspace/log/Cite/ve-cite/reuseRefs.cy.js/Re-using refs in Visual Editor -- sh     (1280x720)
09:03:00      ould display re-used reference in article with correct footnote number and notif               
09:03:00      ication in context dialog (failed) (attempt 2).png                                             
09:03:00   -  /workspace/log/Cite/ve-cite/reuseRefs.cy.js/Re-using refs in Visual Editor -- sh     (1280x720)
09:03:00      ould display re-used reference in article with correct footnote number and notif               
09:03:00      ication in context dialog (failed) (attempt 3).png                                             
09:03:00   -  /workspace/log/Cite/ve-cite/reuseRefs.cy.js/Re-using refs in Visual Editor -- sh     (1280x720)
09:03:00      ould display correct ref content and name attribute for re-used ref with existin               
09:03:00      g name attribute (failed).png                                                                  
09:03:00   -  /workspace/log/Cite/ve-cite/reuseRefs.cy.js/Re-using refs in Visual Editor -- sh     (1280x720)
09:03:00      ould display correct ref content and name attribute for re-used ref with existin               
09:03:00      g name attribute (failed) (attempt 2).png                                                      
09:03:00   -  /workspace/log/Cite/ve-cite/reuseRefs.cy.js/Re-using refs in Visual Editor -- sh     (1280x720)
09:03:00      ould display correct ref content and name attribute for re-used ref with existin               
09:03:00      g name attribute (failed) (attempt 3).png                                                      
09:03:00 
09:03:00 
09:03:00   (Video)
09:03:00 
09:03:00   -  Video output: /workspace/log/Cite/ve-cite/reuseRefs.cy.js.mp4

Details

Subject	Repo	Branch	Lines +/-
[Cypress] Fix VE save	mediawiki/extensions/Cite	master	+12 -19
[Cypress] Always provide ref templates to fix Citoid	mediawiki/extensions/Cite	master	+19 -7
Revert "[Cypress] Wait for VE modules to load"	mediawiki/extensions/Cite	master	+19 -41
[WIP] tests: mark flaky browser tests as skipped	mediawiki/extensions/Cite	master	+2 -2
[Cypress] Disable CI Citoid flaky test	mediawiki/extensions/Cite	master	+21 -0

Customize query in gerrit

Related Objects

Mentioned In: T377666: Flaky ve-cite cypress tests
Mentioned Here: T377666: Flaky ve-cite cypress tests
T360026: Skip and then fix flapping reuseRefs tests

Event Timeline

Novem_Linguae created this task.May 31 2024, 5:14 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 31 2024, 5:14 PM

Novem_Linguae added a project: Citoid.May 31 2024, 5:19 PM

Those are the Cite tests, yet again being flaky, not Citoid.

Last time it was reported under T360026: Skip and then fix flapping reuseRefs tests

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1037555 - This patch looks suspicious. Touches the file with the failing tests, and the timestamp is around the time that Citoid started failing all tests.

Do we want to revert that patch, or mark the 2 tests as skipped, or something else?

Change #1037761 had a related patch set uploaded (by Novem Linguae; author: Novem Linguae):

[mediawiki/extensions/Cite@master] Revert "[Cypress] Wait for VE modules to load"

https://gerrit.wikimedia.org/r/1037761

gerritbot added a project: Patch-For-Review.Jun 1 2024, 11:10 AM

• OKJ04 added a subtask: T366433: CentralAuth tests broken unless you run them inside Quibble.Jun 3 2024, 1:39 AM

JJMC89 removed a subtask: T366433: CentralAuth tests broken unless you run them inside Quibble.Jun 3 2024, 1:44 AM

Please feel free to mark tests with .skip in case they are seriously blocking you. We will look into this for sure but it might take a few days.

Change #1037758 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Novem Linguae):

[mediawiki/extensions/Cite@master] [WIP] tests: mark flaky browser tests as skipped

https://gerrit.wikimedia.org/r/1037758

Change #1038244 had a related patch set uploaded (by WMDE-Fisch; author: WMDE-Fisch):

[mediawiki/extensions/Cite@master] [Cypress] Disable CI Citoid flaky test

https://gerrit.wikimedia.org/r/1038244

Just leaving a note here how weird the tests fail. It seems that only for the reuseRefs.cy.js file Citoid is not really loaded when executing the tests.

You can see that on the screenshot where the icon in the VE top menu is not the Citoid icon but the default Cite one. And that's even the case, when we wait a bit there for scripts to load.

Re-using refs in Visual Editor -- should display correct ref content and name attribute for re-used ref with existing name attribute (failed).png (720×1 px, 133 KB)

Change #1038244 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] [Cypress] Disable CI Citoid flaky test

https://gerrit.wikimedia.org/r/1038244

ReleaseTaggerBot added a project: MW-1.43-notes (1.43.0-wmf.8; 2024-06-04).Jun 3 2024, 11:00 AM

CI seems to be working again. See https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Citoid/+/1020322

Change #1037758 abandoned by WMDE-Fisch:

[mediawiki/extensions/Cite@master] [WIP] tests: mark flaky browser tests as skipped

Reason:

Different approach merged. See Iee9e4dd04e08fd3ddfa77409d037ec80f84ead3e

https://gerrit.wikimedia.org/r/1037758

Change #1037761 abandoned by Thiemo Kreuz (WMDE):

[mediawiki/extensions/Cite@master] Revert "[Cypress] Wait for VE modules to load"

Reason:

We disabled the failing tests for now, see Iee9e4dd. I believe this isn't needed any more.

https://gerrit.wikimedia.org/r/1037761

Maintenance_bot removed a project: Patch-For-Review.Jun 3 2024, 12:30 PM

In T366383#9854479, @WMDE-Fisch wrote:

CI seems to be working again. See https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Citoid/+/1020322

Thanks! There's still a bunch of patches that are failing CI on that repo, but I think it's a big patch chain that has the base patch failing stylelint, so is unrelated to this ticket.

thiemowmde moved this task from Sprint Backlog to Done on the WMDE-TechWish-Sprint-2024-05-29 board.Jun 4 2024, 7:58 AM

Umherirrender unsubscribed.Jun 16 2024, 11:26 AM

Change #1050039 had a related patch set uploaded (by WMDE-Fisch; author: WMDE-Fisch):

[mediawiki/extensions/Cite@master] [Cypress] Fix reuse ref tests for Citoid

https://gerrit.wikimedia.org/r/1050039

gerritbot added a project: Patch-For-Review.Jun 26 2024, 8:19 PM

Change #1050295 had a related patch set uploaded (by WMDE-Fisch; author: WMDE-Fisch):

[mediawiki/extensions/Cite@master] [Cypress] Fix VE save

https://gerrit.wikimedia.org/r/1050295

Change #1050295 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] [Cypress] Fix VE save

https://gerrit.wikimedia.org/r/1050295

Change #1050039 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] [Cypress] Always provide ref templates to fix Citoid

https://gerrit.wikimedia.org/r/1050039

Maintenance_bot removed a project: Patch-For-Review.Jun 28 2024, 8:30 AM

ReleaseTaggerBot edited projects, added MW-1.43-notes (1.43.0-wmf.12; 2024-07-02); removed MW-1.43-notes (1.43.0-wmf.8; 2024-06-04).Jun 28 2024, 9:00 AM

Izno moved this task from Unsorted backlog to Defect backlog on the Cite board.Aug 22 2024, 8:20 PM

These tests failure seem related, e.g https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74/34831/console

18:27:38   Running:  ve-cite/reuseRefs.cy.js                                                         (3 of 6)
18:27:41 
18:27:41 
18:27:41   Re-using refs in Visual Editor
18:27:45     1) "before all" hook for "should display re-used reference in article with correct footnote number and notification in context dialog"
18:27:45 
18:27:45 
18:27:45   0 passing (4s)
18:27:45   1 failing
18:27:45 
18:27:45   1) Re-using refs in Visual Editor
18:27:45        "before all" hook for "should display re-used reference in article with correct footnote number and notification in context dialog":
18:27:45      Error: http
18:27:45 
18:27:45 Because this error occurred during a `before all` hook we are skipping the remaining tests in the current suite: `Re-using refs in Visual Editor`
18:27:45 
18:27:45 Although you have test retries enabled, we do not retry tests when `before all` or `after all` hooks fail

CI for citoid in general is running now, so closing this task.

But it seems still flaky, created T377666 for tracking.

	F54898987: Re-using refs in Visual Editor -- should display correct ref content and name attribute for re-used ref with existing name attribute (failed).png
	Jun 3 2024, 10:08 AM

Citoid continuous integration brokenClosed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

Citoid continuous integration broken
Closed, ResolvedPublic
Actions