Page MenuHomePhabricator
Create Task
Maniphest T227266

Find out why 'old revisions do not have edit links' selenium test is flaky on CI
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

As advised in T227121:
Some CI builds have been failing due to this Wikibase selenium test (currently skipped on master)
https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Wikibase/+/refs/heads/master/repo/tests/selenium/specs/item.js#102

Examples for failed builds (seeme to fail on every second run or so):
https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-hhvm-docker/29044/console
https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-hhvm-docker/18210/console

The video seems to indicate not being able to click on the REVISION_DATE_LINK defined in repo/tests/selenium/wdio-wikibase/pageobjects/item.page.js

Details

ProjectBranchLines +/-Subject
mediawiki/extensions/Wikibasemaster+13 -7Re-Add "item old revisions do not have an edit link"
mediawiki/extensions/Wikibasemaster+2 -7Revert "Reenable test to check no edit links on old revision"
mediawiki/extensions/Wikibasemaster+1 -1Skip browser test checking edit links on old revisions
mediawiki/extensions/Wikibasemaster+1 -1Unskipping test with hopes of flakiness averted
mediawiki/extensions/Wikibasemaster+2 -1Skip flaky wikibase item browser test for now
mediawiki/extensions/Wikibasemaster+2 -1Fix flaky it block to assert no edit link on old revision
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 DuplicateGoalNoneT234012 Eliminate Wikidata Browser Tests Flakiness
 ResolvedPRODUCTION ERRORhooT227266 Find out why 'old revisions do not have edit links' selenium test is flaky on CI

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a project: User-Noarave. · View Herald TranscriptJul 4 2019, 10:39 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
noarave updated the task description. (Show Details)Jul 4 2019, 12:25 PM
gerritbot added a comment.Jul 4 2019, 12:48 PM

Change 520750 had a related patch set uploaded (by Noa wmde; owner: Noa wmde):
[mediawiki/extensions/Wikibase@master] Fix flaky it block to assert no edit link on old revision

https://gerrit.wikimedia.org/r/520750

gerritbot added a project: Patch-For-Review.Jul 4 2019, 12:48 PM
noarave moved this task from To Do (prioritised from top to bottom) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jul 4 2019, 1:14 PM
Ladsgroup moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jul 5 2019, 9:39 AM
gerritbot added a comment.Jul 5 2019, 9:57 AM

Change 520750 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Fix flaky it block to assert no edit link on old revision

https://gerrit.wikimedia.org/r/520750

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.13; 2019-07-09).Jul 5 2019, 10:00 AM
Maintenance_bot removed a project: Patch-For-Review.Jul 5 2019, 10:10 AM
• alaa_wmde added a subscriber: • alaa_wmde.Jul 5 2019, 1:43 PM

Giving it some time for some wikibase builds to pass to ensure this isn't that flaky anymore

Michael added a subscriber: Michael.Jul 23 2019, 8:18 AM
Michael triaged this task as High priority.Jul 23 2019, 8:20 AM
Michael moved this task from Test (Verification) to To Do (prioritised from top to bottom) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.

Still flaky in gate and submit: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/525037

Michael added a project: Wikimedia-production-error (ARCHIVED -- Shared Build Failure).Jul 23 2019, 8:21 AM
• alaa_wmde added a comment.EditedJul 23 2019, 1:19 PM

@noarave if you haven't started looking into this yet (and not planning soon) I suggest the following:

  • skip this test for now
  • look into the original issue and see if we can write a different (non-browser) test

what do you think?

(I'm thinking to try with this and see how we would approach those cases with tests that are not the usual browser tests we write usually, connected in my head to T226975)

noarave added a comment.Jul 24 2019, 7:49 AM

@alaa_wmde i'm fine with this, as a long standing solution seems to be less than obvious by now.

gerritbot added a comment.Jul 24 2019, 9:05 AM

Change 525234 had a related patch set uploaded (by Noa wmde; owner: Noa wmde):
[mediawiki/extensions/Wikibase@master] Skip flaky wikibase item browser test for now

https://gerrit.wikimedia.org/r/525234

gerritbot added a project: Patch-For-Review.Jul 24 2019, 9:05 AM
gerritbot added a comment.Jul 25 2019, 12:58 AM

Change 525234 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Skip flaky wikibase item browser test for now

https://gerrit.wikimedia.org/r/525234

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.16; 2019-07-30); removed MW-1.34-notes (1.34.0-wmf.13; 2019-07-09).Jul 25 2019, 1:00 AM
Maintenance_bot removed a project: Patch-For-Review.Jul 25 2019, 1:10 AM
Umherirrender added a project: ci-test-error.Aug 4 2019, 6:09 PM
Krinkle moved this task from Untriaged to Resolved on the Wikimedia-production-error (ARCHIVED -- Shared Build Failure) board.Aug 5 2019, 5:26 PM
Lucas_Werkmeister_WMDE added a subscriber: Lucas_Werkmeister_WMDE.Aug 7 2019, 12:07 PM
• alaa_wmde mentioned this in T227121: Browser test fails with "old revisions do not have an edit link" blocking merges in Minerva skin.Aug 13 2019, 9:45 AM
noarave added subscribers: Rosalie_WMDE, WMDE-leszek, Ladsgroup.Aug 19 2019, 8:47 AM

I'm really wondering what to do about this, since this test seemed to be failing relatively often on CI but for different reasons than it might be failing locally every once in a while (which it didn't always do but currently does). I think this needs some decision regarding whether it makes sense to test this in any other way (phpunit + Hamcrest) and is this scenario worth this sort of investment. would like opinions @alaa_wmde @WMDE-leszek @Ladsgroup @Rosalie_WMDE

Michael added a comment.Aug 19 2019, 10:01 AM

Chiming in here: At Wikidata Bridge we started using the "new" wdio5, instead of the current wdio4. Is it possible that upgrading might improve the situation?

There is a patch in the making to make wdio-mediawiki compatible with wdio 5: https://gerrit.wikimedia.org/r/528882

noarave removed noarave as the assignee of this task.Aug 27 2019, 8:34 AM
mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:06 PM
Ladsgroup moved this task from To Do (prioritised from top to bottom) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Sep 3 2019, 1:04 PM
Ladsgroup claimed this task.Sep 3 2019, 4:25 PM
Ladsgroup moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.

The patch should have fixed the issue hopefully. Feel free to bring it back if flaky tests keep happening

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptSep 3 2019, 4:25 PM
gerritbot added a comment.Sep 4 2019, 1:09 PM

Change 534440 had a related patch set uploaded (by Noa wmde; owner: Noa wmde):
[mediawiki/extensions/Wikibase@master] Unskipping test with hopes of flakiness averted

https://gerrit.wikimedia.org/r/534440

gerritbot added a project: Patch-For-Review.Sep 4 2019, 1:09 PM
gerritbot added a comment.Sep 4 2019, 9:53 PM

Change 534440 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Unskipping test with hopes of flakiness averted

https://gerrit.wikimedia.org/r/534440

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.22; 2019-09-10); removed MW-1.34-notes (1.34.0-wmf.16; 2019-07-30).Sep 4 2019, 10:00 PM
Maintenance_bot removed a project: Patch-For-Review.Sep 4 2019, 10:10 PM
gerritbot added a comment.Sep 11 2019, 3:47 PM

Change 535879 had a related patch set uploaded (by Jakob; owner: Jakob):
[mediawiki/extensions/Wikibase@master] Skip browser test checking edit links on old revisions

https://gerrit.wikimedia.org/r/535879

gerritbot added a project: Patch-For-Review.Sep 11 2019, 3:47 PM
Lucas_Werkmeister_WMDE mentioned this in T232363: Occasional Wikibase CI failures: Cannot read property 'set' of undefined (test: item old revisions do not have an edit link).Sep 11 2019, 3:48 PM
Jakob_WMDE moved this task from Test (Verification) to To Do (prioritised from top to bottom) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Sep 11 2019, 3:58 PM
Jakob_WMDE added a subscriber: Jakob_WMDE.

This is still flaky, it seems. I uploaded a patch to skip it again.

Probably worth looking into:

In T232363#5484098, @Lucas_Werkmeister_WMDE wrote:

I assume this is caused by the ItemPage.goToPreviousRevision() function, which as far as I can tell is only used by this one test, and looks extremely fragile. Here’s what it does:

  • click on the recent changes link in the sidebar
  • click on the “history” link of the last change in the wiki
  • click on the date link of the change marked as “after” (this class is assigned based on the two diff selection radio buttons – initially, “before” is the current revision and “after” its parent)

I’m not sure what part of this breaks, but since we already know the item ID, at least the first two steps seem unnecessary to me, and for the last step it might also be better to make an API call to get the second-latest revision.

• Pablo-WMDE added a subscriber: • Pablo-WMDE.Sep 11 2019, 4:31 PM

Also dropped something there: https://phabricator.wikimedia.org/T232363#5484262

gerritbot added a comment.Sep 11 2019, 6:51 PM

Change 535879 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Skip browser test checking edit links on old revisions

https://gerrit.wikimedia.org/r/535879

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.23; 2019-09-17); removed MW-1.34-notes (1.34.0-wmf.22; 2019-09-10).Sep 11 2019, 7:00 PM
Ladsgroup removed Ladsgroup as the assignee of this task.Sep 11 2019, 7:01 PM
Maintenance_bot removed a project: Patch-For-Review.Sep 11 2019, 7:11 PM
• alaa_wmde added a parent task: T234012: Eliminate Wikidata Browser Tests Flakiness.Sep 27 2019, 7:35 AM
• alaa_wmde edited projects, added Wikidata-Campsite; removed Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)).Oct 1 2019, 2:31 PM
Addshore moved this task from Incoming to Needs Work on the Wikidata-Campsite board.Jan 16 2020, 11:12 PM
Addshore added a subscriber: Addshore.

So, this is still skipped!

Addshore moved this task from Needs Work to Needs Tech Work on the Wikidata-Campsite board.Jan 27 2020, 3:46 PM
Krinkle edited projects, added ci-test-error (WMF-deployed Build Failure); removed ci-test-error, Wikimedia-production-error (ARCHIVED -- Shared Build Failure).Apr 18 2020, 10:53 PM
Krinkle removed a project: ci-test-error (WMF-deployed Build Failure).Apr 19 2020, 5:37 PM
Addshore lowered the priority of this task from High to Medium.May 5 2020, 11:55 AM
Lucas_Werkmeister_WMDE added a comment.EditedAug 12 2020, 3:37 PM

T232629: Support query parameters in Special:EntityPage was just resolved after being inactive for a few months; I no longer remember the context for that task, but I think it might have been related to this:

In T227266#5484155, @Jakob_WMDE wrote:

This is still flaky, it seems. I uploaded a patch to skip it again.

Probably worth looking into:

In T232363#5484098, @Lucas_Werkmeister_WMDE wrote:

I assume this is caused by the ItemPage.goToPreviousRevision() function, which as far as I can tell is only used by this one test, and looks extremely fragile. Here’s what it does:

  • click on the recent changes link in the sidebar
  • click on the “history” link of the last change in the wiki
  • click on the date link of the change marked as “after” (this class is assigned based on the two diff selection radio buttons – initially, “before” is the current revision and “after” its parent)

I’m not sure what part of this breaks, but since we already know the item ID, at least the first two steps seem unnecessary to me, and for the last step it might also be better to make an API call to get the second-latest revision.

It may be possible to replace ItemPage.goToPreviousRevision() with a more robust implementation now that Special:EntityPage/Q123?revision=456 Special:EntityPage/Q123?oldid=456 is supported.

Addshore added a project: wdwb-tech.Apr 14 2021, 9:38 AM
Addshore moved this task from Inbox to Research on the wdwb-tech board.Apr 14 2021, 4:30 PM
Addshore removed a project: Wikidata-Campsite.Apr 22 2021, 3:59 PM
Michael mentioned this in T281587: Old revisions of Items should not have edit buttons.Apr 30 2021, 4:07 PM
tstarling mentioned this in T282674: AssertionError [ERR_ASSERTION] in "item old revisions do not have an edit link".May 14 2021, 12:55 AM
tstarling merged a task: T282674: AssertionError [ERR_ASSERTION] in "item old revisions do not have an edit link".
tstarling added subscribers: Nikerabbit, tstarling.
tstarling raised the priority of this task from Medium to High.May 14 2021, 1:03 AM

Looks like this test was reenabled a couple of days ago, by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/683914 . Since then it's been a constant nuisance, failing on maybe 20% of core patches. I don't understand why the test was re-enabled prior to proposed fix https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/690044/ being merged.

gerritbot added a comment.May 14 2021, 1:17 AM

Change 690806 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/Wikibase@master] Revert "Reenable test to check no edit links on old revision"

https://gerrit.wikimedia.org/r/690806

gerritbot added a project: Patch-For-Review.May 14 2021, 1:17 AM
gerritbot added a comment.May 14 2021, 2:43 AM

Change 690806 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Revert "Reenable test to check no edit links on old revision"

https://gerrit.wikimedia.org/r/690806

tstarling lowered the priority of this task from High to Medium.May 14 2021, 3:26 AM
gerritbot added a comment.May 14 2021, 1:34 PM

Change 690044 had a related patch set uploaded (by Hoo man; author: Hoo man):

[mediawiki/extensions/Wikibase@master] Re-Add "item old revisions do not have an edit link"

https://gerrit.wikimedia.org/r/690044

hoo claimed this task.May 14 2021, 1:37 PM
hoo added a project: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)).
hoo moved this task from To Do (prioritised from top to bottom) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
Maintenance_bot moved this task from incoming to in progress on the Wikidata board.May 14 2021, 2:15 PM
Maintenance_bot moved this task from Research to Active on the wdwb-tech board.
Maintenance_bot moved this task from Incoming to In progress on the User-Ladsgroup board.
hoo added a comment.May 17 2021, 2:08 PM

https://github.com/wmde/wdio-wikibase/pull/33

gerritbot added a comment.May 20 2021, 7:13 AM

Change 690044 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Re-Add "item old revisions do not have an edit link"

https://gerrit.wikimedia.org/r/690044

Jakob_WMDE moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.May 20 2021, 9:17 AM
Addshore closed this task as Resolved.May 28 2021, 9:28 AM
Jakob_WMDE mentioned this in T289853: Wikibase Selenium tests fail when targeting wiki created by MediaWiki-Docker.Aug 31 2021, 8:53 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL