Page MenuHomePhabricator
Create Task
Maniphest T280491

Wikibase legacy ruby browser tests need to be updated to work on buster/ruby2.5
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

When run in the buster and ruby 2.5 environment, Wikibase's legacy ruby selenium tests fail to run:

08:25:16 undefined method `col_span' for class `Watir::TableCell' (NameError)
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/elements/table_cell.rb:3:in `alias_method'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/elements/table_cell.rb:3:in `<class:TableCell>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/elements/table_cell.rb:2:in `<module:Watir>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/elements/table_cell.rb:1:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver.rb:119:in `require'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver.rb:119:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/mediawiki_selenium-1.7.1/lib/mediawiki_selenium/cucumber/env.rb:3:in `require'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/mediawiki_selenium-1.7.1/lib/mediawiki_selenium/cucumber/env.rb:3:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/mediawiki_selenium-1.7.1/lib/mediawiki_selenium/cucumber.rb:3:in `require'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/mediawiki_selenium-1.7.1/lib/mediawiki_selenium/cucumber.rb:3:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/features/support/env.rb:9:in `require'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/features/support/env.rb:9:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_language.rb:95:in `load'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_language.rb:95:in `load_code_file'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:180:in `load_file'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:83:in `block in load_files!'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:82:in `each'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:82:in `load_files!'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:184:in `load_step_definitions'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:42:in `run!'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/cli/main.rb:47:in `execute!'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/bin/cucumber:13:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/bin/cucumber:23:in `load'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/bin/cucumber:23:in `<main>'

see results of the test job run on a dummy commit in: https://integration.wikimedia.org/ci/job/legacy-quibble-rubyselenium-docker/2

There are also likely issues with the "rake" job (essentially "linting" ruby test code) but it does not seem to be currently run with "check experimental" spell on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/688772. I suspect might be due to rake jobs to run as part of gate-and-submit job only?

Acceptance Criteria:🏕️🌟

  • Wikibase legacy ruby browser tests need to be updated to work on buster/ruby2.5
Notes

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

ItamarWMDE triaged this task as Unbreak Now! priority.Apr 19 2021, 9:14 AM
ItamarWMDE created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 19 2021, 9:14 AM
Jakob_WMDE claimed this task.Apr 19 2021, 9:23 AM
ItamarWMDE added a project: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)).Apr 19 2021, 9:23 AM
Jakob_WMDE moved this task from To Do (prioritised from top to bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Apr 19 2021, 9:23 AM
gerritbot added a comment.Apr 19 2021, 9:40 AM

Change 680996 had a related patch set uploaded (by Jakob; author: Jakob):

[mediawiki/extensions/Wikibase@master] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/680996

gerritbot added a project: Patch-For-Review.Apr 19 2021, 9:40 AM
Maintenance_bot moved this task from incoming to in progress on the Wikidata board.Apr 19 2021, 10:15 AM
Addshore moved this task from Inbox to Active on the [DEPRECATED] wdwb-tech board.Apr 19 2021, 3:11 PM
Stashbot added a comment.Apr 19 2021, 3:24 PM

Mentioned in SAL (#wikimedia-releng) [2021-04-19T15:24:01Z] <James_F> Re-pushing mwselenium-quibble-docker back to master for T280491

Jdforrester-WMF lowered the priority of this task from Unbreak Now! to Medium.Apr 19 2021, 3:31 PM
Jdforrester-WMF subscribed.

Sorry, I pushed the job over to buster, found the issue, and thought I had rolled back, but apparently I failed at the final step.

I've rolled back the job use the stretch image, as can be seen in https://integration.wikimedia.org/ci/job/mwselenium-quibble-docker/35906/console

Jdforrester-WMF added a parent task: T278203: Migrate all CI jobs from stretch to buster or later and drop stretch testing support.Apr 19 2021, 3:32 PM
WMDE-leszek added a comment.Apr 19 2021, 5:27 PM

Thanks for rolling back @Jdforrester-WMF.

We have been looking into making Wikibase legacy ruby browser test to successfully roll on debian buster. It occurred that with the OS update some Ruby dependencies got updates as well. @Jakob_WMDE managed to sort most of it out. The only thing that would still kind of block us from moving this CI job to buster is the issue with video recordings. Namely, tests seemed to be failing due to lack of libav configured on CI container (despite that seemeingly ffmpeg is meant to be used instead).

See e.g. a failures on https://integration.wikimedia.org/ci/job/mwselenium-quibble-docker/35874/console

11:48:39   Background:                                    # features/header.feature:11
11:48:39     Given I am on an item page                   # features/step_definitions/entity_steps.rb:19
11:48:39       avconv not found on your system. Install it or change video recorder provider (Headless::Exception)
11:48:39       features/header.feature:12:in `Given I am on an item page'

@Jakob_WMDE managed to work this around by temporarily skipping the recording off (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/680996/4..5/tests/browser/features/support/env.rb) but probably that is not the preferred long-term approach.

After getting mwselenium-quibble-docker sorted (except the libav issue), we've noticed that mwgate-rake-docker turned red. We believe it is because that CI job still runs using stretch images, hence the ruby et al are in incompatible versions. Getting this job migrated to buster should held. Would it be possible to set up a temporary not voting buster-based job so we could ensure this?

Finally: Yes, I am aware that ruby selenium infrastructure is outdated and ultimately WMDE will stop maintaining those tests. It seems that getting the CI infrastructure updated could be achieved without the nontrivial effort of migrating the whole test suite to node.

gerritbot added a comment.Apr 19 2021, 10:22 PM

Change 681207 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Provide legacy-quibble-rubyselenium-docker

https://gerrit.wikimedia.org/r/681207

gerritbot added a comment.Apr 19 2021, 10:22 PM

Change 681208 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Zuul: Add legacy-quibble-rubyselenium-docker as experimental

https://gerrit.wikimedia.org/r/681208

gerritbot added a comment.Apr 19 2021, 10:37 PM

Change 681207 merged by jenkins-bot:

[integration/config@master] jjb: Provide legacy-quibble-rubyselenium-docker

https://gerrit.wikimedia.org/r/681207

gerritbot added a comment.Apr 19 2021, 11:03 PM

Change 681208 merged by jenkins-bot:

[integration/config@master] Zuul: Add legacy-quibble-rubyselenium-docker as experimental

https://gerrit.wikimedia.org/r/681208

Stashbot added a comment.Apr 19 2021, 11:04 PM

Mentioned in SAL (#wikimedia-releng) [2021-04-19T23:04:38Z] <James_F> Zuul: Add legacy-quibble-rubyselenium-docker as experimental T280491

Jakob_WMDE removed Jakob_WMDE as the assignee of this task.Apr 20 2021, 7:54 AM
Jakob_WMDE moved this task from Doing to Stalled/Waiting on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.

Unassigning myself for now as the immediate problem here was fixed. Plans for next steps and potentially flipping those two jobs two buster are in the making.

gerritbot added a comment.Apr 21 2021, 10:26 PM

Change 681796 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] zuul: Add mwgate-rake-ruby2.5-docker experimentally to mwgate-rake

https://gerrit.wikimedia.org/r/681796

gerritbot added a comment.Apr 21 2021, 10:55 PM

Change 681796 merged by jenkins-bot:

[integration/config@master] zuul: Add mwgate-ruby2.5-rake-docker experimentally to mwgate-rake

https://gerrit.wikimedia.org/r/681796

Jdforrester-WMF added a comment.Apr 21 2021, 10:58 PM
In T280491#7015502, @WMDE-leszek wrote:

@Jakob_WMDE managed to work this around by temporarily skipping the recording off (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/680996/4..5/tests/browser/features/support/env.rb) but probably that is not the preferred long-term approach.

Agreed!

After getting mwselenium-quibble-docker sorted (except the libav issue), we've noticed that mwgate-rake-docker turned red. We believe it is because that CI job still runs using stretch images, hence the ruby et al are in incompatible versions. Getting this job migrated to buster should held. Would it be possible to set up a temporary not voting buster-based job so we could ensure this?

I've added legacy-quibble-rubyselenium-docker and mwgate-ruby2.5-rake-docker as experimental jobs to the Wikibase repo (and a couple of others). Once your team are happy with a patch that passes under these jobs (and so probably fails under the old ones), ping me on here and I can switch Wikibase over to the new jobs and merge the patch.

gerritbot added a comment.Apr 22 2021, 1:32 AM

Change 681853 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] [DNM] Zuul: Switch mwgate-rake and mediawiki-selenium to Ruby 2.5/buster

https://gerrit.wikimedia.org/r/681853

Jdforrester-WMF changed the subtype of this task from "Production Error" to "Task".Apr 22 2021, 2:28 AM

(This was never a production error.)

Jdforrester-WMF renamed this task from Ruby Browser Tests broken after OS update to Wikibase legacy ruby browser tests need to be updated to work on buster/ruby2.5.Apr 22 2021, 2:29 AM
Jdforrester-WMF added a parent task: T280874: Provide buster-based Ruby CI jobs.Apr 22 2021, 2:33 AM
gerritbot added a comment.May 6 2021, 1:51 AM

Change 685631 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] [DNM] Zuul: [mediawiki/extensions/Wikibase] Switch legacy Ruby jobs to 2.5

https://gerrit.wikimedia.org/r/685631

WMDE-leszek updated the task description. (Show Details)May 11 2021, 9:13 AM
WMDE-leszek updated the task description. (Show Details)May 11 2021, 9:21 AM
WMDE-leszek removed a project: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)).May 12 2021, 9:56 AM
Addshore moved this task from Active to Triaged Low (0-50) on the [DEPRECATED] wdwb-tech board.May 25 2021, 11:12 AM
Addshore added a project: Wikidata-Campsite.May 25 2021, 11:15 AM
Addshore moved this task from Incoming to Prioritized Wikidata Tech Backlog (prioritised from top to bottom) on the Wikidata-Campsite board.
Addshore updated the task description. (Show Details)Jun 2 2021, 8:22 AM
WMDE-leszek set the point value for this task to 5.Jun 2 2021, 10:59 AM
WMDE-leszek moved this task from Prioritized Wikidata Tech Backlog (prioritised from top to bottom) to Wikidata-Campsite-Iteration-∞ (On Hold) on the Wikidata-Campsite board.
WMDE-leszek edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed Wikidata-Campsite.
WMDE-leszek moved this task from Stalled/Waiting to To Do (prioritised from top to bottom) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jun 2 2021, 11:02 AM
Jakob_WMDE added a comment.Jun 8 2021, 10:01 AM

Leaving some thoughts here since I looked into this quite a bit before but am not going to work on it myself. I believe the patch I uploaded is ready to go aside from needing the properly released WikidataApiGem. It already passed the the buster mwselenium job (here) and only failed on the rake job because the two were using very different Ruby versions. If both jobs are flipped to the newer images it should just work (tm). These are the remaining steps that (I think) are missing here:

  1. Release and publish a new version of https://github.com/wmde/WikidataApiGem - the necessary changes are already in the master branch.
  2. Use the newly released package in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/680996 instead of the github branch here: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/680996/13/tests/browser/Gemfile#7
  3. Verify somehow that this works. I believe the patch should pass on It's going to fail on the regular ruby jobs because of some version incompatibilities, so maybe we can add experimental jobs running the buster versions of rake-docker and mwselenium-quibble-docker?
  4. Once it's verified, poke @Jdforrester-WMF to do the switch? Also backport it to the relevant release branches (see this comment)?
Ladsgroup moved this task from To Do (prioritised from top to bottom) to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jun 8 2021, 3:34 PM
In T280491#7141596, @Jakob_WMDE wrote:
  1. Release and publish a new version of https://github.com/wmde/WikidataApiGem - the necessary changes are already in the master branch.

Started: https://github.com/wmde/WikidataApiGem/pull/14

  1. Once it's verified, poke @Jdforrester-WMF to do the switch? Also backport it to the relevant release branches (see this comment)?

I can do it too now.

Ladsgroup claimed this task.Jun 8 2021, 3:35 PM
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptJun 8 2021, 3:35 PM
Ladsgroup added a comment.Jun 8 2021, 4:18 PM

Release is fully done: https://rubygems.org/gems/mediawiki_api-wikidata/versions/0.3.0

Jdforrester-WMF added a comment.Jun 8 2021, 5:17 PM
In T280491#7141596, @Jakob_WMDE wrote:
  1. Verify somehow that this works. I believe the patch should pass on It's going to fail on the regular ruby jobs because of some version incompatibilities, so maybe we can add experimental jobs running the buster versions of rake-docker and mwselenium-quibble-docker?

I added the experimental jobs when this need became apparent in April: https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/zuul/layout.yaml#4609

Ladsgroup added a comment.Jun 8 2021, 5:18 PM
In T280491#7143038, @Jdforrester-WMF wrote:
In T280491#7141596, @Jakob_WMDE wrote:
  1. Verify somehow that this works. I believe the patch should pass on It's going to fail on the regular ruby jobs because of some version incompatibilities, so maybe we can add experimental jobs running the buster versions of rake-docker and mwselenium-quibble-docker?

I added the experimental jobs when this need became apparent in April: https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/zuul/layout.yaml#4609

They are passing on the patch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/680996/14#message-300142225eaa4228af6e1c36061074875c736a66

gerritbot added a comment.Jun 8 2021, 5:25 PM

Change 685631 merged by jenkins-bot:

[integration/config@master] Zuul: [mediawiki/extensions/Wikibase] Switch legacy Ruby jobs to 2.5

https://gerrit.wikimedia.org/r/685631

Stashbot added a comment.Jun 8 2021, 5:25 PM

Mentioned in SAL (#wikimedia-releng) [2021-06-08T17:25:42Z] <James_F> Zuul: [mediawiki/extensions/Wikibase] Switch legacy Ruby jobs to 2.5 T280491

gerritbot added a comment.Jun 8 2021, 5:29 PM

Change 681853 abandoned by Jforrester:

[integration/config@master] [DNM] Zuul: Switch mwgate-rake and mediawiki-selenium to Ruby 2.5/buster

Reason:

Done in other patches.

https://gerrit.wikimedia.org/r/681853

gerritbot added a comment.Jun 8 2021, 5:55 PM

Change 698682 had a related patch set uploaded (by Ladsgroup; author: Jakob):

[mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.9] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698682

gerritbot added a comment.Jun 8 2021, 5:56 PM

Change 698683 had a related patch set uploaded (by Ladsgroup; author: Jakob):

[mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.8] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698683

gerritbot added a comment.Jun 8 2021, 5:57 PM

Change 698683 abandoned by Ladsgroup:

[mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.8] ruby tests: update gems for newer watir version

Reason:

wmf.8 was never deployed.

https://gerrit.wikimedia.org/r/698683

gerritbot added a comment.Jun 8 2021, 5:58 PM

Change 698684 had a related patch set uploaded (by Ladsgroup; author: Jakob):

[mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.7] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698684

gerritbot added a comment.Jun 8 2021, 6:00 PM

Change 698685 had a related patch set uploaded (by Ladsgroup; author: Jakob):

[mediawiki/extensions/Wikibase@REL1_36] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698685

gerritbot added a comment.Jun 8 2021, 6:00 PM

Change 698846 had a related patch set uploaded (by Ladsgroup; author: Jakob):

[mediawiki/extensions/Wikibase@REL1_35] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698846

gerritbot added a comment.Jun 8 2021, 6:04 PM

Change 698847 had a related patch set uploaded (by Ladsgroup; author: Jakob):

[mediawiki/extensions/Wikibase@REL1_31] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698847

gerritbot added a comment.Jun 8 2021, 6:21 PM

Change 680996 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/680996

gerritbot added a comment.Jun 8 2021, 6:26 PM

Change 698682 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.9] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698682

gerritbot added a comment.Jun 8 2021, 6:30 PM

Change 698846 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@REL1_35] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698846

gerritbot added a comment.Jun 8 2021, 6:33 PM

Change 698685 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@REL1_36] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698685

gerritbot added a comment.Jun 8 2021, 6:49 PM

Change 698684 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@wmf/1.37.0-wmf.7] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698684

ReleaseTaggerBot added a project: MW-1.37-notes (1.37.0-wmf.11; 2021-06-21).Jun 8 2021, 7:00 PM
gerritbot added a comment.Jun 8 2021, 7:03 PM

Change 698847 merged by Ladsgroup:

[mediawiki/extensions/Wikibase@REL1_31] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/698847

Ladsgroup moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Jun 8 2021, 7:07 PM
Addshore closed this task as Resolved.Jun 9 2021, 7:12 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits