Page MenuHomePhabricator

Wikibase legacy ruby browser tests need to be updated to work on buster/ruby2.5
Open, MediumPublic

Description

When run in the buster and ruby 2.5 environment, Wikibase's legacy ruby selenium tests fail to run:

08:25:16 undefined method `col_span' for class `Watir::TableCell' (NameError)
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/elements/table_cell.rb:3:in `alias_method'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/elements/table_cell.rb:3:in `<class:TableCell>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/elements/table_cell.rb:2:in `<module:Watir>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver/elements/table_cell.rb:1:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver.rb:119:in `require'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/watir-webdriver-0.9.3/lib/watir-webdriver.rb:119:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/mediawiki_selenium-1.7.1/lib/mediawiki_selenium/cucumber/env.rb:3:in `require'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/mediawiki_selenium-1.7.1/lib/mediawiki_selenium/cucumber/env.rb:3:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/mediawiki_selenium-1.7.1/lib/mediawiki_selenium/cucumber.rb:3:in `require'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/mediawiki_selenium-1.7.1/lib/mediawiki_selenium/cucumber.rb:3:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/features/support/env.rb:9:in `require'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/features/support/env.rb:9:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_language.rb:95:in `load'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_language.rb:95:in `load_code_file'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:180:in `load_file'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:83:in `block in load_files!'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:82:in `each'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:82:in `load_files!'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:184:in `load_step_definitions'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:42:in `run!'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/lib/cucumber/cli/main.rb:47:in `execute!'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/gems/cucumber-1.3.20/bin/cucumber:13:in `<top (required)>'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/bin/cucumber:23:in `load'
08:25:16 /workspace/src/extensions/Wikibase/tests/browser/extensions/Wikibase/ruby/2.5.0/bin/cucumber:23:in `<main>'

see results of the test job run on a dummy commit in: https://integration.wikimedia.org/ci/job/legacy-quibble-rubyselenium-docker/2

There are also likely issues with the "rake" job (essentially "linting" ruby test code) but it does not seem to be currently run with "check experimental" spell on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/688772. I suspect might be due to rake jobs to run as part of gate-and-submit job only?

Notes

Event Timeline

ItamarWMDE triaged this task as Unbreak Now! priority.Mon, Apr 19, 9:14 AM
ItamarWMDE created this task.

Change 680996 had a related patch set uploaded (by Jakob; author: Jakob):

[mediawiki/extensions/Wikibase@master] ruby tests: update gems for newer watir version

https://gerrit.wikimedia.org/r/680996

Mentioned in SAL (#wikimedia-releng) [2021-04-19T15:24:01Z] <James_F> Re-pushing mwselenium-quibble-docker back to master for T280491

Jdforrester-WMF lowered the priority of this task from Unbreak Now! to Medium.Mon, Apr 19, 3:31 PM
Jdforrester-WMF added a subscriber: Jdforrester-WMF.

Sorry, I pushed the job over to buster, found the issue, and thought I had rolled back, but apparently I failed at the final step.

I've rolled back the job use the stretch image, as can be seen in https://integration.wikimedia.org/ci/job/mwselenium-quibble-docker/35906/console

Thanks for rolling back @Jdforrester-WMF.

We have been looking into making Wikibase legacy ruby browser test to successfully roll on debian buster. It occurred that with the OS update some Ruby dependencies got updates as well. @Jakob_WMDE managed to sort most of it out. The only thing that would still kind of block us from moving this CI job to buster is the issue with video recordings. Namely, tests seemed to be failing due to lack of libav configured on CI container (despite that seemeingly ffmpeg is meant to be used instead).

See e.g. a failures on https://integration.wikimedia.org/ci/job/mwselenium-quibble-docker/35874/console

11:48:39   Background:                                    # features/header.feature:11
11:48:39     Given I am on an item page                   # features/step_definitions/entity_steps.rb:19
11:48:39       avconv not found on your system. Install it or change video recorder provider (Headless::Exception)
11:48:39       features/header.feature:12:in `Given I am on an item page'

@Jakob_WMDE managed to work this around by temporarily skipping the recording off (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/680996/4..5/tests/browser/features/support/env.rb) but probably that is not the preferred long-term approach.

After getting mwselenium-quibble-docker sorted (except the libav issue), we've noticed that mwgate-rake-docker turned red. We believe it is because that CI job still runs using stretch images, hence the ruby et al are in incompatible versions. Getting this job migrated to buster should held. Would it be possible to set up a temporary not voting buster-based job so we could ensure this?

Finally: Yes, I am aware that ruby selenium infrastructure is outdated and ultimately WMDE will stop maintaining those tests. It seems that getting the CI infrastructure updated could be achieved without the nontrivial effort of migrating the whole test suite to node.

Change 681207 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Provide legacy-quibble-rubyselenium-docker

https://gerrit.wikimedia.org/r/681207

Change 681208 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Zuul: Add legacy-quibble-rubyselenium-docker as experimental

https://gerrit.wikimedia.org/r/681208

Change 681207 merged by jenkins-bot:

[integration/config@master] jjb: Provide legacy-quibble-rubyselenium-docker

https://gerrit.wikimedia.org/r/681207

Change 681208 merged by jenkins-bot:

[integration/config@master] Zuul: Add legacy-quibble-rubyselenium-docker as experimental

https://gerrit.wikimedia.org/r/681208

Mentioned in SAL (#wikimedia-releng) [2021-04-19T23:04:38Z] <James_F> Zuul: Add legacy-quibble-rubyselenium-docker as experimental T280491

Unassigning myself for now as the immediate problem here was fixed. Plans for next steps and potentially flipping those two jobs two buster are in the making.

Change 681796 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] zuul: Add mwgate-rake-ruby2.5-docker experimentally to mwgate-rake

https://gerrit.wikimedia.org/r/681796

Change 681796 merged by jenkins-bot:

[integration/config@master] zuul: Add mwgate-ruby2.5-rake-docker experimentally to mwgate-rake

https://gerrit.wikimedia.org/r/681796

@Jakob_WMDE managed to work this around by temporarily skipping the recording off (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/680996/4..5/tests/browser/features/support/env.rb) but probably that is not the preferred long-term approach.

Agreed!

After getting mwselenium-quibble-docker sorted (except the libav issue), we've noticed that mwgate-rake-docker turned red. We believe it is because that CI job still runs using stretch images, hence the ruby et al are in incompatible versions. Getting this job migrated to buster should held. Would it be possible to set up a temporary not voting buster-based job so we could ensure this?

I've added legacy-quibble-rubyselenium-docker and mwgate-ruby2.5-rake-docker as experimental jobs to the Wikibase repo (and a couple of others). Once your team are happy with a patch that passes under these jobs (and so probably fails under the old ones), ping me on here and I can switch Wikibase over to the new jobs and merge the patch.

Change 681853 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] [DNM] Zuul: Switch mwgate-rake and mediawiki-selenium to Ruby 2.5/buster

https://gerrit.wikimedia.org/r/681853

Jdforrester-WMF changed the subtype of this task from "Production Error" to "Task".Thu, Apr 22, 2:28 AM

(This was never a production error.)

Jdforrester-WMF renamed this task from Ruby Browser Tests broken after OS update to Wikibase legacy ruby browser tests need to be updated to work on buster/ruby2.5.Thu, Apr 22, 2:29 AM

Change 685631 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] [DNM] Zuul: [mediawiki/extensions/Wikibase] Switch legacy Ruby jobs to 2.5

https://gerrit.wikimedia.org/r/685631