Page MenuHomePhabricator

Run Popups Selenium tests daily targeting beta cluster
Closed, DeclinedPublic

Event Timeline

Restricted Application added subscribers: APerson, Aklapper. · View Herald Transcript
zeljkofilipin triaged this task as Medium priority.Oct 11 2017, 10:59 AM
zeljkofilipin edited projects, added Page-Previews; removed Navigation-Popups.
zeljkofilipin moved this task from Backlog 🔙 to In Progress 🔨 on the User-zeljkofilipin board.

Change 381212 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/Popups@master] Run Selenium Ruby tests daily targeting Beta cluster

https://gerrit.wikimedia.org/r/381212

I have created a test Jenkins job, selenium-Popups-381212. It fails with unexpected HTTP response (403) (MediawikiApi::HttpError). The commit (381212) creating the job should not be merged until the job passes.

@Jdlrobson I am not familiar with Selenium tests in Popups. Please let me know if you need help fixing selenium-Popups-381212 job.

Change 375377 had a related patch set uploaded (by Zfilipin; owner: Jdlrobson):
[integration/config@master] Add Jenkins job for browser tests

https://gerrit.wikimedia.org/r/375377

I have created a test Jenkins job, selenium-Popups-381212. It fails with unexpected HTTP response (403) (MediawikiApi::HttpError). The commit (381212) creating the job should not be merged until the job passes.

That looks like a configuration problem to me. Investigating further this is happening in the mediawiki_api-0.7.0 gem file.
Maybe the use of http is the problem..? https://gerrit.wikimedia.org/r/383718

Tried running tests on my machine.

~/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/Popups$ ruby -v
ruby 2.4.2p198 (2017-09-14 revision 59899) [x86_64-darwin16]

~/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/Popups$ bundle install
The latest bundler is 1.16.0.pre.3, but you are currently running 1.15.4.
To update, run `gem install bundler --pre`
Fetching gem metadata from https://rubygems.org/..........
Fetching version metadata from https://rubygems.org/..
Fetching dependency metadata from https://rubygems.org/.
Using rake 10.5.0
Using ast 2.3.0
Using builder 3.2.2
Using bundler 1.15.4
Using ffi 1.9.14
Using chunky_png 1.3.8
Using diff-lcs 1.2.5
Using multi_json 1.12.1
Using multi_test 0.1.2
Using i18n 0.7.0
Using yml_reader 0.7
Using unf_ext 0.0.7.2
Using multipart-post 2.0.0
Using headless 2.3.1
Fetching json 1.8.3
Installing json 1.8.3 with native extensions
Using net-http-persistent 2.9.4
Using rubyzip 1.2.0
Using websocket 1.2.3
Using mime-types 2.99.3
Using netrc 0.11.0
Using rspec-core 2.99.2
Using syntax 1.2.1
Using thor 0.19.4
Using powerpack 0.1.1
Using rainbow 2.1.0
Using ruby-progressbar 1.8.1
Using parser 2.3.3.1
Using childprocess 0.5.9
Using rspec-expectations 2.99.2
Using gherkin 2.12.2
Using faker 1.6.6
Using unf 0.1.4
Using faraday 0.10.0
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /usr/local/lib/ruby/gems/2.4.0/gems/json-1.8.3/ext/json/ext/generator
/usr/local/opt/ruby/bin/ruby -r ./siteconf20171012-2810-1omxdxl.rb extconf.rb
creating Makefile

current directory: /usr/local/lib/ruby/gems/2.4.0/gems/json-1.8.3/ext/json/ext/generator
make "DESTDIR=" clean

current directory: /usr/local/lib/ruby/gems/2.4.0/gems/json-1.8.3/ext/json/ext/generator
make "DESTDIR="
compiling generator.c
generator.c:861:25: error: use of undeclared identifier 'rb_cFixnum'
    } else if (klass == rb_cFixnum) {
                        ^
generator.c:863:25: error: use of undeclared identifier 'rb_cBignum'
    } else if (klass == rb_cBignum) {
                        ^
generator.c:975:5: warning: division by zero is undefined [-Wdivision-by-zero]
    rb_scan_args(argc, argv, "01", &opts);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2151:9: note: expanded from macro 'rb_scan_args'
        rb_scan_args0(argc,argvp,fmt,\
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2324:9: note: expanded from macro 'rb_scan_args0'
                     (rb_scan_args_verify(fmt, varc), vars))
                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2209:11: note: expanded from macro 'rb_scan_args_verify'
        verify = rb_scan_args_verify_count(fmt, varc); \
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: (skipping 4 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2175:6: note: expanded from macro 'rb_scan_args_count_hash'
     rb_scan_args_count_block(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2170:6: note: expanded from macro 'rb_scan_args_count_block'
     rb_scan_args_count_end(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2166:12: note: expanded from macro 'rb_scan_args_count_end'
    ((vari)/(!fmt[ofs] || rb_scan_args_bad_format(fmt)))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
generator.c:975:5: warning: division by zero is undefined [-Wdivision-by-zero]
    rb_scan_args(argc, argv, "01", &opts);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2151:9: note: expanded from macro 'rb_scan_args'
        rb_scan_args0(argc,argvp,fmt,\
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2324:9: note: expanded from macro 'rb_scan_args0'
                     (rb_scan_args_verify(fmt, varc), vars))
                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2209:11: note: expanded from macro 'rb_scan_args_verify'
        verify = rb_scan_args_verify_count(fmt, varc); \
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: (skipping 4 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2175:6: note: expanded from macro 'rb_scan_args_count_hash'
     rb_scan_args_count_block(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2171:6: note: expanded from macro 'rb_scan_args_count_block'
     rb_scan_args_count_end(fmt, ofs+1, varc, vari+1))
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2166:12: note: expanded from macro 'rb_scan_args_count_end'
    ((vari)/(!fmt[ofs] || rb_scan_args_bad_format(fmt)))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
generator.c:975:5: warning: division by zero is undefined [-Wdivision-by-zero]
    rb_scan_args(argc, argv, "01", &opts);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2151:9: note: expanded from macro 'rb_scan_args'
        rb_scan_args0(argc,argvp,fmt,\
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2324:9: note: expanded from macro 'rb_scan_args0'
                     (rb_scan_args_verify(fmt, varc), vars))
                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2209:11: note: expanded from macro 'rb_scan_args_verify'
        verify = rb_scan_args_verify_count(fmt, varc); \
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: (skipping 4 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2176:6: note: expanded from macro 'rb_scan_args_count_hash'
     rb_scan_args_count_block(fmt, ofs+1, varc, vari+1))
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2170:6: note: expanded from macro 'rb_scan_args_count_block'
     rb_scan_args_count_end(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2166:12: note: expanded from macro 'rb_scan_args_count_end'
    ((vari)/(!fmt[ofs] || rb_scan_args_bad_format(fmt)))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
generator.c:975:5: warning: division by zero is undefined [-Wdivision-by-zero]
    rb_scan_args(argc, argv, "01", &opts);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2151:9: note: expanded from macro 'rb_scan_args'
        rb_scan_args0(argc,argvp,fmt,\
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2324:9: note: expanded from macro 'rb_scan_args0'
                     (rb_scan_args_verify(fmt, varc), vars))
                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2209:11: note: expanded from macro 'rb_scan_args_verify'
        verify = rb_scan_args_verify_count(fmt, varc); \
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: (skipping 4 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2175:6: note: expanded from macro 'rb_scan_args_count_hash'
     rb_scan_args_count_block(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2170:6: note: expanded from macro 'rb_scan_args_count_block'
     rb_scan_args_count_end(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2166:12: note: expanded from macro 'rb_scan_args_count_end'
    ((vari)/(!fmt[ofs] || rb_scan_args_bad_format(fmt)))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
generator.c:975:5: warning: division by zero is undefined [-Wdivision-by-zero]
    rb_scan_args(argc, argv, "01", &opts);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2151:9: note: expanded from macro 'rb_scan_args'
        rb_scan_args0(argc,argvp,fmt,\
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2324:9: note: expanded from macro 'rb_scan_args0'
                     (rb_scan_args_verify(fmt, varc), vars))
                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2209:11: note: expanded from macro 'rb_scan_args_verify'
        verify = rb_scan_args_verify_count(fmt, varc); \
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: (skipping 4 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2175:6: note: expanded from macro 'rb_scan_args_count_hash'
     rb_scan_args_count_block(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2170:6: note: expanded from macro 'rb_scan_args_count_block'
     rb_scan_args_count_end(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2166:12: note: expanded from macro 'rb_scan_args_count_end'
    ((vari)/(!fmt[ofs] || rb_scan_args_bad_format(fmt)))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
generator.c:975:5: warning: division by zero is undefined [-Wdivision-by-zero]
    rb_scan_args(argc, argv, "01", &opts);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2151:9: note: expanded from macro 'rb_scan_args'
        rb_scan_args0(argc,argvp,fmt,\
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2324:9: note: expanded from macro 'rb_scan_args0'
                     (rb_scan_args_verify(fmt, varc), vars))
                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2209:11: note: expanded from macro 'rb_scan_args_verify'
        verify = rb_scan_args_verify_count(fmt, varc); \
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: (skipping 5 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2175:6: note: expanded from macro 'rb_scan_args_count_hash'
     rb_scan_args_count_block(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2170:6: note: expanded from macro 'rb_scan_args_count_block'
     rb_scan_args_count_end(fmt, ofs, varc, vari) : \
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/ruby/2.4.2/include/ruby-2.4.0/ruby/ruby.h:2166:12: note: expanded from macro 'rb_scan_args_count_end'
    ((vari)/(!fmt[ofs] || rb_scan_args_bad_format(fmt)))
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6 warnings and 2 errors generated.
make: *** [generator.o] Error 1

make failed, exit code 2

Gem files will remain installed in /usr/local/lib/ruby/gems/2.4.0/gems/json-1.8.3 for inspection.
Results logged to /usr/local/lib/ruby/gems/2.4.0/extensions/x86_64-darwin-16/2.4.0/json-1.8.3/gem_make.out

An error occurred while installing json (1.8.3), and Bundler cannot continue.
Make sure that `gem install json -v '1.8.3'` succeeds before bundling.

In Gemfile:
  mediawiki_selenium was resolved to 1.7.3, which depends on
    json

I have manged to get tests running on my machine with this:

https://gerrit.wikimedia.org/r/#/c/381212/4..5

But, the tests are failing, pretty much the same way as in CI:

https://integration.wikimedia.org/ci/job/mwext-mw-selenium-jessie/6667/console

Change 383934 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/Popups@master] Drop beta feature step testing in browser tests

https://gerrit.wikimedia.org/r/383934

The tests were wired up to run only for wikis where it's enabled as a beta feature. I've simplified the tests as this is not necessary any more and rebased https://gerrit.wikimedia.org/r/#/c/381212 on top of it.

zeljkofilipin lowered the priority of this task from Medium to Low.Oct 13 2017, 9:23 AM

Since this is Ruby framework that will soon be deprecated.

For Popups to have daily job running Selenium tests targeting beta cluster, it also has to use mwext-ruby-jessie job (see T164479: mwext-ruby-jessie Jenkins job runs all Ruby tasks for MediaWiki extensions) instead of mwext-mw-selenium-jessie.

Change 384005 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[integration/config@master] Run mwext-ruby-jessie Popups

https://gerrit.wikimedia.org/r/384005

Change 384005 merged by jenkins-bot:
[integration/config@master] Run mwext-ruby-jessie Popups

https://gerrit.wikimedia.org/r/384005

Change 384014 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[integration/config@master] Run mwext-ruby-jessie job for Popups

https://gerrit.wikimedia.org/r/384014

Now selenium-Popups-381212 is failing with timed out after 30 seconds (Watir::Wait::TimeoutError):

And for whatever reason, there is no link to Sauce Labs in the error message :/ Uh-oh. Could not find link to Sauce Labs job URL, but the job does exist. :\

I was not able to debug on my machine, looks like I have a version of Ruby that does not work with the gems that Popups needs.

Change 384030 had a related patch set uploaded (by Zfilipin; owner: Jdlrobson):
[integration/config@master] Popups Selenium tests should run daily targeting beta cluster

https://gerrit.wikimedia.org/r/384030

Jdlrobson closed this task as Declined.Oct 13 2017, 5:16 PM

Let's decline this then and go straight to the Node.js version. There's no point in debugging something we are about to replace in T177262.

Change 381212 abandoned by Jdlrobson:
WIP Run Selenium Ruby tests daily targeting Beta cluster

Reason:
Task has been declined

https://gerrit.wikimedia.org/r/381212

Change 384030 abandoned by Zfilipin:
Popups Selenium tests should run daily targeting beta cluster

Reason:
T177924 has been declined.

https://gerrit.wikimedia.org/r/384030

Change 384014 abandoned by Zfilipin:
Run mwext-ruby-jessie job for Popups

Reason:
T177924 has been declined.

https://gerrit.wikimedia.org/r/384014