| zeljkofilipin | |
| Jul 7 2016, 2:05 PM |
| F4280164: VisualEditor_category_item-en.png | |
| Jul 16 2016, 10:24 AM |
We used to have a matrix jenkins job that would run the script for multiple languages. The job was deleted because it was broken. It should be recreated. Existing Ruby script and credentials (User:LanguageScreenshotBot) for uploading to Commons could be reused. The uploading script could be rewritten in JavaScript later.
Relevant pages:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | zeljkofilipin | T94150 Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily | |||
| Resolved | zeljkofilipin | T94162 Fix VisualEditor scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily | |||
| Invalid | None | T138751 Update VisualEditor user guide | |||
| Resolved | Esanders | T136675 Make new screenshots to show off the LaTeX editor | |||
| Resolved | Esanders | T119482 Fix language screenshots browser test on VisualEditor | |||
| Resolved | zeljkofilipin | T132574 Port language screenshots for VisualEditor from Ruby to JavaScript | |||
| Resolved | zeljkofilipin | T139613 Run language screenshots script for VisualEditor in Jenkins | |||
| Resolved | zeljkofilipin | T141369 Fix font support on SauceLabs VE screenshots | |||
| Resolved | zeljkofilipin | T143563 Upload language screenshot files using commons_upload ruby gem | |||
| Resolved | zeljkofilipin | T143655 internal_api_error_LocalFileLockError while uploading file via API to commons.wikimedia.beta.wmflabs.org |
Change 298013 had a related patch set uploaded (by Zfilipin):
WIP use chromedriver from npm package
Running the job at ci-jessie-wikimedia. Unfortunatelly, looks like there is a problem, or the number of instances is really low. :(
Parent job:
https://integration.wikimedia.org/ci/view/Selenium/job/selenium-VisualEditor-T139613/12/console
10:04:39 Triggering selenium-VisualEditor-T139613 » firefox,beta,Linux,ci-jessie-wikimedia
Child job (at 10:33):
#12 (pending—All nodes of label ‘ci-jessie-wikimedia’ are offline)
Change 298013 abandoned by Zfilipin:
WIP use chromedriver from npm package
Reason:
Looks like chromedriver is already present at ci-jessie-wikimedia nodes, so there is no need for this.
00:00:17.027 /usr/lib/chromium/chromedriver -v
00:00:17.093 ChromeDriver 2.21 (undefined)
I am doing something wrong. :(
... Xvfb :94 & export DISPLAY=:94 /usr/lib/chromium/chromium https://www.wikipedia.org/ ...
causes (selenium-VisualEditor-T139613...7)
... 00:00:56.105 + export DISPLAY=:94 00:00:56.106 + DISPLAY=:94 00:00:56.106 + /usr/lib/chromium/chromium https://www.wikipedia.org/ 00:00:56.106 + Xvfb :94 00:00:56.108 (EE) 00:00:56.108 Fatal server error: 00:00:56.109 (EE) Server is already active for display 94 00:00:56.109 If this server is no longer running, remove /tmp/.X94-lock 00:00:56.109 and start again. 00:00:56.110 (EE) 00:00:56.309 Xlib: extension "RANDR" missing on display ":94". 00:00:56.583 [1273:1281:0715/132057:ERROR:bus.cc(432)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory 00:00:56.685 Xlib: extension "RANDR" missing on display ":94". 00:00:57.132 [1310:1310:0715/132058:ERROR:gl_surface_glx.cc(381)] glxQueryVersion failed 00:00:57.132 [1310:1310:0715/132058:ERROR:gl_surface_x11.cc(62)] GLSurfaceGLX::InitializeOneOff failed. 00:00:57.146 [1310:1310:0715/132058:ERROR:gpu_child_thread.cc(397)] Exiting GPU process due to errors during initialization 00:00:57.239 [1273:1292:0715/132058:ERROR:browser_gpu_channel_host_factory.cc(119)] Failed to launch GPU process. 00:02:54.628 Build was aborted 00:02:54.628 Aborted by Zeljko Filipin 00:02:54.727 Finished: ABORTED
If I remove Xvfb :94 & and leave only
... export DISPLAY=:94 /usr/lib/chromium/chromium https://www.wikipedia.org/ ...
I get (selenium-VisualEditor-T139613...8)
... 00:00:12.201 + export DISPLAY=:94 00:00:12.201 + DISPLAY=:94 00:00:12.201 + /usr/lib/chromium/chromium https://www.wikipedia.org/ 00:00:12.455 Xlib: extension "RANDR" missing on display ":94". 00:00:12.632 [1267:1275:0715/132310:ERROR:bus.cc(432)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory 00:00:12.712 Xlib: extension "RANDR" missing on display ":94". 00:00:12.906 [1303:1303:0715/132310:ERROR:gl_surface_glx.cc(381)] glxQueryVersion failed 00:00:12.906 [1303:1303:0715/132310:ERROR:gl_surface_x11.cc(62)] GLSurfaceGLX::InitializeOneOff failed. 00:00:12.932 [1303:1303:0715/132310:ERROR:gpu_child_thread.cc(397)] Exiting GPU process due to errors during initialization 00:00:13.072 [1267:1286:0715/132311:ERROR:browser_gpu_channel_host_factory.cc(119)] Failed to launch GPU process. 00:00:13.081 [1267:1286:0715/132311:ERROR:browser_gpu_channel_host_factory.cc(119)] Failed to launch GPU process. 00:00:38.424 Build was aborted 00:00:38.424 Aborted by Zeljko Filipin 00:00:38.522 Finished: ABORTED
Change 299159 had a related patch set uploaded (by Zfilipin):
WIP Run language screenshots from Jenkins
Problems installing ruby gems on ci-jessie-wikimedia instance (cc @hashar), details at selenium-VisualEditor-299159...8/console
00:00:12.134 + bundle install --verbose ... 00:00:16.926 We trust you have received the usual lecture from the local System 00:00:16.926 Administrator. It usually boils down to these three things: 00:00:16.926 00:00:16.926 #1) Respect the privacy of others. 00:00:16.926 #2) Think before you type. 00:00:16.926 #3) With great power comes great responsibility. 00:00:16.926 00:00:16.927 sudo: no tty present and no askpass program specified 00:00:16.930 00:00:16.930 Bundler::GemspecError: Could not read gem at /var/lib/gems/2.1.0/cache/rake-10.4.2.gem. It may be corrupted. 00:00:16.930 /usr/lib/ruby/vendor_ruby/bundler/rubygems_integration.rb:177:in `rescue in spec_from_gem' 00:00:16.930 /usr/lib/ruby/vendor_ruby/bundler/rubygems_integration.rb:174:in `spec_from_gem' 00:00:16.930 /usr/lib/ruby/vendor_ruby/bundler/source/rubygems.rb:92:in `install' 00:00:16.930 /usr/lib/ruby/vendor_ruby/bundler/installer.rb:104:in `block in install_gem_from_spec' 00:00:16.930 /usr/lib/ruby/vendor_ruby/bundler/rubygems_integration.rb:162:in `with_build_args' 00:00:16.931 /usr/lib/ruby/vendor_ruby/bundler/installer.rb:103:in `install_gem_from_spec' 00:00:16.931 /usr/lib/ruby/vendor_ruby/bundler/installer.rb:264:in `block in install_sequentially' 00:00:16.931 /usr/lib/ruby/2.1.0/forwardable.rb:183:in `each' 00:00:16.931 /usr/lib/ruby/2.1.0/forwardable.rb:183:in `each' 00:00:16.931 /usr/lib/ruby/vendor_ruby/bundler/installer.rb:263:in `install_sequentially' 00:00:16.931 /usr/lib/ruby/vendor_ruby/bundler/installer.rb:90:in `run' 00:00:16.931 /usr/lib/ruby/vendor_ruby/bundler/installer.rb:18:in `install' 00:00:16.931 /usr/lib/ruby/vendor_ruby/bundler/cli/install.rb:79:in `run' 00:00:16.932 /usr/lib/ruby/vendor_ruby/bundler/cli.rb:145:in `install' 00:00:16.932 /usr/lib/ruby/vendor_ruby/thor/command.rb:27:in `run' 00:00:16.932 /usr/lib/ruby/vendor_ruby/thor/invocation.rb:126:in `invoke_command' 00:00:16.932 /usr/lib/ruby/vendor_ruby/thor.rb:359:in `dispatch' 00:00:16.932 /usr/lib/ruby/vendor_ruby/thor/base.rb:440:in `start' 00:00:16.932 /usr/lib/ruby/vendor_ruby/bundler/cli.rb:9:in `start' 00:00:16.932 /usr/bin/bundle:20:in `block in <main>' 00:00:16.933 /usr/lib/ruby/vendor_ruby/bundler/friendly_errors.rb:5:in `with_friendly_errors' 00:00:16.933 /usr/bin/bundle:18:in `<main>' 00:00:16.939 An error occurred while installing rake (10.4.2), and Bundler cannot continue. 00:00:16.939 Make sure that `gem install rake -v '10.4.2'` succeeds before bundling. 00:00:16.976 Build step 'Execute shell' marked build as failure 00:00:17.155 Finished: FAILURE
bundler invokes gem which, on Debian, default to /usr/ which is not writable.
What we did on Nodepool instance is to set BUNDLE_PATH to have gems installed under the jenkins homedir, that is then saved/restored by the package manager cache system that load/save for builds (castor).
In integration/config zuul/parameter_functions.py the Nodepool jobs have:
params['BUNDLE_PATH'] = '/home/jenkins/workspace/vendor/bundle'
Make your job template to use BUNDLE_PATH='/home/jenkins/workspace/vendor/bundle' and you will be fine.
Bonus point for using the builder castor-load and publisher castor-save. That speeds up the build by loading dependencies from the central cache.
Thanks @hashar, Ruby part is now working fine.
The first job that managed to upload images to commons.wikimedia.beta.wmflabs.org is selenium-VisualEditor-299159...16.
Example upload is VisualEditor_references_list-en.png.
Files uploaded to commons.wikimedia.beta.wmflabs.org by Selenium user.
Change 300035 had a related patch set uploaded (by Zfilipin):
WIP Run language screenshots script for VisualEditor in Jenkins
Change 300882 had a related patch set uploaded (by Zfilipin):
WIP Run all language screenshots script for VisualEditor in Jenkins
Change 300882 abandoned by Zfilipin:
WIP Run all language screenshots script for VisualEditor in Jenkins
Change 305966 had a related patch set uploaded (by Zfilipin):
WIP Fix Error: timeout of 20000ms exceeded
Change 305979 had a related patch set uploaded (by Zfilipin):
WIP Dots separate language code in language screenshot file name
Change 305979 abandoned by Zfilipin:
WIP Dots separate language code in language screenshot file name
Reason:
This change would not be backwards compatible with existing file names.
https://commons.wikimedia.org/wiki/File:VisualEditor_Insert_Menu-en.png
Change 306455 had a related patch set uploaded (by Zfilipin):
The latest version of commons_upload Ruby gem is needed for language screenshots
Change 306455 merged by jenkins-bot:
The latest version of commons_upload Ruby gem is needed for language screenshots
Change 306459 had a related patch set uploaded (by Zfilipin):
WIP Fix failing Selenium tests
Change 306908 had a related patch set uploaded (by Zfilipin):
WIP Run language screenshots for 10 languages
Change 307280 had a related patch set uploaded (by Zfilipin):
WIP Run language screenshots script for VisualEditor in Jenkins
Change 307313 had a related patch set uploaded (by Zfilipin):
WIP Test not failing the script on Mocha failure
Change 307313 abandoned by Zfilipin:
WIP Test not failing the script on Mocha failure
Change 307280 abandoned by Zfilipin:
WIP Run grunt and upload separately for two languages
Change 307318 had a related patch set uploaded (by Zfilipin):
WIP Increase Mocha and Selenium timeout to 40 seconds
Change 307318 merged by jenkins-bot:
Increase Mocha and Selenium timeout for language screenshots to 40 s
Change 300035 merged by jenkins-bot:
Run language screenshots script for VisualEditor in Jenkins
