Configure the CI job that runs WikibaseLexeme's browser tests against beta wikidata
Open, HighPublic3 Story Points

Description

Two jobs should be created, one running Ruby Selenium tests (selenium-WikibaseLexeme-chrome), one running Node.js Selenium tests (WikibaseLexeme-npm-browser-run-selenium-daily-node-6-docker). Both jobs should run daily and target beta wikidata.

TODO

Ruby selenium-WikibaseLexeme-chrome

πŸ’£ T197875: LoadError when running selenium-WikibaseLexeme-chrome Jenkins job

Node.js WikibaseLexeme-npm-browser-run-selenium-daily-node-6-docker

πŸ’£ T200011: `ERROR: Cannot find module '../../../../Wikibase/repo/tests/selenium` when running WikibaseLexeme-npm-browser-run-selenium-daily-node-6-docker Jenkins job

Done

βœ… selenium-WikibaseLexeme-chrome (Ruby) - Added WikibaseLexeme for a daily Ruby selenium test run, 434016, integration/config
⚰️ Added selenium project for WikibaseLexeme extension, 432983, integration/config
βœ… Added beta environment for Ruby browser tests, 432982, mediawiki/extensions/WikibaseLexeme
βœ… Parametrize target site in selenium-{name}-jessie, 434024, integration/config
βœ… Add missing configuration file for Ruby Selenium tests, 437718, mediawiki/extensions/WikibaseLexeme
βœ… Move RubyGems configuration files to root of the repository, 437737, mediawiki/extensions/WikibaseLexeme
βœ… Add missing Rake configuration file, 437741, mediawiki/extensions/WikibaseLexeme
βœ… 441035 (integration/config): Install dependency extensions when running selenium-WikibaseLexeme-chrome
βœ… 441042 (integration/config): Revert "Install dependency extensions when running selenium-WikibaseLexeme-chrome"
βœ… 440541 (mediawiki/extensions/WikibaseLexeme): Update installation instructions in readme file
βœ… 441393 (mediawiki/extensions/WikibaseLexeme): Selenium: update json Ruby gem
⚰️ Added WikibaseLexeme project for a daily Node.js Selenium test run against beta cluster, 434025, integration/config
⚰️ 441827 (integration/config): Job running WikibaseLexeme Selenium tests daily
βœ… Get WikibaseLexeme running locally, following instructions at addshore/mediawiki-docker-dev and WikibaseLexeme
βœ… 445652 Selenium: add selenium-daily NPM script

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes
zeljkofilipin added a comment.EditedJun 6 2018, 2:48 PM

I've put the instructions to README as https://gerrit.wikimedia.org/r/#/c/436270/. Feel free to review the instructions by trying them out!

I am following instructions and things seem to work fine up until this point.

README.md:102

~/Documents/gerrit/mediawiki/core$ docker run -it --rm --user $(id -u):$(id -g) -v ~/.composer:/composer -v $(pwd):/app docker.io/composer install
> ComposerHookHandler::onPreInstall
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - The requested package wikibase/data-model could not be found in any version, there may be a typo in the package name.
  Problem 2
    - The requested package wikibase/data-model-serialization could not be found in any version, there may be a typo in the package name.
  Problem 3
    - The requested package wikibase/data-model-services could not be found in any version, there may be a typo in the package name.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.
 - It's a private package and you forgot to add a custom repository to find it

Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)

Change 440541 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/WikibaseLexeme@master] WIP Update installation instructions in readme file

https://gerrit.wikimedia.org/r/440541

Change 440541 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Update installation instructions in readme file

https://gerrit.wikimedia.org/r/440541

Change 441035 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[integration/config@master] Install dependency extensions when running selenium-WikibaseLexeme-chrome

https://gerrit.wikimedia.org/r/441035

Change 441035 merged by jenkins-bot:
[integration/config@master] Install dependency extensions when running selenium-WikibaseLexeme-chrome

https://gerrit.wikimedia.org/r/441035

Change 441042 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Revert "Install dependency extensions when running selenium-WikibaseLexeme-chrome"

https://gerrit.wikimedia.org/r/441042

Change 441042 merged by jenkins-bot:
[integration/config@master] Revert "Install dependency extensions when running selenium-WikibaseLexeme-chrome"

https://gerrit.wikimedia.org/r/441042

https://gerrit.wikimedia.org/r/c/integration/config/+/441035 had Zuul to inject the dependencies by the job is triggered by Jenkins itself. Hence the patch would do nothing.

The job would need to define a parameter EXT_DEPENDENCIES with a default set to the list of extensions (separated by \n). Currently the WikibaseLexeme jobs triggered by Zuul receive:

mediawiki/extensions/BetaFeatures\nmediawiki/extensions/Capiunto\nmediawiki/extensions/CentralAuth\nmediawiki/extensions/CirrusSearch\nmediawiki/extensions/Cite\nmediawiki/extensions/Echo\nmediawiki/extensions/EducationProgram\nmediawiki/extensions/Elastica\nmediawiki/extensions/EventLogging\nmediawiki/extensions/GeoData\nmediawiki/extensions/MwEmbedSupport\nmediawiki/extensions/PdfHandler\nmediawiki/extensions/PropertySuggester\nmediawiki/extensions/Scribunto\nmediawiki/extensions/SiteMatrix\nmediawiki/extensions/SyntaxHighlight_GeSHi\nmediawiki/extensions/TimedMediaHandler\nmediawiki/extensions/VisualEditor\nmediawiki/extensions/WikiEditor\nmediawiki/extensions/Wikibase\nmediawiki/extensions/WikibaseQuality\nmediawiki/extensions/WikibaseQualityConstraints\nmediawiki/extensions/WikimediaBadges\nmediawiki/extensions/cldr

You probably only need a subset of all of them. Wikibase might be sufficient.

Note that the job would be to be adjusted to support cloning the repositories listed in EXT_DEPENDENCIES. We have a couple legacy macros to do that AND install mediawiki:

jjb/mediawiki-extensions.yaml
- builder:
    name: prepare-mediawiki-zuul-project
    builders:
     - shell: "echo $ZUUL_PROJECT > deps.txt"
     - shell: "echo -e $EXT_DEPENDENCIES >> deps.txt"
     - shell: "echo -e $SKIN_DEPENDENCIES > deps_skins.txt"
     # Clone both extensions and skins, but only extensions get listed for the
     # extensions autoloader in integration/jenkins.git, skins are
     # automatically injected by MediaWiki upon installation.
     - zuul-cloner:
         projects: >
             mediawiki/core
             mediawiki/vendor
             $(cat deps.txt)
             $(cat deps_skins.txt)
     - ext-skins-submodules-update
     - shell: "mv deps.txt src/extensions_load.txt"
     - mw-install-mysql
     - mw-apply-settings
     - mw-run-update-script

I think you will be fine by copying the first few lines until, and including, the zuul-cloner one:

`
     - shell: "echo -e $EXT_DEPENDENCIES >> deps.txt"
     - zuul-cloner:
         projects: >
             mediawiki/core
             mediawiki/vendor
             $(cat deps.txt)
             $(cat deps_skins.txt)
     - ext-skins-submodules-update
`

ext-skins-submodules-update is to process submodules in all extensions which is probably not needed.

zeljkofilipin updated the task description. (Show Details)

Thanks @hashar for the insight. I am trying to do something along the lines of what you've described. Expect patches soon :)

ext-skins-submodules-update is to process submodules in all extensions which is probably not needed.

Wikibase has some UI things as submodules, so that would be needed here I think.

Change 441393 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/WikibaseLexeme@master] Selenium: update json Ruby gem

https://gerrit.wikimedia.org/r/441393

Change 441393 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Selenium: update json Ruby gem

https://gerrit.wikimedia.org/r/441393

Change 441827 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[integration/config@master] Job running WikibaseLexeme Selenium tests daily

https://gerrit.wikimedia.org/r/441827

Change 434025 abandoned by Zfilipin:
WIP Added WikibaseLexeme project for a daily node selenium test run against beta cluster

Reason:
Simpler and working implementation at https://gerrit.wikimedia.org/r/c/integration/config/ /441827

https://gerrit.wikimedia.org/r/434025

β€’ Vvjjkkii renamed this task from Configure the CI job that runs WikibaseLexeme's browser tests against beta wikidata to fbdaaaaaaa.Jul 1 2018, 1:11 AM
β€’ Vvjjkkii removed WMDE-leszek as the assignee of this task.
β€’ Vvjjkkii updated the task description. (Show Details)
β€’ Vvjjkkii removed the point value for this task.
β€’ Vvjjkkii removed subscribers: gerritbot, Aklapper.
CommunityTechBot set the point value for this task to 3.Jul 2 2018, 1:50 PM
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot assigned this task to WMDE-leszek.
CommunityTechBot renamed this task from fbdaaaaaaa to Configure the CI job that runs WikibaseLexeme's browser tests against beta wikidata.
CommunityTechBot added subscribers: gerritbot, Aklapper.

Change 441827 abandoned by Zfilipin:
Job running WikibaseLexeme Selenium tests daily

Reason:
Replaced with https://gerrit.wikimedia.org/r/c/integration/config/ /443931 and https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseLexeme/ /445652

https://gerrit.wikimedia.org/r/441827

Change 445652 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/WikibaseLexeme@master] Selenium: add selenium-daily NPM script

https://gerrit.wikimedia.org/r/445652

Change 445652 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Selenium: add selenium-daily NPM script

https://gerrit.wikimedia.org/r/445652