Create a CI task for MCS live system tests which runs on a predetermined schedule (at least workdaily) and/or on demand when a Parsoid deploy is done against the main WMF projects.
Bonus points if this could be also triggered automatically after a Parsoid deployment.
It would be great if test runs are logged on IRC, similarly to what is done for Android CI (channel \#wikimedia-android-ci).
Examples for tests that should ran against live data:
- a modified version of featured-image-large (-/+ 30 days of current day)
- diff-test (we could have another version of it run against sepia fixtures as part of the medium tests)
- news test but against live data instead of fixture. The current news test could probably be rewritten to use the sepia fixtures as well.
- tests running against Beta Cluster (if we still want to keep them) but they should be easily identifiable in the test report
Once set this up we might be able to move more system tests under test/features there.
Current status
A mobileapps-periodic-test job has been created. It runs hourly, immediately failing each time. It needs an environment with npm and currently doesn't have one. Test results should be sent to #wikimedia-ri-ci but aren't.
The config is in integration-config/jjb/mediawiki-services.yaml:
- job-template:
name: 'mobileapps-periodic-test'
node: DebianJessieDocker
concurrent: true
triggers:
- timed: '@hourly'
scm:
- git:
url: https://phabricator.wikimedia.org/diffusion/GMOA
branches:
- master
builders:
- shell: npm run -s periodic
publishers:
- irc-wikimedia-ri-ci
- publisher:
name: irc-wikimedia-ri-ci
publishers:
- ircbot:
message-type: summary-scm-fail
channels:
- name: '#wikimedia-ri-ci'
notify-only: trueNext steps
- Get the job running in the correct environment
- Get the job passing
- Fix the irc-wikimedia-ri-ci publisher definition so results are sent to #wikimedia-ri-ci