Create a CI task for MCS live system tests which runs on a predetermined schedule (at least workdaily) and/or on demand when a Parsoid deploy is done against the main WMF projects.
Bonus points if this could be also triggered automatically after a Parsoid deployment.
It would be great if test runs are logged on IRC, similarly to what is done for Android CI (channel \#wikimedia-android-ci).
Examples for tests that should ran against live data:
* a modified version of featured-image-large (-/+ 30 days of current day)
* `diff-test` (we could have another version of it run against sepia fixtures as part of the medium tests)
* `news` test but against live data instead of fixture. The current news test could probably be rewritten to use the sepia fixtures as well.
* tests running against Beta Cluster (if we still want to keep them) but they should be easily identifiable in the test report
Once set this up we might be able to move more system tests under `test/features` there.
== Current status
A [[ https://integration.wikimedia.org/ci/view/Mobile/job/mobileapps-periodic-test/ | mobileapps-periodic-test ]] job has been created. It runs hourly, [[ https://phabricator.wikimedia.org/P7297 | immediately failing ]] each time. It needs an environment with `npm` and currently doesn't have one. Test results should be sent to #wikimedia-ri-ci but aren't.
The config is [[ https://github.com/wikimedia/integration-config/blob/master/jjb/mediawiki-services.yaml#L81-L104 | in integration-config/jjb/mediawiki-services.yaml ]]:
```
- job-template:
name: 'mobileapps-periodic-test'
node: DebianJessieDocker
concurrent: true
triggers:
- timed: '@hourly'
scm:
- git:
url: https://phabricator.wikimedia.org/diffusion/GMOA
branches:
- master
builders:
- shell: npm run -s periodic
publishers:
- irc-wikimedia-ri-ci
- publisher:
name: irc-wikimedia-ri-ci
publishers:
- ircbot:
message-type: summary-scm-fail
channels:
- name: '#wikimedia-ri-ci'
notify-only: true
```
== Next steps
[] Get the job running in the correct node type (not `DebianJessieDocker`); needs `npm`
[] Get the job passing
[] Fix the `irc-wikimedia-ri-ci` publisher definition so results are sent to #wikimedia-ri-ci