We should make sure all code hitting the MediaWiki deployment train is good and all our tests are passing every Monday. Not sure how to automate this but we could always do this manually.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | zeljkofilipin | T91669 Make browser tests voting for all repos of WMF deployed code | |||
| Resolved | zeljkofilipin | T55697 [EPIC] trigger browser tests from Gerrit (tracking) | |||
| Resolved | • Jhernandez | T104315 [GOAL]: Set up testing environments that run browser tests on all patches. | |||
| Resolved | Jdlrobson | T100293 [EPIC] Run browser tests against every MobileFrontend/Gather patch | |||
| Duplicate | phuedx | T101865 Ensure tests are green before they hit production | |||
| Resolved | Jdlrobson | T108973 Spike: How do we ensure tests are green before they hit production? |
Event Timeline
Comment Actions
@Jdlrobson: @phuedx marked mondays as QA days with the intention of educating the team on doing this before the train get's cut on tuesdays. I guess he didn't send an email about it :p
Comment Actions
If we want to do this manually, Sam should write up a report on email/mediawiki.org and close this bug.
Comment Actions
@Jhernandez: I figured a meeting invite, which is an email, would be enough. I'll write a proper email about it.
Comment Actions
@phuedx about running browser tests before cut should do it manually and report back?
Comment Actions
Jenkins can run planned jobs, too[1], so maybe Unit (and browser?) tests can run in the night so you can evaluate the results on monday?
[1] In general, i'm not sure, if it's possible in wmf setup, but I think so :)
Comment Actions
@phuedx about running browser tests before cut should do it manually and report back?
No. That should be a thing though.
Comment Actions
Neato. I think we can look into this if the late Friday email has a positive impact. With @Jdlrobson's work on the per-patch work and our commitment to improving the coverage/quality of the codebase, we'd be able to make this email, automated or not, redundant.
Comment Actions
@greg is this something that release engineering could organised?
I'm imagining an e-mail sent at time of branch cut reporting all the tests that are failing organisation wide pre-deployment. I assume there is a window between branch cut and deploying to production that could be utilised to make quick regression fixes.
It would be a great way to increase visibility of failing tests....
Comment Actions
Alternatively, the tests could be run when we merge from development to master, which we (Web-Team-Backlog) control, when we do.