Page MenuHomePhabricator

Ensure tests are green before they hit production
Closed, DuplicatePublic

Description

We should make sure all code hitting the MediaWiki deployment train is good and all our tests are passing every Monday. Not sure how to automate this but we could always do this manually.

Event Timeline

Jdlrobson created this task.Jun 9 2015, 5:44 PM
Jdlrobson raised the priority of this task from to Needs Triage.
Jdlrobson updated the task description. (Show Details)
Jdlrobson added subscribers: Jdlrobson, Legoktm, Jhernandez and 2 others.

@Jdlrobson: @phuedx marked mondays as QA days with the intention of educating the team on doing this before the train get's cut on tuesdays. I guess he didn't send an email about it :p

Jdlrobson assigned this task to phuedx.Jun 22 2015, 5:50 PM

If we want to do this manually, Sam should write up a report on email/mediawiki.org and close this bug.

@Jhernandez: I figured a meeting invite, which is an email, would be enough. I'll write a proper email about it.

@phuedx about running browser tests before cut should do it manually and report back?

Jenkins can run planned jobs, too[1], so maybe Unit (and browser?) tests can run in the night so you can evaluate the results on monday?

[1] In general, i'm not sure, if it's possible in wmf setup, but I think so :)

@phuedx about running browser tests before cut should do it manually and report back?

No. That should be a thing though.

Jenkins can run planned jobs, too[1], so maybe Unit (and browser?) tests can run in the night so you can evaluate the results on monday?

[1] In general, i'm not sure, if it's possible in wmf setup, but I think so :)

Neato. I think we can look into this if the late Friday email has a positive impact. With @Jdlrobson's work on the per-patch work and our commitment to improving the coverage/quality of the codebase, we'd be able to make this email, automated or not, redundant.

Jdlrobson added a subscriber: greg.Jul 1 2015, 9:15 PM

@greg is this something that release engineering could organised?

I'm imagining an e-mail sent at time of branch cut reporting all the tests that are failing organisation wide pre-deployment. I assume there is a window between branch cut and deploying to production that could be utilised to make quick regression fixes.

It would be a great way to increase visibility of failing tests....

phuedx added a comment.EditedJul 2 2015, 3:49 PM

Alternatively, the tests could be run when we merge from development to master, which we (Readers-Web-Backlog) control, when we do.

Jdlrobson moved this task from Backlog to Tasks on the MobileFrontend board.Aug 4 2015, 6:37 PM
Jhernandez triaged this task as Medium priority.Aug 13 2015, 6:05 PM

Created T108973 for starting a discussion on how we could solve this task.