Page MenuHomePhabricator

Run @integration tests on new deploy branch creation
Closed, DeclinedPublic

Description

<quote name="Jon Robson" date="2015-09-03" time="16:45:30 -0700">

On Thu, Sep 3, 2015 at 1:59 PM, Greg Grossmeier <greg@wikimedia.org> wrote:

<quote name="Jon Robson" date="2015-09-03" time="11:45:47 -0700">

  1. We should run @integration tests prior to deployments to the

cluster via the train and communicate out when we have failures (and
make a decision to push broken code)

The way I hear this is: Run @integration tests on merge to wmfXX
branches. Is that an accurate rephrasing?

I'm not too clear with how we currently do deployments but as I
understand it we manually cut a branch for wmf. I'd like part of this
process to involve triggering all the @integration tests and
generating a url that we can share with each deployment.

Event Timeline

greg raised the priority of this task from to Needs Triage.
greg updated the task description. (Show Details)
greg added subscribers: greg, Jdlrobson.

@greg any chance of this happening within the next month? If not what can be done to speed that up? I have no knowledge of the deploy process so have no idea how trivial this would be nor whether it needs to be manual. I'm keen to report back to the reading team whether we can expect this to happen any time soon.

It all depends on what you mean by "this". :) What'd you have in mind with "generating a url we can share with each deployment"? A jenkins build URL? or something fancier?

Since this is new and unplanned work, and we're already rushing to finish everything we've committed to (with other interruptions along the way) before the EOQ (aka EOM), I can't make a commitment for any ETA right now (just like any other team has to say when new unplanned work comes up). But...

I'll talk with the team today and see what they think.

I think right now something simple along the lines of this would be a good start:

"mediawiki 1.26 was deployed today to the cluster. At the time of deployment 3/200 browser tests were failing: <link>"

(either posted to wikitech or on a wiki page somewhere that we can subscribe to)

Ah, I see.

We'll put this in our "best effort, but no promise" category for the next couple months; I really don't want to jeopardize our other work right now that we've made commitments to. (/me stops repeating himself)

I think we all agree it's a great idea, though, which is why we're even putting it in the "best effort" category :)

zeljkofilipin lowered the priority of this task from Medium to Low.May 29 2017, 10:31 AM
hashar subscribed.

The context was that we had browser tests running on a daily basis that would fail whenever we deploy. Nowadays we run them on each patchset and have thus have a guarantee they all pass when they are merged in the master branch. The wmf branch cut is merely a copy of that state and is thus guaranteed to pass. So I don't think there is any point in keeping this task.