Problem

Right now code does not follow the same pipeline universally. For example:

• Code merged to master on Friday will be on the Beta Cluster for one full week before going out on the testwikis
• Code merged to master on Thursday early morning will be on the Beta Cluster for all of an hour (or a minute) before going to testwikis.

This is not ok. Code should propagate from master -> Beta Cluster -> test production -> other wikis in a consistent fashion.

Proposal

''Nightly test tags'' - tracking: https://bugzilla.wikimedia.org/show_bug.cgi?id=65126

• Every night at a specified time we tag master in core and all extensions in use on the cluster.
• pre/post merge unit tests are run against the tag
• the tag is deployed to the Beta Cluster using multiversion - {{bugzilla|65127}}
• browser tests are run against the beta cluster hitting the nightly tag version - {{bugzilla|65128}}
• in the morning we review the browser tests and unit test output, determine suitability for deploy
• deploy the tag as the new wmfXX deployed version on Thursday morning
  • If the latest tag is not suitable (due to failed browser tests etc) we use the last suitable nightly (eg Tues night's)