Kubernetes 1.2 is out now, so we can use the opportunity to upgrade + use new setup.

Here's what needs to happen when we 'deploy a new version of kubernetes':

On a deployment/build host:

1. Push a tag to our kubernetes gerrit repo with the version we want to build
2. Kick off a build on the deployment host. Note that this downloads a significant portion of the internet and also requires Docker (https://github.com/kubernetes/kubernetes/tree/master/build has more info)
3. Wait for the build to finish (this can take a while!)
4. The build produces a tarball, which we'll extract to a web root under a versioned path.

and then...

On master node:

1. Stop the three services that will be running (apiserver, scheduler, controller-manager)
2. wget the three binaries of given version, replace the current binaries
3. Start them back up, and make sure they're working ok!

On worker nodes:

1. Stop the two services that are running (kubelet, kube-proxy)
2. wget the three binaries of given version, replace current binaries
3. Start them back up, make sure they are working ok!

So the deploy should take a version as a parameter. This allows us to 'roll back' by just deploying an earlier version.

Things Scap needs to have to make this possible:

1. Ability to setup a deployment server without bringing in all of mediawiki, salt, trebuchet, trebuchet-trigger, redis, etc.
2. Ability to just curl from a webserver instead of using git (because binaries!)

Change 279648 had a related patch set uploaded (by Yuvipanda):
tools: Add class that helps build kubernetes

https://gerrit.wikimedia.org/r/279648

Intermediate alternative if we can't actually get scap to do this in the meantime:

1. The k8s build script extracts the tar into a well known location with version numbers
2. A simple static file server serves these binary blobs
3. We've simple scripts on the master and worker nodes that take a parameterized version and pull these files down and restart services.

We'll run these commands on the nodes via old fashioned ssh / clustershell / yuvi-types-furiously-on-multiple-tabs methods.

In T129311#2151640, @yuvipanda wrote:

Things Scap needs to have to make this possible:

1. Ability to setup a deployment server without bringing in all of mediawiki, salt, trebuchet, trebuchet-trigger, redis, etc.

This is just a minor refactor of the scap3 classes in puppet. Actually it's mostly there already but probably needs a bit more separation of concerns.

2. Ability to just curl from a webserver instead of using git (because binaries!)

git-fat, git-annex or git-lfs all provide good ways to do something like this. We already support git-fat, plan to support git-annex and are also looking into git-lfs. Of course, it wouldn't be difficult to provide a way to just curl a file either.

I think almost all of the requirements for a really minimal deployment server can be satisfied by scap::deploy_host once this merges: https://gerrit.wikimedia.org/r/#/c/279198/5

Change 279648 merged by Yuvipanda:
tools: Add class that helps build kubernetes

https://gerrit.wikimedia.org/r/279648

Cleaning up some old workboards.

Is this task still needed? If so, is scap3 still a part of the discussion?