Page MenuHomePhabricator

Initial Deployment of Kubernetes to Tool Labs
Closed, ResolvedPublic

Description

Tracking ticket for initial deployment of Kubernetes to Tool Labs.

The initial deployment will allow whitelisted tools to run arbitrary docker containers in NFS-free instances directly via the kubectl tool.

Should have:

  1. Debian packages
  2. Authentication setup
    1. Helper scripts to create authentication tokens and namespace
  3. Authorization setup
    1. ABAC rules to restrict users to their own namespace only
  4. DNS for services, available from rest of toollabs
  5. Webproxy from tools.wmflabs.org/<toolname> to a running webservice container, if there is one.
    1. Define what are web services and what are not.

Things that will be missing:

  1. NFS access - Kubernetes doesn't allow gid to be specified explicitly, preventing us from writing an admission controller for this
  2. One-off jobs
  3. Scheduled jobs (cron-like)
  4. Compatibility layer for current commands (jsub, webservice, jstart)
  5. Custom docker image building + local docker repository

Related Objects

StatusSubtypeAssignedTask
Resolvedyuvipanda
Resolvedyuvipanda
Resolvedvalhallasw
Resolvedyuvipanda
ResolvedBstorm
ResolvedJoe
Resolvedyuvipanda
Resolvedyuvipanda
Resolvedyuvipanda
OpenNone
Resolvedbd808
Resolvedbd808
Resolvedmmodell
Resolvedmmodell
Resolvedbd808
Resolved dpatrick
Resolvedbd808
Resolvedmmodell
Resolvedjcrespo
Resolvedbd808
Resolvedbd808
Resolvedbd808
Resolvedbd808
DuplicateNone

Event Timeline

yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added a project: Toolforge.
yuvipanda added a subscriber: yuvipanda.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I've an etcd + k8s master setup on tools-k8s-master-01 and worker nodes on tools-worker-01 and tools-worker-02 running \o/. Flannel is hitting etcd over https, and all binaries are deployed via debian packages.

yuvipanda set Security to None.

tools.wmflabs.org/nagf is now running on kubernetes! \o/ So is grrrrit-wm.

yuvipanda claimed this task.

I think this is done - despite DNS not working yet :)

Phabricator_maintenance renamed this task from Initial Deployment of Kubernetes to Tool Labs (Tracking) to Initial Deployment of Kubernetes to Tool Labs.Aug 13 2016, 9:17 PM