Page MenuHomePhabricator
Create Task
Maniphest T111885

Initial Deployment of Kubernetes to Tool Labs
Closed, ResolvedPublic
Actions

Description

Tracking ticket for initial deployment of Kubernetes to Tool Labs.

The initial deployment will allow whitelisted tools to run arbitrary docker containers in NFS-free instances directly via the kubectl tool.

Should have:

  1. Debian packages
  2. Authentication setup
    1. Helper scripts to create authentication tokens and namespace
  3. Authorization setup
    1. ABAC rules to restrict users to their own namespace only
  4. DNS for services, available from rest of toollabs
  5. Webproxy from tools.wmflabs.org/<toolname> to a running webservice container, if there is one.
    1. Define what are web services and what are not.

Things that will be missing:

  1. NFS access - Kubernetes doesn't allow gid to be specified explicitly, preventing us from writing an admission controller for this
  2. One-off jobs
  3. Scheduled jobs (cron-like)
  4. Compatibility layer for current commands (jsub, webservice, jstart)
  5. Custom docker image building + local docker repository

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedyuvipandaT111885 Initial Deployment of Kubernetes to Tool Labs
 ResolvedyuvipandaT111888 Create debian packages for kubernetes
ResolvedvalhallaswT111893 Make sure that docker0 bridge comes up after flannel network is established
 ResolvedyuvipandaT111904 Setup and verify authentication for Kubernetes
 Resolved• BstormT111914 Setup DNS for kubernetes services
 ResolvedJoeT111916 Add support to dynamicproxy for kubernetes based web services
 ResolvedyuvipandaT112005 Setup a way to store secrets and access them from puppet inside the Tool Labs project
 ResolvedyuvipandaT112718 Write admission controller disabling mounting of unauthorized volumes
 ResolvedyuvipandaT116504 Enforce that containers from a user run with the uid assigned to that user
 ResolveddcaroT117071 Figure out a git hosting solution for tools/kubernetes
 Resolvedbd808T133252 Create application "Striker" to manage Diffusion repositories for a Tool Labs project
 Resolvedbd808T134188 Create application that allows associating a LDAP account with a SUL account
 Resolved• mmodellT134959 LTC needed for user.ldapquery conduit api
 Resolved• mmodellT135249 create conduit method for the creation of phabricator policy objects
 Resolvedbd808T136256 Deploy "Striker" Tool Labs console to WMF production
 Resolved• dpatrickT135784 Security review of Tool Labs console application
 Resolvedbd808T141014 Create Puppet module and role for Striker
 Resolved• mmodellT142539 Create Phabricator bot account for Striker
 Resolved• jcrespoT142545 Create production database and users for Striker
 Resolvedbd808T143253 Ensure that scap3 from tin can access californium
 Resolvedbd808T143172 Send Striker logs to Logstash
 Resolvedbd808T137001 Create Conduit API method to lookup Phabricator accounts by MediaWiki user name
 Resolvedbd808T137004 Create a Conduit API method to lookup Policy information
 DuplicateNoneT123628 Install a docker registry to be used by kubernetes

Event Timeline

yuvipanda created this task.Sep 9 2015, 1:28 AM
yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added a project: Toolforge.
yuvipanda subscribed.
Restricted Application added a project: Cloud-Services. · View Herald TranscriptSep 9 2015, 1:28 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
yuvipanda added a comment.Sep 9 2015, 2:57 AM

I've an etcd + k8s master setup on tools-k8s-master-01 and worker nodes on tools-worker-01 and tools-worker-02 running \o/. Flannel is hitting etcd over https, and all binaries are deployed via debian packages.

yuvipanda updated the task description. (Show Details)Sep 9 2015, 7:08 AM
yuvipanda set Security to None.
yuvipanda updated the task description. (Show Details)Sep 9 2015, 8:29 AM
Joe subscribed.Sep 14 2015, 7:07 AM
JanZerebecki subscribed.Sep 17 2015, 4:07 PM
yuvipanda added a comment.Oct 2 2015, 4:21 AM

tools.wmflabs.org/nagf is now running on kubernetes! \o/ So is grrrrit-wm.

valhallasw triaged this task as Low priority.Oct 4 2015, 11:48 AM
valhallasw closed subtask T111893: Make sure that docker0 bridge comes up after flannel network is established as Resolved.
valhallasw subscribed.
valhallasw closed subtask T111914: Setup DNS for kubernetes services as Resolved.Oct 4 2015, 11:50 AM
valhallasw moved this task from Backlog to Ready to be worked on on the Toolforge board.
zhuyifei1999 subscribed.Oct 7 2015, 9:55 AM
yuvipanda closed subtask T112005: Setup a way to store secrets and access them from puppet inside the Tool Labs project as Resolved.Oct 10 2015, 6:15 AM
yuvipanda closed subtask T111904: Setup and verify authentication for Kubernetes as Resolved.
Krenair moved this task from Triage to Tracking on the Cloud-Services board.Oct 10 2015, 5:13 PM
intracer subscribed.Oct 24 2015, 4:48 PM
yuvipanda closed subtask T111916: Add support to dynamicproxy for kubernetes based web services as Resolved.Oct 24 2015, 11:10 PM
valhallasw mentioned this in T116917: Static content should be always up, or there should be a place where I can put static content and that place should be always up..Oct 28 2015, 2:52 PM
yuvipanda closed subtask T116504: Enforce that containers from a user run with the uid assigned to that user as Resolved.Nov 29 2015, 1:46 AM
yuvipanda reopened subtask T111914: Setup DNS for kubernetes services as Open.Nov 29 2015, 2:14 AM
yuvipanda closed subtask T111888: Create debian packages for kubernetes as Resolved.
Ricordisamoa subscribed.Dec 29 2015, 5:41 PM
Joe added a subtask: T117071: Figure out a git hosting solution for tools/kubernetes.Jan 14 2016, 12:50 PM
bd808 added a project: Tracking-Neverending.Feb 21 2016, 6:43 PM
bd808 moved this task from Ready to be worked on to Tracking on the Toolforge board.
yuvipanda closed this task as Resolved.Apr 12 2016, 7:27 PM
yuvipanda claimed this task.
yuvipanda closed subtask T112718: Write admission controller disabling mounting of unauthorized volumes as Resolved.

I think this is done - despite DNS not working yet :)

bd808 mentioned this in T132610: Develop vision and roadmap for Tool Labs enhancements.Apr 13 2016, 6:13 PM
yuvipanda reopened subtask T112718: Write admission controller disabling mounting of unauthorized volumes as Open.May 14 2016, 11:46 PM
yuvipanda closed subtask T112718: Write admission controller disabling mounting of unauthorized volumes as Resolved.May 18 2016, 2:47 PM
Danny_B moved this task from Tag to Should be Goal instead on the Tracking-Neverending board.Jul 11 2016, 6:39 PM
Danny_B added a project: Tools-Kubernetes.
• Phabricator_maintenance added a project: Goal.Aug 13 2016, 8:41 PM
• Phabricator_maintenance renamed this task from Initial Deployment of Kubernetes to Tool Labs (Tracking) to Initial Deployment of Kubernetes to Tool Labs.Aug 13 2016, 9:17 PM
• Phabricator_maintenance removed a project: Tracking-Neverending.
• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:47 PM
• Bstorm mentioned this in T214513: Deploy and migrate tools to a Kubernetes v1.15 or newer cluster.Jan 23 2019, 8:26 PM
• Bstorm closed subtask T111914: Setup DNS for kubernetes services as Resolved.Feb 25 2020, 4:29 PM
dcaro closed subtask T117071: Figure out a git hosting solution for tools/kubernetes as Resolved.Aug 26 2022, 8:30 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits