Page MenuHomePhabricator

Create a proof-of-concept new model for running tools
Closed, ResolvedPublic

Description

Our goal is to create a proof-of-concept of what could be a more modern paradigm to run tools and/or services.

More in practice, on a correctly-configured python repository:

git push deploy

deploys the python-uwsgi app through a docker on marathon/mesos, available to wide world through a http proxy

Intermediary steps:

  • push deploy will just push to a repository under our control
  • post receive hook that:
    1. checks to determine type of push (currently just support python-uwsgi)
      • A simple heroku-like manifest at the root of the project should help. The format we're using for toollabs currently should be good
    2. build a new docker container image that's based off a base image, with addition of:
      • code files in repo
      • mysql config path
      • other things as needed
      • install things from virtualenv too, if necessary - based off requirements.txt
    3. The mysql server and other common resources will be managed via env variables, that makes it easy to separate dev/staging/"prod"
    4. deploy said container image to a local docker repo
    5. deploy an instance of container through marathon
    6. switch proxy to serve stuff through new container
    7. kill old container
  • The proxy can just be urlproxy from toollabs. Is quite flexible, we can use the same code / scheme!

To figure out:

  • building docker images programmatically. Should we just shell out?
  • Speed of building new image, especially if there are local dependencies tobe installed (virtualenv)

Event Timeline

Joe created this task.May 21 2015, 4:28 PM
Joe raised the priority of this task from to Lowest.
Joe updated the task description. (Show Details)
Joe added a project: Wikimedia-Hackathon-2015.
Joe added subscribers: Joe, yuvipanda.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 21 2015, 4:28 PM

Change 212861 had a related patch set uploaded (by Yuvipanda):
mesos: Install docker on all slaves

https://gerrit.wikimedia.org/r/212861

Change 212861 merged by Yuvipanda:
mesos: Install docker on all slaves

https://gerrit.wikimedia.org/r/212861

Change 212866 had a related patch set uploaded (by Yuvipanda):
mesos: Enable docker as containerization mechanism for mesos

https://gerrit.wikimedia.org/r/212866

Change 212866 merged by Yuvipanda:
mesos: Enable docker as containerization mechanism for mesos

https://gerrit.wikimedia.org/r/212866

Change 212997 had a related patch set uploaded (by Yuvipanda):
dynamicproxy: Add redundanturl dynamicproxy

https://gerrit.wikimedia.org/r/212997

Change 213191 had a related patch set uploaded (by Yuvipanda):
mesos: Setup a marathon event reciever on mesos proxy

https://gerrit.wikimedia.org/r/213191

Change 213189 had a related patch set uploaded (by Yuvipanda):
mesos: Add marathon master class

https://gerrit.wikimedia.org/r/213189

Change 212997 merged by Yuvipanda:
dynamicproxy: Add redundanturl dynamicproxy

https://gerrit.wikimedia.org/r/212997

Change 213189 merged by Yuvipanda:
mesos: Add marathon master class

https://gerrit.wikimedia.org/r/213189

Change 213191 merged by Yuvipanda:
mesos: Setup a marathon event receiver on mesos proxy

https://gerrit.wikimedia.org/r/213191

Nemo_bis closed this task as Resolved.May 31 2015, 5:54 PM
Nemo_bis claimed this task.
Nemo_bis added a subscriber: Nemo_bis.

The POC was completed and we saw it. :)