Page MenuHomePhabricator

productionize 'sremap' and 'filter_victorops_calendar' under sretools.wikimedia.org
Open, Needs TriagePublic

Description

Proposal:

  • create sretools.wikimedia.org in DNS
  • Puppetize microsite / httpd::site for this site, run on role::miscweb
  • puppetize the filter_victorops_calendar script + cron job, running under a sysuser on miscweb, serving from a URL like sretools.wikimedia.org/oncallcalendar.ics
  • puppetize sremap, serving from a URL like sretools.wikimedia.org/sremap/ (protected by CAS)

Event Timeline

@Dzahn please let me know what you think of the above!

Change 815376 had a related patch set uploaded (by CDanis; author: CDanis):

[operations/dns@master] add sretools.wikimedia.org

https://gerrit.wikimedia.org/r/815376

@CDanis The name name "sretools" and not having a separate one for each tool seems good to me. I like that. The DNS change also looks good, it copies other static sites on miscweb. I +1ed that. Puppetizing the site and including that as a profile in the miscweb role is also just fine.

The only thing that gives me a little pause is the "script + cron job, running under a sysuser on miscweb". So far I have been trying to keep miscweb* mostly limited to being a webserver. Originally we even had 2 separate sets of VMs, one for really just static HTML and no apps at all and one for "misc apps". (And then we unified it again.) Meanwhile the place to run maintenance crons has been mwmaint. miscweb* is also in the process of moving to kubernetes so everything that is happening there will at one point also need to go into a container there. I guess in my ideal world the generation of content would happen elsewhere but .. not trying to make it difficult. Do the scripts have a lot of dependencies?

Considering the plan to migrate away from miscweb, are there any reasons not to deploy this to K8s from the get go?

filter_victorops_calendar requires some persistent storage, ideally a plain filesystem although we could figure out something else. Not sure what the current state of the art for that is on our k8s?

The backing data for sremap is currently generated on my laptop but I'd like to make that automatic as well.

Both scripts have pretty minimal dependencies -- the former, just a dependency on Python requests and also the icalendar module, and the latter, just a dependency on Google's API libraries.

I guess as another option we could host these things on role::microsites::peopleweb ? I'm guessing that's going to remain a traditional server for longer than the other microsites?

ping @Dzahn and also @Joe -- would love some advice on stateful services on k8s

filter_victorops_calendar requires some persistent storage, ideally a plain filesystem although we could figure out something else. Not sure what the current state of the art for that is on our k8s?

We currently don't have that in k8s and generally refer to storing data in databases wherever possible. Depending on your use case, maybe swift is an option?

The backing data for sremap is currently generated on my laptop but I'd like to make that automatic as well.

Would it be an option to do that during docker image build and bake the data into the image?