Page MenuHomePhabricator

Move to using scap3 for deployment for ORES service
Closed, ResolvedPublic

Description

This task is done when the functionality in ores-wikimedia-config's fabfile.py is converted to scap3.

Current fabfile.py: https://github.com/wiki-ai/ores-wikimedia-config/blob/master/fabfile.py

Scap3 docs: https://wikitech.wikimedia.org/wiki/Scap3 and https://doc.wikimedia.org/mw-tools-scap/index.html

Event Timeline

Talked to @demon in #wikimedia-releng. He said that (1) it looks like scap3 will be require for prod deployment and (2) the docs @ https://doc.wikimedia.org/mw-tools-scap/index.html are up to date.

I also leaned that our deployments will be run by the "deploy master". Here's a snippet of chat with @demon

tldr for how the master works is you'll ssh to tin (or mira, depending on dc). All the deployments live in /srv/deployments. You hop into your directory, do whatever git magic you need (checkout, pull, etc etc etc), then deploy does all the work
It'll go through the list of steps configured for your repo (pull code, restart services, status checks, etc) and then report back.
There's also rollback logic in case it goes BOOM halfway thru

Ladsgroup renamed this task from Move to using scap3 for deployment to Move to using scap3 for deployment for ORES service.Mar 15 2016, 6:28 PM
Ladsgroup added a project: Scap.

So @Ladsgroup and I talked a bit more on IRC. He's got a work in progress scap.cfg that's mostly right, and based off their existing fabric script. We need T129420: Support git-fat in scap3 deployment packaged and live though, they've got some ~30M pickle files.

Other than that, pretty straightforward...git pull with submodules, restart 2 services, bam we're set.

Change 280403 had a related patch set uploaded (by Ladsgroup):
ores: Scap3 deployment configurations

https://gerrit.wikimedia.org/r/280403

With merging this patch, we are good to go to prod (the whole system works in beta cluster)

Halfak closed this task as a duplicate of T124201: Setup ores on scb cluster.