Page MenuHomePhabricator

Unify ::production / ::beta roles for *oid
Closed, ResolvedPublic

Description

Parsoid, citoid, mathoid, cxserver, apertium all seem to have ::beta and ::production roles that are a lot of duplicated code except for some jenkins access.

Unify them into just a single role + an additional role for jenkins access.

Event Timeline

yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added a subscriber: yuvipanda.

Change 184586 had a related patch set uploaded (by Yuvipanda):
apertium: Unify production and beta roles

https://gerrit.wikimedia.org/r/184586

Patch-For-Review

Change 184590 had a related patch set uploaded (by Yuvipanda):
cxserver: Unify production and beta roles

https://gerrit.wikimedia.org/r/184590

Patch-For-Review

Note that hiera changes for deployment-prep must be made before any of
these get merged.

Change 184605 had a related patch set uploaded (by Yuvipanda):
citoid: Unify production and beta roles

https://gerrit.wikimedia.org/r/184605

Patch-For-Review

greg triaged this task as Normal priority.Jan 13 2015, 4:28 PM
greg moved this task from To Triage to In-progress on the Beta-Cluster-Infrastructure board.

Change 184884 had a related patch set uploaded (by Yuvipanda):
Unify mathoid production and beta roles

https://gerrit.wikimedia.org/r/184884

Patch-For-Review

Change 184586 merged by Yuvipanda:
apertium: Unify production and beta roles

https://gerrit.wikimedia.org/r/184586

Change 184590 merged by Yuvipanda:
cxserver: Unify production and beta roles

https://gerrit.wikimedia.org/r/184590

Change 184605 merged by Alexandros Kosiaris:
citoid: Unify production and beta roles

https://gerrit.wikimedia.org/r/184605

Change 184884 merged by Yuvipanda:
mathoid: Unify mathoid production and beta roles

https://gerrit.wikimedia.org/r/184884

is parsoid still left on the to do list?

ArielGlenn set Security to None.

@ArielGlenn: Yes, parsoid is the only one left.

the parsoid code seems a fair bit more hairy - is it being deployed via different methods in prod vs labs (trebuchet vs jenkins) exclusively? I'm not fully sure. @GWicke thoughts?

Also lots of /var/lib vs /srv issues.

Change 193082 had a related patch set uploaded (by Yuvipanda):
parsoid: Attempt to unify prod and beta roles

https://gerrit.wikimedia.org/r/193082

https://gerrit.wikimedia.org/r/#/c/193082/ has basically killed the current beta role and just picked up the prod role, and testing on deployment-parsoid01-test seems to work fine, but I *think* it is configured to use prod. I see that beta used to have /srv/deployment/parsoid/deploy/conf/wmf/betalabs.localsettings.js as its settings file but I have no idea what puts it there / where it comes from.

And https://github.com/search?q=betalabs.localsettings.js&type=Code&utf8=%E2%9C%93 tells me it isn't really mentioned anywhere other than the role. Is this file hand maintained? @hashar / @GWicke?

https://gerrit.wikimedia.org/r/#/c/193082/ has basically killed the current beta role and just picked up the prod role, and testing on deployment-parsoid01-test seems to work fine, but I *think* it is configured to use prod. I see that beta used to have /srv/deployment/parsoid/deploy/conf/wmf/betalabs.localsettings.js as its settings file but I have no idea what puts it there / where it comes from.

It's kept in the parsoid/deploy repository: https://git.wikimedia.org/blob/mediawiki%2Fservices%2Fparsoid%2Fdeploy/ee3529b23fa8e56b06a5090c73ab6bbb96402b47/conf%2Fwmf%2Fbetalabs.localsettings.js

Right, so the file itself is in the parsoid deploy repository, but unsure why the deployment host actually gets the betalabs file.

fgiunchedi added a subscriber: fgiunchedi.
yuvipanda removed yuvipanda as the assignee of this task.Jun 7 2015, 7:00 PM

@yuvipanda what's left on this task? also is it blocked on anything?

Restricted Application added subscribers: Luke081515, Matanya. · View Herald TranscriptJul 20 2015, 10:49 AM

@yuvipanda what's left on this task? also is it blocked on anything?

Parsoid is left to be done from what I see. We could couple this with moving parsoid to service::node in the long run (@GWicke, @mobrovac - can you think of any reason not to?) or just fix the current manifests (in role namespace unfortunately so it is a bit of work)

akosiaris lowered the priority of this task from Normal to Lowest.Aug 6 2015, 1:43 PM

We could couple this with moving parsoid to service::node in the long run

Yes, definitely. There's already a task for that - T90668: Replace custom server.js with service-runner, but we need to do a bit of more work to satisfy Parsoid's requirements. After the switch to service-runner, Parsoid can start using service::node as well.

or just fix the current manifests (in role namespace unfortunately so it is a bit of work)

I don't know if that's worth the trouble, honestly, given the info above.

GWicke added a comment.EditedAug 6 2015, 2:05 PM

@akosiaris, a precondition for parsoid moving is T90668: Replace custom server.js with service-runner.

Edit: Clearly I was too slow to save ;)

Change 193082 abandoned by Hashar:
parsoid: Remove parsoid beta role

Reason:
The bits in the parsoid::beta class are actually needed. When we switch the deployment of services on beta to scap3, we will be able to do the cleanup.

https://gerrit.wikimedia.org/r/193082

There is still role::parsoid::beta left over. We probably want to audit what is left in puppet.git but afaik there is nothing left to do.

https://gerrit.wikimedia.org/r/#/c/300067/ addresses this. Will amend the commit to link it to this bug too.

Change 300067 had a related patch set uploaded (by Mobrovac):
Parsoid: clean up the manifests and files

https://gerrit.wikimedia.org/r/300067

Change 300067 merged by Giuseppe Lavagetto:
Parsoid: clean up the manifests and files

https://gerrit.wikimedia.org/r/300067

hashar closed this task as Resolved.Jul 27 2016, 9:03 AM
hashar claimed this task.

Imho there is nothing left to do. All services got transitioned :-}

hashar updated the task description. (Show Details)Jul 27 2016, 9:05 AM
This comment was removed by hashar.
hashar updated the task description. (Show Details)Jul 27 2016, 9:06 AM