Page MenuHomePhabricator
Create Task
Maniphest T86633

Unify ::production / ::beta roles for *oid
Closed, ResolvedPublic
Actions

Description

Parsoid, citoid, mathoid, cxserver, apertium all seem to have ::beta and ::production roles that are a lot of duplicated code except for some jenkins access.

Unify them into just a single role + an additional role for jenkins access.

Details

SubjectRepoBranchLines +/-
Parsoid: clean up the manifests and filesoperations/puppetproduction+1 -395
parsoid: Remove parsoid beta roleoperations/puppetproduction+15 -126
mathoid: Unify mathoid production and beta rolesoperations/puppetproduction+8 -49
citoid: Unify production and beta rolesoperations/puppetproduction+8 -15
cxserver: Unify production and beta rolesoperations/puppetproduction+17 -54
apertium: Unify production and beta rolesoperations/puppetproduction+10 -33
Customize query in gerrit

Related Objects
Search...

Event Timeline

yuvipanda created this task.Jan 13 2015, 9:59 AM
yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added projects: acl*sre-team, Beta-Cluster-Infrastructure.
yuvipanda subscribed.
gerritbot added a project: Patch-For-Review.Jan 13 2015, 10:00 AM

Change 184586 had a related patch set uploaded (by Yuvipanda):
apertium: Unify production and beta roles

https://gerrit.wikimedia.org/r/184586

Patch-For-Review

gerritbot added a comment.Jan 13 2015, 10:06 AM

Change 184590 had a related patch set uploaded (by Yuvipanda):
cxserver: Unify production and beta roles

https://gerrit.wikimedia.org/r/184590

Patch-For-Review

yuvipanda added a comment.Jan 13 2015, 10:09 AM

Note that hiera changes for deployment-prep must be made before any of
these get merged.

gerritbot added a comment.Jan 13 2015, 11:24 AM

Change 184605 had a related patch set uploaded (by Yuvipanda):
citoid: Unify production and beta roles

https://gerrit.wikimedia.org/r/184605

Patch-For-Review

greg triaged this task as Medium priority.Jan 13 2015, 4:28 PM
greg moved this task from To Triage to In-progress on the Beta-Cluster-Infrastructure board.
gerritbot added a comment.Jan 14 2015, 12:53 PM

Change 184884 had a related patch set uploaded (by Yuvipanda):
Unify mathoid production and beta roles

https://gerrit.wikimedia.org/r/184884

Patch-For-Review

gerritbot subscribed.Jan 14 2015, 3:26 PM

Change 184586 merged by Yuvipanda:
apertium: Unify production and beta roles

https://gerrit.wikimedia.org/r/184586

gerritbot added a comment.Jan 14 2015, 3:33 PM

Change 184590 merged by Yuvipanda:
cxserver: Unify production and beta roles

https://gerrit.wikimedia.org/r/184590

gerritbot added a comment.Jan 14 2015, 3:49 PM

Change 184605 merged by Alexandros Kosiaris:
citoid: Unify production and beta roles

https://gerrit.wikimedia.org/r/184605

gerritbot added a comment.Jan 14 2015, 3:55 PM

Change 184884 merged by Yuvipanda:
mathoid: Unify mathoid production and beta roles

https://gerrit.wikimedia.org/r/184884

yuvipanda mentioned this in rOPUPf4de4071d587: citoid: Unify production and beta roles.Jan 14 2015, 4:22 PM
yuvipanda mentioned this in rOPUPe422d3fe994f: apertium: Unify production and beta roles.
yuvipanda mentioned this in rOPUP285181034bca: cxserver: Unify production and beta roles.
yuvipanda mentioned this in rOPUP21846bf86b01: mathoid: Unify mathoid production and beta roles.
yuvipanda added a parent task: T87220: Minimize infrastructure differences between Beta Cluster and production.Jan 20 2015, 8:22 AM
yuvipanda removed a parent task: T87220: Minimize infrastructure differences between Beta Cluster and production.Jan 20 2015, 8:25 AM
ArielGlenn subscribed.Feb 23 2015, 3:22 PM

is parsoid still left on the to do list?

ArielGlenn removed a project: Patch-For-Review.Feb 23 2015, 3:22 PM
ArielGlenn set Security to None.
yuvipanda added a comment.Feb 26 2015, 11:26 AM

@ArielGlenn: Yes, parsoid is the only one left.

yuvipanda added a subscriber: • GWicke.Feb 26 2015, 11:28 AM

the parsoid code seems a fair bit more hairy - is it being deployed via different methods in prod vs labs (trebuchet vs jenkins) exclusively? I'm not fully sure. @GWicke thoughts?

yuvipanda added a comment.Feb 26 2015, 11:29 AM

Also lots of /var/lib vs /srv issues.

gerritbot added a comment.Feb 26 2015, 11:40 AM

Change 193082 had a related patch set uploaded (by Yuvipanda):
parsoid: Attempt to unify prod and beta roles

https://gerrit.wikimedia.org/r/193082

gerritbot added a project: Patch-For-Review.Feb 26 2015, 11:40 AM
yuvipanda added a comment.Feb 26 2015, 12:35 PM

https://gerrit.wikimedia.org/r/#/c/193082/ has basically killed the current beta role and just picked up the prod role, and testing on deployment-parsoid01-test seems to work fine, but I *think* it is configured to use prod. I see that beta used to have /srv/deployment/parsoid/deploy/conf/wmf/betalabs.localsettings.js as its settings file but I have no idea what puts it there / where it comes from.

yuvipanda added a subscriber: hashar.Feb 26 2015, 12:37 PM

And https://github.com/search?q=betalabs.localsettings.js&type=Code&utf8=%E2%9C%93 tells me it isn't really mentioned anywhere other than the role. Is this file hand maintained? @hashar / @GWicke?

• marcoil subscribed.Feb 26 2015, 4:49 PM
In T86633#1069583, @yuvipanda wrote:

https://gerrit.wikimedia.org/r/#/c/193082/ has basically killed the current beta role and just picked up the prod role, and testing on deployment-parsoid01-test seems to work fine, but I *think* it is configured to use prod. I see that beta used to have /srv/deployment/parsoid/deploy/conf/wmf/betalabs.localsettings.js as its settings file but I have no idea what puts it there / where it comes from.

It's kept in the parsoid/deploy repository: https://git.wikimedia.org/blob/mediawiki%2Fservices%2Fparsoid%2Fdeploy/ee3529b23fa8e56b06a5090c73ab6bbb96402b47/conf%2Fwmf%2Fbetalabs.localsettings.js

yuvipanda added a comment.Feb 26 2015, 5:36 PM

Right, so the file itself is in the parsoid deploy repository, but unsure why the deployment host actually gets the betalabs file.

• mobrovac subscribed.Mar 13 2015, 3:28 PM
• mobrovac mentioned this in T92636: Move Parsoid config into ops/puppet.Mar 13 2015, 4:11 PM
fgiunchedi assigned this task to yuvipanda.Mar 31 2015, 4:01 PM
fgiunchedi subscribed.
thcipriani subscribed.Apr 13 2015, 10:54 PM
yuvipanda removed yuvipanda as the assignee of this task.Jun 7 2015, 7:00 PM
thcipriani moved this task from In-progress to Next: Maintenance on the Beta-Cluster-Infrastructure board.Jun 15 2015, 7:09 PM
fgiunchedi added a comment.Jul 20 2015, 10:49 AM

@yuvipanda what's left on this task? also is it blocked on anything?

Restricted Application added subscribers: Luke081515, Matanya. · View Herald TranscriptJul 20 2015, 10:49 AM
Aklapper added a comment.Jul 31 2015, 3:47 PM

@yuvipanda what's left on this task? also is it blocked on anything?

akosiaris subscribed.Aug 6 2015, 1:43 PM

Parsoid is left to be done from what I see. We could couple this with moving parsoid to service::node in the long run (@GWicke, @mobrovac - can you think of any reason not to?) or just fix the current manifests (in role namespace unfortunately so it is a bit of work)

akosiaris lowered the priority of this task from Medium to Lowest.Aug 6 2015, 1:43 PM
• mobrovac added a comment.Aug 6 2015, 2:03 PM
In T86633#1514460, @akosiaris wrote:

We could couple this with moving parsoid to service::node in the long run

Yes, definitely. There's already a task for that - T90668: Replace custom server.js with service-runner, but we need to do a bit of more work to satisfy Parsoid's requirements. After the switch to service-runner, Parsoid can start using service::node as well.

or just fix the current manifests (in role namespace unfortunately so it is a bit of work)

I don't know if that's worth the trouble, honestly, given the info above.

• mobrovac added a subtask: T90668: Replace custom server.js with service-runner.Aug 6 2015, 2:04 PM
• GWicke added a comment.EditedAug 6 2015, 2:05 PM

@akosiaris, a precondition for parsoid moving is T90668: Replace custom server.js with service-runner.

Edit: Clearly I was too slow to save ;)

• mobrovac added a subtask: T101272: Standardise CXServer deployment.Aug 6 2015, 2:05 PM
gerritbot added a comment.Sep 29 2015, 11:10 AM

Change 193082 abandoned by Hashar:
parsoid: Remove parsoid beta role

Reason:
The bits in the parsoid::beta class are actually needed. When we switch the deployment of services on beta to scap3, we will be able to do the cleanup.

https://gerrit.wikimedia.org/r/193082

• mobrovac closed subtask T101272: Standardise CXServer deployment as Resolved.Dec 18 2015, 9:05 AM
akosiaris mentioned this in T135176: Migrate Parsoid cluster to Jessie / node 4.x.May 17 2016, 3:35 PM
ssastry closed subtask T90668: Replace custom server.js with service-runner as Resolved.Jul 20 2016, 6:37 PM
hashar added a comment.Jul 21 2016, 8:24 AM

There is still role::parsoid::beta left over. We probably want to audit what is left in puppet.git but afaik there is nothing left to do.

• mobrovac added a comment.Jul 21 2016, 11:44 AM

https://gerrit.wikimedia.org/r/#/c/300067/ addresses this. Will amend the commit to link it to this bug too.

gerritbot added a comment.Jul 21 2016, 11:45 AM

Change 300067 had a related patch set uploaded (by Mobrovac):
Parsoid: clean up the manifests and files

https://gerrit.wikimedia.org/r/300067

• mobrovac mentioned this in rOPUPe66a985a74d3: Parsoid: clean up the manifests and files.Jul 21 2016, 11:50 AM
Joe mentioned this in rOPUP37e1c561749f: Parsoid: clean up the manifests and files.Jul 27 2016, 7:46 AM
gerritbot added a comment.Jul 27 2016, 8:08 AM

Change 300067 merged by Giuseppe Lavagetto:
Parsoid: clean up the manifests and files

https://gerrit.wikimedia.org/r/300067

hashar closed this task as Resolved.Jul 27 2016, 9:03 AM
hashar claimed this task.

Imho there is nothing left to do. All services got transitioned :-}

hashar updated the task description. (Show Details)Jul 27 2016, 9:05 AM
This comment was removed by hashar.
hashar updated the task description. (Show Details)Jul 27 2016, 9:06 AM
• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:55 PM
Restricted Application added a project: Release-Engineering-Team (Kanban). · View Herald TranscriptJun 7 2017, 6:55 PM
• Phabricator_maintenance edited projects, added RelEng-Archive-FY201718-Q1; removed Release-Engineering-Team (Kanban).Sep 26 2017, 11:48 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL