Page MenuHomePhabricator
Create Task
Maniphest T86644

Remove all ::beta roles in puppet
Closed, ResolvedPublic
Actions

Description

Since we now have hiera, lots of prior code that had to be copy pasted / realm branched can now instead be just modified via puppet. This tracks all such changes.

As of May 4th 2017 in puppet.git:

$ git grep -E "class .*::beta"
modules/mediawiki/manifests/web/beta_sites.pp:class mediawiki::web::beta_sites {
modules/role/manifests/beta/availability_collector.pp:class role::beta::availability_collector {
modules/role/manifests/beta/deploymentserver.pp:class role::beta::deploymentserver {
modules/role/manifests/beta/mediawiki.pp:class role::beta::mediawiki {
modules/role/manifests/beta/puppetmaster.pp:class role::beta::puppetmaster {
modules/role/manifests/elasticsearch/beta.pp:class role::elasticsearch::beta {
modules/role/manifests/mariadb/beta.pp:class role::mariadb::beta {
modules/role/manifests/prometheus/beta.pp:class role::prometheus::beta {

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

yuvipanda created this task.Jan 13 2015, 12:44 PM
yuvipanda raised the priority of this task from to Needs Triage.
yuvipanda updated the task description. (Show Details)
yuvipanda added projects: Puppet, Beta-Cluster-Infrastructure, Tracking-Neverending.
yuvipanda added subtasks: T86633: Unify ::production / ::beta roles for *oid, T86642: Kill role::logstash::beta.
yuvipanda added subscribers: yuvipanda, Joe.
yuvipanda closed subtask T86645: Kill role::syslog::centralserver::beta as Resolved.Jan 13 2015, 1:07 PM
yuvipanda added subscribers: greg, akosiaris, hashar.Jan 13 2015, 1:22 PM
yuvipanda closed subtask T86642: Kill role::logstash::beta as Resolved.Jan 14 2015, 9:05 AM
yuvipanda added a parent task: T86885: Unify labs and prod roles for role::deployment::deployment_servers.Jan 15 2015, 8:05 AM
yuvipanda removed a parent task: T86885: Unify labs and prod roles for role::deployment::deployment_servers.Jan 15 2015, 8:36 AM
yuvipanda added a subtask: T86885: Unify labs and prod roles for role::deployment::deployment_servers.
yuvipanda added a parent task: T87220: Minimize infrastructure differences between Beta Cluster and production.Jan 20 2015, 8:25 AM
greg triaged this task as Medium priority.Jan 29 2015, 5:36 AM
greg moved this task from To Triage to Next: Maintenance on the Beta-Cluster-Infrastructure board.
gerritbot added a project: Patch-For-Review.Jan 31 2015, 5:32 AM
gerritbot subscribed.

Change 187885 had a related patch set uploaded (by Yuvipanda):
apertium: Remove jenkins_access class

https://gerrit.wikimedia.org/r/187885

Patch-For-Review

gerritbot added a comment.Jan 31 2015, 5:37 AM

Change 187885 merged by Yuvipanda:
apertium: Remove jenkins_access class

https://gerrit.wikimedia.org/r/187885

yuvipanda mentioned this in rOPUP089455a7a54c: apertium: Remove jenkins_access class.Jan 31 2015, 5:38 AM
yuvipanda closed subtask T86885: Unify labs and prod roles for role::deployment::deployment_servers as Resolved.Feb 2 2015, 5:59 AM
hashar reopened subtask T86885: Unify labs and prod roles for role::deployment::deployment_servers as Open.Feb 2 2015, 9:28 AM
yuvipanda closed subtask T86885: Unify labs and prod roles for role::deployment::deployment_servers as Resolved.Mar 14 2015, 7:22 AM
Liuxinyu970226 subscribed.Apr 18 2015, 1:31 AM
gerritbot added a comment.Oct 20 2015, 2:19 PM

Change 244228 had a related patch set uploaded (by Alex Monk):
Rename mediawiki::web::sites to mediawiki::web::prod_sites to make room for a new generic sites.pp

https://gerrit.wikimedia.org/r/244228

gerritbot added a comment.Oct 20 2015, 2:20 PM

Change 244237 had a related patch set uploaded (by Alex Monk):
Begin to merge production and beta apache config, starting with nonexistent.conf

https://gerrit.wikimedia.org/r/244237

gerritbot added a comment.Oct 27 2015, 4:20 PM

Change 244228 merged by Jcrespo:
Rename mediawiki::web::sites to mediawiki::web::prod_sites to make room for a new generic sites.pp

https://gerrit.wikimedia.org/r/244228

gerritbot added a comment.Jan 19 2016, 5:33 PM

Change 244237 merged by Giuseppe Lavagetto:
Begin to merge production and beta apache config, starting with nonexistent.conf

https://gerrit.wikimedia.org/r/244237

gerritbot added a comment.Jan 19 2016, 6:09 PM

Change 264978 had a related patch set uploaded (by Alex Monk):
[WIP] Move portals into generic sites.pp

https://gerrit.wikimedia.org/r/264978

gerritbot added a comment.Jan 19 2016, 6:44 PM

Change 247324 had a related patch set uploaded (by Chad):
Generate mediawiki-installation dsh group file from hiera data

https://gerrit.wikimedia.org/r/247324

gerritbot added a comment.Jan 21 2016, 5:33 PM

Change 264978 merged by Alexandros Kosiaris:
Move portals into generic sites.pp

https://gerrit.wikimedia.org/r/264978

gerritbot added a comment.Jan 21 2016, 6:48 PM

Change 265532 had a related patch set uploaded (by Alex Monk):
Move apache includes into generic sites.pp

https://gerrit.wikimedia.org/r/265532

gerritbot added a comment.Jan 22 2016, 1:56 PM

Change 265532 merged by Alexandros Kosiaris:
Move apache includes into generic sites.pp

https://gerrit.wikimedia.org/r/265532

Krenair added a subtask: T1256: Rework beta apache config.Jan 26 2016, 8:53 PM
Danny_B subscribed.Feb 14 2016, 2:49 PM
NOTE: This tracking task should be converted to Goal, any objections?
Danny_B moved this task from Tag to Transition suggested on the Tracking-Neverending board.Feb 14 2016, 2:49 PM
Aklapper added a comment.EditedFeb 14 2016, 3:57 PM
In T86644#2026667, @Danny_B wrote:
NOTE: This tracking task should be converted to Goal, any objections?

@Danny_B: I'm surprised about the "should" in that sentence - it's news to me.
I don't see much sense why it's proposed to create a project for two open tasks. Hence I'd like to object, as we cannot express the "Blocks: T87220" via a project only, plus this task is in certain columns on existing workboards.

Danny_B added a comment.Feb 14 2016, 4:15 PM
In T86644#2026956, @Aklapper wrote:
In T86644#2026667, @Danny_B wrote:
NOTE: This tracking task should be converted to Goal, any objections?

@Danny_B: I'm surprised about the "should" in that sentence - it's news to me.

"Projects are favored over 'tracking tasks' with dependencies."

I don't see much sense why it's proposed to create a project for two open tasks.

Currently.

Hence I'd like to object, as we cannot express the "Blocks: T87220" via a project only,

Valid point, if the goal creation was 1:1 to this task. But since it's a part of bigger task/goal, such goal should be that higher task. It does not make a sense to crumble that much.

plus this task is in certain columns on existing workboards.

Patch-For-Review, Puppet and Tracking-Neverending are tags, Beta-Cluster-Infrastructure contains all blockers.

hashar added a comment.Feb 15 2016, 8:38 AM

@Danny_B please bring the discussion on some list. Tracking-Neverending vs Goal has nothing to do with the task at hand Remove all ::beta roles in puppet. Thanks!

greg added a comment.Feb 16 2016, 4:36 PM

Please do not change to a goal. The reverse dependencies, as @Aklapper pointed out, are very important.

Dzahn subscribed.Mar 3 2016, 5:51 AM

@Joe your opinion on https://gerrit.wikimedia.org/r/#/c/247324/ would be appreciated

• mmodell subscribed.Mar 3 2016, 9:08 AM
gerritbot added a comment.Jul 12 2016, 10:52 PM

Change 247324 abandoned by Chad:
Generate mediawiki-installation dsh group file from hiera data

Reason:
meh

https://gerrit.wikimedia.org/r/247324

Danny_B moved this task from Transition suggested to Should be Goal instead on the Tracking-Neverending board.Jul 13 2016, 12:32 AM
Danny_B unsubscribed.
hashar closed subtask T86633: Unify ::production / ::beta roles for *oid as Resolved.Jul 27 2016, 9:03 AM
hashar updated the task description. (Show Details)Jul 27 2016, 9:07 AM
hashar set Security to None.

As of July 27th 2016 in puppet.git:

$ find . -type f -path '*role*beta*'
./modules/role/manifests/beta/availability_collector.pp
./modules/role/manifests/beta/bastion.pp
./modules/role/manifests/beta/puppetmaster.pp
./modules/role/manifests/beta/trebuchet_testing.pp
./modules/role/manifests/beta/uploadservice.pp
hashar removed a project: Patch-For-Review.Jul 27 2016, 9:08 AM
greg moved this task from Next: Maintenance to Epics / Tracking on the Beta-Cluster-Infrastructure board.Aug 5 2016, 8:49 PM
• Phabricator_maintenance added a project: Goal.Aug 13 2016, 8:41 PM
• Phabricator_maintenance removed a project: Tracking-Neverending.Aug 13 2016, 9:57 PM
Krenair subscribed.Nov 20 2016, 10:06 PM

I've been doing some of this recently:
https://gerrit.wikimedia.org/r/#/c/322403/ - role::beta::uploadservice
https://gerrit.wikimedia.org/r/#/c/322404/ - role::beta::bastion
https://gerrit.wikimedia.org/r/#/c/322405/ - role::beta::trebuchet_testing
https://gerrit.wikimedia.org/r/#/c/322406/ - beta::config
https://gerrit.wikimedia.org/r/#/c/322407/ - beta::deployaccess

gerritbot added a comment.Nov 21 2016, 1:29 PM

Change 322403 had a related patch set uploaded (by Hashar):
Remove role::beta::uploadservice

https://gerrit.wikimedia.org/r/322403

gerritbot added a project: Patch-For-Review.Nov 21 2016, 1:29 PM

Change 322406 had a related patch set uploaded (by Hashar):
Remove beta::config

https://gerrit.wikimedia.org/r/322406

gerritbot added a comment.Nov 21 2016, 1:29 PM

Change 322407 had a related patch set uploaded (by Hashar):
Remove beta::deployaccess

https://gerrit.wikimedia.org/r/322407

gerritbot added a comment.Nov 21 2016, 1:29 PM

Change 322404 had a related patch set uploaded (by Hashar):
Remove role::beta::bastion

https://gerrit.wikimedia.org/r/322404

gerritbot added a comment.Nov 21 2016, 1:29 PM

Change 322405 had a related patch set uploaded (by Hashar):
Remove role::beta::trebuchet_testing

https://gerrit.wikimedia.org/r/322405

gerritbot added a comment.Nov 21 2016, 7:38 PM

Change 322403 merged by Filippo Giunchedi:
Remove role::beta::uploadservice

https://gerrit.wikimedia.org/r/322403

gerritbot added a comment.Nov 22 2016, 9:14 PM

Change 322404 merged by Filippo Giunchedi:
Remove role::beta::bastion

https://gerrit.wikimedia.org/r/322404

gerritbot added a comment.Nov 22 2016, 9:15 PM

Change 322405 merged by Filippo Giunchedi:
Remove role::beta::trebuchet_testing

https://gerrit.wikimedia.org/r/322405

gerritbot added a comment.Nov 23 2016, 2:28 AM

Change 322406 merged by Filippo Giunchedi:
Remove beta::config

https://gerrit.wikimedia.org/r/322406

gerritbot added a comment.Nov 23 2016, 11:37 PM

Change 322407 merged by Dzahn:
Remove beta::deployaccess

https://gerrit.wikimedia.org/r/322407

Krenair added a comment.Nov 24 2016, 1:39 AM

So now we have:

alex@alex-laptop:~/Development/Wikimedia/Operations-Puppet (production)$ git grep -E "class .*::beta"
modules/mediawiki/manifests/web/beta_sites.pp:class mediawiki::web::beta_sites {
modules/role/manifests/beta/availability_collector.pp:class role::beta::availability_collector {
modules/role/manifests/beta/deploymentserver.pp:class role::beta::deploymentserver {
modules/role/manifests/beta/mediawiki.pp:class role::beta::mediawiki {
modules/role/manifests/beta/puppetmaster.pp:class role::beta::puppetmaster {
modules/role/manifests/mariadb.pp:class role::mariadb::beta {
modules/role/manifests/prometheus/beta.pp:class role::prometheus::beta {

mediawiki::web::beta_sites is being taken care of in T1256.
role::beta::availability_collector is applied on deployment-cache-(text|upload)04. For the rest it's all fairly obvious where they would be applied.
The role::prometheus::beta one is fairly new - https://gerrit.wikimedia.org/r/#/c/313816/

• demon subscribed.Nov 24 2016, 5:23 AM

Just came across beta::saltmaster::tools. It doesn't even appear used...

Krenair added a comment.Nov 24 2016, 5:26 AM

It's used on deployment-salt02: https://wikitech.wikimedia.org/wiki/Hiera:Deployment-prep/host/deployment-salt02

• demon added a comment.Nov 24 2016, 5:27 AM

Do we even need salt in beta? ;-)

Krenair added a comment.Nov 24 2016, 5:31 AM

That's way out of the scope of this task, but yes, I have used it many times

hashar closed this task as Resolved.Nov 24 2016, 8:24 AM
hashar claimed this task.
In T86644#2820032, @demon wrote:

Do we even need salt in beta? ;-)

When you get 40+ instances. Yes definitely.

Liuxinyu970226 unsubscribed.Nov 24 2016, 8:28 AM
Krenair reopened this task as Open.Nov 24 2016, 8:28 AM
Krenair removed hashar as the assignee of this task.

This task is not complete. I listed 7 existing cases above.

hashar added a comment.Nov 24 2016, 8:34 AM

Sorry made a mistake when replying :D

hashar updated the task description. (Show Details)May 4 2017, 7:40 AM
Dzahn unsubscribed.May 4 2017, 5:23 PM
• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:55 PM
hashar closed this task as Resolved.Oct 23 2017, 11:03 PM
hashar assigned this task to yuvipanda.
hashar added a subscriber: yuvipanda.

It has been nicely cleaned up by @yuvipanda
There are still some classes left but not much we can implement solely based on hiera.

Krenair closed subtask T1256: Rework beta apache config as Resolved.Oct 22 2018, 2:17 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits