Page MenuHomePhabricator

Set up roles for new wiki replicas layout
Closed, ResolvedPublic

Description

We likely need to base it on role(mariadb::dbstore_multiinstance), and when adding the specific requirements from role(labs::db::wikireplica_web), role(labs::db::wikireplica_analytics) and role(labs::db::wikireplica_analytics::dedicated), this is a good opportunity to remove yet another use of the word "labs".

Related Objects

Event Timeline

Bstorm created this task.Aug 19 2020, 6:39 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 19 2020, 6:39 PM
Bstorm triaged this task as Medium priority.Aug 19 2020, 6:39 PM
Bstorm moved this task from Inbox to Soon! on the cloud-services-team (Kanban) board.
Bstorm moved this task from Backlog to Wiki replicas on the Data-Services board.

I'm thinking something like role(mariadb::wikireplica_multiinstance::web) and role(mariadb::wikireplica_multiinstance::analytics) to be more consistent with other DB roles, and I could try to move the WMCS/wiki-replicas specific stuff over to profiles under profile::wmcs::wikireplicas. That would start getting things out of role::labs and profile::labs and put those namespaces to bed.

I could start getting those profiles ready if that works for you @Marostegui .

Obviously the existing views scripts will need changes to work with multi-instance, but I'll get there.

Change 621618 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] wikireplicas: refactor to eliminate confusing "labsdb" naming

https://gerrit.wikimedia.org/r/621618

Change 621777 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[labs/private@master] wikireplicas: add new location for these fake passwords

https://gerrit.wikimedia.org/r/621777

Change 621777 merged by Bstorm:
[labs/private@master] wikireplicas: add new location for these fake passwords

https://gerrit.wikimedia.org/r/621777

I could start getting those profiles ready if that works for you @Marostegui .

Absolutely! Thank you very much

Change 621618 merged by Bstorm:
[operations/puppet@production] wikireplicas: refactor to eliminate confusing "labsdb" naming

https://gerrit.wikimedia.org/r/621618

Change 622387 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] wikireplicas: fix path typo for the heartbeat-views file

https://gerrit.wikimedia.org/r/622387

Change 622387 merged by Bstorm:
[operations/puppet@production] wikireplicas: fix path typo for the heartbeat-views file

https://gerrit.wikimedia.org/r/622387

Ok, that sorts out the name spacing. Now to add something to incorporate a multi-instance option.

Change 622444 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] wikireplicas: create multiinstance roles and profiles

https://gerrit.wikimedia.org/r/622444

Change 623623 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] wikireplicas: test removing deprecated passwords module from role

https://gerrit.wikimedia.org/r/623623

Change 623623 merged by Bstorm:
[operations/puppet@production] wikireplicas: test removing deprecated passwords module from role

https://gerrit.wikimedia.org/r/623623

jbond added a subscriber: jbond.Sep 4 2020, 2:04 PM

Change 622444 merged by Bstorm:
[operations/puppet@production] wikireplicas: create multiinstance roles and profiles

https://gerrit.wikimedia.org/r/622444

@Bstorm how's this work progressing? Should I assign for now the mariadb::dbstore_multiinstance for now (when the hosts get installed) so we can get the data copied and mysql up while you keep refining the final role?

Bstorm added a comment.Oct 9 2020, 4:25 PM

@Bstorm how's this work progressing? Should I assign for now the mariadb::dbstore_multiinstance for now (when the hosts get installed) so we can get the data copied and mysql up while you keep refining the final role?

I'm distracted with the work on Quarry at the moment to get it ready, but I feel good about where I left wmcs::db::wikireplicas::web_multiinstance and wmcs::db::wikireplicas::analytics_multiinstance so go ahead and try using them if you can. I'll help fix it if it's not actually good 😁

If we end up with a proxy somewhere that we can put 8 IP addresses on, I can really try out the suggested proxy role when there's things to connect to. The proxy setup is the part that I'm a bit less sure of. I will be circling back to view scripts shortly, but that doesn't block any of your stuff I don't think @Marostegui

@Bstorm how's this work progressing? Should I assign for now the mariadb::dbstore_multiinstance for now (when the hosts get installed) so we can get the data copied and mysql up while you keep refining the final role?

I'm distracted with the work on Quarry at the moment to get it ready, but I feel good about where I left wmcs::db::wikireplicas::web_multiinstance and wmcs::db::wikireplicas::analytics_multiinstance so go ahead and try using them if you can. I'll help fix it if it's not actually good 😁

Will do thanks. We still need to decide on T265135: wikireplicas: Define MW sections per host

If we end up with a proxy somewhere that we can put 8 IP addresses on, I can really try out the suggested proxy role when there's things to connect to. The proxy setup is the part that I'm a bit less sure of. I will be circling back to view scripts shortly, but that doesn't block any of your stuff I don't think @Marostegui

Nope, that won't be blocking me. Thank you!

Marostegui raised the priority of this task from Medium to High.Nov 3 2020, 6:40 AM

@Bstorm new clouddb hosts are racked and online. I have checked the HW and it looks good, they are ready to get the data (T267090)
Can you set them up on puppet (you might want to also check the data we're going to put on them before: T265135#6598952 ).
Once you are happy with puppet running there, I can start the data transfer.

Note: it is ok to set up one or two host at the time, I don't need all of them with puppet to start transferring data :-)

Change 639815 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] wikireplicas: set up site.pp and hosts hiera for new servers

https://gerrit.wikimedia.org/r/639815

Change 639815 merged by Marostegui:
[operations/puppet@production] wikireplicas: set up site.pp and hosts hiera for new servers

https://gerrit.wikimedia.org/r/639815

Marostegui closed this task as Resolved.Nov 17 2020, 3:39 PM
Marostegui assigned this task to Bstorm.

Going to close this as fixed as it looks like they work fine for clouddb1013 and clouddb1017.
If something arises I will reopen!

Thank you so much Brooke!