Page MenuHomePhabricator

Decouple roles from mariadb.pp into their own file
Closed, ResolvedPublic

Description

There has been some discussions and some initial attempts (https://gerrit.wikimedia.org/r/#/c/315343/) to split all the roles under manifests/role/mariadb.pp into small files under modules/role/manifests/mariadb/

This is a huge task but it will bring some good benefits like flexibility and a lot-easier-to-maintain structure in puppet.

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+206 -238
operations/puppetproduction+57 -57
operations/puppetproduction+68 -66
operations/puppetproduction+17 -19
operations/puppetproduction+17 -18
operations/puppetproduction+43 -43
operations/puppetproduction+84 -85
operations/puppetproduction+28 -28
operations/puppetproduction+156 -156
operations/puppetproduction+46 -45
operations/puppetproduction+54 -54
operations/puppetproduction+51 -50
operations/puppetproduction+57 -48
operations/puppetproduction+94 -89
operations/puppetproduction+267 -50
operations/puppetproduction+21 -22
operations/puppetproduction+21 -21
operations/puppetproduction+139 -133
Show related patches Customize query in gerrit

Event Timeline

Change 324915 had a related patch set uploaded (by Jcrespo):
[WIP] Starting to cleanup mariadb templating structure

https://gerrit.wikimedia.org/r/324915

Change 327536 had a related patch set uploaded (by Jcrespo):
Move role::mariadb::client to a separate file

https://gerrit.wikimedia.org/r/327536

Change 327536 abandoned by Jcrespo:
Move role::mariadb::client to a separate file

Reason:
It's just easier to create a new patch, as this is just moving files around.

https://gerrit.wikimedia.org/r/327536

Change 327556 had a related patch set uploaded (by Jcrespo):
mariadb: Move role::mariadb::client to a separate file

https://gerrit.wikimedia.org/r/327556

Change 327556 merged by Jcrespo:
mariadb: Move role::mariadb::client to a separate file

https://gerrit.wikimedia.org/r/327556

Can you see^, @Dzahn? Slowly, but steadily.

Change 332228 had a related patch set uploaded (by Marostegui):
mariadb: Split dbstore role classes

https://gerrit.wikimedia.org/r/332228

Change 332228 merged by Marostegui:
mariadb: Split dbstore role classes

https://gerrit.wikimedia.org/r/332228

I have pushed the change that decouples dbstore from mariadb.pp: https://gerrit.wikimedia.org/r/332228
It split it into dbstore (runs tokuDB) and dbstore2 (runs InnoDB)

I have added 2 special comment lines to make it ignore the lint-warnings for the remaining ones until this is resolved - https://gerrit.wikimedia.org/r/#/c/336731/

That way we could remove the global exception for this as it is the last file that has them. https://gerrit.wikimedia.org/r/#/c/336735/

Change 341558 had a related patch set uploaded (by jynus):
[operations/puppet] mariadb: separate sanitarium2 role to a separate file

https://gerrit.wikimedia.org/r/341558

Change 341557 had a related patch set uploaded (by jynus):
[operations/puppet] mariadb: Separate sanitarium role && monitor it on prometheus

https://gerrit.wikimedia.org/r/341557

Change 341557 merged by Jcrespo:
[operations/puppet] mariadb: Separate sanitarium role && monitor it on prometheus

https://gerrit.wikimedia.org/r/341557

Change 341558 merged by Jcrespo:
[operations/puppet] mariadb: separate sanitarium2 role to a separate file

https://gerrit.wikimedia.org/r/341558

Change 341565 had a related patch set uploaded (by jynus):
[operations/puppet] mariadb: Decouple parsercache role to a separate file

https://gerrit.wikimedia.org/r/341565

Change 341569 had a related patch set uploaded (by jynus):
[operations/puppet] mariadb: Decouple beta role to a separate file

https://gerrit.wikimedia.org/r/341569

Change 341565 merged by Jcrespo:
[operations/puppet] mariadb: Decouple parsercache role to a separate file

https://gerrit.wikimedia.org/r/341565

Change 341825 had a related patch set uploaded (by Jcrespo):
[operations/puppet] mariadb: Decouple mariadb::misc role to a separate file

https://gerrit.wikimedia.org/r/341825

Change 341825 merged by Jcrespo:
[operations/puppet] mariadb: Decouple mariadb::misc role to a separate file

https://gerrit.wikimedia.org/r/341825

Change 341569 merged by Jcrespo:
[operations/puppet] mariadb: Decouple beta role to a separate file

https://gerrit.wikimedia.org/r/341569

Change 342014 had a related patch set uploaded (by Jcrespo):
[operations/puppet] mariadb: Decouple misc role classes into separate files

https://gerrit.wikimedia.org/r/342014

Change 342014 merged by Jcrespo:
[operations/puppet] mariadb: Decouple misc role classes into separate files

https://gerrit.wikimedia.org/r/342014

Change 342050 had a related patch set uploaded (by Jcrespo):
[operations/puppet] mariadb: Decouple tendril mariadb role to a separate file

https://gerrit.wikimedia.org/r/342050

Change 342054 had a related patch set uploaded (by Jcrespo):
[operations/puppet] mariadb: Decouple core (mediawiki) role on a separate file

https://gerrit.wikimedia.org/r/342054

Change 342050 merged by Jcrespo:
[operations/puppet] mariadb: Decouple tendril mariadb role to a separate file

https://gerrit.wikimedia.org/r/342050

Change 342060 had a related patch set uploaded (by Jcrespo):
[operations/puppet] mariadb: Decouple labsdb mariadb role (deprecated) to a separate file

https://gerrit.wikimedia.org/r/342060

Change 342064 had a related patch set uploaded (by Jcrespo):
[operations/puppet] mariadb: Decouple mariadb wikitech role to a separate file

https://gerrit.wikimedia.org/r/342064

Change 342054 merged by Jcrespo:
[operations/puppet] mariadb: Decouple core (mediawiki) role on a separate file

https://gerrit.wikimedia.org/r/342054

Change 342064 merged by Jcrespo:
[operations/puppet] mariadb: Decouple mariadb wikitech role to a separate file

https://gerrit.wikimedia.org/r/342064

Change 342188 had a related patch set uploaded (by Jcrespo):
[operations/puppet] mariadb: Decouple maintenance mariadb role to a separate file

https://gerrit.wikimedia.org/r/342188

Change 342188 merged by Jcrespo:
[operations/puppet] mariadb: Decouple maintenance mariadb role to a separate file

https://gerrit.wikimedia.org/r/342188

Change 342217 had a related patch set uploaded (by Jcrespo):
[operations/puppet] mariadb: Decouple ferm mariadb common class into a separate file

https://gerrit.wikimedia.org/r/342217

Change 342217 merged by Jcrespo:
[operations/puppet] mariadb: Decouple ferm mariadb common class into a separate file

https://gerrit.wikimedia.org/r/342217

Change 342436 had a related patch set uploaded (by Jcrespo):
[operations/puppet] mariadb: Decouple db proxy role classes to separate files

https://gerrit.wikimedia.org/r/342436

Change 342436 merged by Jcrespo:
[operations/puppet] mariadb: Decouple db proxy role classes to separate files

https://gerrit.wikimedia.org/r/342436

Change 342060 merged by Jcrespo:
[operations/puppet@production] mariadb: Decouple labsdb mariadb role (deprecated) to a separate file

https://gerrit.wikimedia.org/r/342060

Change 394541 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Remove mariadb.pp and move some old roles to profiles

https://gerrit.wikimedia.org/r/394541

Change 394541 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Remove mariadb.pp and move some old roles to profiles

https://gerrit.wikimedia.org/r/394541

Change 394541 merged by Jcrespo:
[operations/puppet@production] mariadb: Remove mariadb.pp and move some old roles to profiles

https://gerrit.wikimedia.org/r/394541

jcrespo claimed this task.

Finally https://gerrit.wikimedia.org/r/394541 was deployed- but there is still many things pending to refactor.

Marostegui added a subscriber: chasemp.

Looks like @chasemp found some issues and this patch might be the cause - on labservices1001:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: {"message":"Server Error: Evaluation Error: Error while evaluating a Function Call, Could not find class ::role::mariadb::monitor::dba for labservices1001.wikimedia.org at /etc/puppet/modules/profile/manifests/openstack/base/pdns/auth/db.pp:13:5 on node labservices1001.wikimedia.org","issue_kind":"RUNTIME_ERROR","stacktrace":["Warning: The 'stacktrace' property is deprecated and will be removed in a future version of Puppet. For security reasons, stacktraces are not returned with Puppet HTTP Error responses."]}
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

The powerdns authoritative servers have database backends

This should be fully fixed now? No cloud issues?

This should be fully fixed now? No cloud issues?

Everything seems good as of now :)

Resolved, we can reopen if we see something broken.