Page MenuHomePhabricator

Prepare mysql hosts for stretch
Closed, ResolvedPublic

Description

Perform the tweaks necessary on puppet, packaging and other operations-related tasks to make sure mysql/mariadb hosts work well on stretch.

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+2 -4
operations/puppetproduction+24 -12
operations/puppetproduction+1 -1
operations/puppetproduction+4 -1
operations/puppetproduction+49 -6
operations/puppetproduction+16 -0
operations/puppetproduction+3 -158
operations/puppetproduction+8 -4
operations/puppetproduction+5 -5
operations/puppetproduction+5 -3
operations/puppetproduction+25 -10
operations/puppetproduction+8 -6
operations/puppetproduction+440 -292
operations/puppetproduction+16 -17
operations/puppetproduction+16 -3
operations/puppetproduction+7 -0
operations/softwaremaster+1 -0
operations/puppetproduction+2 -7
operations/puppetproduction+2 -0
Show related patches Customize query in gerrit

Event Timeline

jcrespo created this task.Jun 20 2017, 6:48 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 20 2017, 6:48 AM

Change 360167 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] install_server: reimage db2072 as stretch

https://gerrit.wikimedia.org/r/360167

Change 360172 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: move db2072 to s1 shard (enwiki)

https://gerrit.wikimedia.org/r/360172

Change 360167 merged by Jcrespo:
[operations/puppet@production] install_server: reimage db2072 as stretch

https://gerrit.wikimedia.org/r/360167

Change 360172 merged by Jcrespo:
[operations/puppet@production] mariadb: move db2072 to s1 shard (enwiki)

https://gerrit.wikimedia.org/r/360172

Change 360318 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s1.hosts: Add db2072 to s1

https://gerrit.wikimedia.org/r/360318

Marostegui moved this task from Triage to In progress on the DBA board.Jun 20 2017, 12:32 PM

Change 360318 merged by jenkins-bot:
[operations/software@master] s1.hosts: Add db2072 to s1

https://gerrit.wikimedia.org/r/360318

Change 360845 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Initial stretch support for wmf package with systemd

https://gerrit.wikimedia.org/r/360845

Change 360845 merged by Jcrespo:
[operations/puppet@production] mariadb: Initial stretch support for wmf package with systemd

https://gerrit.wikimedia.org/r/360845

Change 360859 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Remove installation of percona-xtrabackup on stretch

https://gerrit.wikimedia.org/r/360859

Change 360859 merged by Jcrespo:
[operations/puppet@production] mariadb: Remove installation of percona-xtrabackup on stretch

https://gerrit.wikimedia.org/r/360859

Change 360883 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb-core: Test systemd and stretch support

https://gerrit.wikimedia.org/r/360883

Change 360883 merged by Jcrespo:
[operations/puppet@production] mariadb-core: Test systemd and stretch support

https://gerrit.wikimedia.org/r/360883

Aside from socket, datadir, basedir configurable on hiera: T148507 we need to create user systemd customizable templates (e.g. to increase the number of max file connections).

jcrespo claimed this task.Jun 22 2017, 7:25 PM

For now, db2062 and db2072 are stretch hosts with systemd-driven mysqls.

Change 361456 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Add basedir support and change default socket location

https://gerrit.wikimedia.org/r/361456

Change 361456 merged by Jcrespo:
[operations/puppet@production] mariadb: refactor option support and move it to hiera

https://gerrit.wikimedia.org/r/361456

Change 361824 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Followup to gerrit:361456

https://gerrit.wikimedia.org/r/361824

Change 361824 merged by Jcrespo:
[operations/puppet@production] mariadb: Followup to gerrit:361456

https://gerrit.wikimedia.org/r/361824

Change 361831 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] Add basedir option to all templates; restore option on init.d

https://gerrit.wikimedia.org/r/361831

Change 361831 merged by Jcrespo:
[operations/puppet@production] Add basedir option to all templates; restore option on init.d

https://gerrit.wikimedia.org/r/361831

Change 361836 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Change init.d template variable to avoid defaults changes

https://gerrit.wikimedia.org/r/361836

Change 361836 merged by Jcrespo:
[operations/puppet@production] mariadb: Change init.d template variable to avoid defaults changes

https://gerrit.wikimedia.org/r/361836

Change 361838 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Service followup to gerrit:361836

https://gerrit.wikimedia.org/r/361838

Change 361838 merged by Jcrespo:
[operations/puppet@production] mariadb: Service followup to gerrit:361836

https://gerrit.wikimedia.org/r/361838

Change 361841 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] labsdb-replica: Change default basedir to the 10.1 package

https://gerrit.wikimedia.org/r/361841

Change 361841 merged by Jcrespo:
[operations/puppet@production] labsdb-replica: Change default basedir to the 10.1 package

https://gerrit.wikimedia.org/r/361841

Change 361846 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Avoid the usage of undef or '' due to its falsey nature

https://gerrit.wikimedia.org/r/361846

Change 361846 merged by Jcrespo:
[operations/puppet@production] mariadb: patch mariadb.service to include the basedir

https://gerrit.wikimedia.org/r/361846

Change 361857 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mysql-hiera: Add hosts for codfw:s2, which were missing

https://gerrit.wikimedia.org/r/361857

Change 361857 merged by Jcrespo:
[operations/puppet@production] mysql-hiera: Add hosts for codfw:s2, which were missing

https://gerrit.wikimedia.org/r/361857

With the new puppet refactoring, hosts just work- that doesn't mean the puppet structure is ideal- we need to change many things such as multi-instance support and fix the hiera bits, but the basic support for core is already there for stretch. The last bit missing here is to allow for custom systemd subunits mostly to set the ulimits for max allowed open files, which has to be customized for s3 hosts, dbstores, labs and other large databases.

Change 362156 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: handle service for systemd -autostart and overrides

https://gerrit.wikimedia.org/r/362156

Change 362204 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Set default limits for systemd core databases

https://gerrit.wikimedia.org/r/362204

Change 362156 merged by Jcrespo:
[operations/puppet@production] mariadb: handle service for systemd -autostart and overrides

https://gerrit.wikimedia.org/r/362156

Change 362204 merged by Jcrespo:
[operations/puppet@production] mariadb: Set default limits for systemd core databases

https://gerrit.wikimedia.org/r/362204

Change 362353 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Followup to Set default limits for systemd core databases

https://gerrit.wikimedia.org/r/362353

Change 362353 merged by Jcrespo:
[operations/puppet@production] mariadb: Followup to Set default limits for systemd core databases

https://gerrit.wikimedia.org/r/362353

jcrespo closed this task as Resolved.EditedJun 30 2017, 11:00 AM

This is in no way a closed issue, but the initial scope is covered- pending tidying up puppet and hiera code. But the support is working, at least as good as for trusty and jessie on the module. misc services will have to transition on the role slowly.

Change 364466 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Fix default package installation for stretch

https://gerrit.wikimedia.org/r/364466

Change 364466 merged by Jcrespo:
[operations/puppet@production] mariadb: Fix default package installation for stretch

https://gerrit.wikimedia.org/r/364466

Change 364671 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Delete override- changed default on package

https://gerrit.wikimedia.org/r/364671

Change 364671 merged by Jcrespo:
[operations/puppet@production] mariadb: Delete override- changed default on package

https://gerrit.wikimedia.org/r/364671

Marostegui closed subtask Restricted Task as Resolved.Oct 4 2018, 4:09 PM