Page MenuHomePhabricator
Create Task
Maniphest T428930

Set up deployment-db15 with Trixie and wmf-mariadb1011
Closed, ResolvedPublic
Actions

Description

In order to resolve T428910 we need new deployment-db* hosts running wmf-mariadb1011.

Flavor: g4.cores8.ram16.disk20
Security groups: default and sql

Puppet classes

role::mariadb::beta

Hiera Config

mariadb::config::basedir: /opt/wmf-mariadb1011
mariadb::package: wmf-mariadb1011
profile::mariadb::section_ports: {}

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
beta: Point remaining db11 references at deployment-db15Ahmon Dancyoperations/mediawiki-configmaster+3 -3
beta: Promote deployment-db15 to master, drop deployment-db11Ahmon Dancyoperations/mediawiki-configmaster+5 -14
beta: Fix modules/beta/files/receive_replica.shAhmon Dancyoperations/puppetproduction+3 -0
beta: Add replica provisioning scriptsAhmon Dancyoperations/puppetproduction+235 -0
beta: Replace deployment-db14 with deployment-db15Ahmon Dancyoperations/mediawiki-configmaster+9 -9
beta: Add deployment-db15 to db-labs config at weight 0Ahmon Dancyoperations/mediawiki-configmaster+9 -0
profile::mariadb::beta: Initialize system schema on fresh hostsAhmon Dancyoperations/puppetproduction+15 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

dancy created this task.Jun 11 2026, 4:05 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 11 2026, 4:05 PM
dancy triaged this task as High priority.Jun 11 2026, 4:06 PM
dancy added a parent task: T428910: Beta Cluster MariaDB is still 10.6.17, MW now requires 10.11.
dancy updated the task description. (Show Details)
dancy updated the task description. (Show Details)Jun 11 2026, 4:09 PM
dancy updated the task description. (Show Details)Jun 11 2026, 4:11 PM
dancy added a comment.EditedJun 11 2026, 4:48 PM

Current run-pupet-agent output:

Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for deployment-db15.deployment-prep.eqiad1.wikimedia.cloud
Info: Applying configuration version '(ce31ebb450) gitpuppet - beta: Add a wmf-beta-update-all timer and script'
Error: Systemd start for mariadb failed!
journalctl log for mariadb:
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: log sequence number 35867; transaction id 4
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: Loading buffer pool(s) from /srv/sqldata/ib_buffer_pool
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] Plugin 'FEEDBACK' is disabled.
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: Buffer pool(s) load completed at 260611 16:46:44
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 1 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1017: Can't find file: './mysql/' (errno: 2 "No such file or directory")
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] Server socket created on IP: '0.0.0.0', port: '3306'.
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] Server socket created on IP: '::', port: '3306'.
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.db' doesn't exist
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [ERROR] Aborting
Jun 11 16:46:44 deployment-db15 systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 16:46:44 deployment-db15 systemd[1]: mariadb.service: Failed with result 'exit-code'.
Jun 11 16:46:44 deployment-db15 systemd[1]: Failed to start mariadb.service - mariadb database server.
Jun 11 16:46:44 deployment-db15 systemd[1]: mariadb.service: Consumed 1.032s CPU time, 3.1G memory peak.
Jun 11 16:47:13 deployment-db15 systemd[1]: Starting mariadb.service - mariadb database server...
Jun 11 16:47:13 deployment-db15 (mysqld)[27848]: mariadb.service: Referenced but unset environment variable evaluates to an empty string: MYSQLD_OPTS
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] Starting MariaDB 10.11.16-MariaDB-log source revision 3218602d3100db9ce7a875511a591cddc173cc16 server_uid 4n0lQhoyEx0/YbaDH1FbSfawYUs= as process 27848
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used;  This may cause repliction to break when this server acts as a master and has its hostname changed! Please use '--log-basename=deployment-db15' or '--log-bin=deployment-db15-bin' to avoid this problem.
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [ERROR] mysqld: Plugin 'unix_socket' already installed
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [ERROR] Couldn't load plugin 'unix_socket' from 'auth_socket.so'.
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: Compressed tables use zlib 1.3.1
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: Number of transaction pools: 1
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: Using Linux native AIO
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: innodb_buffer_pool_size_max=12288m, innodb_buffer_pool_size=12288m
Jun 11 16:47:14 deployment-db15 mysqld[27848]: 2026-06-11 16:47:14 0 [Note] InnoDB: Initialized memory pressure event listener
Jun 11 16:47:14 deployment-db15 mysqld[27848]: 2026-06-11 16:47:14 0 [Note] InnoDB: Completed initialization of buffer pool
Jun 11 16:47:14 deployment-db15 mysqld[27848]: 2026-06-11 16:47:14 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
Jun 11 16:47:14 deployment-db15 mysqld[27848]: 2026-06-11 16:47:14 0 [Note] InnoDB: End of log at LSN=35867
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: 128 rollback segments are active.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: log sequence number 35867; transaction id 4
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] Plugin 'FEEDBACK' is disabled.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: Loading buffer pool(s) from /srv/sqldata/ib_buffer_pool
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 1 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1017: Can't find file: './mysql/' (errno: 2 "No such file or directory")
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: Buffer pool(s) load completed at 260611 16:47:15
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] Server socket created on IP: '0.0.0.0', port: '3306'.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] Server socket created on IP: '::', port: '3306'.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.db' doesn't exist
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [ERROR] Aborting
Jun 11 16:47:16 deployment-db15 systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 16:47:16 deployment-db15 systemd[1]: mariadb.service: Failed with result 'exit-code'.
Jun 11 16:47:16 deployment-db15 systemd[1]: Failed to start mariadb.service - mariadb database server.
Jun 11 16:47:16 deployment-db15 systemd[1]: mariadb.service: Consumed 1.782s CPU time, 3.1G memory peak.

Error: /Stage[main]/Mariadb::Service/Service[mariadb]/ensure: change from 'stopped' to 'running' failed: Systemd start for mariadb failed!
journalctl log for mariadb:
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: log sequence number 35867; transaction id 4
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: Loading buffer pool(s) from /srv/sqldata/ib_buffer_pool
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] Plugin 'FEEDBACK' is disabled.
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] InnoDB: Buffer pool(s) load completed at 260611 16:46:44
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 1 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1017: Can't find file: './mysql/' (errno: 2 "No such file or directory")
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] Server socket created on IP: '0.0.0.0', port: '3306'.
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [Note] Server socket created on IP: '::', port: '3306'.
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.db' doesn't exist
Jun 11 16:46:44 deployment-db15 mysqld[27254]: 2026-06-11 16:46:44 0 [ERROR] Aborting
Jun 11 16:46:44 deployment-db15 systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 16:46:44 deployment-db15 systemd[1]: mariadb.service: Failed with result 'exit-code'.
Jun 11 16:46:44 deployment-db15 systemd[1]: Failed to start mariadb.service - mariadb database server.
Jun 11 16:46:44 deployment-db15 systemd[1]: mariadb.service: Consumed 1.032s CPU time, 3.1G memory peak.
Jun 11 16:47:13 deployment-db15 systemd[1]: Starting mariadb.service - mariadb database server...
Jun 11 16:47:13 deployment-db15 (mysqld)[27848]: mariadb.service: Referenced but unset environment variable evaluates to an empty string: MYSQLD_OPTS
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] Starting MariaDB 10.11.16-MariaDB-log source revision 3218602d3100db9ce7a875511a591cddc173cc16 server_uid 4n0lQhoyEx0/YbaDH1FbSfawYUs= as process 27848
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used;  This may cause repliction to break when this server acts as a master and has its hostname changed! Please use '--log-basename=deployment-db15' or '--log-bin=deployment-db15-bin' to avoid this problem.
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [ERROR] mysqld: Plugin 'unix_socket' already installed
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [ERROR] Couldn't load plugin 'unix_socket' from 'auth_socket.so'.
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: Compressed tables use zlib 1.3.1
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: Number of transaction pools: 1
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: Using Linux native AIO
Jun 11 16:47:13 deployment-db15 mysqld[27848]: 2026-06-11 16:47:13 0 [Note] InnoDB: innodb_buffer_pool_size_max=12288m, innodb_buffer_pool_size=12288m
Jun 11 16:47:14 deployment-db15 mysqld[27848]: 2026-06-11 16:47:14 0 [Note] InnoDB: Initialized memory pressure event listener
Jun 11 16:47:14 deployment-db15 mysqld[27848]: 2026-06-11 16:47:14 0 [Note] InnoDB: Completed initialization of buffer pool
Jun 11 16:47:14 deployment-db15 mysqld[27848]: 2026-06-11 16:47:14 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
Jun 11 16:47:14 deployment-db15 mysqld[27848]: 2026-06-11 16:47:14 0 [Note] InnoDB: End of log at LSN=35867
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: 128 rollback segments are active.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: log sequence number 35867; transaction id 4
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] Plugin 'FEEDBACK' is disabled.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: Loading buffer pool(s) from /srv/sqldata/ib_buffer_pool
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 1 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1017: Can't find file: './mysql/' (errno: 2 "No such file or directory")
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] InnoDB: Buffer pool(s) load completed at 260611 16:47:15
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] Server socket created on IP: '0.0.0.0', port: '3306'.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [Note] Server socket created on IP: '::', port: '3306'.
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.db' doesn't exist
Jun 11 16:47:15 deployment-db15 mysqld[27848]: 2026-06-11 16:47:15 0 [ERROR] Aborting
Jun 11 16:47:16 deployment-db15 systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 16:47:16 deployment-db15 systemd[1]: mariadb.service: Failed with result 'exit-code'.
Jun 11 16:47:16 deployment-db15 systemd[1]: Failed to start mariadb.service - mariadb database server.
Jun 11 16:47:16 deployment-db15 systemd[1]: mariadb.service: Consumed 1.782s CPU time, 3.1G memory peak.

Notice: Applied catalog in 9.68 seconds
dancy added a subscriber: Zabe.Jun 11 2026, 7:46 PM

@Zabe I saw that you handled T329577 a few years ago and I'm wondering if you can help me bring deployment-db15 online to take over for deployment-db14 (xref
T428910#12010836).

dancy added a comment.Jun 11 2026, 8:00 PM

I have a 120Gib db15 volume mounted on /srv.

dancy added a comment.EditedJun 11 2026, 8:31 PM

I used the following command to get past the puppet problem:

sudo -u mysql /opt/wmf-mariadb1011/scripts/mariadb-install-db \
    --basedir=/opt/wmf-mariadb1011 \
    --datadir=/srv/sqldata
dancy added a comment.Jun 11 2026, 8:34 PM
root@deployment-db11:~# time /opt/wmf-mariadb106/bin/mariadb-dump --all-databases --single-transaction --gtid --triggers | gzip > /srv/db11-seed.sql.gz
gerritbot added a comment.Jun 11 2026, 8:59 PM

Change #1300914 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/puppet@production] profile::mariadb::beta: Initialize system schema on fresh hosts

https://gerrit.wikimedia.org/r/1300914

gerritbot added a project: Patch-For-Review.Jun 11 2026, 8:59 PM
dancy added a comment.Jun 11 2026, 9:43 PM

I copied /srv/db11-seed.sql.gz from deployment-db11 to deployment-db15 and I'm running

zcat /srv/db11-seed.sql.gz | sudo mysql
• bd808 moved this task from To Triage to Future on the Beta-Cluster-Infrastructure board.Jun 11 2026, 10:04 PM
Zabe added a comment.Jun 11 2026, 10:04 PM
In T428930#12011975, @dancy wrote:

@Zabe I saw that you handled T329577 a few years ago and I'm wondering if you can help me bring deployment-db15 online to take over for deployment-db14 (xref
T428910#12010836).

Sure. Please tell me if I can do something. :)

• bd808 merged a task: T428934: Last Puppet run was over 24 hours ago on instance deployment-db15 in project deployment-prep.Jun 11 2026, 10:08 PM
• bd808 merged a task: T428944: Puppet agent failure detected on instance deployment-db15 in project deployment-prep.
• bd808 added a subscriber: wmcs-alerts.
Stashbot added a comment.Jun 12 2026, 3:07 PM

Mentioned in SAL (#wikimedia-releng) [2026-06-12T15:07:18Z] <dancy> deployment-db15 configured as a replica of deployment-db11 (T428930)

gerritbot added a comment.Jun 12 2026, 9:23 PM

Change #1301445 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/mediawiki-config@master] beta: Add deployment-db15 to db-labs config at weight 0

https://gerrit.wikimedia.org/r/1301445

gerritbot added a comment.Jun 12 2026, 9:30 PM

Change #1301445 merged by jenkins-bot:

[operations/mediawiki-config@master] beta: Add deployment-db15 to db-labs config at weight 0

https://gerrit.wikimedia.org/r/1301445

Zabe mentioned this in T429099: Editing on Beta Cluster does not work.Jun 14 2026, 4:08 PM
gerritbot added a comment.Jun 15 2026, 3:05 PM

Change #1300914 merged by Ladsgroup:

[operations/puppet@production] profile::mariadb::beta: Initialize system schema on fresh hosts

https://gerrit.wikimedia.org/r/1300914

Maintenance_bot removed a project: Patch-For-Review.Jun 15 2026, 3:32 PM
gerritbot added a comment.Jun 15 2026, 4:08 PM

Change #1302202 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/mediawiki-config@master] beta: Replace deployment-db14 with deployment-db15

https://gerrit.wikimedia.org/r/1302202

gerritbot added a project: Patch-For-Review.Jun 15 2026, 4:08 PM
gerritbot added a comment.Jun 15 2026, 4:31 PM

Change #1302202 merged by jenkins-bot:

[operations/mediawiki-config@master] beta: Replace deployment-db14 with deployment-db15

https://gerrit.wikimedia.org/r/1302202

Maintenance_bot removed a project: Patch-For-Review.Jun 15 2026, 4:32 PM
dancy added a comment.Jun 15 2026, 5:04 PM
In T428930#12012467, @Zabe wrote:
In T428930#12011975, @dancy wrote:

@Zabe I saw that you handled T329577 a few years ago and I'm wondering if you can help me bring deployment-db15 online to take over for deployment-db14 (xref
T428910#12010836).

Sure. Please tell me if I can do something. :)

Thanks Zabe. I was initially worried about getting stuck but I was able to eventually figure everything out, but thanks for being available!

gerritbot added a comment.Jun 15 2026, 6:11 PM

Change #1302224 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/puppet@production] beta: Add replica provisioning scripts

https://gerrit.wikimedia.org/r/1302224

gerritbot added a project: Patch-For-Review.Jun 15 2026, 6:11 PM
dancy closed this task as Resolved.Jun 15 2026, 6:24 PM
dancy claimed this task.
gerritbot added a comment.Jun 15 2026, 7:33 PM

Change #1302224 merged by Ladsgroup:

[operations/puppet@production] beta: Add replica provisioning scripts

https://gerrit.wikimedia.org/r/1302224

gerritbot added a comment.Jun 15 2026, 7:57 PM

Change #1302253 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/puppet@production] beta: Fix modules/beta/files/receive_replica.sh

https://gerrit.wikimedia.org/r/1302253

gerritbot added a comment.Jun 15 2026, 7:59 PM

Change #1302253 merged by Ladsgroup:

[operations/puppet@production] beta: Fix modules/beta/files/receive_replica.sh

https://gerrit.wikimedia.org/r/1302253

Maintenance_bot removed a project: Patch-For-Review.Jun 15 2026, 8:32 PM
gerritbot added a comment.Jun 15 2026, 8:52 PM

Change #1302263 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/mediawiki-config@master] beta: Promote deployment-db15 to master, drop deployment-db11

https://gerrit.wikimedia.org/r/1302263

gerritbot added a project: Patch-For-Review.Jun 15 2026, 8:52 PM
gerritbot added a comment.Jun 15 2026, 9:00 PM

Change #1302263 merged by jenkins-bot:

[operations/mediawiki-config@master] beta: Promote deployment-db15 to master, drop deployment-db11

https://gerrit.wikimedia.org/r/1302263

Stashbot added a comment.Jun 15 2026, 9:00 PM

Mentioned in SAL (#wikimedia-releng) [2026-06-15T21:00:49Z] <dancy> deployment-db15 promoted to master (T428930)

Maintenance_bot removed a project: Patch-For-Review.Jun 15 2026, 9:31 PM
gerritbot added a comment.Jun 15 2026, 9:39 PM

Change #1302277 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/mediawiki-config@master] beta: Point remaining db11 references at deployment-db15

https://gerrit.wikimedia.org/r/1302277

gerritbot added a project: Patch-For-Review.Jun 15 2026, 9:39 PM
gerritbot added a comment.Jun 15 2026, 9:41 PM

Change #1302277 merged by jenkins-bot:

[operations/mediawiki-config@master] beta: Point remaining db11 references at deployment-db15

https://gerrit.wikimedia.org/r/1302277

Stashbot added a comment.Jun 15 2026, 9:48 PM

Mentioned in SAL (#wikimedia-operations) [2026-06-15T21:48:29Z] <dancy@deploy1003> Started scap sync-world: Backport for [[gerrit:1302277|beta: Point remaining db11 references at deployment-db15 (T428930)]]

Stashbot added a comment.Jun 15 2026, 9:49 PM

Mentioned in SAL (#wikimedia-operations) [2026-06-15T21:49:23Z] <dancy@deploy1003> dancy: Backport for [[gerrit:1302277|beta: Point remaining db11 references at deployment-db15 (T428930)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Stashbot added a comment.Jun 15 2026, 9:54 PM

Mentioned in SAL (#wikimedia-operations) [2026-06-15T21:54:55Z] <dancy@deploy1003> Finished scap sync-world: Backport for [[gerrit:1302277|beta: Point remaining db11 references at deployment-db15 (T428930)]] (duration: 12m 27s)

Maintenance_bot removed a project: Patch-For-Review.Jun 15 2026, 10:31 PM
Stashbot added a comment.Jun 15 2026, 11:40 PM

Mentioned in SAL (#wikimedia-releng) [2026-06-15T23:40:05Z] <dancy> systemctl restart php8.3-fpm on deployment-jobrunner05 to reload beta db configuration (T428930)

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits