Page MenuHomePhabricator

Upgrade cloudinfra database hosts off of Bullseye
Closed, ResolvedPublic

Description

The cloudinfra server has a pair of MariaDB hosts that hosts data for the ENC and web proxy services. The cluster is running Bullseye and MariaDB 10.4 and is in need of an upgrade.

  • cloudinfra-db03.cloudinfra.eqiad1.wikimedia.cloud
  • cloudinfra-db04.cloudinfra.eqiad1.wikimedia.cloud

Event Timeline

taavi added a subscriber: fnegri.

@fnegri Is this something of interest to you?

taavi triaged this task as Medium priority.Aug 15 2025, 1:13 PM

@taavi yes, I can add it to my backlog. I have never played with those dbs until now, do we have any docs about them?

I have never played with those dbs until now, do we have any docs about them?

Technically yes: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Cloudinfra#cloudinfra-db, although I'm not sure how helpful that is.

I thought this was the same db as T409395: Uprade cloudservices1005 and cloudservices1006 to MariaDB 10.11 but I was wrong. I added the hostnames to the description. Sorry for the noise.

I won't be able to work on this in the near future, unassigning myself in case somebody else wants to do it.

On cloudinfra-db04:

MariaDB [(none)]> stop slave;
Query OK, 0 rows affected (0.029 sec)

MariaDB [(none)]> show slave status\G
*************************** 1. row ***************************
                Slave_IO_State: 
                   Master_Host: cloudinfra-db03.cloudinfra.eqiad1.wikimedia.cloud
                   Master_User: repl
                   Master_Port: 3306
                 Connect_Retry: 60
               Master_Log_File: cloudinfra-db03-bin.000058
           Read_Master_Log_Pos: 18819443
                Relay_Log_File: cloudinfra-db04-relay-bin.000047
                 Relay_Log_Pos: 18819752
         Relay_Master_Log_File: cloudinfra-db03-bin.000058
              Slave_IO_Running: No
             Slave_SQL_Running: No
               Replicate_Do_DB: 
           Replicate_Ignore_DB: 
            Replicate_Do_Table: 
        Replicate_Ignore_Table: 
       Replicate_Wild_Do_Table: 
   Replicate_Wild_Ignore_Table: 
                    Last_Errno: 0
                    Last_Error: 
                  Skip_Counter: 0
           Exec_Master_Log_Pos: 18819443
               Relay_Log_Space: 18820128
               Until_Condition: None
                Until_Log_File: 
                 Until_Log_Pos: 0
            Master_SSL_Allowed: No
            Master_SSL_CA_File: 
            Master_SSL_CA_Path: 
               Master_SSL_Cert: 
             Master_SSL_Cipher: 
                Master_SSL_Key: 
         Seconds_Behind_Master: NULL
 Master_SSL_Verify_Server_Cert: No
                 Last_IO_Errno: 0
                 Last_IO_Error: 
                Last_SQL_Errno: 0
                Last_SQL_Error: 
   Replicate_Ignore_Server_Ids: 
              Master_Server_Id: 2886730617
                Master_SSL_Crl: 
            Master_SSL_Crlpath: 
                    Using_Gtid: No
                   Gtid_IO_Pos: 
       Replicate_Do_Domain_Ids: 
   Replicate_Ignore_Domain_Ids: 
                 Parallel_Mode: conservative
                     SQL_Delay: 0
           SQL_Remaining_Delay: NULL
       Slave_SQL_Running_State: 
              Slave_DDL_Groups: 14
Slave_Non_Transactional_Groups: 0
    Slave_Transactional_Groups: 36352133
1 row in set (0.000 sec)

Change #1308696 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] hieradata: Replace cloudinfra-db04 with cloudinfra-db-5

https://gerrit.wikimedia.org/r/1308696

Change #1308696 merged by Majavah:

[operations/puppet@production] hieradata: Replace cloudinfra-db04 with cloudinfra-db-5

https://gerrit.wikimedia.org/r/1308696

Mentioned in SAL (#wikimedia-cloud) [2026-07-08T18:08:29Z] <taavi> Replace cloudinfra-db04 with cloudinfra-db-5 replicating off of db03 T402005

Change #1309109 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] hieradata: cloudinfra: Make cloudinfra-db-5 the primary one

https://gerrit.wikimedia.org/r/1309109

Change #1309109 merged by Majavah:

[operations/puppet@production] hieradata: cloudinfra: Make cloudinfra-db-5 the primary one

https://gerrit.wikimedia.org/r/1309109

Mentioned in SAL (#wikimedia-cloud) [2026-07-09T09:45:49Z] <taavi> make cloudinfra-db-5 the new primary T402005

Change #1309110 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] hieradata: cloudinfra: Replace cloudinfra-db03 with cloudinfra-db-6

https://gerrit.wikimedia.org/r/1309110

Change #1309110 merged by Majavah:

[operations/puppet@production] hieradata: cloudinfra: Replace cloudinfra-db03 with cloudinfra-db-6

https://gerrit.wikimedia.org/r/1309110

Mentioned in SAL (#wikimedia-cloud) [2026-07-09T10:16:45Z] <taavi> Replace cloudinfra-db03 with cloudinfra-db-6 replicating off of db-5 T402005

Done except I need to clean up the VMs in a couple of days.