Apply schema change to add 3D filetype for STL files
Open, Stalled, NormalPublic

Description

Change: https://gerrit.wikimedia.org/r/336454 (just merged)
Where: All wikis (all.dblist)
When: Any time (the filetype isn't currently enabled on wikis but we might as well be ready when it is)
Backwards compatibility: As far as I know, the only problem might be that files with the 3D type will not have any support - but again, we don't allow these files yet so it should be fine for now. If you're worried about it, wait for the above patch to get deployed.
Testing: Currently underway on Beta.
New columns/tables: No.

Done on:

  • s1
  • s2
  • s3
  • s4 (only primary master pending - we need read_only time to upgrade the master to 10.0.32 which fixes the MariaDB bug that locks a table to add an enum T168661#3378782 and T168661#3576481)
  • s5
  • s6
  • s7
  • All labswiki dbs
There are a very large number of changes, so older changes are hidden. Show Older Changes

Mentioned in SAL (#wikimedia-operations) [2017-08-28T08:16:22Z] <marostegui> Ugprade MariaDB on s4 codfw master - db2051 to 10.0.32 - T168661

db2051 - codfw master has been upgraded to 10.0.32

Change 374476 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1091

https://gerrit.wikimedia.org/r/374476

Change 374476 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1091

https://gerrit.wikimedia.org/r/374476

Mentioned in SAL (#wikimedia-operations) [2017-08-29T07:16:02Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1097 - T168661 (duration: 00m 42s)

Mentioned in SAL (#wikimedia-operations) [2017-08-29T07:16:02Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1097 - T168661 (duration: 00m 42s)

that was actually db1091

Mentioned in SAL (#wikimedia-operations) [2017-08-29T07:41:18Z] <marostegui> Upgrade MariaDB on db1091 to 10.0.32 - T168661

Change 374501 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1091 with low weight

https://gerrit.wikimedia.org/r/374501

Change 374501 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1091 with low weight

https://gerrit.wikimedia.org/r/374501

Mentioned in SAL (#wikimedia-operations) [2017-08-29T08:09:18Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1091 with low weight - T168661 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-08-29T08:41:47Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1091 weight - T168661 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-08-29T09:19:27Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1091 original weight - T168661 (duration: 00m 43s)

Change 374510 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1064 for MariaDB upgrade

https://gerrit.wikimedia.org/r/374510

Change 374510 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1064 for MariaDB upgrade

https://gerrit.wikimedia.org/r/374510

Mentioned in SAL (#wikimedia-operations) [2017-08-29T09:38:43Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1064 for a mariadb upgrade - T168661 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-08-29T09:39:41Z] <marostegui> Update MariaDB on db1064 to 10.0.32 - T168661

Mentioned in SAL (#wikimedia-operations) [2017-08-29T09:59:25Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1064 - T168661 (duration: 00m 43s)

Change 374705 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1059

https://gerrit.wikimedia.org/r/374705

Change 374705 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1059

https://gerrit.wikimedia.org/r/374705

Mentioned in SAL (#wikimedia-operations) [2017-08-30T07:44:05Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1059 for a MariaDB upgrade - T168661 (duration: 00m 53s)

Mentioned in SAL (#wikimedia-operations) [2017-08-30T07:47:11Z] <marostegui> Upgrade MariaDB on db1059 to 10.0.32 - T168661

Change 374764 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1059

https://gerrit.wikimedia.org/r/374764

Change 374764 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1059

https://gerrit.wikimedia.org/r/374764

Mentioned in SAL (#wikimedia-operations) [2017-08-30T09:17:14Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1059 - T168661 (duration: 00m 47s)

Change 374792 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1053

https://gerrit.wikimedia.org/r/374792

Change 374792 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1053

https://gerrit.wikimedia.org/r/374792

Mentioned in SAL (#wikimedia-operations) [2017-08-30T12:05:30Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1053 - T168661 (duration: 00m 47s)

Mentioned in SAL (#wikimedia-operations) [2017-08-30T12:05:43Z] <marostegui> Upgrade MariaDB to 10.0.32 on db1053 - T168661

Change 374796 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1053 with low weight

https://gerrit.wikimedia.org/r/374796

Change 374796 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1053 with low weight

https://gerrit.wikimedia.org/r/374796

Mentioned in SAL (#wikimedia-operations) [2017-08-30T12:26:44Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1053 with low weight - T168661 (duration: 00m 46s)

Change 374802 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase weight on db1053

https://gerrit.wikimedia.org/r/374802

Change 374802 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase weight on db1053

https://gerrit.wikimedia.org/r/374802

Mentioned in SAL (#wikimedia-operations) [2017-08-30T12:40:21Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1053 weight - T168661 (duration: 00m 47s)

Change 374804 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase db1053 weight

https://gerrit.wikimedia.org/r/374804

Change 374804 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase db1053 weight

https://gerrit.wikimedia.org/r/374804

Mentioned in SAL (#wikimedia-operations) [2017-08-30T12:55:40Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1053 weight - T168661 (duration: 00m 45s)

Change 374809 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase weight for db1053

https://gerrit.wikimedia.org/r/374809

Change 374809 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase weight for db1053

https://gerrit.wikimedia.org/r/374809

Mentioned in SAL (#wikimedia-operations) [2017-08-30T13:21:50Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1053 weight - T168661 (duration: 00m 46s)

Change 374819 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Restore db1053 original weight

https://gerrit.wikimedia.org/r/374819

Change 374819 merged by Marostegui:
[operations/mediawiki-config@master] db-eqiad.php: Restore db1053 original weight

https://gerrit.wikimedia.org/r/374819

Mentioned in SAL (#wikimedia-operations) [2017-08-30T13:57:39Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1053 original weight - T168661 (duration: 00m 46s)

Change 374926 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1056

https://gerrit.wikimedia.org/r/374926

Change 374926 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1056

https://gerrit.wikimedia.org/r/374926

Mentioned in SAL (#wikimedia-operations) [2017-08-31T06:16:42Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1056 - T168661 (duration: 00m 47s)

Mentioned in SAL (#wikimedia-operations) [2017-08-31T06:18:17Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1056 - T168661 (duration: 00m 47s)

Mentioned in SAL (#wikimedia-operations) [2017-08-31T06:19:12Z] <marostegui> Upgrade MariaDB to 10.0.32 on db1056 - T168661

Change 374945 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1056 with low weight

https://gerrit.wikimedia.org/r/374945

Change 374945 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1056 with low weight

https://gerrit.wikimedia.org/r/374945

Mentioned in SAL (#wikimedia-operations) [2017-08-31T06:45:27Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1056 with low weight - T168661 (duration: 00m 46s)

Change 374948 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase weight on db1056

https://gerrit.wikimedia.org/r/374948

Change 374948 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase weight on db1056

https://gerrit.wikimedia.org/r/374948

Mentioned in SAL (#wikimedia-operations) [2017-08-31T07:32:52Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic on db1056 - T168661 (duration: 00m 47s)

Change 374953 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase traffic for db1056

https://gerrit.wikimedia.org/r/374953

Change 374953 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase traffic for db1056

https://gerrit.wikimedia.org/r/374953

Mentioned in SAL (#wikimedia-operations) [2017-08-31T08:16:37Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic on db1056 - T168661 (duration: 00m 47s)

Change 374958 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase db1056 weight

https://gerrit.wikimedia.org/r/374958

Change 374958 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase db1056 weight

https://gerrit.wikimedia.org/r/374958

Change 374969 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Restore db1056 original weight

https://gerrit.wikimedia.org/r/374969

Change 374969 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Restore db1056 original weight

https://gerrit.wikimedia.org/r/374969

Mentioned in SAL (#wikimedia-operations) [2017-08-31T11:18:22Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1056 original weight - T168661 (duration: 00m 47s)

Change 374982 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1084

https://gerrit.wikimedia.org/r/374982

Change 374982 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1084

https://gerrit.wikimedia.org/r/374982

Mentioned in SAL (#wikimedia-operations) [2017-08-31T12:11:46Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1084 - T168661 (duration: 00m 47s)

Mentioned in SAL (#wikimedia-operations) [2017-08-31T12:14:26Z] <marostegui> Upgrade MariaDB on db1084 - T168661

Change 374985 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1084 with low weight

https://gerrit.wikimedia.org/r/374985

Change 374985 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1084 with low weight

https://gerrit.wikimedia.org/r/374985

Mentioned in SAL (#wikimedia-operations) [2017-08-31T12:34:18Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1084 with low weight - T168661 (duration: 00m 47s)

Change 374988 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase weight on db1084

https://gerrit.wikimedia.org/r/374988

Change 374988 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase weight on db1084

https://gerrit.wikimedia.org/r/374988

Mentioned in SAL (#wikimedia-operations) [2017-08-31T13:10:34Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1084 weight - T168661 (duration: 00m 47s)

Change 374990 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Restore db1084 original weight

https://gerrit.wikimedia.org/r/374990

Change 374990 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Restore db1084 original weight

https://gerrit.wikimedia.org/r/374990

Mentioned in SAL (#wikimedia-operations) [2017-08-31T13:36:01Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1084 original weight - T168661 (duration: 00m 47s)

Change 375358 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1081

https://gerrit.wikimedia.org/r/375358

Change 375358 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1081

https://gerrit.wikimedia.org/r/375358

Mentioned in SAL (#wikimedia-operations) [2017-09-01T10:29:45Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1081 - T168661 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-09-01T10:31:18Z] <marostegui> Upgrade MariaDB to 10.0.32 on db1081 - T168661

Change 375362 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1081 with low weight

https://gerrit.wikimedia.org/r/375362

Change 375362 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1081 with low weight

https://gerrit.wikimedia.org/r/375362

Mentioned in SAL (#wikimedia-operations) [2017-09-01T11:23:25Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1081 with low weight - T168661 (duration: 00m 44s)

All s4 hosts have been upgraded to 10.0.32.
Obviously not the master (see below)

So the remaining steps are:

  • DBAs: to alter s1 master (db1052) early in the morning (I will attempt to do this on Monday at 5:00 UTC)
  • @MarkTraceur to help arranging a read-only time for s4 so we can upgrade s4 master to 10.0.32 and run the instant alter there.

Change 375363 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase traffic for db1081

https://gerrit.wikimedia.org/r/375363

Change 375363 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase traffic for db1081

https://gerrit.wikimedia.org/r/375363

Mentioned in SAL (#wikimedia-operations) [2017-09-01T11:50:03Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1081 weight - T168661 (duration: 00m 43s)

Change 375367 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Restore db1081 original weight

https://gerrit.wikimedia.org/r/375367

Change 375367 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Restore db1081 original weight

https://gerrit.wikimedia.org/r/375367

Mentioned in SAL (#wikimedia-operations) [2017-09-01T12:32:39Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1081 original weight - T168661 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-09-04T04:52:00Z] <marostegui> Deploy alter table on db1052 - T168661

I have done the alter table on enwiki master (db1052) and it has gone fine. I pre-warmed the big tables (image and filearchive) before and it went thru relatively quickly.
Before running the alter I checked the tables last modifications and indeed they were having low traffic at this time, so while running the alter I didn't see any connection waiting or errors happening so looks like it didn't impact anyone.
Image table took around 1 minute to be altered, and filearchive around 3 minutes.

root@neodymium:~# for i in filearchive image oldimage uploadstash; do echo $i; mysql --skip-ssl -hdb1052 enwiki -e "show create table $i\G" | grep "3D";one
> ^C
root@neodymium:~# for i in filearchive image oldimage uploadstash; do echo $i; mysql --skip-ssl -hdb1052 enwiki -e "show create table $i\G" | grep "3D";done
filearchive
  `fa_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
image
  `img_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
oldimage
  `oi_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
uploadstash
  `us_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
Marostegui updated the task description. (Show Details)Mon, Sep 4, 5:04 AM

Just for the record, I am checking again that the alter has been done across all the host and all the wikis and the only pending one would be commonswiki on s4 master - db1068

Mentioned in SAL (#wikimedia-operations) [2017-09-04T07:03:18Z] <marostegui> Deploy alter table on s3 codfw master with replication on kpbwiki - T168661

The new 3D enum value was missing on the new created wiki kbpwiki on s3, as probably the patch was merged after it was created.
The rest of wikis look good across all the shards.

@MarkTraceur I am going to re-assign this task back to you to see if you can arrange a read-only time for commonswiki.
I will be on holidays from 7th to 25th, so anytime after 25th of Sept (and probably best to avoid it being a Friday) would be good for me.

My initial idea would be to get 30 minutes of read only for commons, so we can upgrade the master and run the alter. And if possible, maybe around 5:00AM UTC, so the traffic is low.

Let me know what you think!

Marostegui updated the task description. (Show Details)Mon, Sep 4, 8:22 AM
Marostegui updated the task description. (Show Details)
Marostegui moved this task from In progress to Blocked external/Not db team on the DBA board.