Apply schema change to add 3D filetype for STL files
Closed, ResolvedPublic

Description

Change: https://gerrit.wikimedia.org/r/336454 (just merged)
Where: All wikis (all.dblist)
When: Any time (the filetype isn't currently enabled on wikis but we might as well be ready when it is)
Backwards compatibility: As far as I know, the only problem might be that files with the 3D type will not have any support - but again, we don't allow these files yet so it should be fine for now. If you're worried about it, wait for the above patch to get deployed.
Testing: Currently underway on Beta.
New columns/tables: No.

Done on:

  • s1
  • s2
  • s3
  • s4 (only primary master pending - we need read_only time to upgrade the master to 10.0.32 which fixes the MariaDB bug that locks a table to add an enum T168661#3378782 and T168661#3576481)
  • s5
  • s6
  • s7
  • All labswiki dbs
There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 374953 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase traffic for db1056

https://gerrit.wikimedia.org/r/374953

Mentioned in SAL (#wikimedia-operations) [2017-08-31T08:16:37Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic on db1056 - T168661 (duration: 00m 47s)

Change 374958 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase db1056 weight

https://gerrit.wikimedia.org/r/374958

Change 374958 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase db1056 weight

https://gerrit.wikimedia.org/r/374958

Change 374969 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Restore db1056 original weight

https://gerrit.wikimedia.org/r/374969

Change 374969 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Restore db1056 original weight

https://gerrit.wikimedia.org/r/374969

Mentioned in SAL (#wikimedia-operations) [2017-08-31T11:18:22Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1056 original weight - T168661 (duration: 00m 47s)

Change 374982 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1084

https://gerrit.wikimedia.org/r/374982

Change 374982 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1084

https://gerrit.wikimedia.org/r/374982

Mentioned in SAL (#wikimedia-operations) [2017-08-31T12:11:46Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1084 - T168661 (duration: 00m 47s)

Mentioned in SAL (#wikimedia-operations) [2017-08-31T12:14:26Z] <marostegui> Upgrade MariaDB on db1084 - T168661

Change 374985 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1084 with low weight

https://gerrit.wikimedia.org/r/374985

Change 374985 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1084 with low weight

https://gerrit.wikimedia.org/r/374985

Mentioned in SAL (#wikimedia-operations) [2017-08-31T12:34:18Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1084 with low weight - T168661 (duration: 00m 47s)

Change 374988 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase weight on db1084

https://gerrit.wikimedia.org/r/374988

Change 374988 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase weight on db1084

https://gerrit.wikimedia.org/r/374988

Mentioned in SAL (#wikimedia-operations) [2017-08-31T13:10:34Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1084 weight - T168661 (duration: 00m 47s)

Change 374990 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Restore db1084 original weight

https://gerrit.wikimedia.org/r/374990

Change 374990 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Restore db1084 original weight

https://gerrit.wikimedia.org/r/374990

Mentioned in SAL (#wikimedia-operations) [2017-08-31T13:36:01Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1084 original weight - T168661 (duration: 00m 47s)

Change 375358 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1081

https://gerrit.wikimedia.org/r/375358

Change 375358 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1081

https://gerrit.wikimedia.org/r/375358

Mentioned in SAL (#wikimedia-operations) [2017-09-01T10:29:45Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1081 - T168661 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-09-01T10:31:18Z] <marostegui> Upgrade MariaDB to 10.0.32 on db1081 - T168661

Change 375362 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Repool db1081 with low weight

https://gerrit.wikimedia.org/r/375362

Change 375362 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Repool db1081 with low weight

https://gerrit.wikimedia.org/r/375362

Mentioned in SAL (#wikimedia-operations) [2017-09-01T11:23:25Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1081 with low weight - T168661 (duration: 00m 44s)

All s4 hosts have been upgraded to 10.0.32.
Obviously not the master (see below)

So the remaining steps are:

  • DBAs: to alter s1 master (db1052) early in the morning (I will attempt to do this on Monday at 5:00 UTC)
  • @MarkTraceur to help arranging a read-only time for s4 so we can upgrade s4 master to 10.0.32 and run the instant alter there.

Change 375363 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase traffic for db1081

https://gerrit.wikimedia.org/r/375363

Change 375363 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase traffic for db1081

https://gerrit.wikimedia.org/r/375363

Mentioned in SAL (#wikimedia-operations) [2017-09-01T11:50:03Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1081 weight - T168661 (duration: 00m 43s)

Change 375367 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Restore db1081 original weight

https://gerrit.wikimedia.org/r/375367

Change 375367 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Restore db1081 original weight

https://gerrit.wikimedia.org/r/375367

Mentioned in SAL (#wikimedia-operations) [2017-09-01T12:32:39Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1081 original weight - T168661 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-09-04T04:52:00Z] <marostegui> Deploy alter table on db1052 - T168661

I have done the alter table on enwiki master (db1052) and it has gone fine. I pre-warmed the big tables (image and filearchive) before and it went thru relatively quickly.
Before running the alter I checked the tables last modifications and indeed they were having low traffic at this time, so while running the alter I didn't see any connection waiting or errors happening so looks like it didn't impact anyone.
Image table took around 1 minute to be altered, and filearchive around 3 minutes.

root@neodymium:~# for i in filearchive image oldimage uploadstash; do echo $i; mysql --skip-ssl -hdb1052 enwiki -e "show create table $i\G" | grep "3D";one
> ^C
root@neodymium:~# for i in filearchive image oldimage uploadstash; do echo $i; mysql --skip-ssl -hdb1052 enwiki -e "show create table $i\G" | grep "3D";done
filearchive
  `fa_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
image
  `img_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
oldimage
  `oi_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
uploadstash
  `us_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
Marostegui updated the task description. (Show Details)Sep 4 2017, 5:04 AM

Just for the record, I am checking again that the alter has been done across all the host and all the wikis and the only pending one would be commonswiki on s4 master - db1068

Mentioned in SAL (#wikimedia-operations) [2017-09-04T07:03:18Z] <marostegui> Deploy alter table on s3 codfw master with replication on kpbwiki - T168661

The new 3D enum value was missing on the new created wiki kbpwiki on s3, as probably the patch was merged after it was created.
The rest of wikis look good across all the shards.

@MarkTraceur I am going to re-assign this task back to you to see if you can arrange a read-only time for commonswiki.
I will be on holidays from 7th to 25th, so anytime after 25th of Sept (and probably best to avoid it being a Friday) would be good for me.

My initial idea would be to get 30 minutes of read only for commons, so we can upgrade the master and run the alter. And if possible, maybe around 5:00AM UTC, so the traffic is low.

Let me know what you think!

Marostegui updated the task description. (Show Details)Sep 4 2017, 8:22 AM
Marostegui updated the task description. (Show Details)
Marostegui moved this task from In progress to Blocked external/Not db team on the DBA board.

@MarkTraceur any update on this? Were you able to try to arrange a read-only time for s4?

Oh, I'm sorry, I missed some updates here (lots of noise on this ticket), and misunderstood that I needed to go through some process to complete it. I've never requested read-only time before, is it appreciably more complex than e-mailing @greg and telling him what's up? We already have a 1-hour deploy window for testwiki this Wednesday at noon UTC, maybe we can pull together a 30-minute window before that or on Tuesday? What works best?

Oh, I'm sorry, I missed some updates here (lots of noise on this ticket), and misunderstood that I needed to go through some process to complete it. I've never requested read-only time before, is it appreciably more complex than e-mailing @greg and telling him what's up? We already have a 1-hour deploy window for testwiki this Wednesday at noon UTC, maybe we can pull together a 30-minute window before that or on Tuesday? What works best?

I assume it is a bit more complicate than that, because it implies getting commonswiki on read only, meaning that no one would be able to upload files :-)
I guess that needs to be announced somewhere so people are aware of that maintenance coming on.
@klove maybe can point us on the right direction here?

greg added a comment.Sep 25 2017, 5:23 PM

What @Marostegui said, the only really more complicated part is the announce/delay so that people are aware.

I'm not sure @klove is the right person to ask here? Who were you thinking of Manuel?

What @Marostegui said, the only really more complicated part is the announce/delay so that people are aware.

I'm not sure @klove is the right person to ask here? Who were you thinking of Manuel?

Hey @greg!
I might be wrong indeed, but I thought @klove was the one coordinating the announcements of the DC switchover for instance?
Basically I was looking for some that can coordinate the announcement of commons going read only for a bit with us :-)

Who do you think can help us here?

greg added a comment.Sep 26 2017, 4:50 PM

I would ask the Multimedia team who their standard community liaison is (they must have one due to the Structured Data on Commons work). cc @MarkTraceur

Marostegui claimed this task.

As per: T176883#3658963 we will be upgrading the master on Wednesday 11th at 6:00UTC

Change 382380 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1068: Update socket path

https://gerrit.wikimedia.org/r/382380

Legoktm added a subscriber: Legoktm.Oct 5 2017, 7:49 AM

This ticket has a lot of comments so I might have missed it, but has someone verified that putting Commons into read only mode won't affect other wikis when you try and add an image to an article - IIRC the globalimagelinks table is also on commonswiki database and we write to that every time a new Commons image is used in an article.

Also there's features like cross-wiki image upload - how will those handle read only mode?

The procedure I had in mind to upgrade the master:

@jcrespo can you double check this procedure and let me know if it looks good to you?

This ticket has a lot of comments so I might have missed it, but has someone verified that putting Commons into read only mode won't affect other wikis when you try and add an image to an article - IIRC the globalimagelinks table is also on commonswiki database and we write to that every time a new Commons image is used in an article.

Also there's features like cross-wiki image upload - how will those handle read only mode?

Thanks for the input!
I don't think anyone has checked that, who could be a good starting point to double check it on mediawiki code?

This ticket has a lot of comments so I might have missed it, but has someone verified that putting Commons into read only mode won't affect other wikis when you try and add an image to an article - IIRC the globalimagelinks table is also on commonswiki database and we write to that every time a new Commons image is used in an article.

Also there's features like cross-wiki image upload - how will those handle read only mode?

Thanks for the input!
I don't think anyone has checked that, who could be a good starting point to double check it on mediawiki code?

Probably @aaron or @Bawolff ?

Marostegui changed the task status from Stalled to Open.Oct 6 2017, 6:52 AM

OK, I tested it by putting Commons into read-only mode using the same method that is planned. My hunch was correct, and that adding a file to a page while Commons is read only throws an exception after the page is saved, and the globalimagelinks table is *not* updated:

1​2017-10-06 07:13:52 [WdctLApAAC4AAHcBSC0AAAAF] mwdebug1002 enwiki 1.31.0-wmf.2 exception ERROR: [WdctLApAAC4AAHcBSC0AAAAF] /w/index.php?title=User:Legoktm/san
2​dbox&action=submit Wikimedia\Rdbms\DBReadOnlyError from line 904 of /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/database/Database.php: Database is r
3​ead-only: TEST - This request is served by a passive datacenter. If you see this something is really wrong. {"exception_id":"WdctLApAAC4AAHcBSC0AAAAF","except
4​ion_url":"/w/index.php?title=User:Legoktm/sandbox&action=submit","caught_by":"mwe_handler"}
5​[Exception Wikimedia\Rdbms\DBReadOnlyError] (/srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/database/Database.php:904) Database is read-only: TEST - This
6​ request is served by a passive datacenter. If you see this something is really wrong.
7​ #0 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/database/Database.php(1608): Wikimedia\Rdbms\Database->query(string, string)
8​ #1 [internal function]: Wikimedia\Rdbms\Database->insert(string, array, string, string)
9​ #2 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/database/DBConnRef.php(49): call_user_func_array(array, array)
10​ #3 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/database/DBConnRef.php(312): Wikimedia\Rdbms\DBConnRef->__call(string, array)
11​ #4 /srv/mediawiki/php-1.31.0-wmf.2/extensions/GlobalUsage/includes/GlobalUsage.php(52): Wikimedia\Rdbms\DBConnRef->insert(string, array, string, array)
12​ #5 /srv/mediawiki/php-1.31.0-wmf.2/extensions/GlobalUsage/includes/GlobalUsageHooks.php(44): GlobalUsage->insertLinks(Title, array, integer, integer)
13​ #6 /srv/mediawiki/php-1.31.0-wmf.2/includes/Hooks.php(177): GlobalUsageHooks::onLinksUpdateComplete(LinksUpdate, integer)
14​ #7 /srv/mediawiki/php-1.31.0-wmf.2/includes/Hooks.php(205): Hooks::callHook(string, array, array, NULL)
15​ #8 /srv/mediawiki/php-1.31.0-wmf.2/includes/deferred/LinksUpdate.php(185): Hooks::run(string, array)
16​ #9 [internal function]: Closure$LinksUpdate::doUpdate(integer)
17​ #10 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/database/Database.php(2797): call_user_func_array(Closure$LinksUpdate::doUpdate;3476, array)
18​ #11 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1311): Wikimedia\Rdbms\Database->runOnTransactionIdleCallbacks(integer
19​)
20​ #12 [internal function]: Closure$Wikimedia\Rdbms\LoadBalancer::runMasterPostTrxCallbacks(Wikimedia\Rdbms\DatabaseMysqli)
21​ #13 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1546): call_user_func_array(Closure$Wikimedia\Rdbms\LoadBalancer::runM
22​asterPostTrxCallbacks;2288, array)
23​ #14 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1320): Wikimedia\Rdbms\LoadBalancer->forEachOpenMasterConnection(Closure$Wikimedia\Rdbms\LoadBalancer::runMasterPostTrxCallbacks;2288)
24​ #15 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/lbfactory/LBFactory.php(232): Wikimedia\Rdbms\LoadBalancer->runMasterPostTrxCallbacks(integer)
25​ #16 [internal function]: Closure$Wikimedia\Rdbms\LBFactory::commitMasterChanges(Wikimedia\Rdbms\LoadBalancer)
26​ #17 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/lbfactory/LBFactoryMulti.php(417): call_user_func_array(Closure$Wikimedia\Rdbms\LBFactory::commitMasterChanges;2240, array)
27​ #18 /srv/mediawiki/php-1.31.0-wmf.2/includes/libs/rdbms/lbfactory/LBFactory.php(234): Wikimedia\Rdbms\LBFactoryMulti->forEachLB(Closure$Wikimedia\Rdbms\LBFactory::commitMasterChanges;2240)
28​ #19 /srv/mediawiki/php-1.31.0-wmf.2/includes/deferred/DeferredUpdates.php(258): Wikimedia\Rdbms\LBFactory->commitMasterChanges(string)
29​ #20 /srv/mediawiki/php-1.31.0-wmf.2/includes/deferred/DeferredUpdates.php(210): DeferredUpdates::runUpdate(LinksUpdate, Wikimedia\Rdbms\LBFactoryMulti, string, integer)
30​ #21 /srv/mediawiki/php-1.31.0-wmf.2/includes/deferred/DeferredUpdates.php(131): DeferredUpdates::execute(array, string, integer)
31​ #22 /srv/mediawiki/php-1.31.0-wmf.2/includes/MediaWiki.php(897): DeferredUpdates::doUpdates(string)
32​ #23 /srv/mediawiki/php-1.31.0-wmf.2/includes/MediaWiki.php(719): MediaWiki->restInPeace(string, boolean)
33​ #24 [internal function]: Closure$MediaWiki::doPostOutputShutdown()
34​ #25 {main}

Second, for whatever reason, the setting being used to mark Commons as read only doesn't actually display the provided message to the user:

Thanks @Legoktm for testing this.
@aaron @Bawolff any advise?

Legoktm added a comment.EditedOct 6 2017, 7:47 AM

Once a Commons administrator implements https://commons.wikimedia.org/wiki/MediaWiki_talk:Readonlywarning that will fix the issue of the read only message not being displayed to users.

I would compress the steps:

Set commonswiki on read-only by merging: https://gerrit.wikimedia.org/r/#/c/382379/1
Double check indeed writes have stopped on the master.
Once checked: stop mysql on the master
apt-get update && apt-get upgrade
Update socket path by merging: https://gerrit.wikimedia.org/r/#/c/382380/
Run puppet
Start MySQL on the master
mysql_upgrade on the master
Stop MySQL
Start MySQL
Make sure the master has read_only=OFF
Make sure all the slave reconnect

Into

apt update && apt full-upgrade
Set commonswiki on read-only by merging: https://gerrit.wikimedia.org/r/#/c/382379/1
Double check indeed writes have stopped on the master.
Once checked: stop mysql on the master
Update socket path by merging: https://gerrit.wikimedia.org/r/#/c/382380/
Run puppet
Start MySQL on the master
Make sure the master has read_only=OFF
Make sure all the slave reconnect
Revert read-only by reverting: https://gerrit.wikimedia.org/r/#/c/382379/
Run mysql_upgrade

mysql_upgrade, in this case, is unlikely to change any inner structure 10.0.23 -> 10.0.32, and by double-rebooting it, you are losing the buffer pool dump. We can also speed up the warmup by running a SELECT/ALTER TABLE on image just after reboot (metadata locking may be problematic once it is fully pooled).

Into

apt update && apt full-upgrade
Set commonswiki on read-only by merging: https://gerrit.wikimedia.org/r/#/c/382379/1
Double check indeed writes have stopped on the master.
Once checked: stop mysql on the master
Update socket path by merging: https://gerrit.wikimedia.org/r/#/c/382380/
Run puppet
Start MySQL on the master
Make sure the master has read_only=OFF
Make sure all the slave reconnect
Revert read-only by reverting: https://gerrit.wikimedia.org/r/#/c/382379/
Run mysql_upgrade

mysql_upgrade, in this case, is unlikely to change any inner structure 10.0.23 -> 10.0.32, and by double-rebooting it, you are losing the buffer pool dump. We can also speed up the warmup by running a SELECT/ALTER TABLE on image just after reboot (metadata locking may be problematic once it is fully pooled).

One note: you would still want to run mysql_upgrade after read-only is gone?

jcrespo added a comment.EditedOct 6 2017, 8:51 AM

you would still want to run mysql_upgrade after read-only is gone?

Yes, precisely because it will be noop, so no need to block on that. Not necessarily after, reversion and run should happen at the same time, most likely it will end before all deployment is done.

Marostegui added a comment.EditedOct 6 2017, 8:52 AM

Cool! Fine by me, so let's do that procedure then - thanks!:

Disable all replication alerts on s4.
apt update && apt full-upgrade
Set commonswiki on read-only by merging: https://gerrit.wikimedia.org/r/#/c/382379/1
Double check indeed writes have stopped on the master.
Once checked: stop mysql on the master
Update socket path by merging: https://gerrit.wikimedia.org/r/#/c/382380/
Run puppet
Start MySQL on the master
ALTER the tables
Make sure the master has read_only=OFF
Make sure all the slave reconnect
Revert read-only by reverting: https://gerrit.wikimedia.org/r/#/c/382379/
Run mysql_upgrade

I have tested the procedure described at T168661#3663752 with another core host and it worked perfectly.

Mentioned in SAL (#wikimedia-operations) [2017-10-11T05:06:42Z] <marostegui> Dump buffer pool on s4 primary master db1068 - T168661

Mentioned in SAL (#wikimedia-operations) [2017-10-11T05:13:39Z] <marostegui> Disable alerts on s4 for 2 hours - T168661

I'm worried that T168661#3663516 is still an issue, but I suppose we can just clean it up afterwards by running refreshLinks.php everywhere.

I'm worried that T168661#3663516 is still an issue, but I suppose we can just clean it up afterwards by running refreshLinks.php everywhere.

Yeah, let's do that.
We are hoping not to be in RO a long time though.

Mentioned in SAL (#wikimedia-operations) [2017-10-11T05:35:14Z] <marostegui> Disable puppet on db1068 (s4 primary master) - T168661

Change 382380 merged by Marostegui:
[operations/puppet@production] db1068: Update socket path

https://gerrit.wikimedia.org/r/382380

Mentioned in SAL (#wikimedia-operations) [2017-10-11T05:48:54Z] <marostegui> Package update on db1068 s4 primary master - T168661

Mentioned in SAL (#wikimedia-operations) [2017-10-11T06:00:56Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Set s4 in read-only for maintenance - T168661 (duration: 00m 47s)

Mentioned in SAL (#wikimedia-operations) [2017-10-11T06:01:03Z] <marostegui> Set read-only on db1068 s4 primary master - T168661

Mentioned in SAL (#wikimedia-operations) [2017-10-11T06:07:14Z] <marostegui> Deploy alter table on s4 primary master - T168661

Mentioned in SAL (#wikimedia-operations) [2017-10-11T06:14:11Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Set s4 to writable mode after maintenance - T168661 (duration: 00m 47s)

Mentioned in SAL (#wikimedia-operations) [2017-10-11T06:14:18Z] <marostegui> Set db1068 s4 primary master to read-only OFF - T168661

This maintenance is finished

Read only time started at: 6:01
Read only time finished at: 6:14

Total 13 minutes

The master has now the new types:

filearchive
  `fa_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
image
  `img_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
oldimage
  `oi_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
uploadstash
  `us_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
Marostegui updated the task description. (Show Details)Oct 11 2017, 6:23 AM

This is all done - I am going to check again (T168661#3576396) across all the fleet to make sure this is done everywhere before closing this task.

@Legoktm will you want to file a ticket for something like "smoother read only" for other projects when commons or wikidata are down/ro or are you ok with how things went?

Marostegui closed this task as Resolved.Oct 11 2017, 8:00 AM

It is everywhere, so closing this.
Thanks everyone involved in making possible to close this task!