Page MenuHomePhabricator

Schema change for refactored actor and comment storage
Open, NormalPublic

Description

From the checklist at https://wikitech.wikimedia.org/wiki/Schema_changes:

  1. ALTERs to run: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/482371/3/maintenance/archives/patch-drop-comment-fields.sql and https://gerrit.wikimedia.org/r/c/mediawiki/core/+/525605/8/maintenance/archives/patch-drop-user-fields.sql
  2. Where to run those changes: all.dblist
  3. When to run those changes: No time constraint.
  4. If the schema change is backwards compatible: Yes. The code no longer accesses these fields.
  5. If the schema change has been tested already on some of the test/beta wikis: It appears to have been auto-deployed to Beta. Things seem to be working, and if anyone has complained about things breaking I haven't heard it.
  6. If the data should be made available on the labs replicas and/or dumps: N/A, no new data is being added here. References to these fields from Labs replica views were removed in T212972 and T223406.

Progress:

  • s1
    • codfw
    • eqiad
    • Run maintain-views on labs hosts to recreate the views (sudo /usr/local/sbin/maintain-views --databases jawiki --table protected_titles --replace-all)
  • s2
    • codfw
    • eqiad progress T233135#5578395
    • Run maintain-views on labs hosts to recreate the views
  • s3
    • codfw
    • eqiad
    • Run maintain-views on labs hosts to recreate the views
  • s4
    • codfw
    • eqiad
    • Run maintain-views on labs hosts to recreate the views
  • s5
    • codfw
    • eqiad
    • Run maintain-views on labs hosts to recreate the views
  • s6
    • codfw T233135#5502104
    • eqiad
    • Run maintain-views on labs hosts to recreate the views
  • s7
    • codfw
    • eqiad progress T233135#5571998
    • Run maintain-views on labs hosts to recreate the views
  • s8
    • codfw
    • eqiad
    • Run maintain-views on labs hosts to recreate the views
  • labswiki
  • labtestwiki
  • Remove columns from modules/role/files/mariadb/filtered_tables.txt once everything is done.

Related Objects

Event Timeline

Anomie created this task.Sep 17 2019, 5:39 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 17 2019, 5:39 PM
Marostegui triaged this task as Normal priority.
Marostegui moved this task from Triage to In progress on the DBA board.
Marostegui added a subscriber: Marostegui.

Yay!!

Marostegui updated the task description. (Show Details)
Marostegui updated the task description. (Show Details)Wed, Sep 18, 5:08 AM
Marostegui added a comment.EditedWed, Sep 18, 5:53 AM

I want to alter s6 first host by host, if everything goes fine, I will later do codfw all at once with replication. I am also interested in seeing how much disk space we free up by removing all these columns and indexes.

s6 progress:

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1005
  • db2129
  • db2124
  • db2117
  • db2114
  • db2097
  • db2095
  • db2089 T233135#5502290
  • db2087
  • db2076
  • db1139
  • db1131
  • db1125
  • db1113
  • db1098
  • db1096
  • db1093
  • db1088
  • db1085
  • db1061

Mentioned in SAL (#wikimedia-operations) [2019-09-18T05:58:05Z] <marostegui> Deploy schema change on db2097:3316 - T233135

Mentioned in SAL (#wikimedia-operations) [2019-09-18T05:58:05Z] <marostegui> Deploy schema change on db2097:3316 - T233135

I will change this to db2089 instead, as db2097:3316 is a backup host, and I prefer not to touch those for now.

s6 db2089:3316
I found this error on the three wikis that live there: frwiki jawiki ruwiki

ERROR 1091 (42000) at line 37: Can't DROP 'ar_usertext_timestamp'; check that column/key exists

Which I guess it is the result of: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/346649/1/maintenance/archives/patch-rename-ar_usertext_timestamp.sql ?
@Anomie do you want to drop the KEY usertext_timestamp (ar_user_text,ar_timestamp), key which is the one that really exists?
I have not been able to find that other ar_usertext_timestamp key there, and not on enwiki, for instance (db1089 or db1067)

Size wise: before the schema change

root@db2089:/srv/sqldata.s6# for i in `cat /home/marostegui/tables`; do echo $i; find . -name $i*.ibd | xargs ls -lh ;done
archive
-rw-rw---- 1 mysql mysql 2.1G Sep 18 05:13 ./frwiki/archive.ibd
-rw-rw---- 1 mysql mysql 708M Sep 17 21:27 ./jawiki/archive.ibd
-rw-rw---- 1 mysql mysql 2.4G Sep 18 05:49 ./ruwiki/archive.ibd
filearchive
-rw-rw---- 1 mysql mysql  80M Sep 17 12:11 ./frwiki/filearchive.ibd
-rw-rw---- 1 mysql mysql  32M Sep  4 11:30 ./jawiki/filearchive.ibd
-rw-rw---- 1 mysql mysql 160M Sep 18 05:58 ./ruwiki/filearchive.ibd
image
-rw-rw---- 1 mysql mysql  52M Sep 18 05:11 ./frwiki/image.ibd
-rw-rw---- 1 mysql mysql  76M Sep 17 16:12 ./jawiki/image.ibd
-rw-rw---- 1 mysql mysql 196M Sep 18 06:09 ./ruwiki/image.ibd
ipblocks
-rw-rw---- 1 mysql mysql  18M Sep 18 05:43 ./frwiki/ipblocks.ibd
-rw-rw---- 1 mysql mysql  24M Sep 18 06:07 ./jawiki/ipblocks.ibd
-rw-rw---- 1 mysql mysql 412M Sep 18 06:09 ./ruwiki/ipblocks.ibd
logging
-rw-rw---- 1 mysql mysql 220M Sep 16 21:01 ./frwiki/logging#P#p1000000.ibd
-rw-rw---- 1 mysql mysql  80M Aug 15 14:17 ./frwiki/logging#P#p100000.ibd
-rw-rw---- 1 mysql mysql 340M Sep 16 21:01 ./frwiki/logging#P#p10000.ibd
-rw-rw---- 1 mysql mysql 156M Sep  5 08:43 ./frwiki/logging#P#p125000.ibd
-rw-rw---- 1 mysql mysql 644M Sep 16 21:01 ./frwiki/logging#P#p1500000.ibd
-rw-rw---- 1 mysql mysql 220M Sep  5 08:43 ./frwiki/logging#P#p150000.ibd
-rw-rw---- 1 mysql mysql  88M Sep  5 08:43 ./frwiki/logging#P#p175000.ibd
-rw-rw---- 1 mysql mysql 416M Sep 18 06:09 ./frwiki/logging#P#p1.ibd
-rw-rw---- 1 mysql mysql 1.8G Sep 16 15:06 ./frwiki/logging#P#p2000000.ibd
-rw-rw---- 1 mysql mysql 124M Aug 24 12:40 ./frwiki/logging#P#p200000.ibd
-rw-rw---- 1 mysql mysql 168M Sep  5 08:43 ./frwiki/logging#P#p20000.ibd
-rw-rw---- 1 mysql mysql 132M Sep  5 08:43 ./frwiki/logging#P#p250000.ibd
-rw-rw---- 1 mysql mysql 304M Sep  5 08:43 ./frwiki/logging#P#p300000.ibd
-rw-rw---- 1 mysql mysql 256M Sep  5 08:43 ./frwiki/logging#P#p30000.ibd
-rw-rw---- 1 mysql mysql 344M Sep  5 08:43 ./frwiki/logging#P#p400000.ibd
-rw-rw---- 1 mysql mysql 200M Sep  5 08:43 ./frwiki/logging#P#p40000.ibd
-rw-rw---- 1 mysql mysql 160M Sep 11 13:07 ./frwiki/logging#P#p500000.ibd
-rw-rw---- 1 mysql mysql 176M Aug 15 14:17 ./frwiki/logging#P#p50000.ibd
-rw-rw---- 1 mysql mysql 180M Sep  6 03:51 ./frwiki/logging#P#p600000.ibd
-rw-rw---- 1 mysql mysql 304M Sep 12 10:17 ./frwiki/logging#P#p700000.ibd
-rw-rw---- 1 mysql mysql 268M Sep  5 08:43 ./frwiki/logging#P#p75000.ibd
-rw-rw---- 1 mysql mysql 192M Sep  5 08:43 ./frwiki/logging#P#p800000.ibd
-rw-rw---- 1 mysql mysql 104M Sep 14 15:53 ./frwiki/logging#P#p900000.ibd
-rw-rw---- 1 mysql mysql 1.3G Sep 18 04:32 ./frwiki/logging#P#pMAXVALUE.ibd
-rw-rw---- 1 mysql mysql 232M Sep 16 14:50 ./jawiki/logging#P#p1000000.ibd
-rw-rw---- 1 mysql mysql 108M Sep 17 14:57 ./jawiki/logging#P#p100000.ibd
-rw-rw---- 1 mysql mysql 292M Aug 11 15:53 ./jawiki/logging#P#p10000.ibd
-rw-rw---- 1 mysql mysql  80M Aug 17 15:46 ./jawiki/logging#P#p150000.ibd
-rw-rw---- 1 mysql mysql  84M Sep 18 06:09 ./jawiki/logging#P#p1.ibd
-rw-rw---- 1 mysql mysql 120M Aug 24 01:30 ./jawiki/logging#P#p200000.ibd
-rw-rw---- 1 mysql mysql 100M Aug 10 11:21 ./jawiki/logging#P#p20000.ibd
-rw-rw---- 1 mysql mysql 116M Sep 11 13:07 ./jawiki/logging#P#p300000.ibd
-rw-rw---- 1 mysql mysql  72M Jun  3 16:18 ./jawiki/logging#P#p30000.ibd
-rw-rw---- 1 mysql mysql 168M Sep 16 11:20 ./jawiki/logging#P#p400000.ibd
-rw-rw---- 1 mysql mysql  80M Jul 28 17:14 ./jawiki/logging#P#p40000.ibd
-rw-rw---- 1 mysql mysql  80M Sep 15 16:53 ./jawiki/logging#P#p500000.ibd
-rw-rw---- 1 mysql mysql  27M Jun  3 16:01 ./jawiki/logging#P#p50000.ibd
-rw-rw---- 1 mysql mysql 172M Sep 16 15:06 ./jawiki/logging#P#p750000.ibd
-rw-rw---- 1 mysql mysql 312M Sep 16 21:01 ./jawiki/logging#P#pMAXVALUE.ibd
-rw-rw---- 1 mysql mysql 1.8G Sep 14 15:56 ./ruwiki/logging#P#p1000000.ibd
-rw-rw---- 1 mysql mysql 696M Sep 14 15:55 ./ruwiki/logging#P#p100000.ibd
-rw-rw---- 1 mysql mysql 588M Sep  8 16:28 ./ruwiki/logging#P#p10000.ibd
-rw-rw---- 1 mysql mysql 1.9G Sep  8 12:05 ./ruwiki/logging#P#p150000.ibd
-rw-rw---- 1 mysql mysql 792M Sep 18 06:09 ./ruwiki/logging#P#p1.ibd
-rw-rw---- 1 mysql mysql 6.7G Sep 16 18:32 ./ruwiki/logging#P#p2000000.ibd
-rw-rw---- 1 mysql mysql 1.2G Sep 12 02:02 ./ruwiki/logging#P#p200000.ibd
-rw-rw---- 1 mysql mysql 772M Sep  8 12:05 ./ruwiki/logging#P#p20000.ibd
-rw-rw---- 1 mysql mysql 1.7G Sep 14 15:55 ./ruwiki/logging#P#p300000.ibd
-rw-rw---- 1 mysql mysql 528M Sep 14 15:55 ./ruwiki/logging#P#p30000.ibd
-rw-rw---- 1 mysql mysql 1.9G Sep 14 15:55 ./ruwiki/logging#P#p400000.ibd
-rw-rw---- 1 mysql mysql 592M Aug 23 17:11 ./ruwiki/logging#P#p40000.ibd
-rw-rw---- 1 mysql mysql 1.4G Sep 14 15:55 ./ruwiki/logging#P#p500000.ibd
-rw-rw---- 1 mysql mysql 536M Jun  3 16:03 ./ruwiki/logging#P#p50000.ibd
-rw-rw---- 1 mysql mysql 728M Sep 14 15:55 ./ruwiki/logging#P#p5000.ibd
-rw-rw---- 1 mysql mysql 1.7G Sep 14 15:55 ./ruwiki/logging#P#p600000.ibd
-rw-rw---- 1 mysql mysql 2.7G Sep 14 15:55 ./ruwiki/logging#P#p750000.ibd
-rw-rw---- 1 mysql mysql 904M Aug 25 17:25 ./ruwiki/logging#P#p75000.ibd
-rw-rw---- 1 mysql mysql 804M Sep 17 03:38 ./ruwiki/logging#P#pMAXVALUE.ibd
oldimage
-rw-rw---- 1 mysql mysql 14M Sep 17 11:46 ./frwiki/oldimage.ibd
-rw-rw---- 1 mysql mysql 14M Sep 17 16:12 ./jawiki/oldimage.ibd
-rw-rw---- 1 mysql mysql 22M Sep 18 05:58 ./ruwiki/oldimage.ibd
protected_titles
-rw-rw---- 1 mysql mysql 9.0M Sep 18 00:29 ./frwiki/protected_titles.ibd
-rw-rw---- 1 mysql mysql 9.0M Sep 17 21:18 ./jawiki/protected_titles.ibd
-rw-rw---- 1 mysql mysql  13M Sep 18 05:46 ./ruwiki/protected_titles.ibd
recentchanges
-rw-rw---- 1 mysql mysql 1.1G Sep 18 06:09 ./frwiki/recentchanges.ibd
-rw-rw---- 1 mysql mysql 396M Sep 18 06:09 ./jawiki/recentchanges.ibd
-rw-rw---- 1 mysql mysql 976M Sep 18 06:09 ./ruwiki/recentchanges.ibd

root@db2089:/srv/sqldata.s6# du -sh *wiki
349G	frwiki
140G	jawiki
413G	ruwiki

After the schema change:

archive
-rw-rw---- 1 mysql mysql 1.4G Sep 18 08:47 ./frwiki/archive.ibd
-rw-rw---- 1 mysql mysql 476M Sep 18 08:26 ./jawiki/archive.ibd
-rw-rw---- 1 mysql mysql 1.5G Sep 18 08:42 ./ruwiki/archive.ibd
filearchive
-rw-rw---- 1 mysql mysql 52M Sep 18 06:20 ./frwiki/filearchive.ibd
-rw-rw---- 1 mysql mysql 20M Sep 18 06:53 ./jawiki/filearchive.ibd
-rw-rw---- 1 mysql mysql 96M Sep 18 07:14 ./ruwiki/filearchive.ibd
image
-rw-rw---- 1 mysql mysql  29M Sep 18 08:18 ./frwiki/image.ibd
-rw-rw---- 1 mysql mysql  52M Sep 18 07:41 ./jawiki/image.ibd
-rw-rw---- 1 mysql mysql 120M Sep 18 08:09 ./ruwiki/image.ibd
ipblocks
-rw-rw---- 1 mysql mysql  14M Sep 18 08:23 ./frwiki/ipblocks.ibd
-rw-rw---- 1 mysql mysql  19M Sep 18 08:35 ./jawiki/ipblocks.ibd
-rw-rw---- 1 mysql mysql 232M Sep 18 08:46 ./ruwiki/ipblocks.ibd
logging
-rw-rw---- 1 mysql mysql 140M Sep 18 06:38 ./frwiki/logging#P#p1000000.ibd
-rw-rw---- 1 mysql mysql  48M Sep 18 06:29 ./frwiki/logging#P#p100000.ibd
-rw-rw---- 1 mysql mysql 212M Sep 18 06:25 ./frwiki/logging#P#p10000.ibd
-rw-rw---- 1 mysql mysql 100M Sep 18 06:29 ./frwiki/logging#P#p125000.ibd
-rw-rw---- 1 mysql mysql 412M Sep 18 06:39 ./frwiki/logging#P#p1500000.ibd
-rw-rw---- 1 mysql mysql 140M Sep 18 06:30 ./frwiki/logging#P#p150000.ibd
-rw-rw---- 1 mysql mysql  56M Sep 18 06:30 ./frwiki/logging#P#p175000.ibd
-rw-rw---- 1 mysql mysql 348M Sep 18 08:43 ./frwiki/logging#P#p1.ibd
-rw-rw---- 1 mysql mysql 1.3G Sep 18 06:46 ./frwiki/logging#P#p2000000.ibd
-rw-rw---- 1 mysql mysql  76M Sep 18 06:31 ./frwiki/logging#P#p200000.ibd
-rw-rw---- 1 mysql mysql 108M Sep 18 06:26 ./frwiki/logging#P#p20000.ibd
-rw-rw---- 1 mysql mysql  84M Sep 18 06:31 ./frwiki/logging#P#p250000.ibd
-rw-rw---- 1 mysql mysql 196M Sep 18 06:32 ./frwiki/logging#P#p300000.ibd
-rw-rw---- 1 mysql mysql 160M Sep 18 06:27 ./frwiki/logging#P#p30000.ibd
-rw-rw---- 1 mysql mysql 220M Sep 18 06:33 ./frwiki/logging#P#p400000.ibd
-rw-rw---- 1 mysql mysql 128M Sep 18 06:27 ./frwiki/logging#P#p40000.ibd
-rw-rw---- 1 mysql mysql 100M Sep 18 06:34 ./frwiki/logging#P#p500000.ibd
-rw-rw---- 1 mysql mysql 132M Sep 18 06:28 ./frwiki/logging#P#p50000.ibd
-rw-rw---- 1 mysql mysql 112M Sep 18 06:34 ./frwiki/logging#P#p600000.ibd
-rw-rw---- 1 mysql mysql 192M Sep 18 06:36 ./frwiki/logging#P#p700000.ibd
-rw-rw---- 1 mysql mysql 168M Sep 18 06:29 ./frwiki/logging#P#p75000.ibd
-rw-rw---- 1 mysql mysql 124M Sep 18 06:37 ./frwiki/logging#P#p800000.ibd
-rw-rw---- 1 mysql mysql  64M Sep 18 06:37 ./frwiki/logging#P#p900000.ibd
-rw-rw---- 1 mysql mysql 804M Sep 18 06:50 ./frwiki/logging#P#pMAXVALUE.ibd
-rw-rw---- 1 mysql mysql 140M Sep 18 07:01 ./jawiki/logging#P#p1000000.ibd
-rw-rw---- 1 mysql mysql  64M Sep 18 06:58 ./jawiki/logging#P#p100000.ibd
-rw-rw---- 1 mysql mysql 212M Sep 18 06:56 ./jawiki/logging#P#p10000.ibd
-rw-rw---- 1 mysql mysql  48M Sep 18 06:58 ./jawiki/logging#P#p150000.ibd
-rw-rw---- 1 mysql mysql  72M Sep 18 08:48 ./jawiki/logging#P#p1.ibd
-rw-rw---- 1 mysql mysql  68M Sep 18 06:59 ./jawiki/logging#P#p200000.ibd
-rw-rw---- 1 mysql mysql  64M Sep 18 06:56 ./jawiki/logging#P#p20000.ibd
-rw-rw---- 1 mysql mysql  72M Sep 18 06:59 ./jawiki/logging#P#p300000.ibd
-rw-rw---- 1 mysql mysql  44M Sep 18 06:56 ./jawiki/logging#P#p30000.ibd
-rw-rw---- 1 mysql mysql  88M Sep 18 07:00 ./jawiki/logging#P#p400000.ibd
-rw-rw---- 1 mysql mysql  48M Sep 18 06:56 ./jawiki/logging#P#p40000.ibd
-rw-rw---- 1 mysql mysql  48M Sep 18 07:00 ./jawiki/logging#P#p500000.ibd
-rw-rw---- 1 mysql mysql  15M Sep 18 06:57 ./jawiki/logging#P#p50000.ibd
-rw-rw---- 1 mysql mysql 104M Sep 18 07:01 ./jawiki/logging#P#p750000.ibd
-rw-rw---- 1 mysql mysql 200M Sep 18 07:02 ./jawiki/logging#P#pMAXVALUE.ibd
-rw-rw---- 1 mysql mysql 1.2G Sep 18 08:26 ./ruwiki/logging#P#p1000000.ibd
-rw-rw---- 1 mysql mysql 460M Sep 18 07:37 ./ruwiki/logging#P#p100000.ibd
-rw-rw---- 1 mysql mysql 384M Sep 18 07:24 ./ruwiki/logging#P#p10000.ibd
-rw-rw---- 1 mysql mysql 1.3G Sep 18 07:43 ./ruwiki/logging#P#p150000.ibd
-rw-rw---- 1 mysql mysql 680M Sep 18 08:43 ./ruwiki/logging#P#p1.ibd
-rw-rw---- 1 mysql mysql 4.6G Sep 18 08:49 ./ruwiki/logging#P#p2000000.ibd
-rw-rw---- 1 mysql mysql 764M Sep 18 07:47 ./ruwiki/logging#P#p200000.ibd
-rw-rw---- 1 mysql mysql 500M Sep 18 07:26 ./ruwiki/logging#P#p20000.ibd
-rw-rw---- 1 mysql mysql 1.1G Sep 18 07:52 ./ruwiki/logging#P#p300000.ibd
-rw-rw---- 1 mysql mysql 340M Sep 18 07:28 ./ruwiki/logging#P#p30000.ibd
-rw-rw---- 1 mysql mysql 1.3G Sep 18 07:59 ./ruwiki/logging#P#p400000.ibd
-rw-rw---- 1 mysql mysql 380M Sep 18 07:30 ./ruwiki/logging#P#p40000.ibd
-rw-rw---- 1 mysql mysql 912M Sep 18 08:03 ./ruwiki/logging#P#p500000.ibd
-rw-rw---- 1 mysql mysql 356M Sep 18 07:32 ./ruwiki/logging#P#p50000.ibd
-rw-rw---- 1 mysql mysql 468M Sep 18 07:22 ./ruwiki/logging#P#p5000.ibd
-rw-rw---- 1 mysql mysql 1.1G Sep 18 08:09 ./ruwiki/logging#P#p600000.ibd
-rw-rw---- 1 mysql mysql 1.8G Sep 18 08:17 ./ruwiki/logging#P#p750000.ibd
-rw-rw---- 1 mysql mysql 572M Sep 18 07:35 ./ruwiki/logging#P#p75000.ibd
-rw-rw---- 1 mysql mysql 540M Sep 18 08:43 ./ruwiki/logging#P#pMAXVALUE.ibd
oldimage
-rw-rw---- 1 mysql mysql 7.0M Sep 18 06:20 ./frwiki/oldimage.ibd
-rw-rw---- 1 mysql mysql  11M Sep 18 07:41 ./jawiki/oldimage.ibd
-rw-rw---- 1 mysql mysql  11M Sep 18 07:14 ./ruwiki/oldimage.ibd
protected_titles
-rw-rw---- 1 mysql mysql 320K Sep 18 07:00 ./frwiki/protected_titles.ibd
-rw-rw---- 1 mysql mysql 392K Sep 18 07:02 ./jawiki/protected_titles.ibd
-rw-rw---- 1 mysql mysql 2.0M Sep 18 08:43 ./ruwiki/protected_titles.ibd
recentchanges
-rw-rw---- 1 mysql mysql 668M Sep 18 08:49 ./frwiki/recentchanges.ibd
-rw-rw---- 1 mysql mysql 268M Sep 18 08:49 ./jawiki/recentchanges.ibd
-rw-rw---- 1 mysql mysql 572M Sep 18 08:49 ./ruwiki/recentchanges.ibd
root@db2089:/srv/sqldata.s6# du -sh *wiki
345G	frwiki
139G	jawiki
402G	ruwiki

Once @Anomie confirms what to do with the ar_usertext_timestamp, I will go ahead and alter an active s6 eqiad host, I want to double check that the optimizer doesn't do anything unexpected after we've removed so many indexes, as we've seen that before.

s6 db2089:3316
I found this error on the three wikis that live there: frwiki jawiki ruwiki

ERROR 1091 (42000) at line 37: Can't DROP 'ar_usertext_timestamp'; check that column/key exists

Which I guess it is the result of: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/346649/1/maintenance/archives/patch-rename-ar_usertext_timestamp.sql ?

WTF is going on there?

It looks like that patch made the schema from update.php inconsistent with that from tables.sql. But somehow when I ran update.php on my local wiki it didn't have any problem with the attempted drop, which hints that my local installation still had ar_usertext_timestamp despite that patch and me having run update.php many times.

@Anomie do you want to drop the KEY usertext_timestamp (ar_user_text,ar_timestamp), key which is the one that really exists?

Yes, the index needs to be dropped, because the column is going away.

Also I guess I may have to write a patch to handle this for people using update.php.

Thanks - so I will include an DROP INDEX IF EXISTS usertext_timestamp and will also leave the DROP INDEX IF EXISTS ar_usertext_timestamp so we can cover for both situations in production.

Marostegui added a comment.EditedWed, Sep 18, 2:53 PM

There is another issue: for the logging table, we are dropping the log_user column, which is the one we have the partitions based on:

/*!50100 PARTITION BY RANGE (log_user)
(PARTITION p1 VALUES LESS THAN (1) ENGINE = InnoDB,
 PARTITION p10000 VALUES LESS THAN (10000) ENGINE = InnoDB,
 PARTITION p20000 VALUES LESS THAN (20000) ENGINE = InnoDB,
 PARTITION p30000 VALUES LESS THAN (30000) ENGINE = InnoDB,
 PARTITION p40000 VALUES LESS THAN (40000) ENGINE = InnoDB,
 PARTITION p50000 VALUES LESS THAN (50000) ENGINE = InnoDB,
 PARTITION p75000 VALUES LESS THAN (75000) ENGINE = InnoDB,
 PARTITION p100000 VALUES LESS THAN (100000) ENGINE = InnoDB,
 PARTITION p125000 VALUES LESS THAN (125000) ENGINE = InnoDB,
 PARTITION p150000 VALUES LESS THAN (150000) ENGINE = InnoDB,
 PARTITION p175000 VALUES LESS THAN (175000) ENGINE = InnoDB,
 PARTITION p200000 VALUES LESS THAN (200000) ENGINE = InnoDB,
 PARTITION p250000 VALUES LESS THAN (250000) ENGINE = InnoDB,
 PARTITION p300000 VALUES LESS THAN (300000) ENGINE = InnoDB,
 PARTITION p400000 VALUES LESS THAN (400000) ENGINE = InnoDB,
 PARTITION p500000 VALUES LESS THAN (500000) ENGINE = InnoDB,
 PARTITION p600000 VALUES LESS THAN (600000) ENGINE = InnoDB,
 PARTITION p700000 VALUES LESS THAN (700000) ENGINE = InnoDB,
 PARTITION p800000 VALUES LESS THAN (800000) ENGINE = InnoDB,
 PARTITION p900000 VALUES LESS THAN (900000) ENGINE = InnoDB,
 PARTITION p1000000 VALUES LESS THAN (1000000) ENGINE = InnoDB,
 PARTITION p1500000 VALUES LESS THAN (1500000) ENGINE = InnoDB,
 PARTITION p2000000 VALUES LESS THAN (2000000) ENGINE = InnoDB,
 PARTITION pMAXVALUE VALUES LESS THAN MAXVALUE ENGINE = InnoDB) *

So we would need to also REMOVE the partitions on the recentchanges hosts (which I am more than happy to drop!)
@Anomie, does this require another patch from your side for MW?

No, MediaWiki knows nothing about the paritioning. That's purely a Wikimedia thing.

No, MediaWiki knows nothing about the paritioning. That's purely a Wikimedia thing.

Gotcha - so I will go ahead and remove the partitioning there - we need to keep an eye on the optimizer once I apply the alters to the s6 eqiad host.

Also, see T223151: Review special replica partitioning of certain tables by `xx_user` where we discussed this before since we knew this was coming.

Also, see T223151: Review special replica partitioning of certain tables by `xx_user` where we discussed this before since we knew this was coming.

Which means we also need to change the PK, as that column is part of it, for the special slaves.
This is getting complex :-)

WTF is going on there?

In case you're curious, T233221 has all the details of WTF is going on there.

Change 538030 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] index-conf.yaml: Remove unused index

https://gerrit.wikimedia.org/r/538030

Change 538030 merged by Marostegui:
[operations/puppet@production] index-conf.yaml: Remove unused index

https://gerrit.wikimedia.org/r/538030

Marostegui updated the task description. (Show Details)Fri, Sep 20, 10:05 AM
Marostegui updated the task description. (Show Details)Thu, Oct 3, 6:15 AM

Mentioned in SAL (#wikimedia-operations) [2019-10-03T06:16:03Z] <marostegui> Deploy schema change on db2089:3316 T233135 T234066

This is how the tables look like after this alter and the alter for the special slaves (T233625)

root@cumin1001:/home/marostegui# for i in archive ipblocks image oldimage filearchive recentchanges logging protected_titles; do echo $i; mysql.py -hdb2089:3316 frwiki -e "show create table $i\G";done
archive
*************************** 1. row ***************************
       Table: archive
Create Table: CREATE TABLE `archive` (
  `ar_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `ar_namespace` int(11) NOT NULL DEFAULT '0',
  `ar_title` varbinary(255) NOT NULL DEFAULT '',
  `ar_comment_id` bigint(20) unsigned NOT NULL,
  `ar_actor` bigint(20) unsigned NOT NULL,
  `ar_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `ar_minor_edit` tinyint(1) NOT NULL DEFAULT '0',
  `ar_rev_id` int(10) unsigned NOT NULL,
  `ar_text_id` int(10) unsigned NOT NULL DEFAULT '0',
  `ar_deleted` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `ar_len` int(8) unsigned DEFAULT NULL,
  `ar_page_id` int(10) unsigned DEFAULT NULL,
  `ar_parent_id` int(10) unsigned DEFAULT NULL,
  `ar_sha1` varbinary(32) NOT NULL DEFAULT '',
  `ar_content_model` varbinary(32) DEFAULT NULL,
  `ar_content_format` varbinary(64) DEFAULT NULL,
  PRIMARY KEY (`ar_id`),
  UNIQUE KEY `ar_revid_uniq` (`ar_rev_id`),
  KEY `name_title_timestamp` (`ar_namespace`,`ar_title`,`ar_timestamp`),
  KEY `ar_actor_timestamp` (`ar_actor`,`ar_timestamp`)
) ENGINE=InnoDB AUTO_INCREMENT=10493733 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
ipblocks
*************************** 1. row ***************************
       Table: ipblocks
Create Table: CREATE TABLE `ipblocks` (
  `ipb_id` int(8) NOT NULL AUTO_INCREMENT,
  `ipb_address` tinyblob NOT NULL,
  `ipb_user` int(8) unsigned NOT NULL DEFAULT '0',
  `ipb_reason_id` bigint(20) unsigned NOT NULL,
  `ipb_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `ipb_auto` tinyint(1) NOT NULL DEFAULT '0',
  `ipb_anon_only` tinyint(1) NOT NULL DEFAULT '0',
  `ipb_create_account` tinyint(1) NOT NULL DEFAULT '1',
  `ipb_expiry` varbinary(14) NOT NULL DEFAULT '',
  `ipb_range_start` tinyblob NOT NULL,
  `ipb_range_end` tinyblob NOT NULL,
  `ipb_enable_autoblock` tinyint(1) NOT NULL DEFAULT '1',
  `ipb_deleted` tinyint(1) NOT NULL DEFAULT '0',
  `ipb_block_email` tinyint(1) NOT NULL DEFAULT '0',
  `ipb_by_actor` bigint(20) unsigned NOT NULL,
  `ipb_allow_usertalk` tinyint(1) NOT NULL DEFAULT '1',
  `ipb_parent_block_id` int(11) DEFAULT NULL,
  `ipb_sitewide` tinyint(1) NOT NULL DEFAULT '1',
  PRIMARY KEY (`ipb_id`),
  UNIQUE KEY `ipb_address_unique` (`ipb_address`(255),`ipb_user`,`ipb_auto`),
  KEY `ipb_user` (`ipb_user`),
  KEY `ipb_timestamp` (`ipb_timestamp`),
  KEY `ipb_expiry` (`ipb_expiry`),
  KEY `ipb_range` (`ipb_range_start`(20),`ipb_range_end`(20)),
  KEY `ipb_parent_block_id` (`ipb_parent_block_id`)
) ENGINE=InnoDB AUTO_INCREMENT=3976644 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
image
*************************** 1. row ***************************
       Table: image
Create Table: CREATE TABLE `image` (
  `img_name` varbinary(255) NOT NULL DEFAULT '',
  `img_size` int(8) unsigned NOT NULL DEFAULT '0',
  `img_width` int(5) NOT NULL DEFAULT '0',
  `img_height` int(5) NOT NULL DEFAULT '0',
  `img_metadata` mediumblob NOT NULL,
  `img_bits` int(3) NOT NULL DEFAULT '0',
  `img_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
  `img_major_mime` enum('unknown','application','audio','image','text','video','message','model','multipart') NOT NULL DEFAULT 'unknown',
  `img_minor_mime` varbinary(100) NOT NULL DEFAULT 'unknown',
  `img_description_id` bigint(20) unsigned NOT NULL,
  `img_actor` bigint(20) unsigned NOT NULL,
  `img_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `img_sha1` varbinary(32) NOT NULL DEFAULT '',
  `img_deleted` tinyint(3) unsigned NOT NULL DEFAULT '0',
  PRIMARY KEY (`img_name`),
  KEY `img_size` (`img_size`),
  KEY `img_timestamp` (`img_timestamp`),
  KEY `img_media_mime` (`img_media_type`,`img_major_mime`,`img_minor_mime`),
  KEY `img_actor_timestamp` (`img_actor`,`img_timestamp`),
  KEY `img_sha1` (`img_sha1`(10))
) ENGINE=InnoDB DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
oldimage
*************************** 1. row ***************************
       Table: oldimage
Create Table: CREATE TABLE `oldimage` (
  `oi_name` varbinary(255) NOT NULL DEFAULT '',
  `oi_archive_name` varbinary(255) NOT NULL DEFAULT '',
  `oi_size` int(8) unsigned NOT NULL DEFAULT '0',
  `oi_width` int(5) NOT NULL DEFAULT '0',
  `oi_height` int(5) NOT NULL DEFAULT '0',
  `oi_bits` int(3) NOT NULL DEFAULT '0',
  `oi_description_id` bigint(20) unsigned NOT NULL,
  `oi_actor` bigint(20) unsigned NOT NULL,
  `oi_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `oi_sha1` varbinary(32) NOT NULL DEFAULT '',
  `oi_metadata` mediumblob NOT NULL,
  `oi_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
  `oi_major_mime` enum('unknown','application','audio','image','text','video','message','model','multipart') NOT NULL DEFAULT 'unknown',
  `oi_minor_mime` varbinary(100) NOT NULL DEFAULT 'unknown',
  `oi_deleted` tinyint(3) unsigned NOT NULL DEFAULT '0',
  KEY `oi_name_timestamp` (`oi_name`,`oi_timestamp`),
  KEY `oi_name_archive_name` (`oi_name`,`oi_archive_name`(14)),
  KEY `oi_actor_timestamp` (`oi_actor`,`oi_timestamp`),
  KEY `oi_sha1` (`oi_sha1`(10))
) ENGINE=InnoDB DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
filearchive
*************************** 1. row ***************************
       Table: filearchive
Create Table: CREATE TABLE `filearchive` (
  `fa_id` int(11) NOT NULL AUTO_INCREMENT,
  `fa_name` varbinary(255) NOT NULL DEFAULT '',
  `fa_archive_name` varbinary(255) DEFAULT '',
  `fa_storage_group` varbinary(16) DEFAULT NULL,
  `fa_storage_key` varbinary(64) DEFAULT '',
  `fa_deleted_user` int(11) DEFAULT NULL,
  `fa_deleted_timestamp` varbinary(14) DEFAULT '',
  `fa_deleted_reason_id` bigint(20) unsigned NOT NULL,
  `fa_size` int(8) unsigned DEFAULT '0',
  `fa_width` int(5) DEFAULT '0',
  `fa_height` int(5) DEFAULT '0',
  `fa_metadata` mediumblob,
  `fa_bits` int(3) DEFAULT '0',
  `fa_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
  `fa_major_mime` enum('unknown','application','audio','image','text','video','message','model','multipart') DEFAULT 'unknown',
  `fa_minor_mime` varbinary(100) DEFAULT 'unknown',
  `fa_description_id` bigint(20) unsigned NOT NULL,
  `fa_actor` bigint(20) unsigned NOT NULL,
  `fa_timestamp` varbinary(14) DEFAULT '',
  `fa_deleted` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `fa_sha1` varbinary(32) NOT NULL DEFAULT '',
  PRIMARY KEY (`fa_id`),
  KEY `fa_name` (`fa_name`,`fa_timestamp`),
  KEY `fa_storage_group` (`fa_storage_group`,`fa_storage_key`),
  KEY `fa_deleted_timestamp` (`fa_deleted_timestamp`),
  KEY `fa_sha1` (`fa_sha1`(10)),
  KEY `fa_actor_timestamp` (`fa_actor`,`fa_timestamp`)
) ENGINE=InnoDB AUTO_INCREMENT=140921 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
recentchanges
*************************** 1. row ***************************
       Table: recentchanges
Create Table: CREATE TABLE `recentchanges` (
  `rc_id` int(8) NOT NULL AUTO_INCREMENT,
  `rc_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `rc_actor` bigint(20) unsigned NOT NULL,
  `rc_namespace` int(11) NOT NULL DEFAULT '0',
  `rc_title` varbinary(255) NOT NULL DEFAULT '',
  `rc_comment_id` bigint(20) unsigned NOT NULL,
  `rc_minor` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_bot` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_new` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_cur_id` int(10) unsigned NOT NULL DEFAULT '0',
  `rc_this_oldid` int(10) unsigned NOT NULL DEFAULT '0',
  `rc_last_oldid` int(10) unsigned NOT NULL DEFAULT '0',
  `rc_type` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_source` varbinary(16) NOT NULL DEFAULT '',
  `rc_patrolled` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_ip` varbinary(40) NOT NULL DEFAULT '',
  `rc_old_len` int(10) DEFAULT NULL,
  `rc_new_len` int(10) DEFAULT NULL,
  `rc_deleted` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `rc_logid` int(10) unsigned NOT NULL DEFAULT '0',
  `rc_log_type` varbinary(255) DEFAULT NULL,
  `rc_log_action` varbinary(255) DEFAULT NULL,
  `rc_params` blob,
  PRIMARY KEY (`rc_id`),
  KEY `rc_timestamp` (`rc_timestamp`),
  KEY `rc_cur_id` (`rc_cur_id`),
  KEY `new_name_timestamp` (`rc_new`,`rc_namespace`,`rc_timestamp`),
  KEY `rc_ip` (`rc_ip`),
  KEY `tmp_2` (`rc_bot`,`rc_timestamp`),
  KEY `tmp_3` (`rc_namespace`,`rc_timestamp`),
  KEY `rc_name_type_patrolled_timestamp` (`rc_namespace`,`rc_type`,`rc_patrolled`,`rc_timestamp`),
  KEY `rc_ns_actor` (`rc_namespace`,`rc_actor`),
  KEY `rc_actor` (`rc_actor`,`rc_timestamp`),
  KEY `rc_namespace_title_timestamp` (`rc_namespace`,`rc_title`,`rc_timestamp`),
  KEY `rc_this_oldid` (`rc_this_oldid`)
) ENGINE=InnoDB AUTO_INCREMENT=395253359 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
logging
*************************** 1. row ***************************
       Table: logging
Create Table: CREATE TABLE `logging` (
  `log_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `log_type` varbinary(32) NOT NULL DEFAULT '',
  `log_action` varbinary(32) NOT NULL DEFAULT '',
  `log_timestamp` varbinary(14) NOT NULL DEFAULT '19700101000000',
  `log_namespace` int(11) NOT NULL DEFAULT '0',
  `log_title` varbinary(255) NOT NULL DEFAULT '',
  `log_comment_id` bigint(20) unsigned NOT NULL,
  `log_params` blob NOT NULL,
  `log_deleted` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `log_actor` bigint(20) unsigned NOT NULL,
  `log_page` int(10) unsigned DEFAULT NULL,
  PRIMARY KEY (`log_id`),
  KEY `type_time` (`log_type`,`log_timestamp`),
  KEY `page_time` (`log_namespace`,`log_title`,`log_timestamp`),
  KEY `times` (`log_timestamp`),
  KEY `log_page_id_time` (`log_page`,`log_timestamp`),
  KEY `type_action` (`log_type`,`log_action`,`log_timestamp`),
  KEY `actor_time` (`log_actor`,`log_timestamp`),
  KEY `log_actor_type_time` (`log_actor`,`log_type`,`log_timestamp`),
  KEY `log_type_action` (`log_type`,`log_action`,`log_timestamp`)
) ENGINE=InnoDB AUTO_INCREMENT=102243986 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
protected_titles
*************************** 1. row ***************************
       Table: protected_titles
Create Table: CREATE TABLE `protected_titles` (
  `pt_namespace` int(11) NOT NULL DEFAULT '0',
  `pt_title` varbinary(255) NOT NULL DEFAULT '',
  `pt_user` int(10) unsigned NOT NULL DEFAULT '0',
  `pt_reason_id` bigint(20) unsigned NOT NULL,
  `pt_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `pt_expiry` varbinary(14) NOT NULL DEFAULT '',
  `pt_create_perm` varbinary(60) NOT NULL DEFAULT '',
  PRIMARY KEY (`pt_namespace`,`pt_title`),
  KEY `pt_timestamp` (`pt_timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8

Mentioned in SAL (#wikimedia-operations) [2019-10-03T08:26:52Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1096:3316 for schema change T233135 T234066', diff saved to https://phabricator.wikimedia.org/P9236 and previous config saved to /var/cache/conftool/dbconfig/20191003-082651-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2019-10-03T08:33:13Z] <marostegui> Deploy schema change on db2087:3316 T233135 T234066

Anomie added a comment.Thu, Oct 3, 3:51 PM

This is how the tables look like after this alter and the alter for the special slaves (T233625)

Those all look good as far as this task goes. Thanks!

I see several unrelated schema differences versus tables.sql, which isn't too surprising. We already know about most of them, I think.

This is how the tables look like after this alter and the alter for the special slaves (T233625)

Those all look good as far as this task goes. Thanks!
I see several unrelated schema differences versus tables.sql, which isn't too surprising. We already know about most of them, I think.

Yeah, to be tackle in separate tasks, we still have lots of work ahead of us to make ourselves consistent with tables.sql

Mentioned in SAL (#wikimedia-operations) [2019-10-04T06:16:47Z] <marostegui> Deploy schema change on dbstore1005:3316 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-04T06:40:59Z] <marostegui> Deploy schema change on db2114 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-04T08:41:38Z] <marostegui> Deploy schema change on db2076 (sanitarium master) with replication T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-04T12:28:29Z] <marostegui> Deploy schema change on db2097:3316 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-04T14:03:21Z] <marostegui> Deploy schema change on db2117 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-07T04:54:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1098:3316 for schema change T233135 T234066', diff saved to https://phabricator.wikimedia.org/P9245 and previous config saved to /var/cache/conftool/dbconfig/20191007-045411-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2019-10-07T05:25:27Z] <marostegui> Deploy schema change on db2124 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-07T09:55:01Z] <marostegui> Deploy schema change on db2129 (s6 codfw master), this will generate lag on s6 codfw - T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-07T13:17:22Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1098:3316 schema change T233135 T234066', diff saved to https://phabricator.wikimedia.org/P9248 and previous config saved to /var/cache/conftool/dbconfig/20191007-131720-marostegui.json

Marostegui updated the task description. (Show Details)Mon, Oct 7, 2:12 PM

Mentioned in SAL (#wikimedia-operations) [2019-10-07T14:17:40Z] <marostegui> Deploy schema change on db1139:3316 - T233135 T234066

The only view that needs to be recreated on labs hosts is the one for protected_titles the rest are fine.

for i in archive ipblocks image oldimage filearchive recentchanges logging protected_titles logging; do echo $i; mysql.py -hlabsdb1009 frwiki_p -e "select * from $i limit 1;" | grep ERROR;done
archive
ipblocks
image
oldimage
filearchive
recentchanges
logging
protected_titles
ERROR 1356 (HY000) at line 1: View 'frwiki_p.protected_titles' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them
logging
Marostegui updated the task description. (Show Details)Wed, Oct 9, 9:01 AM
Anomie added a comment.Wed, Oct 9, 2:56 PM

Good catch. That's because protected_titles is in "fullviews" instead of "customviews".

Mentioned in SAL (#wikimedia-operations) [2019-10-10T04:53:54Z] <marostegui> Deploy schema change on db1061 (s6 eqiad master) - T233135 T234066

Marostegui updated the task description. (Show Details)Thu, Oct 10, 10:11 AM

Mentioned in SAL (#wikimedia-operations) [2019-10-10T11:40:53Z] <marostegui> Deploy schema change on s7 codfw master (db2118), this will generate lag on s7 codfw - T234066 T233135

Marostegui updated the task description. (Show Details)Fri, Oct 11, 4:51 AM
Marostegui added a comment.EditedMon, Oct 14, 9:13 AM

s7 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1003
  • db1136
  • db1125
  • db1116
  • db1101
  • db1098
  • db1094
  • db1090
  • db1086
  • db1079
  • db1062

Mentioned in SAL (#wikimedia-operations) [2019-10-14T14:21:34Z] <marostegui> Deploy schema change on db1116:3317 T234066 T233135

Mentioned in SAL (#wikimedia-operations) [2019-10-15T05:28:49Z] <marostegui> Deploy schema change on db1098:3317 T234066 T233135

Mentioned in SAL (#wikimedia-operations) [2019-10-16T05:08:05Z] <marostegui> Deploy schema change on s7 sanitarium master (db1079) this will create lag on s7 labsdb T233135 T234066

Marostegui added a comment.EditedWed, Oct 16, 5:16 AM

s2 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004
  • db1129
  • db1125
  • db1122
  • db1105
  • db1103
  • db1095
  • db1090
  • db1076
  • db1074

Mentioned in SAL (#wikimedia-operations) [2019-10-16T05:18:28Z] <marostegui> Deploy schema change on s2 sanitarium master (db1074) this will create lag on s2 labsdb T233135 T234066

Marostegui updated the task description. (Show Details)Wed, Oct 16, 5:56 AM

Mentioned in SAL (#wikimedia-operations) [2019-10-16T10:17:10Z] <marostegui> Stop replication on s2 codfw master for schema change and to modify sanitarium triggers T234066 T233135 T234704

Mentioned in SAL (#wikimedia-operations) [2019-10-16T15:09:17Z] <marostegui> Recreate views for protected_titles on s2 and s7 on labsdb1009 and labsdb1012 - T233135

Mentioned in SAL (#wikimedia-operations) [2019-10-16T15:17:27Z] <marostegui> Deploy schema change on dbstore1004:3312 - T234066 T233135

Marostegui updated the task description. (Show Details)Thu, Oct 17, 5:04 AM
Marostegui updated the task description. (Show Details)Thu, Oct 17, 5:25 AM
Marostegui updated the task description. (Show Details)Thu, Oct 17, 5:30 AM