Page MenuHomePhabricator

Remove unused bundling DB fields
Closed, ResolvedPublic

Description

notification_bundle_base and notification_bundle_display_hash are no longer used now that T120153: Change bundling system to allow individual treatment is done.

SQL:

Where: all x1 databases that have an echo_event table (same set as echo.dblist)
Backwards compatible: yes, now that the patches are merged and deployed the software doesn't reference these fields anymore

Event Timeline

Catrope created this task.Aug 24 2016, 2:24 AM
Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptAug 24 2016, 2:24 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Catrope claimed this task.Aug 24 2016, 2:40 AM

Change 306845 had a related patch set uploaded (by Catrope):
Remove unused method EchoEventMapper::fetchByUserBundleHash

https://gerrit.wikimedia.org/r/306845

Change 306846 had a related patch set uploaded (by Catrope):
Remove notification_bundle_base

https://gerrit.wikimedia.org/r/306846

Catrope renamed this task from Remove notification_bundle_base to Remove unused bundling DB fields.Aug 25 2016, 11:53 PM
Catrope updated the task description. (Show Details)

Change 306849 had a related patch set uploaded (by Catrope):
Remove notification_bundle_display_hash

https://gerrit.wikimedia.org/r/306849

Change 306845 merged by jenkins-bot:
Remove unused method EchoEventMapper::fetchByUserBundleHash

https://gerrit.wikimedia.org/r/306845

Catrope updated the task description. (Show Details)Mar 4 2019, 10:43 PM
Restricted Application added a project: Growth-Team. · View Herald TranscriptMar 4 2019, 10:43 PM

Change 306846 merged by jenkins-bot:
[mediawiki/extensions/Echo@master] Remove notification_bundle_base

https://gerrit.wikimedia.org/r/306846

Change 306849 merged by jenkins-bot:
[mediawiki/extensions/Echo@master] Remove notification_bundle_display_hash

https://gerrit.wikimedia.org/r/306849

Catrope updated the task description. (Show Details)
Marostegui triaged this task as Normal priority.Mar 15 2019, 6:20 AM
Marostegui added a subscriber: Marostegui.

x1 runs row based replication, which means this may break replication if executed first on the slaves and then on the master.
We can probably put the master to serve reads (once we have the new master in place T211613: rack/setup/install db11[26-38].eqiad.wmnet) and execute it with replication.

I am going to do a test with these schema changes, as the table isn't big, I am going to put the master to serve reads and depool the slaves and execute it for a few small wikis with replication (they are mostly dropping an index and just 2 columns drop).
Will get the patch depooling the slaves reviewed before merging.

Change 500654 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool all slaves in x1

https://gerrit.wikimedia.org/r/500654

Change 500654 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool all slaves in x1

https://gerrit.wikimedia.org/r/500654

Mentioned in SAL (#wikimedia-operations) [2019-04-02T08:16:40Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool all slaves in x1 T219777 T143763 (duration: 00m 53s)

Mentioned in SAL (#wikimedia-operations) [2019-04-02T08:50:00Z] <marostegui> Execute schema change on db1069 x1 master with replication enabled on the following small wikis: aawiki aawikibooks aawiktionary abwiki abwiktionary acewiki advisorswiki advisorywiki adywiki afwiki T143763

Mentioned in SAL (#wikimedia-operations) [2019-04-02T08:58:56Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool all slaves in x1 T219777 T143763 (duration: 00m 53s)

So, I have run both changes directly on the master with replication enabled on the following wikis:

aawiki
aawikibooks
aawiktionary
abwiki
abwiktionary
acewiki
advisorswiki
advisorywiki
adywiki
afwiki

The table now looks like this:

mysql.py -hdb1064 afwiki -e "show create table echo_notification\G"
*************************** 1. row ***************************
       Table: echo_notification
Create Table: CREATE TABLE `echo_notification` (
  `notification_event` int(10) unsigned NOT NULL,
  `notification_user` int(10) unsigned NOT NULL,
  `notification_timestamp` binary(14) NOT NULL,
  `notification_read_timestamp` binary(14) DEFAULT NULL,
  `notification_bundle_hash` varchar(32) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
  `notification_seen` tinyint(1) NOT NULL DEFAULT '0',
  PRIMARY KEY (`notification_user`,`notification_event`),
  KEY `echo_user_timestamp` (`notification_user`,`notification_timestamp`),
  KEY `echo_notification_user_hash_timestamp` (`notification_user`,`notification_bundle_hash`,`notification_timestamp`),
  KEY `echo_notification_event` (`notification_event`),
  KEY `echo_notification_user_read_timestamp` (`notification_user`,`notification_read_timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

The change was pretty fast as the tables there were pretty small. The biggest one was 12M on disk.
I warmed up the tables before hand too.

The master has been serving all x1 traffic for around 45 minutes with no much problem

https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-dc=eqiad%20prometheus%2Fops&var-server=db1069&var-port=9104&from=1554192789545&to=1554195687545

I think we can probably fully proceed with this one day early in the morning although there is not a big decrease in traffic during the day, maybe the best gap would be 4-6AM UTC, but even there, there is not such a big difference in traffic anyways.

I will probably depool all slaves on x1 on Monday early in the morning and go ahead and alter the rest of the wikis.

Change 501147 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool all x1 slaves

https://gerrit.wikimedia.org/r/501147

Change 501147 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool all x1 slaves

https://gerrit.wikimedia.org/r/501147

Mentioned in SAL (#wikimedia-operations) [2019-04-08T05:18:01Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool all slaves in x1 T219777 T143763 (duration: 01m 30s)

Mentioned in SAL (#wikimedia-operations) [2019-04-08T05:20:02Z] <marostegui> Deploy schema change on x1 master with replication, there will be lag on x1 slaves T143763

Mentioned in SAL (#wikimedia-operations) [2019-04-08T07:02:35Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool all slaves in x1 T143763 (duration: 00m 58s)

Marostegui closed this task as Resolved.Apr 8 2019, 7:02 AM

This is all done!