Page MenuHomePhabricator

labswiki: Field 'notification_bundle_display_hash' doesn't have a default value (10.64.0.98)
Closed, ResolvedPublic

Description

Creating this for the record
After m5 switchover yesterday (T260324) I am seeing some errors on logstash related to wikitech. Not many, 13 errors in 24h.

Error 1364 from EchoNotificationMapper::insert, Field 'notification_bundle_display_hash' doesn't have a default value (10.64.0.98) INSERT INTO `echo_notification` (notification_event,notification_user,notification_timestamp,notification_read_timestamp,notification_bundle_hash) VALUES (846840,xxxx,'20200904044316',NULL,'') 10.64.0.98

The table is identical on the new master and on the old master:

mysql.py -hdb1128 -A labswiki -e "show create table echo_notification\G"
*************************** 1. row ***************************
       Table: echo_notification
Create Table: CREATE TABLE `echo_notification` (
  `notification_event` int(10) unsigned NOT NULL,
  `notification_user` int(10) unsigned NOT NULL,
  `notification_timestamp` binary(14) NOT NULL,
  `notification_read_timestamp` binary(14) DEFAULT NULL,
  `notification_bundle_base` tinyint(1) NOT NULL DEFAULT 1,
  `notification_bundle_hash` varbinary(32) NOT NULL,
  `notification_bundle_display_hash` varbinary(32) NOT NULL,
  `notification_seen` tinyint(1) NOT NULL DEFAULT 0,
  UNIQUE KEY `user_event` (`notification_user`,`notification_event`),
  KEY `echo_notification_user_base_read_timestamp` (`notification_user`,`notification_bundle_base`,`notification_read_timestamp`),
  KEY `echo_notification_user_base_timestamp` (`notification_user`,`notification_bundle_base`,`notification_timestamp`,`notification_event`),
  KEY `echo_notification_user_hash_timestamp` (`notification_user`,`notification_bundle_hash`,`notification_timestamp`),
  KEY `echo_notification_user_hash_base_timestamp` (`notification_user`,`notification_bundle_display_hash`,`notification_bundle_base`,`notification_timestamp`),
  KEY `echo_user_timestamp` (`notification_user`,`notification_timestamp`),
  KEY `echo_notification_event` (`notification_event`),
  KEY `echo_notification_user_read_timestamp` (`notification_user`,`notification_read_timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=binary

I noticed that on 10.4 the new master had: STRICT_TRANS_TABLES enabled, while the old one doesn't.
I have changed that on the new master to match what the old one had.

Event Timeline

Marostegui added a subscriber: Andrew.
dduvall added a subscriber: dduvall.

Reopening and categorizing as logspam since we're still seeing this albeit infrequently.

Most recent entry is reqId: dc3b4ca609e1d8c9943a630b at Feb 19, 2021 @ 19:30:12.208

This should fix it for now:

root@db1128.eqiad.wmnet[(none)]> set global sql_mode='IGNORE_BAD_TABLE_OPTIONS';
Query OK, 0 rows affected (0.001 sec)

root@db1128.eqiad.wmnet[(none)]>

@Marostegui How does this compare to the main wiki clusters such as s1, s7 and extension1? Does that have the same ignore mode set? Does it seem good or bad that we ignore this? Is there something Echo could improve to behave better here?

@Marostegui How does this compare to the main wiki clusters such as s1, s7 and extension1? Does that have the same ignore mode set? Does it seem good or bad that we ignore this? Is there something Echo could improve to behave better here?

Yes, those have the same one, it is probably something specific puppet-related to that misc master (I haven't checked yet), as wikitech runs on misc hosts until we complete: T167973

The field was dropped with T143763

It is possible that the drop was not done on labswiki?

Thanks @Umherirrender - thats a very good point. It was definitely not applied on labswiki as the original ticket mentioned x1 databases, and labswiki has those tables within the same host, not on x1.

I will drop that field there.
Thanks

Marostegui added a project: DBA.