Page MenuHomePhabricator

66 rows from external storage (dewiki) gave duplicate key errors on master failover
Closed, ResolvedPublic

Description

On external storage es2 failover, 66 rows written to dewiki es1015 gave a duplicate key error. The rows were deleted only on es1015 (not on the rest of servers of the shard), and then replication continued.

The rows were backed up on es2002. We checked the text tables on dewiki and we saw no duplicate references to those rows or orphan revisions however, so probably those new revisions were discarded and inserted with other ids, but a deeper analysis probably is required to make sure no data was loss.

Event Timeline

Marostegui claimed this task.
Marostegui added a subscriber: Marostegui.

As I am with the "checksumming" state of mind I have also taken care of this.
The table is consistent across all the eqiad hosts.