Page MenuHomePhabricator

Wikimedia\Rdbms\DBQueryError: Error 1062: Duplicate entry '…' for key 'itid_itemid' / 'it_itemname'
Closed, ResolvedPublic

Description

From the last attempt at T315353. 157 occurrences in ~10 minutes before we reverted: https://logstash.wikimedia.org/goto/a734c099589019f597dc3e7b34813884

Sample errors:

Error 1062: Duplicate entry 'h-Tech_News:_2021-07' for key 'itid_itemid'

https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-deploy-1-7.0.0-1-2022.11.08?id=VCE0WYQBkh1O0J80WvXc

Error 1062: Duplicate entry 'h-EranBot-2021-03-06T21:20:00.000Z' for key 'it_itemname'

https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-deploy-1-7.0.0-1-2022.11.08?id=SdU0WYQBGZfvtWAW8_cK

Related Objects

Event Timeline

Change 854647 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/DiscussionTools@master] ThreadItemStore: Handle race conditions when finding/inserting outside of transaction

https://gerrit.wikimedia.org/r/854647

What slightly worries me is that in five minutes it had 70 cases, it is possible to have race conditions but not that many. Maybe we can work on deduplicating the job if a newer edit happens and takes care of it?

Indeed, I was not really expecting this to happen at all. I don't control the jobs though, there could be a deeper issue here but I don't very much want to get into it (I'm not even sure what kind of a job it is, our code just uses the RevisionDataUpdates hook). It could also just have been a fluke, caused by identical content being transcluded on different pages that were processed at (almost) the same time.

I can take a look a bit later. Maybe I manage to find some skeletons

Change 854647 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@master] ThreadItemStore: Handle race conditions when finding/inserting outside of transaction

https://gerrit.wikimedia.org/r/854647

Change 855744 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/DiscussionTools@wmf/1.40.0-wmf.8] ThreadItemStore: Handle race conditions when finding/inserting outside of transaction

https://gerrit.wikimedia.org/r/855744

Change 855744 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@wmf/1.40.0-wmf.8] ThreadItemStore: Handle race conditions when finding/inserting outside of transaction

https://gerrit.wikimedia.org/r/855744

Mentioned in SAL (#wikimedia-operations) [2022-11-14T14:14:24Z] <taavi@deploy1002> Started scap: Backport for [[gerrit:856550|Use legacy DiscussionTools heading markup except on beta cluster (T314714)]], [[gerrit:855744|ThreadItemStore: Handle race conditions when finding/inserting outside of transaction (T322701)]], [[gerrit:855745|persistRevisionThreadItems: Print time taken]]

Mentioned in SAL (#wikimedia-operations) [2022-11-14T14:14:44Z] <taavi@deploy1002> taavi and matmarex: Backport for [[gerrit:856550|Use legacy DiscussionTools heading markup except on beta cluster (T314714)]], [[gerrit:855744|ThreadItemStore: Handle race conditions when finding/inserting outside of transaction (T322701)]], [[gerrit:855745|persistRevisionThreadItems: Print time taken]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmn

Mentioned in SAL (#wikimedia-operations) [2022-11-14T14:20:39Z] <taavi@deploy1002> Finished scap: Backport for [[gerrit:856550|Use legacy DiscussionTools heading markup except on beta cluster (T314714)]], [[gerrit:855744|ThreadItemStore: Handle race conditions when finding/inserting outside of transaction (T322701)]], [[gerrit:855745|persistRevisionThreadItems: Print time taken]] (duration: 06m 14s)