"Duplicate entry 'XXX-YYY' for key 'itr_itemid_id_revision_id'" in "INSERT INTO `discussiontools_item_revisions`"
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	matmarex
	Nov 14 2022, 10:09 PM

Description

(The latest deployment of DiscussionTools permalinks backend to group1 wikis T315353#8392662 has resulted in some new errors, although a lot less numerous this time. It really is a gift that keeps on giving.)

There are a few occurrences of "Duplicate entry 'XXX-YYY' for key 'itr_itemid_id_revision_id'" (where "XXX" and "YYY" are different numbers).

https://logstash.wikimedia.org/goto/fc6ea240e98b381db12b9d275915e001

Error

Error 1062: Duplicate entry 'XXX-YYY' for key 'itr_itemid_id_revision_id'
Function: MediaWiki\Extension\DiscussionTools\ThreadItemStore::insertThreadItems
Query: INSERT INTO `discussiontools_item_revisions` (itr_itemid_id,itr_revision_id,itr_items_id,itr_parent_id,itr_transcludedfrom,itr_level) VALUES ('XXX',YYY,'ZZZ','AAA',NULL,2)

exception.trace

…
#4 /srv/mediawiki/php-1.40.0-wmf.17/extensions/DiscussionTools/includes/ThreadItemStore.php(555): Wikimedia\Rdbms\Database->insert(string, array, string)
#5 /srv/mediawiki/php-1.40.0-wmf.17/includes/libs/rdbms/database/Database.php(2620): MediaWiki\Extension\DiscussionTools\ThreadItemStore->MediaWiki\Extension\DiscussionTools\{closure}(Wikimedia\Rdbms\DatabaseMysqli, string)
#6 /srv/mediawiki/php-1.40.0-wmf.17/includes/libs/rdbms/database/DBConnRef.php(119): Wikimedia\Rdbms\Database->doAtomicSection(string, Closure, string)
#7 /srv/mediawiki/php-1.40.0-wmf.17/includes/libs/rdbms/database/DBConnRef.php(680): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#8 /srv/mediawiki/php-1.40.0-wmf.17/extensions/DiscussionTools/includes/ThreadItemStore.php(566): Wikimedia\Rdbms\DBConnRef->doAtomicSection(string, Closure, string)
#9 /srv/mediawiki/php-1.40.0-wmf.17/extensions/DiscussionTools/includes/Hooks/DataUpdatesHooks.php(47): MediaWiki\Extension\DiscussionTools\ThreadItemStore->insertThreadItems(MediaWiki\Revision\RevisionStoreRecord, MediaWiki\Extension\DiscussionTools\ContentThreadItemSet)
#10 /srv/mediawiki/php-1.40.0-wmf.17/includes/deferred/MWCallableUpdate.php(38): MediaWiki\Extension\DiscussionTools\Hooks\DataUpdatesHooks->MediaWiki\Extension\DiscussionTools\Hooks\{closure}()
#11 /srv/mediawiki/php-1.40.0-wmf.17/includes/deferred/DeferredUpdates.php(474): MWCallableUpdate->doUpdate()
#12 /srv/mediawiki/php-1.40.0-wmf.17/includes/deferred/RefreshSecondaryDataUpdate.php(103): DeferredUpdates::attemptUpdate(MWCallableUpdate, Wikimedia\Rdbms\LBFactoryMulti)
#13 /srv/mediawiki/php-1.40.0-wmf.17/includes/deferred/DeferredUpdates.php(474): RefreshSecondaryDataUpdate->doUpdate()
#14 /srv/mediawiki/php-1.40.0-wmf.17/includes/Storage/DerivedPageDataUpdater.php(1834): DeferredUpdates::attemptUpdate(RefreshSecondaryDataUpdate, Wikimedia\Rdbms\LBFactoryMulti)
#15 /srv/mediawiki/php-1.40.0-wmf.17/includes/page/WikiPage.php(2150): MediaWiki\Storage\DerivedPageDataUpdater->doSecondaryDataUpdates(array)
#16 /srv/mediawiki/php-1.40.0-wmf.17/includes/jobqueue/jobs/RefreshLinksJob.php(243): WikiPage->doSecondaryDataUpdates(array)
#17 /srv/mediawiki/php-1.40.0-wmf.17/includes/jobqueue/jobs/RefreshLinksJob.php(152): RefreshLinksJob->runForTitle(Title)
#18 /srv/mediawiki/php-1.40.0-wmf.17/extensions/EventBus/includes/JobExecutor.php(79): RefreshLinksJob->run()
#19 /srv/mediawiki/rpc/RunSingleJob.php(77): MediaWiki\Extension\EventBus\JobExecutor->execute(array)
#20 {main}

It looks like the revisions are being processed simultaneously by multiple jobs? (as Ladsgroup was guessing in T322701#8382329)

I don't know if we can dig into that, but if that's the case, this should be harmless from our perspective – we should avoid logging errors about it when it happens, though…

(This is distinct from T323079: "Duplicate entry '0-XXX' for key 'itr_itemid_id_revision_id'" in "INSERT INTO `discussiontools_item_revisions`")

Related Objects
Search...

Status	Assigned	Task
Open	None	T249579 [EPIC] Usability improvements: make the actions, activity and content within talk pages easier to understand
Open	None	T269950 [EPIC] Topic Containers
Open	None	T295431 Enable Special:TopicSubscriptions Contents to Update Automatically
Open	None	T314973 Improve Topic Subscription support for discussions that are merged, moved, or deleted
Open	None	T315050 Enable people to visit the page the discussion they were subscribed to has been moved to
Open	None	T294592 [EPIC] Ensure Talk Pages Project "Phase 1" work is easy to maintain, build upon, and learn from
Open	None	T315507 Introduce support for topic and comment permanent links within the Android and iOS apps
Open	None	T318449 [Tracking] Improve the sharing of MediaWiki content
Open	None	T318446 [Tracking] Improve sharing link functionality of pages
Open	None	T302011 [Release Ticket] Introduce permalinks on wikitext talk pages
Open	nayoub	T275729 Comment permalinks: design and implement user experience
Open	None	T306373 Update Special:TopicSubscriptions to use permalinks
Open	None	T273341 Topic permalinks: design and implement user experience
Resolved	matmarex	T274685 Comment IDs that reliably identify the comment across revisions/pages (comment names)
Open	matmarex	T315510 Start maintenance script to backfill talk page comment database
Open	matmarex	T315353 Create database tables for permalinks in production wikis, and enable the feature
Resolved	matmarex	T296801 Introduce a data structure for tracking talk page comments
Resolved	matmarex	T303295 DBA review: data structure for storing permalinks
Open	None	T323080 "Duplicate entry 'XXX-YYY' for key 'itr_itemid_id_revision_id'" in "INSERT INTO `discussiontools_item_revisions`"

Event Timeline

matmarex created this task.Nov 14 2022, 10:09 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 14 2022, 10:09 PM

matmarex mentioned this in T323079: "Duplicate entry '0-XXX' for key 'itr_itemid_id_revision_id'" in "INSERT INTO `discussiontools_item_revisions`".Nov 14 2022, 10:11 PM

matmarex updated the task description. (Show Details)

matmarex added a project: Wikimedia-production-error.

matmarex added a parent task: T315353: Create database tables for permalinks in production wikis, and enable the feature.

brennen added a project: User-brennen.Nov 15 2022, 5:39 PM

brennen moved this task from Backlog to Logs/Train on the User-brennen board.

Looking at an example of it for example https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-mediawiki-1-7.0.0-1-2022.11.15?id=R15SfIQBGZfvtWAWG4lG
The revision is quite old, there is no newer edit on the page so it is very likely caused by refreshlinks job being somehow trying to run the DT update twice (or the update itself is trying it twice).

I might be wrong but it feels like the last update in the transaction (starting line 526 on discussiontools_item_revisions) is suffering from REPEATABLE_READ issue as well?

ppelberg moved this task from Backlog to Triaged on the DiscussionTools board.Nov 17 2022, 4:07 PM

ppelberg added a project: Editing-team.

ppelberg moved this task from Untriaged to Upcoming on the Editing-team board.

Krinkle moved this task from Untriaged to Nov 2022 on the Wikimedia-production-error board.Nov 17 2022, 4:23 PM

Noting that this and T323079: "Duplicate entry '0-XXX' for key 'itr_itemid_id_revision_id'" in "INSERT INTO `discussiontools_item_revisions`" are happening at what feels like a somewhat higher rate than the last few days. Checking logstash, that may not actually be the case, but they're persistent enough to stand out to deployers. (We've filtered them out of the mediawiki-new-errors dashboard in logstash, but they're something of an annoyance in logspam-watch.)

Krinkle updated the task description. (Show Details)Jan 5 2023, 9:43 PM

"Duplicate entry 'XXX-YYY' for key 'itr_itemid_id_revision_id'" in "INSERT INTO `discussiontools_item_revisions`"Open, Needs TriagePublicActions

Description

Related ObjectsSearch...

Event Timeline

"Duplicate entry 'XXX-YYY' for key 'itr_itemid_id_revision_id'" in "INSERT INTO `discussiontools_item_revisions`"
Open, Needs TriagePublic
Actions

Related Objects
Search...