There are many more occurrences now, with different indexes. They seem to be related to processing pages that have headings with no comments (in discussion namespaces, but mostly not actual discussion pages), e.g. https://ie.wikibooks.org/wiki/Wikibooks:Nospam#Pages_locked_from_recreation.

Change 854138 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/DiscussionTools@master] ThreadItemStore: Fix setting parent IDs when parent already existed

https://gerrit.wikimedia.org/r/854138

The error occurred whenever the script processed a revision that was already processed, e.g. by a refreshlinks job. It processes the revision again, in case new data can be generated from it (which can happen if it contains transclusions of other talk pages).

As a result of the bug, when generating new data, it would use an incorrect value for the itr_parent_id field (always NULL, which should only be used for top-level headings – never for comments or sub-headings).

However, the incorrect data was probably never used – it was computed for every row (hence the high volume of errors), but it would only be saved for new rows, and there probably weren't actually any new rows for previously processed revisions on any of the wikis we processed this time (since we just enabled the feature a few hours ago, and there probably aren't any talk pages that contain transclusions of other talk pages on group0 wikis anyway).

Luckily, due to a little bit of redundancy in the schema, this can be verified with the following query, which returns data about all comments that have no parent assigned:

select * from discussiontools_item_revisions
where itr_level != 0 and itr_parent_id is null

– and it returns 0 rows on every wiki I tested.

@matmarex I ran that query across all group0 wikis, and it returns non-zero number of results for testwiki (all other wikis have zero results).

FTR, this is how I did it (from stat1005):

[urbanecm@stat1005 ~]$ source conda-activate-stacked
[...]
(2021-04-07T23.21.00_urbanecm) [urbanecm@stat1005 ~]$ ipython3
[...]
In [1]: from wmfdata import mariadb, utils

[...]


In [6]: dfs = []

In [7]: for wiki in wikis:
   ...:     if wiki == 'labtestwiki' or wiki == 'labswiki':
   ...:         continue
   ...:     dfs.append(mariadb.run('select database(), count(*) from discussiontools_item_revisions where itr_level != 0 and itr_parent_id is null', wiki))
   ...:

In [8]: import pandas as pd

In [9]: df = pd.concat(dfs)

In [10]: df.head()
Out[10]:
     database()  count(*)
0   aawikibooks         0
0  aawiktionary         0
0  abwiktionary         0
0  advisorywiki         0
0   akwikibooks         0

In [11]: df.loc[df['count(*)'] > 0]
Out[11]:
  database()  count(*)
0   testwiki         2

In [12]:

Change 854138 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@master] ThreadItemStore: Fix setting parent IDs when parent already existed

https://gerrit.wikimedia.org/r/854138

Change 854592 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/DiscussionTools@wmf/1.40.0-wmf.8] ThreadItemStore: Fix setting parent IDs when parent already existed

https://gerrit.wikimedia.org/r/854592

@Urbanecm Thanks for double-checking. So on testwiki I get two results corresponding to these comments (or in fact one comment, transcluded on 2 pages):

• https://test.wikipedia.org/wiki/Wikipedia:Requests#c-Xaosflux-20221103215900-Ponor
• https://test.wikipedia.org/wiki/Wikipedia:Requests/Permissions#c-Xaosflux-20221103215900-Ponor

This is exactly the scenario I described. Given that it's testwiki, I don't think it's worth worrying about fixing it (even though they are "real" comments, not just testing).

If we really wanted to fix it, I could tweak the maint. script so that it would update the buggy rows (rather than skip them because they exist), and run it with --current --page 'Wikipedia:Requests' --page 'Wikipedia:Requests/Permissions'.

No problem. I'm not sure what kinds of issues do those invalid rows – if you say that the buggy rows aren't worth fixing at testwiki, no issues with that.

Change 854592 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@wmf/1.40.0-wmf.8] ThreadItemStore: Fix setting parent IDs when parent already existed

https://gerrit.wikimedia.org/r/854592

Mentioned in SAL (#wikimedia-operations) [2022-11-08T21:26:22Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:851182|Bump sampling rate to 0.2 for various editing schemas on a/b test wikis (T321734)]], [[gerrit:854592|ThreadItemStore: Fix setting parent IDs when parent already existed (T322599)]]

Mentioned in SAL (#wikimedia-operations) [2022-11-08T21:26:41Z] <urbanecm@deploy1002> urbanecm and kemayo and matmarex: Backport for [[gerrit:851182|Bump sampling rate to 0.2 for various editing schemas on a/b test wikis (T321734)]], [[gerrit:854592|ThreadItemStore: Fix setting parent IDs when parent already existed (T322599)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2022-11-08T21:32:07Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:851182|Bump sampling rate to 0.2 for various editing schemas on a/b test wikis (T321734)]], [[gerrit:854592|ThreadItemStore: Fix setting parent IDs when parent already existed (T322599)]] (duration: 05m 45s)