Page MenuHomePhabricator

Create database tables for permalinks in production wikis, and enable the feature
Closed, ResolvedPublic

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
Resolvedppelberg
ResolvedDLynch
Resolvedmatmarex
Resolvedmatmarex
Resolvedmatmarex
ResolvedPRODUCTION ERRORmatmarex
ResolvedPRODUCTION ERRORmatmarex
Resolvedmatmarex
ResolvedPRODUCTION ERRORmatmarex
ResolvedLadsgroup
Resolvedmatmarex
Resolvedmatmarex
Resolvedmatmarex
Resolvedmatmarex
Resolvedmatmarex
ResolvedPRODUCTION ERRORmatmarex
Resolvedmatmarex
Resolvedmatmarex
Invalidmatmarex
ResolvedPRODUCTION ERRORmatmarex
Resolvedmatmarex
Resolvedmatmarex
Resolvedmatmarex
Resolvedmatmarex
Resolvedmatmarex
ResolvedSecuritymatmarex

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 854097 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on group0 wikis

https://gerrit.wikimedia.org/r/854097

Change 854097 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on group0 wikis

https://gerrit.wikimedia.org/r/854097

Mentioned in SAL (#wikimedia-operations) [2022-11-07T21:17:34Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:854097|Enable wgDiscussionToolsEnablePermalinksBackend on group0 wikis (T315353)]]

Mentioned in SAL (#wikimedia-operations) [2022-11-07T21:17:54Z] <urbanecm@deploy1002> urbanecm and matmarex: Backport for [[gerrit:854097|Enable wgDiscussionToolsEnablePermalinksBackend on group0 wikis (T315353)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2022-11-07T21:23:22Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:854097|Enable wgDiscussionToolsEnablePermalinksBackend on group0 wikis (T315353)]] (duration: 05m 47s)

Change 854606 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis

https://gerrit.wikimedia.org/r/854606

Change 854606 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis

https://gerrit.wikimedia.org/r/854606

Mentioned in SAL (#wikimedia-operations) [2022-11-08T21:35:15Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:854606|Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis (T315353)]]

Mentioned in SAL (#wikimedia-operations) [2022-11-08T21:35:34Z] <urbanecm@deploy1002> urbanecm and matmarex: Backport for [[gerrit:854606|Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis (T315353)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2022-11-08T21:41:53Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:854606|Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis (T315353)]] (duration: 06m 36s)

Change 856566 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis (#2)

https://gerrit.wikimedia.org/r/856566

Change 856566 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis (#2)

https://gerrit.wikimedia.org/r/856566

Mentioned in SAL (#wikimedia-operations) [2022-11-14T14:23:05Z] <taavi@deploy1002> Started scap: Backport for [[gerrit:856566|Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis (#2) (T315353)]]

Mentioned in SAL (#wikimedia-operations) [2022-11-14T14:23:26Z] <taavi@deploy1002> taavi and matmarex: Backport for [[gerrit:856566|Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis (#2) (T315353)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2022-11-14T14:30:11Z] <taavi@deploy1002> Finished scap: Backport for [[gerrit:856566|Enable wgDiscussionToolsEnablePermalinksBackend on group1 wikis (#2) (T315353)]] (duration: 07m 05s)

I think I might have found the underlying problem here.
As I suspected, I think the DT updates (and probably any other secondary data update) are running twice in the same context.

For example, these two look similar if not almost identical:

But look at the start of stack trace. One is explicit call to $page->doSecondaryDataUpdates( $options ); in the job and one is deferred update being ran implicitly after the job is done which JobExecuter runs regardless of whether secondary data updates are done or not (see JobExecutor.php(105)). The only other possible explanation is that we have another job that completely unrelatedly queues secondary data updates in its context.

That was a red herring but looking at the data, refreshlinks jobs are mostly useless and won't make any change to the output (assuming dt storage is filled) but ThreadItemStore::insertThreadItems is being called regardless, maybe we can do a short-circuit when revision id already exists in dt.

Change 901697 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on group2 wikis

https://gerrit.wikimedia.org/r/901697

Change 901697 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on group2 wikis

https://gerrit.wikimedia.org/r/901697

Mentioned in SAL (#wikimedia-operations) [2023-03-21T20:02:55Z] <taavi@deploy2002> Started scap: Backport for [[gerrit:900331|Enable DiscussionTools_visualenhancements_newsectionlink_enable on labs for testing]], [[gerrit:901697|Enable wgDiscussionToolsEnablePermalinksBackend on group2 wikis (T315353)]]

Mentioned in SAL (#wikimedia-operations) [2023-03-21T20:04:27Z] <taavi@deploy2002> esanders and taavi and matmarex: Backport for [[gerrit:900331|Enable DiscussionTools_visualenhancements_newsectionlink_enable on labs for testing]], [[gerrit:901697|Enable wgDiscussionToolsEnablePermalinksBackend on group2 wikis (T315353)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-03-21T20:20:35Z] <taavi@deploy2002> Finished scap: Backport for [[gerrit:900331|Enable DiscussionTools_visualenhancements_newsectionlink_enable on labs for testing]], [[gerrit:901697|Enable wgDiscussionToolsEnablePermalinksBackend on group2 wikis (T315353)]] (duration: 17m 40s)

matmarex added a project: Skipped QA.

This is done now: all new comments being posted are being indexed in the permalink database. Old comments will be handled in T315510 by the maintenance script runs.

The new plan to re-enable the feature without repeating this incident is: run the maintenance script first, then enable the config option, then run the maintenance script again on just the changes that will have occurred between these two steps and thus wouldn't have been processed.

ppelberg subscribed.

This ticket being resolved depends on permalinks being enabled and permalinks being enabled depends on T315510 being resolved.

Change 946998 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on s2/s3/s5/s6 group2

https://gerrit.wikimedia.org/r/946998

Change 946998 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on s2/s3/s5/s6 group2

https://gerrit.wikimedia.org/r/946998

Mentioned in SAL (#wikimedia-operations) [2023-08-08T20:13:26Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:946998|Enable wgDiscussionToolsEnablePermalinksBackend on s2/s3/s5/s6 group2 (T315353)]]

Mentioned in SAL (#wikimedia-operations) [2023-08-08T20:14:57Z] <urbanecm@deploy1002> urbanecm and matmarex: Backport for [[gerrit:946998|Enable wgDiscussionToolsEnablePermalinksBackend on s2/s3/s5/s6 group2 (T315353)]] synced to the testservers mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option)

Mentioned in SAL (#wikimedia-operations) [2023-08-08T20:24:22Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:946998|Enable wgDiscussionToolsEnablePermalinksBackend on s2/s3/s5/s6 group2 (T315353)]] (duration: 10m 55s)

Mentioned in SAL (#wikimedia-operations) [2023-08-08T20:29:13Z] <urbanecm> mwmaint1002: foreachwikiindblist 'group2 & s2' extensions/DiscussionTools/maintenance/persistRevisionThreadItems.php --current --all --touched-after=20230615000000 (T315353)

Mentioned in SAL (#wikimedia-operations) [2023-08-08T20:30:26Z] <urbanecm> mwmaint1002: foreachwikiindblist 'group2 & s3' extensions/DiscussionTools/maintenance/persistRevisionThreadItems.php --current --all --touched-after=20230615000000 (T315353)

Mentioned in SAL (#wikimedia-operations) [2023-08-08T20:30:37Z] <urbanecm> mwmaint1002: foreachwikiindblist 'group2 & s5' extensions/DiscussionTools/maintenance/persistRevisionThreadItems.php --current --all --touched-after=20230615000000 (T315353)

Change 965562 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on s7 group2

https://gerrit.wikimedia.org/r/965562

Change 965562 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on s7 group2

https://gerrit.wikimedia.org/r/965562

Mentioned in SAL (#wikimedia-operations) [2023-10-12T20:58:22Z] <thcipriani@deploy2002> Started scap: Backport for [[gerrit:965562|Enable wgDiscussionToolsEnablePermalinksBackend on s7 group2 (T315353)]]

Mentioned in SAL (#wikimedia-operations) [2023-10-12T20:59:36Z] <thcipriani@deploy2002> thcipriani and matmarex: Backport for [[gerrit:965562|Enable wgDiscussionToolsEnablePermalinksBackend on s7 group2 (T315353)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-10-12T21:06:17Z] <thcipriani@deploy2002> Finished scap: Backport for [[gerrit:965562|Enable wgDiscussionToolsEnablePermalinksBackend on s7 group2 (T315353)]] (duration: 07m 55s)

Change #1036197 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on enwiki (last one!)

https://gerrit.wikimedia.org/r/1036197

Change #1036197 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable wgDiscussionToolsEnablePermalinksBackend on enwiki

https://gerrit.wikimedia.org/r/1036197

Mentioned in SAL (#wikimedia-operations) [2024-05-27T13:28:21Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Started scap: Backport for [[gerrit:1036197|Enable wgDiscussionToolsEnablePermalinksBackend on enwiki (T315353)]], [[gerrit:1026511|Pre-emptively disable DiscussionToolsEnableThanks (no-op)]]

Mentioned in SAL (#wikimedia-operations) [2024-05-27T13:30:51Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 esanders and matmarex and lucaswerkmeister-wmde: Backport for [[gerrit:1036197|Enable wgDiscussionToolsEnablePermalinksBackend on enwiki (T315353)]], [[gerrit:1026511|Pre-emptively disable DiscussionToolsEnableThanks (no-op)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-05-27T13:46:37Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1036197|Enable wgDiscussionToolsEnablePermalinksBackend on enwiki (T315353)]], [[gerrit:1026511|Pre-emptively disable DiscussionToolsEnableThanks (no-op)]] (duration: 18m 15s)