Page MenuHomePhabricator

Switch GETempLinkRecommendationSwitchTagClearHook to true at all wikis
Closed, ResolvedPublic

Description

Following the investigations in T372337: High number of dangling search index results at fr.wikipedia or it.wikipedia, it is clear the new way of cleaning link recommendation (GETempLinkRecommendationSwitchTagClearHook) is helpful to the wikis. We should enable it globally.

This was discussed in #growth-engine-room on Slack:

Michael: A dangling-records question: If we maybe want to roll-out a 2% version of the Link-Recommendations task to enwiki soon-ish, should we switch GETempLinkRecommendationSwitchTagClearHook for it to true and run the fixLinkRecommendations script for it a bunch of times?

Martin: Good idea, that makes sense. I think it would even make sense to switch it to true by default, I did not notice anything off regarding that switch?
Martin: (I still prefer waiting for EventGate with fixLinkRecommendation, but given this is now merged, we should be able to do that on Monday)
Scope of task
  • Switch GETempLinkRecommendationSwitchTagClearHook to true on all wikis
  • Remove the temporary GETempLinkRecommendationSwitchTagClearHook feature flag if nothing breaks

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Urbanecm_WMF moved this task from Inbox to Up Next on the Growth-Team board.

Change #1089739 had a related patch set uploaded (by Michael Große; author: Michael Große):

[operations/mediawiki-config@master] wikipedias: clear link-recommendations on page save

https://gerrit.wikimedia.org/r/1089739

Change #1089739 merged by jenkins-bot:

[operations/mediawiki-config@master] wikipedias: clear link-recommendations on page save

https://gerrit.wikimedia.org/r/1089739

Mentioned in SAL (#wikimedia-operations) [2024-11-11T14:49:40Z] <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1089739|wikipedias: clear link-recommendations on page save (T379522)]]

Mentioned in SAL (#wikimedia-operations) [2024-11-11T14:51:46Z] <lucaswerkmeister-wmde@deploy2002> migr, lucaswerkmeister-wmde: Backport for [[gerrit:1089739|wikipedias: clear link-recommendations on page save (T379522)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-11-11T15:00:39Z] <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1089739|wikipedias: clear link-recommendations on page save (T379522)]] (duration: 10m 59s)

Michael changed the task status from Open to Stalled.Nov 11 2024, 4:19 PM

Let's wait for at least one, but maybe even 2 branch-cuts before starting the work to remove this feature flag. So I'm setting this to stalled until Wednesday, November 20th, 2024.

Change #1090449 had a related patch set uploaded (by Michael Große; author: Michael Große):

[operations/puppet@production] growthexperiments.pp: track dangling records for cswiki hourly

https://gerrit.wikimedia.org/r/1090449

Change #1090449 merged by RLazarus:

[operations/puppet@production] growthexperiments.pp: track dangling records for cswiki hourly

https://gerrit.wikimedia.org/r/1090449

In principle, this should be ready to be worked on, but I would feel better doing it if we actually had the related metrics. Currently, we don't due to T380635/T380609: Maintenance scripts do not emit StatsLib metrics.

I wonder if we should just revert the change that migrated those metrics to statslib until that system is ready?

I wonder if we should just revert the change that migrated those metrics to statslib until that system is ready?

Generally, such a change would make sense to me, although it would be a step in the opposite direction than T359352 (which we just pulled to the sprint partially).

I'm wondering if it would make sense to call StatsFactory->flush() at the end of execute() temporarily. That would likely be a relatively straightforward change that we could drop once T380609 gets resolved (create GrowthMaintenance that would have this change, and let Growth maintenance scripts inherit from that class rather than Maintenance). OTOH, that'd be a workaround, not a fix, but it would unblock both this and the migration (barring JS instrumentation), which might be a big-enough gain for us to warrant that.

Where in that GrowthMaintenance class would you call StatsFactory->flush() and how would you call that from our maintenance scripts?

That being said, maybe just adding

// HACK: remove after T380609 is resolved
StatsFactory->flush()

at the end of every ::execute() method that potentially logs statistics in our maintenance scripts is a good enough step for the migration.

Michael changed the task status from Stalled to In Progress.Dec 18 2024, 7:06 PM
Michael moved this task from Blocked / Needs Work to Doing on the Growth-Team (Current Sprint) board.

The data looks pretty solid, so I think it is time to remove the old way of doing things and that temporary config.

Change #1105419 had a related patch set uploaded (by Michael Große; author: Michael Große):

[mediawiki/extensions/GrowthExperiments@master] Remove legacy way of clearing link recommendations + temp config

https://gerrit.wikimedia.org/r/1105419

Change #1105420 had a related patch set uploaded (by Michael Große; author: Michael Große):

[operations/mediawiki-config@master] Growth: Remove temporary config for clearing link recommendations

https://gerrit.wikimedia.org/r/1105420

Change #1105419 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Remove legacy way of clearing link recommendations + temp config

https://gerrit.wikimedia.org/r/1105419

This is blocked on time now. Should be actionable with wmf.12 being branched (Tuesday, January 14). The GE patch I just merged will be deployed with wmf.11, and to ensure it won't be rollbacked, we need the wmf.12 train to depart.

Change #1105420 merged by jenkins-bot:

[operations/mediawiki-config@master] Growth: Remove temporary config for clearing link recommendations

https://gerrit.wikimedia.org/r/1105420

Mentioned in SAL (#wikimedia-operations) [2025-01-15T08:28:45Z] <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1109736|Enable abusefilter-log-detail for autoconfirmed users on en.wikibooks (T383332)]], [[gerrit:1105420|Growth: Remove temporary config for clearing link recommendations (T379522)]]

Mentioned in SAL (#wikimedia-operations) [2025-01-15T08:35:45Z] <urbanecm@deploy2002> dreamrimmer, urbanecm, migr: Backport for [[gerrit:1109736|Enable abusefilter-log-detail for autoconfirmed users on en.wikibooks (T383332)]], [[gerrit:1105420|Growth: Remove temporary config for clearing link recommendations (T379522)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-01-15T08:44:55Z] <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1109736|Enable abusefilter-log-detail for autoconfirmed users on en.wikibooks (T383332)]], [[gerrit:1105420|Growth: Remove temporary config for clearing link recommendations (T379522)]] (duration: 16m 09s)

Urbanecm_WMF changed the task status from In Progress to Open.Jan 15 2025, 9:47 AM
Urbanecm_WMF moved this task from Incoming to QA on the Growth-Team (Current Sprint) board.

QA Note: This has been completed. The main thing that changed since ~mid-November is that we removed the old (and by then: unused) way of clearing outdated link-recommendations. So there should be no functional change in production because of this since Mid-November.