Page MenuHomePhabricator

End the Surfacing Structured Tasks experiment
Closed, ResolvedPublic

Description

Background

In T385343: Surfacing "Add a link" Structured Tasks: Experiment Release (FY24/25 WE1.2.9), we deployed the Surfacing Structured Tasks experiment (to eswiki, fawiki, frwiki, idwiki and ptwiki). As part of T392256: [EPIC] "Get Started" notifications, we want to deploy another experiment on some of those wikis (eswiki, frwiki and arwiki). Since we currently cannot deploy more than one experiment on one wiki (T288022), we need to undeploy the Surfacing Structured Tasks experiment on at least eswiki and frwiki.

We collected the necessary data from that experiment and as such, it makes sense to me to end the experiment in full. The final experiment results should be shared soon: T377098: Experiment Analysis: Surfacing Structured Tasks experiment (WE1.2.6 / WE1.2.9)
This task is to (a) determine the right scope of disabling and (b) execute the disabling itself.

Open Questions
  • Should we disable the Surfacing Structured Tasks experiment on all wikis it is enabled on, or only on pilots for the Getting Started notification?
  • We should disable the Surfacing Structured Tasks experiment on all wikis it is enabled on.
  • Do we anticipate the need to enable the Surfacing Structured Tasks experiment again? (Do we need to keep the code to make the experiment possible?)
  • It is very unlikely that we will run this exact experiment again. However, in the future we may run a similar experiment again with a different UI, or a different target audience. For example, this intervention doesn't seem to have a meaningful impact on constructive activation, however in the coming fiscal year, we might want to test this feature as a means to re-engage newer editors after their initial edits. If keeping the existing code will help with that, then let's keep it.
Acceptance Criteria
  • disable the Surfacing Structured Tasks experiment on all wikis
  • code clean up, as appropriate

Event Timeline

Restricted Application added subscribers: hubaishan, Huji, Aklapper. · View Herald Transcript

@KStoller-WMF Before we can deploy T392256: [EPIC] "Get Started" notifications, we will need to end the pre-existing experiment. I filled this task to track that work, but before it becomes actionable from an Engineering standpoint, we need to know the scope of the disabling. From the technical perspective, disabling the experiment on eswiki, frwiki and testwiki (where it conflicts) is sufficient. But if we do not need the experiment to continue running, it might make more sense to terminate it on all wikis. If we do that, it is also important to know whether we want to keep the code that backs the experiment up or whether that can be safely removed.

If we decide not to do the full end now, I'd appreciate if you could fill a task at the time when full end would be appropriate.

Change #1163025 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] Remove the Surfacing Structured Tasks code

https://gerrit.wikimedia.org/r/1163025

Change #1163027 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/mediawiki-config@master] [Growth] Disable the Surfacing Structured Tasks experiment

https://gerrit.wikimedia.org/r/1163027

Change #1163028 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/mediawiki-config@master] [Growth] Remove support code for Surfacing Structured Tasks experiment

https://gerrit.wikimedia.org/r/1163028

Change #1163288 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/mediawiki-config@master] [Growth] Remove feature flags related to Surfacing Structured Tasks

https://gerrit.wikimedia.org/r/1163288

This blocks the Getting Started experiment from getting started.

Change #1163027 merged by jenkins-bot:

[operations/mediawiki-config@master] [Growth] Disable the Surfacing Structured Tasks feature

https://gerrit.wikimedia.org/r/1163027

Mentioned in SAL (#wikimedia-operations) [2025-06-24T12:12:23Z] <urbanecm@deploy1003> Started scap sync-world: Backport for [[gerrit:1163027|[Growth] Disable the Surfacing Structured Tasks feature (T397515)]]

Mentioned in SAL (#wikimedia-operations) [2025-06-24T12:14:38Z] <urbanecm@deploy1003> urbanecm: Backport for [[gerrit:1163027|[Growth] Disable the Surfacing Structured Tasks feature (T397515)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-06-24T12:23:14Z] <urbanecm@deploy1003> Finished scap sync-world: Backport for [[gerrit:1163027|[Growth] Disable the Surfacing Structured Tasks feature (T397515)]] (duration: 10m 51s)

Change #1163025 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Remove the Surfacing Structured Tasks code

https://gerrit.wikimedia.org/r/1163025

I think these are now ready to be deployed, or am I missing something that is still blocking them?

Change #1172312 had a related patch set uploaded (by Michael Große; author: Michael Große):

[mediawiki/extensions/GrowthExperiments@master] refactor: remove unused styles from surfacing experiment

https://gerrit.wikimedia.org/r/1172312

Change #1172312 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] refactor: remove unused styles from surfacing experiment

https://gerrit.wikimedia.org/r/1172312

Change #1163028 merged by jenkins-bot:

[operations/mediawiki-config@master] [Growth] Remove support code for Surfacing Structured Tasks experiment

https://gerrit.wikimedia.org/r/1163028

Change #1163288 merged by jenkins-bot:

[operations/mediawiki-config@master] [Growth] Remove feature flags related to Surfacing Structured Tasks

https://gerrit.wikimedia.org/r/1163288

Mentioned in SAL (#wikimedia-operations) [2025-07-30T12:20:09Z] <urbanecm@deploy1003> Started scap sync-world: Backport for [[gerrit:1163028|[Growth] Remove support code for Surfacing Structured Tasks experiment (T397515)]], [[gerrit:1163288|[Growth] Remove feature flags related to Surfacing Structured Tasks (T397515)]]

Mentioned in SAL (#wikimedia-operations) [2025-07-30T12:22:19Z] <urbanecm@deploy1003> urbanecm: Backport for [[gerrit:1163028|[Growth] Remove support code for Surfacing Structured Tasks experiment (T397515)]], [[gerrit:1163288|[Growth] Remove feature flags related to Surfacing Structured Tasks (T397515)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-07-30T12:29:15Z] <urbanecm@deploy1003> Finished scap sync-world: Backport for [[gerrit:1163028|[Growth] Remove support code for Surfacing Structured Tasks experiment (T397515)]], [[gerrit:1163288|[Growth] Remove feature flags related to Surfacing Structured Tasks (T397515)]] (duration: 09m 06s)

I think these are now ready to be deployed, or am I missing something that is still blocking them?

Yep yep. I just did that last step.

Etonkovidova updated the task description. (Show Details)