Page MenuHomePhabricator

Newcomer tasks: keep task list non-deterministic once topic matching is introduced
Closed, ResolvedPublic

Description

Before topic matching we used random scoring to get a random subset of tasks. Topic matching itself is a kind of sorting, which might be incompatible with this. Since we want to avoid giving the same task to all users who have the same task type / topic configuration, we need to figure out another way to make the task list non-deterministic.

In the short term, we can just shuffle the search results we get. That might be fine in the longer term too, or there might be another solution which makes ElasticSearch combine multiple score types. See the discussion starting at T240512#5773074 for more details.

Details

Related Gerrit Patches:
mediawiki/extensions/GrowthExperiments : wmf/1.35.0-wmf.14Newcomer tasks: Randomize suggested tasks locally
mediawiki/extensions/GrowthExperiments : masterNewcomer tasks: Randomize suggested tasks locally

Event Timeline

Tgr created this task.Jan 7 2020, 3:19 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 7 2020, 3:19 AM

Change 562648 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/GrowthExperiments@master] Newcomer tasks: Randomize suggested tasks locally

https://gerrit.wikimedia.org/r/562648

Added a trivial fix that should be enough to unblock v1.1. I'd like to return to this when we have more time. I think a decent solution would do one of

  • combine the deterministic topic matching score with a random (or even better pseudorandom) score, possibly as a custom search profile
  • use a topic matching implemnetation which does not rely on scoring (something more analogous to hastemplate:)
Tgr claimed this task.Jan 8 2020, 12:55 AM

Change 562648 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Newcomer tasks: Randomize suggested tasks locally

https://gerrit.wikimedia.org/r/562648

Change 564158 had a related patch set uploaded (by Catrope; owner: Gergő Tisza):
[mediawiki/extensions/GrowthExperiments@wmf/1.35.0-wmf.14] Newcomer tasks: Randomize suggested tasks locally

https://gerrit.wikimedia.org/r/564158

Change 564158 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@wmf/1.35.0-wmf.14] Newcomer tasks: Randomize suggested tasks locally

https://gerrit.wikimedia.org/r/564158

Etonkovidova closed this task as Resolved.Thu, Jan 23, 12:42 AM