Page MenuHomePhabricator

Newcomer tasks: fetch ElasticSearch data for search results
Open, Needs TriagePublic

Description

Currently we rely on doing one search per topic, which is expensive and we'll want to move away from it at some point. Once we do a joint search for all topics the user selected, we'll need a way to tell which search result is about which topic. The nicest way would be to fetch this data from the ElasticSearch index; maybe it's possible to do that as an extension to the search API, maybe it would require switching to raw CirrusSearch queries.

In the longer term this might be useful for other things as well (e.g. store the task type definitions in ES as well and fetch them in a similar way so we can also collapse type searches into one).

Event Timeline

Change 578878 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/GrowthExperiments@master] Genralize TemplateProvider

https://gerrit.wikimedia.org/r/578878

Change 578879 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/GrowthExperiments@master] [WIP] Fetch ElasticSearch data for search results

https://gerrit.wikimedia.org/r/578879

I suspect this task and its patches might have gotten lost due to COVID related chaos. @Tgr do you want to rebase and bring these up for review again, or should we remove from the current sprint and put into Upcoming Work on the team board?

Tgr removed Tgr as the assignee of this task.Mar 1 2021, 8:26 PM
Tgr edited projects, added Growth-Team; removed Growth-Team (Sprint 0 (Growth Team)).

I don't think it would make sense to prioritize this above getting link recommendations out.

Faster search would be nice, but it conflicts with our current product requirements (specifically, with interleaving the various topics) so I don't think we can do this.

Change 578878 abandoned by Kosta Harlan:

[mediawiki/extensions/GrowthExperiments@master] Generalize TemplateProvider

Reason:

Cleaning up code review backlog for GrowthExperiments. If you still plan to work on this, then please restore it

https://gerrit.wikimedia.org/r/578878

Faster search would be nice, but it conflicts with our current product requirements (specifically, with interleaving the various topics) so I don't think we can do this.

I wonder if we could impose a limit on the maximum number of topics that a user can select, based on real-world usage (do most users pick e.g. 3 topics? 5 topics?). Given that we now only query for 20 tasks at a time, perhaps it would be feasible to do a joint search for all the topics x task types, perhaps reducing the limit further to 10 tasks at a time.

Change 578879 abandoned by Kosta Harlan:

[mediawiki/extensions/GrowthExperiments@master] [WIP] Fetch ElasticSearch data for search results

Reason:

Cleaning up code review backlog for GrowthExperiments. If you still plan to work on this, then please restore the change.

https://gerrit.wikimedia.org/r/578879