Page MenuHomePhabricator

Newcomer tasks: GrowthExperiments API for template tasks
Closed, ResolvedPublic

Description

This task is for building out an API that returns tasks to display in the suggested edits module. For v1.0, it needs to return tasks based on a grouping of maintenance templates.

The source of the tasks will be hastemplate:{pipe-delimited-template-names} sent to Special:Search.

While the tasks (without topics) could be done purely client-side, creating an API that proxies these requests to Special:Search has some benefits:

  • we can do EventLogging on the server-side
  • post-processing of results, including (not in v1.0) ensuring a threshold for uniqueness of tasks that users see, or supporting task claiming, etc
  • If topic matching ends up involving something other than ElasticSearch for storage, then our backend code switches to using that, without us needing to update the client-side code
  • because we'd probably only return 20 results per task type ("copyediting", "links" etc) then we could probably return the full set of 120 records in a single request, instead of requiring up to 6 requests on the client-side.

That said, this approach isn't strictly necessary to to populate the suggested edits module with tasks grouped by maintenance template types, so if we want to do a client-side approach that's fine too. (Example of the client-side approach can be seen in the prototype).

One difference between the prototype and what we'd use for populating suggested edits is that we will want to set the limit to 20, and srsort to random. We will also want to use -hastemplate keyword to filter out deletion templates, pass in the relevant property to get the protection status and obtain the properties needed to construct the suggested edit card (image, description, page view count, etc).

Event Timeline

kostajh created this task.Oct 2 2019, 11:13 AM
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptOct 2 2019, 11:13 AM
kostajh updated the task description. (Show Details)Oct 2 2019, 11:15 AM

Change 540391 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] (wip) API module for suggested edits

https://gerrit.wikimedia.org/r/540391

Tgr added a comment.Oct 8 2019, 11:29 AM

We will also want to ... obtain the properties needed to construct the suggested edit card (image, description, page view count, etc).

There are two ways to do that: put them directly in the API response, or make the API a generator so that other query-prop APIs can be chained to it. I went with the second approach in the API patch; it's a little more complicated for callers to use, but less code to maintain and gives more flexibility to the caller.

Change 540391 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] API module and service abstraction for suggested edits

https://gerrit.wikimedia.org/r/540391

Tgr added a comment.Oct 8 2019, 12:32 PM

gerrit 540391 does technically build the API, but the underlying service is still a mock implementation. See StaticTaskSuggester on using it for testing.

Change 541277 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/GrowthExperiments@master] [WIP] Add RemoteSearchEditSuggester

https://gerrit.wikimedia.org/r/541277

Tgr added a comment.Oct 8 2019, 10:46 PM

I think the main question here is whether we should try to cram everything in a single search. That would mean doing work on CirrusSearch (e.g. page protection status is not currently indexed), but we'd be able to rely on the search service for totals, sorting and continuation. Otherwise, post-processing (like removing protected pages from a search result set) makes totals and continuation complicated, and merging multiple search result sets makes continuation and sorting complicated. Those are not strictly 1.0 concerns, but we might want to avoid starting down a route we'll have to abandon at some point anyway.

MMiller_WMF renamed this task from GrowthExperiments API for template tasks to Newcomer tasks: GrowthExperiments API for template tasks.Oct 9 2019, 5:46 PM

Change 542059 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Growth tasks API: Update query limit to 200

https://gerrit.wikimedia.org/r/542059

Tgr added a comment.Oct 16 2019, 9:56 PM

I'm declaring this done and splitting using local search to another task. Using HTTP API requests in production is frowned upon so this is tech debt and ideally should be fixed before 1.0; but the current approach *is* operational, so it is lower priority.

Change 542059 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Growth tasks API: Update query limit to 200

https://gerrit.wikimedia.org/r/542059

Tgr added a comment.Oct 16 2019, 10:20 PM

I'm declaring this done and splitting using local search to another task.

T235717: Newcomer tasks: non-HTTP-based ConfigurationLoader and TaskSuggester

Change 541277 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Add RemoteSearchTaskSuggester

https://gerrit.wikimedia.org/r/541277

@Tgr should we make a patch to set a remote configuration in enwiki beta?

@Tgr should we make a patch to set a remote configuration in enwiki beta?

Does it mean that it cannot be tested in betalabs yet? I was trying to look at https://en.wikipedia.beta.wmflabs.org/wiki/Special:ApiSandbox and could not find API options for suggested edits, but it might mean that I just need some help with API requests.

Tgr added a comment.Oct 22 2019, 5:20 AM

Right, sorry, I forgot about that. (The API is available but it will just give an error complaining about lack of configuration.)

Change 545155 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/mediawiki-config@master] Set GrowthExperiments task suggester config on beta

https://gerrit.wikimedia.org/r/545155

Change 545155 merged by jenkins-bot:
[operations/mediawiki-config@master] Set GrowthExperiments task suggester config on beta

https://gerrit.wikimedia.org/r/545155