Page MenuHomePhabricator

Post-edit dialog recommendations should take into account quality gate limits
Closed, ResolvedPublicBUG REPORT

Description

  1. A user selects only "Add image" or "Add links" structured tasks.
  2. A user performs edits and reaches a daily limit on such tasks.
  3. After completing the last structured task, the post-dialog keeps displaying more structured tasks even though a user cannot do them due to the quality gates limits.
"Add link" quality gate reached"Add image" quality gate reached
Screen Shot 2022-07-01 at 5.05.31 PM.png (1×1 px, 175 KB)
Screen Shot 2022-07-01 at 4.00.19 PM.png (1×1 px, 220 KB)

NOTES:

  • The spec for the post-dialog for "Add image" from T293316: Add an image: minimal quality gate, requires that if there are no more "Add image" tasks available due to the quality gate limitations, then a user should see the suggestions for the selected topics but for other task types:

If the user does not have other tasks available in their feed that are not image tasks, then we should fall back to other task types using this logic: choose any other task type, as long as it matches the user's currently selected topics.

  • For "Add link" I did not find the spec for that type of scenario; logically it'd make sense to have the same spec as for "Add image"

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Tagging as Regression as I'm pretty sure this was implemented in T293316: Add an image: minimal quality gate, and probably broke during some refactorings since then.

kostajh triaged this task as High priority.Jul 4 2022, 9:17 AM
kostajh changed the subtype of this task from "Task" to "Bug Report".
Sgs changed the task status from Open to In Progress.Jul 5 2022, 7:51 AM
Sgs claimed this task.

We used to do the quality gate check just in the client in ext.growthExperiments.PostEdit.js#51. Seems that the logic was removed in gerrit 786403 but can't find the reason behind or any other refactored logic handling the quality gate check of the task feed elements (both in homepage and post edit dialog). @Etonkovidova do you remember having done a QA test for this scenario in the homepage feed?

I see three options, although trying to solve this in the server seems the most reasonable ones:

  1. Do the check in the client like prior status quo
  2. Filter the exceeded task type tasks after querying ES
  3. Check the limit before querying ES, and exclude the exceeded task type if needed

Exploring option (3) because it makes sense to avoid requesting exceeded quota types beforehand, I come across this comment in SearchTaskSuggester.php#144:

// We generally don't try to handle task type filtering for the A/B test (T278123) here
// as it is already handled in NewcomerTasksUserOptionsLookup, but we make an exception
// for the case when $taskTypeFilter === [] which would be difficult to handle elsewhere.
if ( !$taskSetFilters->getTaskTypeFilters() ) {
    $taskSetFilters->setTaskTypeFilters(
        $this->newcomerTasksUserOptionsLookup
            ->filterTaskTypes( array_keys( $this->taskTypes ), $user )
    );
}

Since our SearchStrategy iterates over the provided task types, and the user will still have the exceeded quota task type on its preferences by the time of this check, it is impeding us to return 0 results for the scenario Only 1 task type selected, and the quota has been exceeded for it.

Can someone help to identify which scenario is that logic covering? We can also explore option (2) which does not have this problem. @kostajh @Tgr

Change 811944 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] [WIP] Newcomer tasks: avoid returning exceeded quota task types for the task feed

https://gerrit.wikimedia.org/r/811944

Change 811944 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Post edit dialog: avoid showing exceeded task daily limit task types

https://gerrit.wikimedia.org/r/811944

Etonkovidova added a subscriber: Sergio.

Tagging as Regression as I'm pretty sure this was implemented in T293316: Add an image: minimal quality gate, and probably broke during some refactorings since then.

Yes, it's a regression. Answering @Sergio on @Etonkovidova do you remember having done a QA test for this scenario in the homepage feed? - yes, the functional testing was initially done, but there are no regression tests for the quality gates.


Checked link recommendation on testwiki wmf.23 and image recommendation on `cswiki betalabs - all works as expected:

link recommendationimage recommendation
- yes answers
Screen Shot 2022-08-05 at 12.36.21 PM.png (1×1 px, 456 KB)
- no answers
Screen Shot 2022-08-05 at 12.24.30 PM.png (1×1 px, 282 KB)
- other type of tasks are still available
Screen Shot 2022-08-05 at 12.34.49 PM.png (1×1 px, 213 KB)
Screen Shot 2022-08-05 at 12.27.08 PM.png (1×1 px, 369 KB)