Page MenuHomePhabricator

Newcomer tasks: do not refresh suggestions if user cancels difficulty filter
Closed, ResolvedPublic

Description

Currently, if the user opens up the difficulty filter and dismisses it with the "x" (which means "cancel"), the whole set of suggestions is refreshed again from random. The user will probably have a better experience if the set of suggestions is unchanged when the user cancels with the "x".

Event Timeline

This is already fixed in beta labs

Just to clarify - the task description means:
"Currently, if the user opens up the difficulty filter (and does not make any selection/de-selection of filters) and dismisses it with the "x" (which means "cancel"), the whole set of suggestions is refreshed again from random."

The case when a user

  • opens up the difficulty filter and makes selection-deselection
  • clicks on 'X' - the displayed card will change from the initial card that user had before opening the difficulty filter.

In short, the initial state with a specific card that was displayed to a user won't be preserved.

Note: a kind of interesting case

  • two easy filters are selected
  • the difficulty filter is open and another filter is added
  • the underlying card will change but not necessarily to show a card that fetched by just added filter.

I am okay with the current behavior for the release of V1.0.

But I am wondering something for @RHo: do we want the underlying cards to change as the user checks and unchecks filter boxes? Because the user may be able to see the dynamics happening in the background behind the modal? Or should we do nothing to the underlying cards until the user clicks "Done" on the modal?

I am okay with the current behavior for the release of V1.0.

But I am wondering something for @RHo: do we want the underlying cards to change as the user checks and unchecks filter boxes? Because the user may be able to see the dynamics happening in the background behind the modal? Or should we do nothing to the underlying cards until the user clicks "Done" on the modal?

As a general rule when changing the filters in a modal or dialog, we should do nothing until the user clicks "Done", since the expectation is that users shouldn't really notice anything changing in the 'dimmed' area behind the dialog/modal.

I am putting this back in Ready for Development because the user is able to unselect all the difficulty checkboxes, and then cancel the dialog, resulting in no suggestions found. The rule should be that in all cases, cancelling the dialog puts the selections back to the way they were before. See gif below:

newcomer_tasks_selections_2019-12-09.gif (818×1 px, 689 KB)

@kostajh and @Etonkovidova -- this is related to my latest comment in T235042: Newcomer tasks: implement difficulty filters

Change 556172 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Suggested Edits: Do not update state until user clicks "Done"

https://gerrit.wikimedia.org/r/556172

Change 556172 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Suggested Edits: Do not update state until user clicks "Done"

https://gerrit.wikimedia.org/r/556172

@kostajh - all works as expected - clicking 'X' (cancel) will preserve the previous filter selection except for the two following cases (see the description below).

(1)

  1. A user opens the difficulty filter selection and deselect all filter - the warning message correctly informs that some filters have to be selected, but a user clicks on 'X' anywhere and is presented with the initial filter selection (which is a correct behavior).
  2. A user, thinking that the filter is in effect, clicks on one of the arrows to navigate between suggested articles - the selected filter changes to "Select types of edits" and "No suggestions found" is displayed (see the gif below):

no_selected_filters_case.gif (651×877 px, 319 KB)

(2) (minor) about the warning message - "Select at least one type of edit"- not being always present.
The beginning steps are from (1) - a user needs to get to the step when "No suggestions found" is present.

  1. Then user opens the difficulty filter selection and de-select all filters (the warning message 'Select at least one type of edit' appears) and clicks 'X'
  2. Without interacting with Suggested edits module, a user opens the difficulty filter selection again and select one of the filters, then clicks 'X'.
  3. Opening the difficulty filter again will display the overlay without any selected filters and without the message "Select at least one type of edit".

So, it goes from left to right:

Screen Shot 2019-12-16 at 4.02.49 PM.png (613×793 px, 133 KB)
Screen Shot 2019-12-16 at 4.03.13 PM.png (588×815 px, 129 KB)
Screen Shot 2019-12-16 at 3.11.55 PM.png (555×527 px, 86 KB)
kostajh moved this task from QA to Ready for Development on the Growth-Team (Current Sprint) board.

Unassigning myself for now, but will probably get to it tomorrow or the next day unless someone else does first.

(1) seems to happen on cancel, whether you deselected everything or not.

Change 559285 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/extensions/GrowthExperiments@master] Improve DifficultyFiltersDialog state management after cancel

https://gerrit.wikimedia.org/r/559285

(1) seems to happen on cancel, whether you deselected everything or not.

When not everything is deselected, that's a different error, caused by a race condition. Will mostly go away once the API is faster.

(2) (minor) about the warning message - "Select at least one type of edit"- not being always present.
The beginning steps are from (1) - a user needs to get to the step when "No suggestions found" is present.

IIUC that starts from an error state, so it's not really relevant.

A small annoyance is that pressing escape does not save the checkbox state like clicking the cancel icon does; I guess those are different actions. Maybe someone more familiar with OOUI knows offhand how to fix that.

A small annoyance is that pressing escape does not save the checkbox state like clicking the cancel icon does; I guess those are different actions. Maybe someone more familiar with OOUI knows offhand how to fix that.

It looks like we might need to do something like this example from CX

/**
 * @inheritdoc
 */
mw.cx.ui.CaptchaDialog.prototype.onDialogKeyDown = function ( e ) {
	if ( e.which === OO.ui.Keys.ESCAPE && this.constructor.static.escapable ) {
		this.executeAction( 'cancel' );
		e.preventDefault();
		e.stopPropagation();
		return;
	}

	return mw.cx.ui.CaptchaDialog.super.prototype.onDialogKeyDown.call( this, e );
};

After thinking a bit about the result counting inconsistency Kosta pointed out on the patch, I think this is a good time to move away from the decision that the filter dialog should preserve its state (which is distinct from the actual filter state) when cancelled and reopened. It is already a source of bugs and complexity for very limited UX gain, and having a topic filter dialog will make that much worse. At that point we'll have two filter dialogs, the full filter setup (and thus the number of results) will be determined by sum of the two; so if you open up the topic matching dialog and experiment with settings and see how many results they give, those result counts will depend on your difficulty filter settings. Should those be the actual difficulty filter settings or the changed but cancelled state of the dialog? Both are going to be very confusing to the user. (Basing it on actual filter settings would mean that the two dialogs show different result numbers, since for the filter dialog it will be based on the unsaved difficulty filter and saved topic filter, for the topic filter dialog on the saved difficulty filter and unsaved topic filter. Basing it on the settings in the two dialog would mean the numbers are not real; e.g. you manipulate the filters in one dialog, see 100 in the result counter, then save it, and the card queue is actually empty.)

So IMO we should either update settings in real time on checkbox click, or throw away the changes in the dialog when cancel is clicked. @MMiller_WMF @RHo any thoughts?

Apologies but I am having a little trouble understanding what the conflicts might be that have been pointed out. Would you be able review the selected behaviour as I see it outlined in the following scenario, and highlight what is not feasible and/or problematic?

Expected scenario

  1. User starts with 200 suggestions based on all easy edits selected and all topics (i.e., they have not chosen topics yet). In the SE module, "Apple" is the first article result (for a copyedit).
  2. They open the task types filter, where it shows 200 results number as well. They then select a Medium task to "Add references". This refreshes the article results number in the filter dialog to say 250. In the background on the homepage however, the Suggested edits remains the same (still "Apple").
  3. User clicks "Done" to save this set of filters. It's at this point that the new filter results are shown, and it now shows "Banana" as the first article result, and 250 as the number of results. (I know we have the results number capped at 200, but allowing the 250 for the sake of clarity that a new filter result was run)
  4. User now opens the Topics filter. They select "Geography' as a topic and the results number in the filter dialog updates from 250 to 120. As in step 2 though, the SE module in the background still shows the same article (Banana).
  5. User selects the "X" to cancel out of the topics filter without retaining the change they made.
  6. It is expected they will return to the homepage with the same Banana article shown as the first result of 250 articles.

After thinking a bit about the result counting inconsistency Kosta pointed out on the patch, I think this is a good time to move away from the decision that the filter dialog should preserve its state (which is distinct from the actual filter state) when cancelled and reopened. It is already a source of bugs and complexity for very limited UX gain, and having a topic filter dialog will make that much worse. At that point we'll have two filter dialogs, the full filter setup (and thus the number of results) will be determined by sum of the two; so if you open up the topic matching dialog and experiment with settings and see how many results they give, those result counts will depend on your difficulty filter settings. Should those be the actual difficulty filter settings or the changed but cancelled state of the dialog? Both are going to be very confusing to the user. (Basing it on actual filter settings would mean that the two dialogs show different result numbers, since for the filter dialog it will be based on the unsaved difficulty filter and saved topic filter, for the topic filter dialog on the saved difficulty filter and unsaved topic filter. Basing it on the settings in the two dialog would mean the numbers are not real; e.g. you manipulate the filters in one dialog, see 100 in the result counter, then save it, and the card queue is actually empty.)

So IMO we should either update settings in real time on checkbox click, or throw away the changes in the dialog when cancel is clicked. @MMiller_WMF @RHo any thoughts?

@RHo @kostajh @Tgr -- I'm going to put this on our agenda to discuss in a meeting in January.

Here is an example scenario:

Stored user settingsTask type filter dialogTopic filter dialogReal result countResult count shown in task type filter dialogResult count shown in topic filter dialog
StartCopyedit + Add links, no topicsclosedclosed100closedclosed
Opens task type filter dialog, deselects Add linksCopyedit + Add links, no topicsCopyeditclosed10080closed
Cancels task type filter dialogCopyedit + Add links, no topicsclosed, remembers Copyeditclosed100closedclosed
Opens topic filter dialog, selects ArtCopyedit + Add links, no topicsclosed, remembers CopyeditArt100closed???

What should the ??? number be? The number of Copyedit + Add links tasks in Art (which matches what will happen if the user presses Done) or the number of Copyedit tasks in Art (which matches the what the dialogs show / will show when reopened)?

In the second case, the result number does not tell the user what will happen if they save the changes, and therefore does not seem useful.

In the first case, let's say the dialog shows 20 as result count. Assume the user cancels the dialog again. Now if they open the task type dialog (and don't change anything) it will say there are 80 results, if they open the topic dialog (and don't change anything) it will say there are 20 results. Both of these numbers are correct in a way (they show the amount of results you'll get after saving the dialog) but it still feels confusing.

It is also a source of code complexity (e.g. what happens if the user saves the changes to the topic filter dialog? the count remembered by the other dialog is not going to be correct anymore and we need to arrange for it to be updated somehow). Remembering unsaved dialog states while the dialog is closed just does not seem to have value proportional to the mental and code complexity it introduces.

Based on the example scenario I agree it is not useful and I also do not think it is at all expected that the task type dialog remembers copyedit when it was not "saved" by the user. The first case @Tgr mentions (20=number of Copyedit + Add links tasks in Art) matches my expectations, whereas I would never have expected unsaved dialog changes to be remembered as is outlined in the second case.

Here is an example scenario:

Stored user settingsTask type filter dialogTopic filter dialogReal result countResult count shown in task type filter dialogResult count shown in topic filter dialog
StartCopyedit + Add links, no topicsclosedclosed100closedclosed
Opens task type filter dialog, deselects Add linksCopyedit + Add links, no topicsCopyeditclosed10080closed
Cancels task type filter dialogCopyedit + Add links, no topicsclosed, remembers Copyeditclosed100closedclosed
Opens topic filter dialog, selects ArtCopyedit + Add links, no topicsclosed, remembers CopyeditArt100closed???

What should the ??? number be? The number of Copyedit + Add links tasks in Art (which matches what will happen if the user presses Done) or the number of Copyedit tasks in Art (which matches the what the dialogs show / will show when reopened)?

In the second case, the result number does not tell the user what will happen if they save the changes, and therefore does not seem useful.

In the first case, let's say the dialog shows 20 as result count. Assume the user cancels the dialog again. Now if they open the task type dialog (and don't change anything) it will say there are 80 results, if they open the topic dialog (and don't change anything) it will say there are 20 results. Both of these numbers are correct in a way (they show the amount of results you'll get after saving the dialog) but it still feels confusing.

It is also a source of code complexity (e.g. what happens if the user saves the changes to the topic filter dialog? the count remembered by the other dialog is not going to be correct anymore and we need to arrange for it to be updated somehow). Remembering unsaved dialog states while the dialog is closed just does not seem to have value proportional to the mental and code complexity it introduces.

@RHo so if I understand correctly this is the expected behavior?

  1. User saves dialog with Copyedit + Add links (this is stored in user settings, updates the task cards on the homepage etc)
  2. User reopens dialog and unchecks Add links
  3. User cancels dialog
  4. User opens dialog again

-> user should see Copyedit + Add links (matching the state of the task cards) and not Copyedit (which would match the state of the filter when cancelled)

That would make things simpler, code-wise. The current behavior is to remember the last state of the dialog; I thought we are doing that by design, but probably I got confused and it's just an accidental remainder of the old logic which updated/saved everything immediately when a checkbox was clicked (so there wasn't any difference between real and cancelled state).

@RHo so if I understand correctly this is the expected behavior?

  1. User saves dialog with Copyedit + Add links (this is stored in user settings, updates the task cards on the homepage etc)
  2. User reopens dialog and unchecks Add links
  3. User cancels dialog
  4. User opens dialog again

-> user should see Copyedit + Add links (matching the state of the task cards) and not Copyedit (which would match the state of the filter when cancelled)

  • Hi @Tgr, yes this matches my expectations, thanks!

Change 559285 abandoned by Gergő Tisza:
Improve DifficultyFiltersDialog state management after cancel

https://gerrit.wikimedia.org/r/559285

Change 564018 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Suggested Edits: Backup state and restore on cancel

https://gerrit.wikimedia.org/r/564018