Change 893734 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[schemas/event/secondary@master] helppanel: Add support for trynewtask dialog

https://gerrit.wikimedia.org/r/893734

If the user chose “try new task” and the post-edit dialogue changes to show the task list, we need to also capture whether the standard dialogue was shown or whether the combination of task type and user filters resulted in no available tasks.

@nettrom_WMF I didn't implement the feature in this way in T322386: Leveling up: "Try a new task" dialog. The user will only be prompted to try a new task if there is at least one available task matching the new task type and their existing topic filters. I'll ask in T322386 if that's acceptable for this iteration.

Change 893734 merged by jenkins-bot:

[schemas/event/secondary@master] helppanel: Add support for trynewtask dialog

https://gerrit.wikimedia.org/r/893734

In T330637#8660470, @kostajh wrote:

@nettrom_WMF I didn't implement the feature in this way in T322386: Leveling up: "Try a new task" dialog. The user will only be prompted to try a new task if there is at least one available task matching the new task type and their existing topic filters. I'll ask in T322386 if that's acceptable for this iteration.

I don't have a problem with that approach. If things change and we end up with the possibility of the "no suggestions" dialogue popping up after the user has clicked to see suggestions of the new task type, then I don't think we'll need to make any changes as long as the events are connected through the same help_panel_session_id.

From what I can tell, the implemented instrumentation should cover our needs. Let me check that I've understood correctly how this works based on the patch:

• Interactions with the "try new task" dialogue all have context = "postedit-trynewtask"
• Impressions of the dialogue captures the relevant metadata in action_data using two new keys and the existing key savedTaskType to reflect the metadata described above (what the new suggested task was and what previous task they completed and how many edits of said task they reached).
• There are three actions a user can take: close, no thanks, and try new task. All of these capture the state of the "don't show this again" checkbox.
• These events and subsequent events in the post-edit dialogue will all be connected with the same help_panel_session_id (similar to other post-edit dialogue events).

Let me know if I missed anything. Otherwise, this all looks good to me!

In T330637#8678277, @nettrom_WMF wrote:

In T330637#8660470, @kostajh wrote:

@nettrom_WMF I didn't implement the feature in this way in T322386: Leveling up: "Try a new task" dialog. The user will only be prompted to try a new task if there is at least one available task matching the new task type and their existing topic filters. I'll ask in T322386 if that's acceptable for this iteration.

I don't have a problem with that approach. If things change and we end up with the possibility of the "no suggestions" dialogue popping up after the user has clicked to see suggestions of the new task type, then I don't think we'll need to make any changes as long as the events are connected through the same help_panel_session_id.

From what I can tell, the implemented instrumentation should cover our needs. Let me check that I've understood correctly how this works based on the patch:

• Interactions with the "try new task" dialogue all have context = "postedit-trynewtask"
• Impressions of the dialogue captures the relevant metadata in action_data using two new keys and the existing key savedTaskType to reflect the metadata described above (what the new suggested task was and what previous task they completed and how many edits of said task they reached).
• There are three actions a user can take: close, no thanks, and try new task. All of these capture the state of the "don't show this again" checkbox.
• These events and subsequent events in the post-edit dialogue will all be connected with the same help_panel_session_id (similar to other post-edit dialogue events).

Let me know if I missed anything. Otherwise, this all looks good to me!

Yes, that's all correct. Below is a screenshot where the panel appears and two sample events related to its impression. To make the panel appear, I completed 4 copyedit edits. On the 5th edit, I received the "try new task" panel instead of the standard post-edit dialog.

{
	"event": {
		"action": "postedit-toast-message-impression",
		"action_data": "",
		"context": "postedit-trynewtask",
		"editor_interface": "visualeditor",
		"help_panel_session_id": "066f72d5882382588f68",
		"is_mobile": false,
		"is_suggested_task": true,
		"page_id": 0,
		"page_ns": 0,
		"page_protection": "",
		"page_title": "",
		"session_token": "{removed}",
		"user_can_edit": true,
		"user_editcount": 21,
		"user_id": 23688
	}
}

I had just completed my 5th copyedit when I received this panel. The next task type I should try is "links". The edit-count-for-task-type is 4 because that is the value shown when I opened the page. I should probably revise that to show 5.

{
	"event": {
		"action": "trynewtask-impression",
		"action_data": "next-suggested-task-type=links;edit-count-for-task-type=4",
		"context": "postedit-trynewtask",
		"editor_interface": "visualeditor",
		"help_panel_session_id": "066f72d5882382588f68",
		"is_mobile": false,
		"is_suggested_task": true,
		"page_id": 0,
		"page_ns": 0,
		"page_protection": "",
		"page_title": "",
		"session_token": "{removed}",
		"user_can_edit": true,
		"user_editcount": 21,
		"user_id": 23688
	}
}

In T330637#8687140, @kostajh wrote:

I had just completed my 5th copyedit when I received this panel. The next task type I should try is "links". The edit-count-for-task-type is 4 because that is the value shown when I opened the page. I should probably revise that to show 5.

Yeah, if you could add one to that count that would be neat!

I think I expected action_data for the second event to also include savedTaskType=copyedit since that is the type of edit you just completed. Otherwise, I'm not sure we'll capture that anywhere (it might show up in the regular post-edit dialogue impression, but that would require the user to make a choice in the try-new-task dialogue, I think?)

I think I also missed one detail in my previous list, in that the interaction with the try-new-task dialogue will have "context": "postedit-trynewtask" as shown in the events you posted. If the user makes a choice about the new task so the regular post-edit dialogue is shown, then the events follow how that dialogue would normally go (e.g. context changes to postedit and so on). Let me know if my understanding of that is missing something?

In T330637#8689980, @nettrom_WMF wrote:

In T330637#8687140, @kostajh wrote:

I had just completed my 5th copyedit when I received this panel. The next task type I should try is "links". The edit-count-for-task-type is 4 because that is the value shown when I opened the page. I should probably revise that to show 5.

Yeah, if you could add one to that count that would be neat!

Ack, will do.

I think I expected action_data for the second event to also include savedTaskType=copyedit since that is the type of edit you just completed. Otherwise, I'm not sure we'll capture that anywhere (it might show up in the regular post-edit dialogue impression, but that would require the user to make a choice in the try-new-task dialogue, I think?)

Good point, done:

{
  "action": "trynewtask-impression",
  "action_data": "next-suggested-task-type=update;savedTaskType=copyedit;edit-count-for-task-type=47",
  "user_id": 1232,
  "user_editcount": 46,
  "context": "postedit-trynewtask",
  "editor_interface": "visualeditor",
  "is_suggested_task": true,
  "is_mobile": false,
  "page_id": 0,
  "page_title": "",
  "page_ns": 0,
  "user_can_edit": true,
  "page_protection": "",
  "session_token": "{removed}",
  "help_panel_session_id": "e0d3cefea2b6cdaf0f72"
}

I think I also missed one detail in my previous list, in that the interaction with the try-new-task dialogue will have "context": "postedit-trynewtask" as shown in the events you posted. If the user makes a choice about the new task so the regular post-edit dialogue is shown, then the events follow how that dialogue would normally go (e.g. context changes to postedit and so on). Let me know if my understanding of that is missing something?

That's right, after dismissing the "Try new task" panel, the context for the post-edit panel is "postedit".

It seems there's another problem, though, because I see multiple impression events for the post-edit dialog after dismissing the try new task panel:

{
  "action": "postedit-impression",
  "action_data": "type=full;newcomerTaskToken=65fcocs407cno1rdclja807ps7l76a10",
  "user_id": 1232,
  "user_editcount": 46,
  "context": "postedit",
  "editor_interface": "visualeditor",
  "is_suggested_task": true,
  "is_mobile": false,
  "page_id": 3691,
  "page_title": "Stanley_Tucker",
  "page_ns": 0,
  "user_can_edit": true,
  "page_protection": "",
  "session_token": "{removed}",
  "help_panel_session_id": "e0d3cefea2b6cdaf0f72"
},
{
  "action": "postedit-impression",
  "action_data": "type=full;newcomerTaskToken=65fcocs407cno1rdclja807ps7l76a10",
  "user_id": 1232,
  "user_editcount": 46,
  "context": "postedit",
  "editor_interface": "visualeditor",
  "is_suggested_task": true,
  "is_mobile": false,
  "page_id": 3691,
  "page_title": "Stanley_Tucker",
  "page_ns": 0,
  "user_can_edit": true,
  "page_protection": "",
  "session_token": "{removed}",
  "help_panel_session_id": "e0d3cefea2b6cdaf0f72"
},
{
  "action": "postedit-impression",
  "action_data": "type=full;savedTaskType=copyedit;userTaskTypes=copyedit;newcomerTaskToken=fuhuhfsofs8s6uuah3boehhv9n7udsh4",
  "user_id": 1232,
  "user_editcount": 46,
  "context": "postedit",
  "editor_interface": "visualeditor",
  "is_suggested_task": true,
  "is_mobile": false,
  "page_id": 3691,
  "page_title": "Stanley_Tucker",
  "page_ns": 0,
  "user_can_edit": true,
  "page_protection": "",
  "session_token": "{removed}",
  "help_panel_session_id": "e0d3cefea2b6cdaf0f72"
}

so I'll look into seeing why that is happening.

Change 899637 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[schemas/event/secondary@master] helppanel: Document savedTaskType in action_data for trynewtask-impression

https://gerrit.wikimedia.org/r/899637

Change 899639 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] TryNewTaskPanel: Log savedTaskType in impression event

https://gerrit.wikimedia.org/r/899639

In T330637#8698121, @kostajh wrote:

It seems there's another problem, though, because I see multiple impression events for the post-edit dialog after dismissing the try new task panel:

I checked event data, and it looks like this is also happening in production. Found a session where every task navigation click in the post-edit dialogue is followed by four impression events. If needed, I can dig up more information.

I don't see this as a blocker for release, though. Maybe something to dig into the days following deployment?

In T330637#8698462, @nettrom_WMF wrote:

In T330637#8698121, @kostajh wrote:

It seems there's another problem, though, because I see multiple impression events for the post-edit dialog after dismissing the try new task panel:

I checked event data, and it looks like this is also happening in production. Found a session where every task navigation click in the post-edit dialogue is followed by four impression events. If needed, I can dig up more information.

I don't see this as a blocker for release, though. Maybe something to dig into the days following deployment?

Hmm, it looks like it may date back to rEGREf9e68da4cb75: Suggested Edits: Add refresh button on post-edit dialog from June 2021. Or some code added then is now being triggered more often than we intended to.

In T330637#8698562, @kostajh wrote:

In T330637#8698462, @nettrom_WMF wrote:

In T330637#8698121, @kostajh wrote:

It seems there's another problem, though, because I see multiple impression events for the post-edit dialog after dismissing the try new task panel:

I checked event data, and it looks like this is also happening in production. Found a session where every task navigation click in the post-edit dialogue is followed by four impression events. If needed, I can dig up more information.

I don't see this as a blocker for release, though. Maybe something to dig into the days following deployment?

Hmm, it looks like it may date back to rEGREf9e68da4cb75: Suggested Edits: Add refresh button on post-edit dialog from June 2021. Or some code added then is now being triggered more often than we intended to.

I made a patch for it in T332190: Help panel: Fix duplicate postedit-impression events, and we can track code review / QA of that in that task.

Change 899637 merged by jenkins-bot:

[schemas/event/secondary@master] helppanel: Document savedTaskType in action_data for trynewtask-impression

https://gerrit.wikimedia.org/r/899637

Change 899639 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] TryNewTaskPanel: Log savedTaskType in impression event

https://gerrit.wikimedia.org/r/899639

The off-by-one error in action data is still there. I've made a separate task for that: T332319