Change Details

The help panel feature is already instrumented, per the work done in {T206719}. In using it to display guidance for newcomer tasks, we have modifications and additions to make. This task will track the work, and the individual tasks about the different components of guidance will also contain notes on the relevant instrumentation for those components. [] Changes to EventLogging schemas [] Code for instrumentation [] Post-deployment QA by data analyst [] Publish changes to measurement plan The specifications are recorded in the existing measurement plan for newcomer tasks at these headers: * [[ https://docs.google.com/document/d/1dEE_XicyBWOKPu8op4t6neIHncSqlxubUgFa82bZQxI/edit#heading=h.f1gzx2ak03t6 | Measurements ]]: the numbers we want to be able to produce * [[ https://docs.google.com/document/d/1dEE_XicyBWOKPu8op4t6neIHncSqlxubUgFa82bZQxI/edit#heading=h.6z9kmrrq4bzz | Rules ]]: specific interactions to be recorded. These are also pasted below for convenience. But note that the Google Doc contains useful comments from team members. **Additional rules for version 1.1 (topic matching)** * General ** Since guidance will largely be implemented via the help panel, we want to record all the same things we already record via the help panel, in terms of it being opened and closed, preferences cog being clicked, back button being clicked, etc. The below points highlight things that are new with the guidance feature (and also includes some of the standard elements, just for completeness). ** For all events, we need to be able to tell that they happened as part of the suggested edits workflow, and not as part of the help panel displaying in a normal editing session. ** For all events, we need to be able to tell whether they happened in read mode or in edit mode. * [[ https://phabricator.wikimedia.org/T244541 | Suggested edit screen in read mode ]] ** This screen displays on desktop when the user arrives on the article. On mobile, it displays when the user has tapped the mobile peek. ** impression: we want to record that the user saw the help panel and which set of content it displays, in terms of which “task type” it is guiding, and whether it is showing the desktop VE, mobile VE, desktop wikitext, or mobile wikitext content. ** clicks: we want to record exactly which tab the user clicks on when they click through the quick tips. ** auto-advance: the quick start tabs will advance automatically every five seconds until the user clicks one. We want to record an event for each auto-advance, indicating which tab it advanced to. ** link clicks: the suggested edit screen will contain links to help documents, to which we want to record clicks. * [[ https://phabricator.wikimedia.org/T244434 | Mobile peek ]] ** peek impression: we want to record that the user saw the mobile peek and which set of content it displays, in terms of which “task type” it is guiding, and whether it is showing the desktop VE, mobile VE, desktop wikitext, or mobile wikitext content. ** tap on peek: whether the user taps the mobile peek. ** dismiss peek: whether the user taps outside the mobile peek, which dismisses it. * [[ https://phabricator.wikimedia.org/T244435 | Clicking edit ]] ** We want to record when the user clicks the “Edit” button. In two-tab wikis on desktop, we need to know whether they chose the VE or wikitext option. One way to figure this out may be with the editattemptstep schema with appropriate oversampling. * [[ https://phabricator.wikimedia.org/T244546 | In edit mode ]] ** impression: we want to record whether the user saw the help panel in edit mode. ** root screen button click: the new design of the “root screen” will contain several buttons. We want to record an event indicating which one the user clicks. ** Once the user has clicked one of the buttons, normal help panel logging ensues. * [[ https://phabricator.wikimedia.org/T245790 | Post-edit dialog ]] ** impression: we want to record that the user received the post-edit dialog. Because this dialog will contain a next article suggestion, we want to record the same details of that article suggestion that we record in se-task-impression events (e.g. title, whether it has an image, task type, maintenance template, etc.), while also recording that they received the impression via this post-edit dialog, as opposed to from the suggested edits module. ** close: whether the user clicks away or clicks the “X” to dismiss the dialog. ** clicks: whether the user clicks the link to go to their homepage. ** task click: if the user clicks the suggested article, we want to record that click event along with the article details, similar to se-task-click events on the homepage, but specifying that they clicked via this post-edit dialog, as opposed to from the suggested edits module. ** homepage visit: in the homepageVisit schema, we record the referer_route. We should add a referer_route for arriving on the homepage via the link in the post-edit dialog.

The help panel feature is already instrumented, per the work done in {T206719}. In using it to display guidance for newcomer tasks, we have modifications and additions to make. This task will track the work, and the individual tasks about the different components of guidance will also contain notes on the relevant instrumentation for those components. [] Changes to EventLogging schemas [] Code for instrumentation [] Post-deployment QA by data analyst [] Publish changes to measurement plan The specifications are recorded in the existing measurement plan for newcomer tasks at these headers: * [[ https://docs.google.com/document/d/1dEE_XicyBWOKPu8op4t6neIHncSqlxubUgFa82bZQxI/edit#heading=h.f1gzx2ak03t6 | Measurements ]]: the numbers we want to be able to produce * [[ https://docs.google.com/document/d/1dEE_XicyBWOKPu8op4t6neIHncSqlxubUgFa82bZQxI/edit#heading=h.6z9kmrrq4bzz | Rules ]]: specific interactions to be recorded. These are also pasted below for convenience. But note that the Google Doc contains useful comments from team members. **Additional rules for version 1.2 (guidance)** * General ** Since guidance will largely be implemented via the help panel, we want to record all the same things we already record via the help panel, in terms of it being opened and closed, preferences cog being clicked, back button being clicked, etc. The below points highlight things that are new with the guidance feature (and also includes some of the standard elements, just for completeness). ** For all events, we need to be able to tell that they happened as part of the suggested edits workflow, and not as part of the help panel displaying in a normal editing session. ** For all events, we need to be able to tell whether they happened in read mode or in edit mode. * [[ https://phabricator.wikimedia.org/T244541 | Suggested edit screen in read mode ]] ** This screen displays on desktop when the user arrives on the article. On mobile, it displays when the user has tapped the mobile peek. ** impression: we want to record that the user saw the help panel and which set of content it displays, in terms of which “task type” it is guiding, and whether it is showing the desktop VE, mobile VE, desktop wikitext, or mobile wikitext content. ** clicks: we want to record exactly which tab the user clicks on when they click through the quick tips. ** auto-advance: the quick start tabs will advance automatically every five seconds until the user clicks one. We want to record an event for each auto-advance, indicating which tab it advanced to. ** link clicks: the suggested edit screen will contain links to help documents, to which we want to record clicks. * [[ https://phabricator.wikimedia.org/T244434 | Mobile peek ]] ** peek impression: we want to record that the user saw the mobile peek and which set of content it displays, in terms of which “task type” it is guiding, and whether it is showing the desktop VE, mobile VE, desktop wikitext, or mobile wikitext content. ** tap on peek: whether the user taps the mobile peek. ** dismiss peek: whether the user taps outside the mobile peek, which dismisses it. * [[ https://phabricator.wikimedia.org/T244435 | Clicking edit ]] ** We want to record when the user clicks the “Edit” button. In two-tab wikis on desktop, we need to know whether they chose the VE or wikitext option. One way to figure this out may be with the editattemptstep schema with appropriate oversampling. * [[ https://phabricator.wikimedia.org/T244546 | In edit mode ]] ** impression: we want to record whether the user saw the help panel in edit mode. ** root screen button click: the new design of the “root screen” will contain several buttons. We want to record an event indicating which one the user clicks. ** Once the user has clicked one of the buttons, normal help panel logging ensues. * [[ https://phabricator.wikimedia.org/T245790 | Post-edit dialog ]] ** impression: we want to record that the user received the post-edit dialog. Because this dialog will contain a next article suggestion, we want to record the same details of that article suggestion that we record in se-task-impression events (e.g. title, whether it has an image, task type, maintenance template, etc.), while also recording that they received the impression via this post-edit dialog, as opposed to from the suggested edits module. ** close: whether the user clicks away or clicks the “X” to dismiss the dialog. ** clicks: whether the user clicks the link to go to their homepage. ** task click: if the user clicks the suggested article, we want to record that click event along with the article details, similar to se-task-click events on the homepage, but specifying that they clicked via this post-edit dialog, as opposed to from the suggested edits module. ** homepage visit: in the homepageVisit schema, we record the referer_route. We should add a referer_route for arriving on the homepage via the link in the post-edit dialog.

The help panel feature is already instrumented, per the work done in {T206719}. In using it to display guidance for newcomer tasks, we have modifications and additions to make. This task will track the work, and the individual tasks about the different components of guidance will also contain notes on the relevant instrumentation for those components. [] Changes to EventLogging schemas [] Code for instrumentation [] Post-deployment QA by data analyst [] Publish changes to measurement plan The specifications are recorded in the existing measurement plan for newcomer tasks at these headers: * [[ https://docs.google.com/document/d/1dEE_XicyBWOKPu8op4t6neIHncSqlxubUgFa82bZQxI/edit#heading=h.f1gzx2ak03t6 | Measurements ]]: the numbers we want to be able to produce * [[ https://docs.google.com/document/d/1dEE_XicyBWOKPu8op4t6neIHncSqlxubUgFa82bZQxI/edit#heading=h.6z9kmrrq4bzz | Rules ]]: specific interactions to be recorded. These are also pasted below for convenience. But note that the Google Doc contains useful comments from team members. **Additional rules for version 1.1 (topic matching)**2 (guidance)** * General ** Since guidance will largely be implemented via the help panel, we want to record all the same things we already record via the help panel, in terms of it being opened and closed, preferences cog being clicked, back button being clicked, etc. The below points highlight things that are new with the guidance feature (and also includes some of the standard elements, just for completeness). ** For all events, we need to be able to tell that they happened as part of the suggested edits workflow, and not as part of the help panel displaying in a normal editing session. ** For all events, we need to be able to tell whether they happened in read mode or in edit mode. * [[ https://phabricator.wikimedia.org/T244541 | Suggested edit screen in read mode ]] ** This screen displays on desktop when the user arrives on the article. On mobile, it displays when the user has tapped the mobile peek. ** impression: we want to record that the user saw the help panel and which set of content it displays, in terms of which “task type” it is guiding, and whether it is showing the desktop VE, mobile VE, desktop wikitext, or mobile wikitext content. ** clicks: we want to record exactly which tab the user clicks on when they click through the quick tips. ** auto-advance: the quick start tabs will advance automatically every five seconds until the user clicks one. We want to record an event for each auto-advance, indicating which tab it advanced to. ** link clicks: the suggested edit screen will contain links to help documents, to which we want to record clicks. * [[ https://phabricator.wikimedia.org/T244434 | Mobile peek ]] ** peek impression: we want to record that the user saw the mobile peek and which set of content it displays, in terms of which “task type” it is guiding, and whether it is showing the desktop VE, mobile VE, desktop wikitext, or mobile wikitext content. ** tap on peek: whether the user taps the mobile peek. ** dismiss peek: whether the user taps outside the mobile peek, which dismisses it. * [[ https://phabricator.wikimedia.org/T244435 | Clicking edit ]] ** We want to record when the user clicks the “Edit” button. In two-tab wikis on desktop, we need to know whether they chose the VE or wikitext option. One way to figure this out may be with the editattemptstep schema with appropriate oversampling. * [[ https://phabricator.wikimedia.org/T244546 | In edit mode ]] ** impression: we want to record whether the user saw the help panel in edit mode. ** root screen button click: the new design of the “root screen” will contain several buttons. We want to record an event indicating which one the user clicks. ** Once the user has clicked one of the buttons, normal help panel logging ensues. * [[ https://phabricator.wikimedia.org/T245790 | Post-edit dialog ]] ** impression: we want to record that the user received the post-edit dialog. Because this dialog will contain a next article suggestion, we want to record the same details of that article suggestion that we record in se-task-impression events (e.g. title, whether it has an image, task type, maintenance template, etc.), while also recording that they received the impression via this post-edit dialog, as opposed to from the suggested edits module. ** close: whether the user clicks away or clicks the “X” to dismiss the dialog. ** clicks: whether the user clicks the link to go to their homepage. ** task click: if the user clicks the suggested article, we want to record that click event along with the article details, similar to se-task-click events on the homepage, but specifying that they clicked via this post-edit dialog, as opposed to from the suggested edits module. ** homepage visit: in the homepageVisit schema, we record the referer_route. We should add a referer_route for arriving on the homepage via the link in the post-edit dialog.