The newcomer tasks feature is already instrumented, per the work done in {T230068}. In adding topic matching to the workflow, we will need to extend the existing instrumentation. Since this is a modification to an existing instrumentation effort and not a whole new effort, we'll try to use this one task to track all the steps.
[] Changes to EventLogging schema
[] Code for instrumentation
[] Post-deployment QA by data analyst
[] Publish changes to measurement plan
The specifications are recorded in the existing measurement plan for newcomer tasks at these headers:
* [[ https://docs.google.com/document/d/1dEE_XicyBWOKPu8op4t6neIHncSqlxubUgFa82bZQxI/edit#heading=h.7fi11obawyft | Measurements ]]: the numbers we want to be able to produce
* [[ https://docs.google.com/document/d/1dEE_XicyBWOKPu8op4t6neIHncSqlxubUgFa82bZQxI/edit#heading=h.yasztcizry6q | Rules ]]: specific interactions to be recorded. These are also pasted below for convenience.
The measurements and rules were written with the use case of ORES models in mind -- not with the interim "morelike" approach. These should be pretty compatible in terms of instrumentation, but if they're not, we should discuss on this task.
**Additional rules for version 1.1 (topic matching)**
* Intro overlay: version 1.1 will evolve the topic overlay from purely informative to including options to set topics of interest.
** We want to record the topics the user has chosen once they advance beyond this overlay. We should also record which topics the user chose from above the “show more” link, and which they chose from below the “show more” link. Individual clicks on topics do not need to be recorded, and no record of topics chosen needs to be made if the user cancels the overlay.
** We should record as a separate event that the user clicked to “show more” in the overlay.
* Topic filters:
** topic open: the user clicks to open the filter selection to change topics. We record the topic selection upon open.
** topic done: the user clicks “done” on the filter selection to change topics. We record the topic selection and the number of matching suggestions. We should also record which topics the user chose from above the “show more” link, and which they chose from below the “show more” link.
** topic close: the user clicks “cancel” on the filter selection to change topics. We record the topic selection and the number of matching suggestions.
** We should record as a separate event that the user clicked to “show more” in the overlay.
* Task impression and click: in addition to the data being recorded for these events for version 1.0, we should add the following elements.
** topic assignment: the topic associated with that task. This should be determined as the highest-scoring topic that the user has selected a filter for. For example, if an article scores 0.96 for Physics and 0.83 for Chemistry, and the user has selected “Chemistry” in their topic filters, then we should record Chemistry. If the user has selected “Physics” and “Chemistry”, we should record Physics.
** match score: any sort of match confidence or score available with the topic.