Updating task description: 13-Dec

• ADDED: background information about this task, why it is important and relevant links

@DLynch + @MNeisler, if you think the description could be made more clear, please let me know or boldly affect it as you see fit.

Met with Peter and David over this week to finalize project kpi, replying kpi, impact kpi as well as Workflow Engagement metrics. Also discussed workflow events and scenarios that we will need to track.
Topics discussed in 01/17 meeting between DL-PP-MK:

• What talk page and replying-related events would we like to track?
• In what schema should these events be logged?
  • DL: seeming more and more like a new schema is necessary; contexts are sufficiently different; dashboarding would become tricky if metrics for talk pages and articles were collapsed into one schema
• Current talk pages tracking
  • Assume things we are tracking on articles we are tracking on talk pages EXCEPT for VE-related metrics (all happens in Schema:EditAttemptStep - Meta)
  • Mostly this is just EditAttemptStep. People using NWE will also be submitting VisualEditorFeatureUse for any wikitext tools they use.
• Talk pages/Replying/Instrumentation spec
  • Workflow completion timing
    • Complicated, but potentially not impossible to create a 1:1 comparison between replying workflow and full page b/c full page workflow could – in the case of contributors using classic wikitext editor – involve a contributor visiting 3 distinct pages. Then becomes difficult to work out when exactly the session started
    • Also: more thought needs to be put to when exactly the workflow starts in both experience (full page and new replying workflow)

Peter has scheduled meeting with Jason, Mikhail, David L., Maya on 01/21 for schema discussions:
Agenda

• Where should all talk page, and talk page-related events, should be logged?
• Who and how will we document, implement and test talk page instrumentation?
• Bonus: how might we be able to better understand the kinds of edits contributors are making using full talk page editing?
  • E.g. how do we tell whether a contributor is replying to a comment or fixing a typo?

Notes

• Talk page schema requirements:
  • Reliable
  • Extensible
  • Maintainable
  • Contextual (e.g. "editing" a talk page is diff. than "editing" an article)
• Considerations
  • We'd like to have the ability to dashboard talk page metrics

Meeting notes recorded in this document

21-Jan update

Notes from today's meeting below...

Decisions

• We will experiment with logging the new DiscussionTools-related events in a new schema

Resulting actions

• Editing-team and Product-Analytics will collaborate to "mockup" a schema design for the new schema and its relationship to "EditAttemptStep" that follows the pattern established by EditAttemptStep and VisualEditorFeatureUse schemas.
  • Once this mockup is created, we can revisit the "Open questions" listed below.

Open questions

• Where should full talk page editing events live? In EditAttemptStep? In the new schema? Some other configuration?
• If we proceed with introducing a new schema, what are the different schema those events should be sent to?
• What is the relationship between new schema and EditAttemptStep?

Full meeting notes: Meeting/Product Analytics ⇄ Editing

24-Jan Update

Met with David and Peter and to understand how the EditAttemptStep schema can be used for Replying Talk pages.
Will discuss next week during All Hands or after, regarding the new talk page schema.

Action
Which events in the events tab of the instrumentation spec are deserving of a schema that is distinct from EditAttemptstep?

06-Feb Update

Met with @DLynch over the last week to discuss, (focussing only on the current release V1.0) which events need to be instrumented and which schema they will live. We have come to the conclusion that for the features being implemented as a part of Replying V1.0, we will not need a new schema and everything we need can be tracked using EditAttemptStep.
T243363 is not a blocker for V1.0

Please note that as we move forward with the release of V1.1 and V2.0, we will discuss and may require a new schema.

19-February

Documenting the answers to our remaining open questions...

• Where should full talk page editing events live? In EditAttemptStep? In the new schema?
  • For Replying v1.0, all events will be logged in EditAttemptStep. See task: T243467
• If we proceed with introducing a new schema, what are the different schema those events should be sent to?
  • We do not need to answer this question for v1.0. Instead, we will answer this question as part of v2.0. That work will happen in this task: T243363
• What is the relationship between new schema and EditAttemptStep?
  • This will be determined as part of the work to complete his tasks T243363