This task is about analyzing how people are engaging with the V2.0 of the new Replying feature to help determine:
- Should the replying feature be deployed as an opt-out user Preference on the four target wikis?
- What – if any – changes should be made to the workflow before the feature is deployed to more people?
This task/analysis, comes as a follow up to the initial analysis the team did as part of the release of V1.0. More information about that analysis can be found in this ticket: T247138.
=== Open questions
- [ ] How long after v2.0 is released should this analysis happen?
=== Workflow engagement metrics
//This section is still being drafted. Once complete, it will continue the information needed make the two decisions listed above.//
**1. Are people publishing the replies they start?**
- % of people who click reply AND who start typing a comment
- % of people who start typing a comment AND who publish their comment to page
**2. Are people abandoning the replies they start?**
- % of people who start typing a comment AND who actively abandon their comment
**3. Are people taking a long time to publish their replies?**
- Distribution of elapsed times between when a person starts typing their comment and when they publish their comment
**4. Are people needing to correct mistakes the tool produced?**
- % of edits made using Replying feature that are "immediately" followed by a non-DiscussionTools edits to the same talk page, by the same person
**5. Are people satisfied with the feature?**
- % of people who publish at least one comment and turn off DiscussionTools in Beta Features
- Of the people who have made at least 1 edit with the Replying workflow, the % of people who posted 1, 2-5, 5-10 and 10+ comments
**6. Are people posting disruptive comments?**
- % of comments posted using Replying workflow that are reverted
- # of people blocked after posting a reply using DiscussionTools
- % of comments posted using Replying workflow that are made in good faith
**7. Are people satisfied with the text input they are being shown by default?**
- % of contributors who switch to a text input mode that is different from the one they are shown when they first click "Reply" //after// having started to type a comment
- % of contributors who switch to a text input mode that is different from the one they are shown when they first click "Reply" //before// having started to type a comment
**8. Are people able to successfully edit their comments?**
- % of people who initiate the "edit specific comment workflow" and successfully save a change
- % of people who initiate the "edit specific comment workflow," make a change to their original comment and DO NOT publish it
---
//The metrics in this section are best considered to be "exploratory." We will determine whether they are in/out of scope of this particular task once we get closer to working on it.//
**How do the comment completion rates vary between the text input modes?**
- % of published comments drafted using the `Visual` mode
- % of published comments drafted using the `Source` mode
- % of published comments drafted that involved a contributor switching from the `Source` to the `Visual` and vice versa
=== Done
- [ ] Analysis is completed that contains the "Workflow engagement metrics" listed above