Page MenuHomePhabricator

Implement way to identify people who have opened the New Discussion Tool before
Closed, ResolvedPublic

Description

This task is about implementing a way to identify people who have opened the New Discussion Tool before.

Use Case(s)

Distinguishing people in the New Discussion Tool A/B test (T277825) who have opened the New Discussion Tool before the test started from people who have NOT opened the New Discussion Tool before the A/B test started.

Requirements

  • Whatever approach is taken to log whether people have opened the New Discussion Tool or not needs to be easily related to EditAttemptStep.
    • Note: "opened the New Discussion Tool" in this context means clicking an affordance that causes people to see the New Discussion Tool open on a particular wiki.
  • Whatever trigger/event/value we use should NOT collide with anything that's set by the reply tool.
    • Said another way: we are looking for a trigger that'd ignore usage of the reply tool. E.g. discussiontools-editmode preference is not empty would not let us know which tool (new discussion or reply tool) the person used.

Minimum test cast

  1. Log in to the beta cluster on desktop
  2. Verify the New Discussion Tool is enabled on your account
  3. Visit a talk page on the beta cluster. E.g. https://en.wikipedia.beta.wmflabs.org/wiki/Talk:Cats.
  4. Click the New section button to open the New Discussion Tool
  5. Open the browser's console
  6. Verify the state of the 'discussiontools-newtopictool-opened' preference is '1'

Done

  • 1. Editing Engineering implements an approach that satisfies the ===Requirements above and can be used in the cases described in ===Use cases (also above)
  • 2. Editing QA verifies the preference Editing Engineering implemented is being changed/created at the expected moment.
    • QA: Current state of the preference can be tested via mw.user.options.get( 'discussiontools-newtopictool-opened' ) in the console.
  • 3. Product Analytics verifies the preference Editing Engineering implemented is being stored in the expected way/place

Event Timeline

I'll note that if the two use-cases are tied together, this basically becomes a minor sub-detail of implementing T293962. (Basically "make sure to implement this in a way that the A/B test can see it", which mostly just constrains that task away from using localstorage or similar.) If we wanted to roll out the A/B test before the hint was implemented (or simultaneously with its release), we'd need to consider those separately.

That said, regarding use case 2, "across wikis" is actually really unusual for us, and I'll have to make sure that GlobalPreferences is acceptable to use for such a thing. (There's a lot of initial-tutorial stuff that you'll see every time you visit a new wiki. There's even a use-case for this remaining so, because the individual communities might want to customize the messages in these things away from a literal translation.)

@DLynch: I think the information you shared in T294167#7460407 contains the missing information we needed to converge on the requirements be ready to implement. As such, I've updated the ===Requirements in the task description.
Please comment here if anything strikes you as unclear/unexpected/etc.

Note: below are the direct responses to the points you raised.


I'll note that if the two use-cases are tied together, this basically becomes a minor sub-detail of implementing T293962. (Basically "make sure to implement this in a way that the A/B test can see it", which mostly just constrains that task away from using localstorage or similar.)

The details you shared above are helping me to more clearly see the relationship between this work and T293962...thank you for elaborating, David.

If we wanted to roll out the A/B test before the hint was implemented (or simultaneously with its release), we'd need to consider those separately.

We do want to have the flexibility to roll out the A/B Test BEFORE the hint (T293962) is implemented. As such, let's explicitly consider this task as unrelated to T293962.[i]

That said, regarding use case 2, "across wikis" is actually really unusual for us, and I'll have to make sure that GlobalPreferences is acceptable to use for such a thing. (There's a lot of initial-tutorial stuff that you'll see every time you visit a new wiki. There's even a use-case for this remaining so, because the individual communities might want to customize the messages in these things away from a literal translation.)

Noted. For the purpose of this task, I'm going to explicitly EXCLUDE any requirement about the method we use to log peoples' prior use of the New Discussion Tool across wikis.


i. The primary reason for us treating this task and T293962 as distinct is so that we can start gathering data about who's used the New Discussion Tool as soon as possible...I'm glad you (David) raised this consideration in T291536#7424732.

Change 738610 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/DiscussionTools@master] Set a user preference to record whether the new topic tool has been used

https://gerrit.wikimedia.org/r/738610

Change 738610 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@master] Set a user preference to record whether the new topic tool has been used

https://gerrit.wikimedia.org/r/738610

DLynch added a project: Editing QA.

QA: Current state of the preference can be tested via mw.user.options.get( 'discussiontools-newtopictool-opened' ) in the console.

ppelberg updated the task description. (Show Details)
ppelberg moved this task from Inbox to High Priority on the Editing QA board.

Change 745672 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/DiscussionTools@master] Update abtest bucketing for newtopictool usage

https://gerrit.wikimedia.org/r/745672

Ignore that, I copied from the wrong tab when writing my commit message.