Page MenuHomePhabricator

Determine if new instrumentation is necessary based on talk page designs
Closed, ResolvedPublic

Description

1. Background

The Android team made significant visual enhancements to talk pages in the app. The changes may result in necessary alterations to our team's instrumentation.

2. User stories

  • When viewing user and article talk pages, I want a visual distinction between the two, so that I realize that they have different purposes and have better orientation.
  • When viewing user and article talk pages, I want to have a preview of the discussion topics in the list view, so that I can get a better idea on the topics that are discussed.

3. Designs (Figma)

Disclaimer: Only lists differences from user talk page designs.

  • App bar:
    • Features link to article talk page in app bar (blue)
    • Overflow menu features link to article page
  • Header image
    • Uses the article’s header image (if available). Height: 84dp.
  • Template (yellow): previews contents of the template on three lines in plain text. Notice the updates to the designs.
  • List item differences to user talk pages
    • Don’t contain usernames
    • Don’t contain message previews
  • Footer
    • Contains link to the article’s native edit history
    • Contains a link to the article page

4.0. Task for Analytics

Review the Figma file for the User and article talk page changes [T297617] as well as the new threading and subject view [T303254] and define any new changes to instrumentation or schemas that may be necessary as a result of the refresh.

Event Timeline

We will want to measure user interaction with new features on both User and Article Talk re-designs.

For both User and Talk:
Search bar usage (per unique user & total usage relevant to overall DAU)
'Sort by' icon clicks (per unique user & total usage relevant to overall DAU)

Will discuss with @Sharvaniharan on best method to integrate these metrics and document here.

TBD @JTannerWMF, @scblr if we want more in-depth metrics for usage of:
Sorting order selections/settings (User and Article)
Messages Read/Unread (User Talk)
Show reply indicators (Article Talk)
Expand thread indicators (Article Talk)

Thanks @SNowick_WMF

TBD @JTannerWMF, @scblr if we want more in-depth metrics for usage of:
Sorting order selections/settings (User and Article)
Messages Read/Unread (User Talk)
Show reply indicators (Article Talk)
Expand thread indicators (Article Talk)

  • The sort order selection instrumentation would be very interesting to me, as it would indicate a possible shift to making it the default after a certain while (towards a more modern approach to communication) – if we see people are using it often.
  • Expanding thread data in both article and user talk pages is interesting too – as conversations haven’t been threaded before. Besides expanding individually, there’s also a global option, available in the overflow menu to expand all threads at once (see screens below). Might be interesting to instrument this one as well:
Talk pages redesign1.png (1×720 px, 167 KB)
Talk pages redesign2.png (1×720 px, 166 KB)

I agree with what Robin shared above.

We will add tracking for both additional in-depth metrics; sort order selection (and what options user selects) and thread expand/collapse (both global and by item). @Sharvaniharan will follow up when implementation is ready to be tested.

Noting here what action values we are adding to the MobileWikiAppTalk dataset.

Talk pages redesign.png (1×720 px, 97 KB)

For sort order usage, change tracking:
action = open_sort (user clicks on 'Sort By')
action = sort_order_published (user selects 'Date published')
action = sort_order_updated (user selects 'Date updated')
action = sort_order_topic (user selects 'Topic name')

For thread expand/collapse:
Sidebar expand/collapse:

  • action = global_collapse
  • action = global_expand

Item expand/collapse:

  • action = item_collapse
  • action = item_expand

Hey @SNowick_WMF it appears you've added your feedback so I will move this into ready for dev for the analytics eng, def move it back if that is incorrect.

Hi @SNowick_WMF

For the following actions:

action = sort_order_published (user selects 'Date published')
action = sort_order_updated (user selects 'Date updated')
action = sort_order_topic (user selects 'Topic name')

They are able to sort by ascending order or descending order, should we add them into the action like sort_order_published_ascending and sort_order_published_descending?

@cooltey sounds good, yes please use for those events.