Page MenuHomePhabricator

Instrument new feature for iOS Watchlist
Closed, ResolvedPublic

Description

Background
This task is to wire up the event logging for the new features that will be created for watchlist.

The task
Shay to add further details below:

  • Instrument new Diff screen and new actions
  • Instrument Watchlist home
  • Instrument entry point for feature

Requirements

  • Ensure data signs off on instrumentation task
  • Shay should reference KPIs and questions we want to answer to ensure all data that is wired up meet her needs for pulling data

References
The Deepdive deck

Designs on Figma

Event Timeline

Update on data availability - I have counts from Legacy schemas that can be used for action frequency distribution but not actual active DAUs, etc because this is data sent from users using older app versions.

For Android I can get some current user data for Watchlists from event.android_breadcrumbs_event schema, working with Sharvani now to map events to action/screen_name. See: https://phabricator.wikimedia.org/T335167

iOS data for Thanks, Diff Compare views/actions
MobileWikiAppiOSEditHistoryCompare
https://docs.google.com/spreadsheets/d/1GYEzHrTA-6cBuKD1dnOC50_4_ZlXoqoiGGFRtrxt880/edit#gid=1822923207

For Android I have data from older version users for Watchlist actions:
MobileWikiAppwatchlist
https://docs.google.com/spreadsheets/d/1GYEzHrTA-6cBuKD1dnOC50_4_ZlXoqoiGGFRtrxt880/edit#gid=0

mpopov triaged this task as Medium priority.Apr 25 2023, 5:09 PM
mpopov moved this task from Triage to Current Quarter on the Product-Analytics board.

A question that may come up is what are the most popular things tapped on in our toolbars on iOS so we can consider entry points.

RWambua-WMF renamed this task from Report baseline metrics to iOS Watchlist Report baseline metrics.Apr 26 2023, 6:47 PM
RWambua-WMF updated the task description. (Show Details)
RWambua-WMF lowered the priority of this task from Medium to Low.May 1 2023, 7:24 PM
RWambua-WMF updated the task description. (Show Details)
RWambua-WMF added a subscriber: SNowick_WMF.
RWambua-WMF renamed this task from iOS Watchlist Report baseline metrics to Instrument new feature for iOS Watchlist.May 9 2023, 6:49 PM

Design docs for reference:
Figma

When finalized I will make a proper schema doc to document what will track, plan on creating new schema ios_watchlists

Watchlist Instrumentation Docs:
For diff page analytics see end of deck and last rows in Instrumentation spreadsheet

iOS Watchlist Schema deck
iOS Watchlist Instrumentation Process and Spec doc

Change 955405 had a related patch set uploaded (by Tsevener; author: Tsevener):

[schemas/event/secondary@master] Add watchlist-specific properties to ios_watchlists

https://gerrit.wikimedia.org/r/955405

@SNowick_WMF Hi Shay, I think the design has changed a bit since the last time this schema was made, so here's some changes I'm proposing:

  1. Empty states have changed - T342552. We are no longer showing the Needs Login state, and the "Explore articles" button has been changed to "Search articles". I suggest removing watchlist_login, watchlist_signup, watchlist_create_account, watchlist_account_success, watchlist_account_fail and changing watchlist_explore to watchlist_search.
  2. Users can now enter the Watchlist view from the success toast displayed after successfully watching an article on the article view or the diff view. I suggest adding a open_watchlist_article_added_confirm and open_watchlist_article_diff_confirm action (feel free to improve the name) to capture users entering through that path.
  3. The Article or Diff > Watch flow has changed a bit. Now the user is immediately prompted with an expiry option before the article is watched, there is no longer a "change expiration date?" toast. I suggest removing expiry_change.

Analytics aren't in here yet, but the feature is available in TestFlight build 7.4.2 (2622) if you want to reference it while considering these schema tweaks.

Officially moving to blocked for final events and thank action work, but this is getting closer! WIP is in linked Gerrit patch and T334968 branches in app and Components repo.

Thank you for updating me, I agree we should go with the changes you are suggesting, I am updating the docs now.

Notes:

  1. Took out watchlist_login, watchlist_signup, watchlist_create_account, watchlist_account_success, watchlist_account_fail and changed watchlist_explore to watchlist_search
  2. I'm ok with the action names, just need to determine best location in slides to add these, let's discuss
  3. Done, expiry_change was removed.
Tsevener subscribed.

Change 955405 merged by jenkins-bot:

[schemas/event/secondary@master] Add watchlist-specific properties to ios_watchlists

https://gerrit.wikimedia.org/r/955405

Verifying that we are seeing most events in ios_watchlists corresponding to the changes mentioned above. There are a few events from overflow menu and specific expiry selections we have not gotten data for yet so not signing off 100% yet but wanted to note initial validation is looking good. Validation so far.

Will update ticket when all events have been confirmed.

Also verifying we are seeing edit init/success events in Editattemptstep for successful edits tracked in ios_watchlists.

Reopening so we can see which release it went out in