Page MenuHomePhabricator

Track basic usage of new Talk pages
Closed, ResolvedPublic

Description

With the revision to talk pages we'd like to track basic engagement and usage of this feature. Specifically:

  • How many users access the Talk and User Talk pages?
  • What actions do they take on the Talk pages?
  • What is the language and project distribution of usage?

Other data included in analysis:

  • Talk page editing event data
  • Platform comparison Talk page edit data

Event Timeline

LGoto triaged this task as Medium priority.

I have a current.yaml file for this ready for schema when engineering is ready.

Uploading the current.yaml file here but note we will need to determine and add the values for source field, they may different from what Android uses* so I left that out.

Also note we now need to specify when we want to track geocoded_data in MP - if it's not currently set up you will likely need to add the /fragment/http/client_ip/1.0.0# ; then we can chain inherit the fragments in the MEP system. @Sharvaniharan noted "i have included it in a common fragment called wikipedia_android_app and have in turn used it in all schemas. You can see how it is used in one of the android schemas and follow the same". Contact her if you run into issues with this.

*Current values in use on Android MobileWikiAppTalk:
page
talkActivity
notification
diffActivity
navMenu
watchlist
archivedTalkActivity

Hi @SNowick_WMF! Thanks for the yaml file. I have a few questions and notes:

  1. I noticed it's at version for 2.0.0 - is that intentional? $id: /analytics/mobile_apps/ios_talk_page_interaction/2.0.0. This will be a first for us so I'm thinking maybe we should change that one to 1.0.0.
  2. refresh - we don't have this option currently, but we may in the future so I think it's fine to leave this here.
  3. open_talk - I'm a little unsure of when we should log this. We can log it any time the talk page appears, so that we capture the event no matter the source (notifications center, article footer, link in an article, link in another talk page, deep link, etc). Or are we supposed log this action only when they tap a particular talk button somewhere?
  4. submit - This is for whenever they submit a brand new topic or and inline reply to an existing thread, correct?
  5. Several different projects can display these native talk pages now (German Wikipedia, Wikidata, Spanish Wikiquote. etc). Maybe we should log some identifying project string? We did something similar for our notifications schema.
  6. time_spent - Just confirming - we'll start a timer as soon as they land on the talk page, and send the seconds elapsed with each action event?

we will need to determine and add the values for source field, they may different from what Android uses* so I left that out.

Sounds good, I'll drum up a list source options here. If it's okay I'd like to add an unknown or other fallback option once I get final list, just in case for some reason we're unable to determine it. It should be unlikely but could indicate a bug that we missed setting a source somewhere before logging.

Also note we now need to specify when we want to track geocoded_data in MP

Sounds good, we'll take a look, thanks!

@SNowick_WMF these are the source values I came up with for routes that take you to talk pages:

talkPage
article
notificationsCenter
deepLink
account
search
inAppWebView
unknown

Let me know if this looks good to you, thanks!

I should probably explain a bit more :)

  • talkPage - tapping a talk page link within a talk page
  • article - tapping a talk page link within an article, or the article footer button
  • notificationsCenter - tapping a talk page button in the notifications center detail view or the swipe > more menu from each cell
  • deepLink - if a user is deep linked to a talk page from outside the app
  • account - if user goes to their talk page from Account
  • search - if user explicitly searches for a talk page and taps a search result
  • inAppWebView - if a user taps a talk page link from within an in-app web view (such as a user page)
  • unknown - unable to determine source

Hi @Tsevener

Thank you for the source definitions, I wasn't sure where there was overlap with Android. Can we add these to the description in the current.yaml?

Answering Qs inline:

  1. I noticed it's at version for 2.0.0 - is that intentional? $id: /analytics/mobile_apps/ios_talk_page_interaction/2.0.0. This will be a first for us so I'm thinking maybe we should change that one to 1.0.0.

Yes, that was copied over from Android, lets stay with 1.0.0

  1. refresh - we don't have this option currently, but we may in the future so I think it's fine to leave this here.

Ok, makes sense to keep it.

  1. open_talk - I'm a little unsure of when we should log this. We can log it any time the talk page appears, so that we capture the event no matter the source (notifications center, article footer, link in an article, link in another talk page,

This is Android specific since we have a Talk entry point so yes, it may be extra with iOS, ok to leave out.

  1. submit - This is for whenever they submit a brand new topic or and inline reply to an existing thread, correct?

Yes for both.

  1. Several different projects can display these native talk pages now (German Wikipedia, Wikidata, Spanish Wikiquote. etc). Maybe we should log some identifying project string? We did something similar for our notifications schema

Makes sense, we can add a wiki_db column to the schema (type: string) to cover this.

  1. time_spent - Just confirming - we'll start a timer as soon as they land on the talk page, and send the seconds elapsed with each action event?

Yes, we measure in seconds on Android.

Change 857759 had a related patch set uploaded (by Mazevedo; author: Mazevedo):

[schemas/event/secondary@master] Add ios talk page interaction schema

https://gerrit.wikimedia.org/r/857759

@SNowick_WMF For wiki_db, I plan on using the same identifier that we used with our notifications logging, which maps up with the how the Notifications API identifies it (e.g. enwiki, wikidatawiki, eswiktionary, etc.). I'm open to setting up different identifiers if you'd rather have something else, though. Thanks!

@Mazevedo Just a reminder (I nearly forgot myself until I started working on the client-side piece of this) - after your schema patch is merged, you'll need to make another small patch adding the stream ID. Here's an example patch we did for notifications. This stream will be called ios.talk_page_interaction.

Tsevener moved this task from Doing to Needs Code Review on the ios-app-v7.0.1 board.

@Mazevedo Just a reminder (I nearly forgot myself until I started working on the client-side piece of this) - after your schema patch is merged, you'll need to make another small patch adding the stream ID. Here's an example patch we did for notifications. This stream will be called ios.talk_page_interaction.

I'll create a separate ticket to help me organize the patches.

Hi @Tsevener that wiki_db value works, as long as we have an identifier, we are good.

I do want to note that the standard for column name should be wiki_id, Marina noted that we were requested to change it in the schema but you're probably already aware, just noting for posterity. Gerrit

Hi @Ottomata! - I was wondering if we could get a +2 soon on this patch. We're hoping to be out with the app version that calls these events in a couple of weeks, and I'd like to get it in before any holiday deployment freeze. We also have a follow up config patch - I realize we need to go through a different deployment process for it, so +1 before we try to initiate that would be great. Let me know if that sounds doable, thanks!

Oh hello! You do not need to block on me for +2 in the schemas/events/secondary repository, nor on the stream config path. I don't mind being added as reviewer for quick spot checks, but for instrumentation schemas and streams, please feel to review/approve/merge however your team sees fit! :)

I've gone ahead and +2ed anyway, but next time you all can. :)

Change 857759 merged by jenkins-bot:

[schemas/event/secondary@master] Add ios talk page interaction schema

https://gerrit.wikimedia.org/r/857759

@Ottomata ok perfect, thanks for clearing that up!

Thanks @Tsevener - verifying I am now seeing test data in ios_talk_page_interaction schema, every possible action variable is showing up, looks like we are good to go.

@SNowick_WMF Great, thanks! I'll move this over to PM signoff. Are you seeing geocoded_data coming through the schemas as well? If so I'll move T322790 over.

@Tsevener yes, I verified the geocoded_data for all 3 schemas just now and made a note in T322790, ok to move over.

JTannerWMF subscribed.

Shay is working on this task now?