Page MenuHomePhabricator

As a product analyst I would like to know how people are using the Wikidata Descriptions editing features
Closed, ResolvedPublic

Description

See https://phabricator.wikimedia.org/T149915 and https://www.mediawiki.org/wiki/Wikimedia_Apps/Short_descriptions/Research for prior work by Android.

Here are the core product questions:

  • How many Wikidata edits are made via the iOS? What is the distribution of languages?
  • Do iPad or iPhone users make proportionally more edits?
  • What is the 7 day revert rate of edits from iOS?
  • How many/what percent of users start the description editing process?
  • How many/what percent of them make it to the point of saving the edit?
  • How many see the revert notification?
  • How many read or click through on the revert notification?
  • How many of these edits are anonymous?
  • What is the distribution of edits per editor (ie. how many people edit 1 description, how many edit more than 5, etc)

"these users" below refers to people who have submitted one edit description...

Harder:

  • How many of these users go on to edit in other ways, either on app or desktop?
  • What is the distribution of 30 day retention of these editors, as editors?
  • Are their any correlations between completing an edit (or making multiple edits) and other trackable factors, such as device type, primary language or days since install? Ie. are long time readers on iPads more likely to edit these successfully? Not the reason, just the correlations if any...

Event Timeline

JMinor triaged this task as High priority.Sep 6 2018, 10:37 PM
JMinor created this task.

@JMinor Thanks for the list of the product questions!

Except the followings, other questions can be answered by mediawiki database:

  • Do iPad or iPhone users make proportionally more edits?

I remember the raw user agent strings are saved somewhere.. @Neil_P._Quinn_WMF Do you know about it?

  • How many/what percent of users start the description editing process?
  • How many/what percent of them make it to the point of saving the edit?
  • How many see the revert notification?
  • How many read or click through on the revert notification?

These questions would require eventlogging instrumentation. @mpopov Has Android implemented similar schema?

  • Do iPad or iPhone users make proportionally more edits?

I remember the raw user agent strings are saved somewhere.. @Neil_P._Quinn_WMF Do you know about it?

Yeah, the user agent for each edit is saved for 90 days by the CheckUser extension. The data is available in the MediaWiki tables in each project's cu_changes table, and in the Data Lake in a single wmf_raw.mediawiki_private_cu_changes table 😀

JMinor lowered the priority of this task from High to Medium.Nov 14 2018, 12:26 AM

I created T210970 for the instrumentation work.
And since the rest of this task are analysis work for @chelsyx, I removed it from the v6.2 board.

chelsyx moved this task from Next Up to Doing on the Product-Analytics board.Dec 3 2018, 8:00 PM
chelsyx moved this task from Doing to Backlog on the Product-Analytics board.Dec 12 2018, 6:54 AM

The analysis task is partly done. See the slide deck: https://docs.google.com/presentation/d/1xEKa_iHkg6Rb9UBOlGGG9quZGX7kBhpzhd5lcIhRVrE/edit?usp=sharing

Regarding other questions need to be answered in this task:
1, We need to implement eventlogging to answer the following questions:

  • How many/what percent of users start the description editing process?
  • How many/what percent of them make it to the point of saving the edit?
  • Are their any correlations between completing an edit (or making multiple edits) and other trackable factors, such as device type, primary language or days since install? Ie. are long time readers on iPads more likely to edit these successfully? Not the reason, just the correlations if any...
  • How many see the revert notification?
  • How many read or click through on the revert notification?

Thee first three can be answer after T210970 is resolved. We still need to create new schema to record notification related behavior, if we want to answer the last two questions above.

2, The following question required table joins across databases, which is doable, but very slow given our current technology. This question is also not critical at this moment to my knowledge. Please let me know if you think differently.

  • How many of these users go on to edit in other ways, either on app or desktop?

I'm moving this task back to the backlog.

chelsyx moved this task from Backlog to Epics on the Product-Analytics board.Feb 12 2019, 10:32 PM

Moving to blocked as I'm currently waiting for 6.2 release (T210970)

chelsyx moved this task from Epics to Next Up on the Product-Analytics board.Mar 12 2019, 10:24 PM
chelsyx moved this task from Next Up to Doing on the Product-Analytics board.Apr 4 2019, 5:07 PM
chelsyx closed this task as Resolved.Apr 17 2019, 12:53 AM

T210970 is resolved and here is the answer for these questions:

How many/what percent of users start the description editing process?

1.28%

How many/what percent of them make it to the point of saving the edit?

After they start, 41.7% of sessions save the edit.

Conversion rate in the funnel:

Please note that these numbers are calculated using event logging data, therefore it only includes users who opt-in to share their usage with us.
Feel free to re-open this ticket if you have any other questions.