Page MenuHomePhabricator

[EPIC] AMC Metrics
Open, NormalPublic

Description

NOTE: The following do not include metrics for individual components of the project - these will be reflected in the tasks themselves

Questions we want to answer

  • Does advanced mode help advanced users be more effective on mobile? (proxied by moderation action count, edit rate)
  • Does advanced mode reduce the necessity of switching to desktop mode?
  • Does advanced mode increase traffic and edits to the talk page of an article?

Metrics

  • Mobile web edit rate on target wikis: looking for 10% increase (over time, according to existing edit tag definition. We'll tag edits from the new interface - T212959 - and see if they reach 1/11th of total mobile edits)
  • Mobile web edit rate on target wikis limited to users with 100+ edits: 10% increase (over time, according to existing edit tag definition. We'll tag edits from the new interface -T212959 - and see if they reach 1/11th of mobile web edits from those “100+” editors)
  • Rates of access to special pages overall (i.e. pageviews, repeating the query from T198218, relying on new X-Analytics tag for AMC, analogous to the mf-m key or as a new value for it: T212961 )
  • Rates of access and edits to article talk pages (relying on AMC X-Analytics tag and AMC edit tag)
  • (Feature) retention rate overall (for users with less than 100+ edits, via instrumenting the opt-in button - done in T211197 using the PrefUpdate EventLogging schema, related spike: T211195). Retention period TBD
  • Rate of moderation actions on mobile web on target wikis: 10% increase (moderation actions having been defined in T213461), determined via AMC edit tag as for edits (above)
  • Rate of switching from desktop to mobile? (T208457)
  • (future iteration) Effectiveness of call to action methods

See also the sections about metrics in the December 2018 epics notes, November 2018 notes, the project page, and the annual plan.

Target wikis

While we will be reaching out and are open to feedback from all projects, we have created a list of projects that we would like to specifically focus on. We looked at wikis that already had a decent amount of editors on mobile, indicating that editing on mobile is a comfortable concept for users of these projects. As the target audience for this project is medium and high-volume contributors, our assumption was that projects with a decently large number (1000+) of monthly editors would be likely to have a higher percentage of editors with 100+ edits. Finally, in terms of priority, we wanted to focus on projects that had the largest gap in mobile editing - projects whose ratio of mobile edits to new active editors was the highest:

English Wikipedia
Spanish Wikipedia
Japanese Wikipedia
Chinese Wikipedia
Portuguese Wikipedia
Italian Wikipedia
Persian Wikipedia
Hebrew Wikipedia
Arabic Wikipedia
Finnish Wikipedia
Vietnamese Wikipedia
Indonesian Wikipedia
English Wiktionary
Thai Wikipedia
Hindi Wikipedia

Progressive Rollouts

We will rollout progressively to the following wikis (selected as wikis with greatest % of mobile edits):

  • Spanish Wikipedia
  • Arabic Wikipedia
  • Indonesian Wikipedia

We will consult with the community for confirmation prior to deployments

Related Objects

Event Timeline

ovasileva triaged this task as Normal priority.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 28 2018, 8:35 PM
Tbayer updated the task description. (Show Details)Dec 5 2018, 6:22 PM
Tbayer updated the task description. (Show Details)Dec 14 2018, 1:07 AM
Tbayer updated the task description. (Show Details)Jan 8 2019, 1:30 AM
Tbayer added a comment.Jan 8 2019, 1:39 AM

We have been creating different versions of this list in at least six different places now (see the four links in the task description and most recently T212960). This is confusing and can lead to oversights - e.g. I just noticed that the number of moderation actions (one of the three core metrics for this project set forth in the annual plan, and required to answer the first question in this task) was not yet listed here, which I assume is unintentional.

How about we regard this task as the reference for the time being? Or is there another "one place of truth" we should use?

Tbayer updated the task description. (Show Details)Jan 8 2019, 4:28 AM
Tbayer updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)Jan 8 2019, 9:39 PM
ovasileva updated the task description. (Show Details)Jan 9 2019, 4:18 PM
Tbayer updated the task description. (Show Details)Jan 10 2019, 9:51 PM
ovasileva updated the task description. (Show Details)Jan 11 2019, 11:05 PM
Restricted Application added a subscriber: Cosine02. · View Herald TranscriptJan 11 2019, 11:05 PM
Tbayer updated the task description. (Show Details)Jan 11 2019, 11:13 PM
Tbayer updated the task description. (Show Details)Mon, Feb 25, 10:23 PM
pmiazga added a subscriber: pmiazga.EditedMon, Mar 11, 6:32 PM
Actioncreates Recent Change objectcodebase
undo/revertYEScore
rollbackYEScore
blocking (Special:Blog, tag should show up in the block log)YEScore
unblocking (Special:Unblock, tag should show up in the block log)YEScore
deleting&undelete (tag should show up in the deletion log)YEScore
protecting pages (tag should show up in the protection log)YEScore
moving pages (Special:MovePage, tag should show up in the move log)YEScore
reviewing new pages (cf. Special:NewPagesFeed)YESPageTriage
thanksNOThanks
pending changes (cf. log on enwiki)sometimes (move, updateStability)FlaggedRevisions

Currently we support all log entries that create the RecentChange object.