Page MenuHomePhabricator

[EPIC] AMC Metrics
Closed, ResolvedPublic

Description

NOTE: The following do not include metrics for individual components of the project - these will be reflected in the tasks themselves

Questions we want to answer

  • Does advanced mode help advanced users be more effective on mobile? (proxied by moderation action count, edit rate)
  • Does advanced mode reduce the necessity of switching to desktop mode?
  • Does advanced mode increase traffic and edits to the talk page of an article?

Metrics

(see also data dictionary and list of target wikis)

  • [annual plan target] Mobile web edit rate on target wikis :
    • looking for 10% increase (We'll tag edits from the new interface - T212959 - and see if they reach 1/11th of total mobile web edits, defined using the existing edit tag.)
  • Mobile web edit rate on target wikis limited to users with 100+ edits:
    • 10% increase (We'll tag edits from the new interface - T212959 - and see if they reach 1/11th of mobile web edits - defined using the existing edit tag - from those “100+” editors.)
  • Rates of access to special pages overall
    • i.e. pageviews, repeating the query from T198218, relying on new X-Analytics tag for AMC, analogous to the mf-m key or as a new value for it: T212961
  • Rates of access and edits to article talk pages
  • [annual plan target] Feature retention rate overall for users with 100+ edits
    • via instrumenting the opt-in button - done in T211197 using the PrefUpdate EventLogging schema, related spike: T211195
    • Retention period TBD.
    • Target: at least 60% retention
  • [annual plan target] Rate of moderation actions on mobile web on target wikis
    • Target: 10% increase
    • moderation actions are defined in T213461
    • measured using the AMC edit tag as for edits (above)
  • (future iteration) Effectiveness of call to action methods

See also the sections about metrics in the December 2018 epics notes, November 2018 notes, the project page, and the annual plan.

Target wikis

While we will be reaching out and are open to feedback from all projects, we have created a list of projects that we would like to specifically focus on. We looked at wikis that already had a decent amount of editors on mobile, indicating that editing on mobile is a comfortable concept for users of these projects. As the target audience for this project is medium and high-volume contributors, our assumption was that projects with a decently large number (1000+) of monthly editors would be likely to have a higher percentage of editors with 100+ edits. Finally, in terms of priority, we wanted to focus on projects that had the largest gap in mobile editing - projects whose ratio of mobile edits to new active editors was the highest.

See here for the resulting list of target wikis.

Progressive Rollouts

In T217643, we will rollout progressively to the following wikis (selected as wikis with greatest % of mobile edits):

  • Spanish Wikipedia
  • Arabic Wikipedia
  • Indonesian Wikipedia

We will consult with the community for confirmation prior to deployments.

Related Objects

Event Timeline

ovasileva triaged this task as Medium priority.Nov 28 2018, 8:35 PM
ovasileva created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 28 2018, 8:35 PM
Tbayer updated the task description. (Show Details)Dec 5 2018, 6:22 PM
Tbayer updated the task description. (Show Details)Dec 14 2018, 1:07 AM
Tbayer updated the task description. (Show Details)Jan 8 2019, 1:30 AM

We have been creating different versions of this list in at least six different places now (see the four links in the task description and most recently T212960). This is confusing and can lead to oversights - e.g. I just noticed that the number of moderation actions (one of the three core metrics for this project set forth in the annual plan, and required to answer the first question in this task) was not yet listed here, which I assume is unintentional.

How about we regard this task as the reference for the time being? Or is there another "one place of truth" we should use?

Tbayer updated the task description. (Show Details)Jan 8 2019, 4:28 AM
Tbayer updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)Jan 8 2019, 9:39 PM
ovasileva updated the task description. (Show Details)Jan 9 2019, 4:18 PM
Tbayer updated the task description. (Show Details)Jan 10 2019, 9:51 PM
ovasileva updated the task description. (Show Details)Jan 11 2019, 11:05 PM
Restricted Application added a subscriber: Cosine02. · View Herald TranscriptJan 11 2019, 11:05 PM
Tbayer updated the task description. (Show Details)Jan 11 2019, 11:13 PM
Tbayer updated the task description. (Show Details)Feb 25 2019, 10:23 PM
pmiazga added a subscriber: pmiazga.EditedMar 11 2019, 6:32 PM
Actioncreates Recent Change objectcodebase
undo/revertYEScore
rollbackYEScore
blocking (Special:Blog, tag should show up in the block log)YEScore
unblocking (Special:Unblock, tag should show up in the block log)YEScore
deleting&undelete (tag should show up in the deletion log)YEScore
protecting pages (tag should show up in the protection log)YEScore
moving pages (Special:MovePage, tag should show up in the move log)YEScore
reviewing new pages (cf. Special:NewPagesFeed)YESPageTriage
thanksNOThanks
pending changes (cf. log on enwiki)sometimes (move, updateStability)FlaggedRevisions

Currently we support all log entries that create the RecentChange object.

Tbayer updated the task description. (Show Details)Apr 3 2019, 9:47 PM
Tbayer changed the edit policy from "Custom Policy" to "All Users".
Tbayer updated the task description. (Show Details)Apr 3 2019, 10:40 PM
Tbayer updated the task description. (Show Details)Apr 3 2019, 11:07 PM
ovasileva closed this task as Resolved.Apr 16 2019, 10:42 AM
ovasileva claimed this task.

Looks like we're all done here! Some QA remaining that is tracked elsewhere.

Shizhao moved this task from Backlog to Closed on the Chinese-Sites board.Apr 17 2019, 3:02 AM