Page MenuHomePhabricator

Measure % of edits and pvs coming from users without JS
Closed, ResolvedPublic

Description

As part of an ongoing discussion of support levels for different browser capabilities (https://www.mediawiki.org/wiki/Compatibility#Browsers), we want to take a snapshot of current usage by people who, for whatever reason, are using our projects without JS-support turned on.

Here are the metrics desired, in order of priority:

  • % of user edits done with no JS support
  • % of anon user edits done with no JS support
  • % pageviews with no JS support

2nd tier, but still very useful

  • % of the above by country
  • % of the above by project

Here is a similar ticket for measuring and reporting this on an ongoing basis. For now, we probably just need a snapshot. This might be accomplished via a no-script pixel or something like that.

Event Timeline

JKatzWMF created this task.Oct 4 2019, 9:29 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 4 2019, 9:29 PM
JKatzWMF updated the task description. (Show Details)Oct 4 2019, 9:30 PM
LGoto triaged this task as Medium priority.Oct 7 2019, 4:53 PM
LGoto moved this task from Triage to Next Up on the Product-Analytics board.
kzimmerman added a comment.EditedOct 7 2019, 4:54 PM

@JKatzWMF @ovasileva We have a lot on our plate at the moment. What is the date by which you need this data? How critical is it to decision making?

kzimmerman moved this task from Next Up to Backlog on the Product-Analytics board.Oct 7 2019, 11:08 PM

@kzimmerman - Since JK is out for a while, I'll chime in here. This data is basically needed for any future editing-related features, as we need to decide whether or not to continue building no-JS fallbacks for those features. For example, the Editing team is currently working on Discussion Tools, which is a group of Javascript editing features for talk pages. Since we don't have a good idea of how much editing is done on no-JS browsers, we don't really know what the impact will be of not providing a no-JS fallback and whether that may impact some communities more than others.

The real thing that is driving this push, however, is our desire to finally publish some comprehensive guidelines about no-JS support across all our features to inform Engineering teams in their future work (especially around projects like Desktop Refresh where no-JS support is a big cost). These guidelines were written up way back in September 2019, but have never been finalized or published mainly because we are stalled on the editing question: Should editing be a feature we continue to provide no-JS support for? In the meantime, Engineering teams are continuing to make these discussions on an ad-hoc basis, often wasting lots of time and resources in the process.

So basically, there is no specific deadline, but this data will have a very large impact on decision making across Product, if for no other reason than it will unblock our publication of the no-JS guidelines.

ovasileva raised the priority of this task from Medium to High.Apr 14 2020, 7:11 PM
Mayakp.wiki moved this task from Backlog to Triage on the Product-Analytics board.Apr 14 2020, 7:13 PM

@kaldari Maya and Olga chatted about this and have raised the priority to high; we'll review it in our team's next triage meeting (scheduled for Monday, April 20) and determine a timeline for tackling this at that time.

kzimmerman moved this task from Triage to Epics on the Product-Analytics board.
kzimmerman closed this task as Resolved.Sep 21 2020, 4:40 PM
kzimmerman claimed this task.

Resolved in T240697 and T253033

To summarize the outcome of those two tickets:

  • T240697: We cannot accurately determine the percentage of edits made without JS due to tracking blockers. T263505 would be the next step forward on this.
  • T253033: We did not measure percent of pageviews without JS, but instead used a proxy measurement of percent of pageviews from grade A browsers vs. lower grade browsers:
access_methodcategorypageviewspercent of pageviews
desktopGrade A127550021886.16%
desktopGrade C or X20495864013.84%
mobile webGrade A151942679363.52%
mobile webGrade C or X87273439736.48%

It would be good to break out Grade C and Grade X since they are treated very differently.