Page MenuHomePhabricator

Add X-Analytics tag for AMC webrequests
Closed, ResolvedPublic3 Story Points

Description

Background

In principle, the view-based metrics planned for AMC (see T210660: [EPIC] AMC Metrics ) can be determined from the data that's collected in the wmf.webrequest Hive table provided that we can distinguish between regular users and those users who've enabled AMC.

We already achieve the same for users who have enabled Wikipedia's mobile beta mode by adding a well-known key-value pair to the X-Analytics header: https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/blob/3231dee54e6abc08080627f64157848542dd323d/includes/MobileContext.php#L1022-L1034

QA

  1. Login and visit https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=Special:MobileOptions&returnto=Dog
  2. Check the X-Analytics header in DevTools for index.php. Note: this header may not show up when using DevTools' search feature!

  1. Try the four different modes: AMC on / beta on, AMC off / beta on, AMC on / beta off, AMC off / beta off.
  2. Verify each mode matches the documented output.

Acceptance Criteria

  • Assess privacy implications
  • Add key-value pair to X-Analytics header when the user has enabled AMC
  • Document the key-value pair

Notes

  1. MobileFrontend's server-side manipulation of the X-Analytics header is already under test (see MobileContext::addAnalyticsLogItem).
  2. The value of the X-Analytics response header is logged and processed by WMF's analytics pipeline automatically, i.e. if a key-value pair is added to the header, it'll be reflected in the wmf.webrequest Hive table once it's been processed.
  3. MobileContext::logMobileMode exists and seems like the appropriate place to add this change.
  4. As @Tbayer notes in T212961#4861533, it might be easiest to add a new value to the mf-m X-Analytics key, e.g. when the user has enabled both the mobile beta mode and AMC, we set the value to mf-m=b,amc (the other possible values being =b, =amc).

QA Results

StatusDetails
✅ PassedT212961#5020949

QA Results:Production (arwiki)

StatusDetails
✅ PassedT212961#5064944

Event Timeline

ovasileva created this task.Jan 4 2019, 6:30 PM
ovasileva triaged this task as Normal priority.
Tbayer renamed this task from Allow AMC to appear in hive to Add X-Analytics tag for AMC webrequests.Jan 8 2019, 4:21 AM
Tbayer updated the task description. (Show Details)
Tbayer changed the edit policy from "Custom Policy" to "All Users".

Did we already decide how AMC corresponds to Mobile Beta? (I.e. does AMC always imply - or exclude - Mobile Beta?) In that case it might be worth considering to add a new value to the existing mf-m key instead of adding a new key. @phuedx, thoughts ?

phuedx updated the task description. (Show Details)Jan 8 2019, 11:41 AM
phuedx updated the task description. (Show Details)Jan 8 2019, 11:43 AM
phuedx updated the task description. (Show Details)

Do note that this task focusses on making the change in our application code, which is made possible by restricting AMC to logged-in users (who are always served by the application servers). It may be prudent to home all of our X-Analytics-related logic in the Varnish VCL's themselves.

Jdlrobson added a subscriber: Jdlrobson.

Can you take a look at this task when you have a spare moment and check that you wouldn't have any questions prior to implementing this?

ovasileva raised the priority of this task from Normal to High.Jan 15 2019, 4:29 PM

Looks clear to me. It's ready to work on.

pmiazga removed pmiazga as the assignee of this task.
pmiazga added a subscriber: pmiazga.
phuedx added a comment.EditedJan 16 2019, 12:36 PM

Edit

This is currently blocked on a privacy review.

@Tbayer: Should we contact Legal for this?

phuedx assigned this task to ovasileva.

Olga's currently drafting the request.

@ovasileva has sent the request. We're waiting for a response.

For the record, to track progress: We received followup questions from Legal on January 24, and replied on the same day. (Last week was taken up by the WMF All Hands meeting.)

Update: still waiting on a reply from Legal. Sent them a note to check in on the status Feb 11.

Tbayer moved this task from Triage to Tracking on the Product-Analytics board.

The task has now been reviewed for privacy by the Legal team. No concerns were raised. Moving to to-do.

ovasileva removed ovasileva as the assignee of this task.Feb 15 2019, 8:39 AM

Change 492924 had a related patch set uploaded (by Niedzielski; owner: Stephen Niedzielski):
[mediawiki/extensions/MobileFrontend@master] WIP: Analytics: add AMC tag to X-Analytics header

https://gerrit.wikimedia.org/r/492924

I believe this is ready for review. ^Extra thanks to those who contributed to technical details on this task. It was useful!

pmiazga claimed this task.Mar 6 2019, 11:34 AM
pmiazga added a comment.EditedMar 7 2019, 6:11 PM

The X-Analitycs header when AMC and Beta modes are on:
X-Analytics: mf-m=b%2Camc;ns=-1;special=MobileOptions;loggedIn=1

Note: We urlencode the ,character.

/cc @Tbayer

Change 492924 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Analytics: add AMC tag to X-Analytics header

https://gerrit.wikimedia.org/r/492924

Niedzielski updated the task description. (Show Details)
Edtadros reassigned this task from Edtadros to ovasileva.Mar 13 2019, 2:17 PM
Edtadros added a subscriber: Edtadros.

Test Result

Status: ✅ PASS
OS: macOS Mojave
Browser: Chrome DevTools Device Emulator (iPhone X)

Test Artifact(s):

  • AMC on / beta on
  • AMC off / beta on
  • AMC on / beta off
  • AMC off / beta off

We still need to check at some point that this is correctly processed on Varnish and stored in the webrequest table (where we need it to conduct the analysis). But that will have to wait until the change is in production, because the domain en.m.wikipedia.beta.wmflabs.org is not captured in webrequest. Probably best to create a separate task for that, and close this one?

For later reference, here is a query for checking, which should yield amc values:

SELECT DISTINCT x_analytics_map['mf-m'] AS values FROM wmf.webrequest WHERE year = 2019 AND month = 3 AND day = 12 AND hour = 0;

OK
values
NULL
b
Time taken: 26.1 seconds, Fetched: 2 row(s)
ovasileva updated the task description. (Show Details)Mar 14 2019, 10:50 AM
ovasileva closed this task as Resolved.

We still need to check at some point that this is correctly processed on Varnish and stored in the webrequest table (where we need it to conduct the analysis). But that will have to wait until the change is in production, because the domain en.m.wikipedia.beta.wmflabs.org is not captured in webrequest. Probably best to create a separate task for that, and close this one?

For later reference, here is a query for checking, which should yield amc values:

SELECT DISTINCT x_analytics_map['mf-m'] AS values FROM wmf.webrequest WHERE year = 2019 AND month = 3 AND day = 12 AND hour = 0;

OK
values
NULL
b
Time taken: 26.1 seconds, Fetched: 2 row(s)

Resolving this. Created followup task for testing in production: T218286

phuedx removed a subscriber: phuedx.Mar 14 2019, 11:08 AM

Test Result:Production

Status: ✅ PASS
OS: macOS Mojave
Browser: Chrome DevTools Device Emulator (iPhone X)

Test Artifact(s):

  • AMC on / beta on
  • AMC off / beta on
  • AMC on / beta off
  • AMC off / beta off
Edtadros updated the task description. (Show Details)Mar 28 2019, 8:49 AM