Page MenuHomePhabricator

Add X-Analytics tag for AMC webrequests
Closed, ResolvedPublic3 Estimated Story Points

Assigned To
Authored By
ovasileva
Jan 4 2019, 6:30 PM
Referenced Files
F28493243: Screen Shot 2019-03-28 at 1.29.36 AM.png
Mar 28 2019, 8:45 AM
F28493244: Screen Shot 2019-03-28 at 1.31.12 AM.png
Mar 28 2019, 8:45 AM
F28493245: Screen Shot 2019-03-28 at 1.32.08 AM.png
Mar 28 2019, 8:45 AM
F28493246: Screen Shot 2019-03-28 at 1.30.14 AM.png
Mar 28 2019, 8:45 AM
F28380984: Screen Shot 2019-03-13 at 7.06.32 AM.png
Mar 13 2019, 2:17 PM
F28381002: Screen Shot 2019-03-13 at 7.12.02 AM.png
Mar 13 2019, 2:17 PM
F28380989: Screen Shot 2019-03-13 at 7.08.38 AM.png
Mar 13 2019, 2:17 PM
F28380997: Screen Shot 2019-03-13 at 7.10.39 AM.png
Mar 13 2019, 2:17 PM

Description

Background

In principle, the view-based metrics planned for AMC (see T210660: [EPIC] AMC Metrics ) can be determined from the data that's collected in the [[https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Webrequest | wmf.webrequest Hive table ]] provided that we can distinguish between regular users and those users who've enabled AMC.

We already achieve the same for users who have enabled Wikipedia's mobile beta mode by adding a well-known key-value pair to the X-Analytics header: https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/blob/3231dee54e6abc08080627f64157848542dd323d/includes/MobileContext.php#L1022-L1034

QA

  1. Login and visit https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=Special:MobileOptions&returnto=Dog
  2. Check the X-Analytics header in DevTools for index.php. Note: this header may not show up when using DevTools' search feature!

Screenshot from 2019-03-11 12-06-40.png (2×3 px, 1 MB)

  1. Try the four different modes: AMC on / beta on, AMC off / beta on, AMC on / beta off, AMC off / beta off.
  2. Verify each mode matches the documented output.

Acceptance Criteria

  • Assess privacy implications
  • Add key-value pair to X-Analytics header when the user has enabled AMC
  • Document the key-value pair

Notes

  1. MobileFrontend's server-side manipulation of the X-Analytics header is already under test (see MobileContext::addAnalyticsLogItem).
  2. The value of the X-Analytics response header is logged and processed by WMF's analytics pipeline automatically, i.e. if a key-value pair is added to the header, it'll be reflected in the wmf.webrequest Hive table once it's been processed.
  3. MobileContext::logMobileMode exists and seems like the appropriate place to add this change.
  4. As @Tbayer notes in T212961#4861533, it might be easiest to add a new value to the mf-m X-Analytics key, e.g. when the user has enabled both the mobile beta mode and AMC, we set the value to mf-m=b,amc (the other possible values being =b, =amc).

QA Results

StatusDetails
✅ PassedT212961#5020949

QA Results:Production (arwiki)

StatusDetails
✅ PassedT212961#5064944

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
ovasileva triaged this task as Medium priority.Jan 4 2019, 6:30 PM
Tbayer renamed this task from Allow AMC to appear in hive to Add X-Analytics tag for AMC webrequests.Jan 8 2019, 4:21 AM
Tbayer updated the task description. (Show Details)
Tbayer changed the edit policy from "Custom Policy" to "All Users".

Did we already decide how AMC corresponds to Mobile Beta? (I.e. does AMC always imply - or exclude - Mobile Beta?) In that case it might be worth considering to add a new value to the existing mf-m key instead of adding a new key. @phuedx, thoughts ?

phuedx updated the task description. (Show Details)

Do note that this task focusses on making the change in our application code, which is made possible by restricting AMC to logged-in users (who are always served by the application servers). It may be prudent to home all of our X-Analytics-related logic in the Varnish VCL's themselves.

Jdlrobson subscribed.

Can you take a look at this task when you have a spare moment and check that you wouldn't have any questions prior to implementing this?

ovasileva raised the priority of this task from Medium to High.Jan 15 2019, 4:29 PM

Looks clear to me. It's ready to work on.

pmiazga moved this task from Needs Prioritization to Upcoming on the Web-Team-Backlog board.
pmiazga subscribed.

Edit

This is currently blocked on a privacy review.

@Tbayer: Should we contact Legal for this?

For the record, to track progress: We received followup questions from Legal on January 24, and replied on the same day. (Last week was taken up by the WMF All Hands meeting.)

Update: still waiting on a reply from Legal. Sent them a note to check in on the status Feb 11.

The task has now been reviewed for privacy by the Legal team. No concerns were raised. Moving to to-do.

Change 492924 had a related patch set uploaded (by Niedzielski; owner: Stephen Niedzielski):
[mediawiki/extensions/MobileFrontend@master] WIP: Analytics: add AMC tag to X-Analytics header

https://gerrit.wikimedia.org/r/492924

I believe this is ready for review. ^Extra thanks to those who contributed to technical details on this task. It was useful!

The X-Analitycs header when AMC and Beta modes are on:
X-Analytics: mf-m=b%2Camc;ns=-1;special=MobileOptions;loggedIn=1

Note: We urlencode the ,character.

/cc @Tbayer

Change 492924 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Analytics: add AMC tag to X-Analytics header

https://gerrit.wikimedia.org/r/492924

Edtadros subscribed.

Test Result

Status: ✅ PASS
OS: macOS Mojave
Browser: Chrome DevTools Device Emulator (iPhone X)

Test Artifact(s):

  • AMC on / beta on
    Screen Shot 2019-03-13 at 7.06.32 AM.png (1×3 px, 942 KB)
  • AMC off / beta on
    Screen Shot 2019-03-13 at 7.08.38 AM.png (1×3 px, 936 KB)
  • AMC on / beta off
    Screen Shot 2019-03-13 at 7.10.39 AM.png (1×3 px, 933 KB)
  • AMC off / beta off
    Screen Shot 2019-03-13 at 7.12.02 AM.png (1×3 px, 933 KB)

We still need to check at some point that this is correctly processed on Varnish and stored in the webrequest table (where we need it to conduct the analysis). But that will have to wait until the change is in production, because the domain en.m.wikipedia.beta.wmflabs.org is not captured in webrequest. Probably best to create a separate task for that, and close this one?

For later reference, here is a query for checking, which should yield amc values:

SELECT DISTINCT x_analytics_map['mf-m'] AS values FROM wmf.webrequest WHERE year = 2019 AND month = 3 AND day = 12 AND hour = 0;

OK
values
NULL
b
Time taken: 26.1 seconds, Fetched: 2 row(s)

We still need to check at some point that this is correctly processed on Varnish and stored in the webrequest table (where we need it to conduct the analysis). But that will have to wait until the change is in production, because the domain en.m.wikipedia.beta.wmflabs.org is not captured in webrequest. Probably best to create a separate task for that, and close this one?

For later reference, here is a query for checking, which should yield amc values:

SELECT DISTINCT x_analytics_map['mf-m'] AS values FROM wmf.webrequest WHERE year = 2019 AND month = 3 AND day = 12 AND hour = 0;

OK
values
NULL
b
Time taken: 26.1 seconds, Fetched: 2 row(s)

Resolving this. Created followup task for testing in production: T218286

Test Result:Production

Status: ✅ PASS
OS: macOS Mojave
Browser: Chrome DevTools Device Emulator (iPhone X)

Test Artifact(s):

  • AMC on / beta on
    Screen Shot 2019-03-28 at 1.29.36 AM.png (2×3 px, 1 MB)
  • AMC off / beta on
    Screen Shot 2019-03-28 at 1.30.14 AM.png (2×3 px, 1 MB)
  • AMC on / beta off
    Screen Shot 2019-03-28 at 1.31.12 AM.png (2×3 px, 1 MB)
  • AMC off / beta off
    Screen Shot 2019-03-28 at 1.32.08 AM.png (2×3 px, 1 MB)