Page MenuHomePhabricator

Feature Policy Reporting origin trial
Closed, ResolvedPublic

Event Timeline

While the Feature policies have been implemented in Chrome some time ago, and the reporting feature added as well, the "report only" mode hasn't been implemented yet:

Capture d'écran 2018-11-19 13.36.53.png (148×1 px, 52 KB)

Tests have been added a few days ago: https://chromium.googlesource.com/chromium/src/+/f482e77e59ef0a506e745182ca6843080cd8d1ea but as far as I can tell in Canary, the feature doesn't work yet.

Gilles triaged this task as Medium priority.
Gilles changed the task status from Open to Stalled.Nov 19 2018, 12:48 PM
Gilles changed the task status from Stalled to Open.Feb 19 2019, 9:46 PM
Gilles updated the task description. (Show Details)
Gilles moved this task from Blocked (old) to Doing (old) on the Performance-Team board.
Gilles renamed this task from Use Feature Policies in Report-Only mode to detect bad practices to Feature Policy Reporting origin trial.Feb 20 2019, 8:42 AM

Change 494959 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@master] [WIP] Reporting API and Feature Policy reporting support

https://gerrit.wikimedia.org/r/494959

Tried it with sync-xhr, couldn't get it to work. Left a comment on the Chromium bug: https://bugs.chromium.org/p/chromium/issues/detail?id=867471&can=2&start=0&num=100&q=&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified&groupby=&sort=

This is why the patch is marked as WIP for now, maybe I got something wrong with the syntax?

@JBennett FYI, we're experimenting with two new browser features.

  1. "Origin Trials" is a meta feature of Google Chrome, it's how they ship experimental features to a select audience (opt-in) with an expiry for when it turns itself off again.
  2. "Feature Policy", is meta feature being worked on by W3C webappsec, it revolves around a header (kind of like CSP) which allows sites to disable JavaScript APIs normally exposed by the browser (examples).

The Feature Policy API itself is experimental (W3C draft), and shipped in Chrome stable as an origin trial.

Change 494959 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@master] Reporting API and Feature Policy reporting support

https://gerrit.wikimedia.org/r/494959

Change 499709 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/vagrant@master] Enable various origin trials

https://gerrit.wikimedia.org/r/499709

Change 499710 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/vagrant@master] Update origin trials tokens

https://gerrit.wikimedia.org/r/499710

Change 499709 merged by jenkins-bot:
[mediawiki/vagrant@master] Enable various origin trials

https://gerrit.wikimedia.org/r/499709

Change 499710 merged by jenkins-bot:
[mediawiki/vagrant@master] Update origin trials tokens

https://gerrit.wikimedia.org/r/499710

Change 494959 merged by jenkins-bot:
[mediawiki/core@master] Reporting API and Feature Policy reporting support

https://gerrit.wikimedia.org/r/494959

Change 507555 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/mediawiki-config@master] Enable Feature Policy Reporting origin trial

https://gerrit.wikimedia.org/r/507555

Change 507555 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable Feature Policy Reporting origin trial

https://gerrit.wikimedia.org/r/507555

Change 507559 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/mediawiki-config@master] Fix syntax of wgFeaturePolicyReportOnly fields

https://gerrit.wikimedia.org/r/507559

Change 507559 merged by jenkins-bot:
[operations/mediawiki-config@master] Fix syntax of wgFeaturePolicyReportOnly fields

https://gerrit.wikimedia.org/r/507559

Change 507563 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/mediawiki-config@master] Remove unsupported wgFeaturePolicyReportOnly types

https://gerrit.wikimedia.org/r/507563

Change 507563 merged by jenkins-bot:
[operations/mediawiki-config@master] Remove unsupported wgFeaturePolicyReportOnly types

https://gerrit.wikimedia.org/r/507563

Mentioned in SAL (#wikimedia-operations) [2019-05-01T13:31:45Z] <gilles@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T209572 Enable Feature Policy Reporting origin trial (duration: 01m 01s)

The API is definitely working, we're getting reports of sync XHRs. Unfortunately since the reports are sent as POSTS, varnishlog alone can't let us inspect the contents, as it's unable to look at POST request bodies (because those aren't cacheable, by definition). In order to see what we're actually getting, we need to set up a Varnish backend for this.

Hah, actually a workaround is to set up a ReportingObserver, and then we could ship them with EventLogging. Obviously this only works for reports like sync-xhr where the page actually works. But this allows us to look at sync-xhr without having to set up the whole infrastructure required to capture all report types via the reporting endpoint.

We might even be able to disable the endpoint for now, I'll check if the observer gets the report without the report-to header being set.

Change 507571 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/mediawiki-config@master] Disable Reporting API endpoint

https://gerrit.wikimedia.org/r/507571

Change 507571 merged by jenkins-bot:
[operations/mediawiki-config@master] Disable Reporting API endpoint

https://gerrit.wikimedia.org/r/507571

Indeed, the observer can work without the reporting endpoint. This will be sufficient for this origin trial. And we have a pretty good idea now of what kind of pipeline we'll need when we want to collect actual Reporting API reports.

Mentioned in SAL (#wikimedia-operations) [2019-05-01T13:57:50Z] <gilles@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T209572 Disable Reporting API endpoint (duration: 00m 59s)

Change 507584 had a related patch set uploaded (by Krinkle; owner: Gilles):
[mediawiki/extensions/NavigationTiming@master] Observe and report feature policy violations

https://gerrit.wikimedia.org/r/507584

Change 509630 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/vagrant@master] Renew origin trial tokens

https://gerrit.wikimedia.org/r/509630

Change 509630 merged by jenkins-bot:
[mediawiki/vagrant@master] Renew origin trial tokens

https://gerrit.wikimedia.org/r/509630

Gilles raised the priority of this task from Medium to High.May 27 2019, 5:04 AM

Change 507584 merged by jenkins-bot:
[mediawiki/extensions/NavigationTiming@master] Observe and report feature policy violations

https://gerrit.wikimedia.org/r/507584

We should start getting reports onces eswiki and ruwiki have moved onto wmf.10 tomorrow

We have sync-xhr reports, as expected. For the most part (%) they come from Chrome extensions.

This one comes up a lot: https://chrome.google.com/webstore/detail/s3translator/debnnjfbneojbmioajinefnflopdohjk?hl=en

I'm also seeing some weird source files hosted on 3rd-parties: https://sfops.ru/xxxoxx/23f84e9c86281c112ece8d87854dd435/ru.wikipedia.org?ref=ru.wikipedia.org&adb=false&ref_path=%252Fwiki%252F%2525D0%25259F%2525D0%2525B0%2525D1%252580%2525D0%2525BB%2525D0%2525B0%2525D0%2525BC%2525D0%2525B5%2525D0%2525BD%2525D1%252582%2525D1%252581%2525D0%2525BA%2525D0%2525B8%2525D0%2525B5_%2525D0%2525B2%2525D1%25258B%2525D0%2525B1%2525D0%2525BE%2525D1%252580%2525D1%25258B_%2525D0%2525BD%2525D0%2525B0_%2525D0%2525A3%2525D0%2525BA%2525D1%252580%2525D0%2525B0%2525D0%2525B8%2525D0%2525BD%2525D0%2525B5_(2019)&r=1563812951317

https://aldamva.ru/xxxoxx/25d02a4c13429dfd313ce11b0d90b47e/ru.wikipedia.org?ref=ru.wikipedia.org&adb=false&ref_path=%252Fwiki%252F%2525D0%252591%2525D1%25258B%2525D1%252581%2525D1%252582%2525D1%252580%2525D0%2525B8%2525D1%252586%2525D0%2525BA%2525D0%2525B0%2525D1%25258F%252C_%2525D0%2525AD%2525D0%2525BB%2525D0%2525B8%2525D0%2525BD%2525D0%2525B0_%2525D0%252590%2525D0%2525B2%2525D1%252580%2525D0%2525B0%2525D0%2525B0%2525D0%2525BC%2525D0%2525BE%2525D0%2525B2%2525D0%2525BD%2525D0%2525B0&r=1561915117425

That second one gets flagged as a phishing site by cloudflare.

Looks like code to inject ads. I wonder if it's also coming from a nasty Chrome extension that injects ads into the page. I'm seeing a lot of Chrome extension whose hash isn't found on the Chrome web store, so that really smells like virus/adware.

Some Kaspersky stuff: https://gc.kis.scr.kaspersky-labs.com/1B74BD89-2A22-4B93-B451-1C9E1052A0EC/main.js

This might come from an extension as well.

A user script shows up: https://ru.wikipedia.org/w/index.php?title=User:%D0%9C%D0%B0%D1%81%D1%82%D0%B5%D1%80_%D1%82%D0%B5%D0%BD%D0%B5%D0%B9/%D0%9F%D0%BE%D0%BC%D0%BE%D1%89%D0%BD%D0%B8%D0%BA_%D0%BF%D0%B0%D1%82%D1%80%D1%83%D0%BB%D1%8F.js&action=raw&ctype=text/javascript nothing surprising there.

Finally, a lot of the entries don't have a "sourceFile" value, suggesting that it came from JS on the page. I'll look at those next to see if there's any pattern.

Looking at entries without sourceFile attribution, I notice that there are some patterns in terms of line/column number. Here are the most common:

linenumber columnnumber count
1 72 486
1 100 179
2 95 105
3 72 48
11 95 56
12 95 50
28 13 854
122 95 14

When opening the articles that these reports are coming from, I'm not seeing any sync xhr on pageload.

Looking at the most common one (28/13), it seems like it tends to have multiple sync xhrs per pageload, as I'm seeing only 227 distinct pageviewToken values for it.

Given the prominence of what looks like adware in reports that do get attribution, I'm going to guess that it's more of the same.

I haven't been able to find any occurrence of a sync xhr that seemed to come from our code. Reports are coming from anonymous users, which makes it unlikely that the xhr is a result of page interaction.

This will be a useful API when released, particularly when it starts supporting other types of reports. However, it's likely that the bulk of the reports will be coming from browser extensions, adware, etc.

CSP is moving to using the reports-to header. At some point it might make sense to make our CSP reporting use this style of report-to with the varnishkafka beacon instead of the current solution involving the api module.

Change 908018 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/extensions/NavigationTiming@master] Remove FeaturePolicyViolation instrumentation

https://gerrit.wikimedia.org/r/908018

Change 908018 merged by jenkins-bot:

[mediawiki/extensions/NavigationTiming@master] Remove FeaturePolicyViolation instrumentation

https://gerrit.wikimedia.org/r/908018

Change 908382 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/puppet@production] eventlogging: Remove unused FeaturePolicyViolation schema

https://gerrit.wikimedia.org/r/908382

Change 908382 merged by RLazarus:

[operations/puppet@production] eventlogging: Remove obsolete FeaturePolicyViolation schema

https://gerrit.wikimedia.org/r/908382

Change 976303 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/puppet@production] refine - Put SpecialMuteSubmit and FeaturePolicyViolation in eventlogging analytics exclude list

https://gerrit.wikimedia.org/r/976303

Change 976303 merged by Ottomata:

[operations/puppet@production] refine - Put SpecialMuteSubmit and FeaturePolicyViolation in exclude list

https://gerrit.wikimedia.org/r/976303