In the Android app, add x-analytics request header with value pageview=1 for both RESTBase and MediaWiki php.api endpoints.
(Might as well add the missing preview=1 values to the link preview requests.)
In the analytics-refinery-source the PageviewDefinition for is isAppPageview() should be updated to use the pageview=1 value.
It would be great to get this completed and rolled out to the Android production app before we rollout RESTBase usage to the Android production app.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Dbrant | T115484 Display definitions of highlighted words from Wiktionary | |||
| Resolved | • bearND | T126934 Roll out Restbase to production | |||
| Resolved | • Nuria | T128612 Count requests to RESTBase from the Android app | |||
| Resolved | • bearND | T128887 Add x-analytics header for page view and preview requests |
Please use x-analytics header to tag whether a request is a pageview. See for example how is this handled for preview requests on the app:
Value should be pageview=1
https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-core/src/main/java/org/wikimedia/analytics/refinery/core/PageviewDefinition.java#L195
https://wikitech.wikimedia.org/wiki/X-Analytics
Homework is to make sure that this header passes through restbase and comes into hive
I think we should do this for both MW API and RESTBase page content requests; either on the lead section or remaining sections requests.
this would certainly simplify the pageview definition if all clients adopted it, so let's do it.
All RB requests pass through the text varnishes, so all accesses (including X-Analytics) are already available as part of the regular log stream. If all text varnish logs are treated equally for page view purposes, then requests with the header set would already be counted.
All RB requests pass through the text varnishes, so all accesses (including X-Analytics) are already available as part of the regular log stream.
then we should see those in webrequest table.
@bearND : let us know when you have it implemented it on your end (test app requests are fine) and we can check details. We would also need to update documentation regarding this "pageview=1
Change 274644 had a related patch set uploaded (by BearND):
Use x-analytics pageview to count mobile apps pageviews
Change 274645 had a related patch set uploaded (by BearND):
Add X-Analytics request headers for previews and pageviews
Change 274645 merged by jenkins-bot:
Add X-Analytics request headers for previews and pageviews
@Dbrant If this is really an Analytics task (evidenced by @Milimetric story-pointing it), I would recommend that the Android team subtask for their portion of the work, and stash this task outside of their sprint, to avoid skewing Android's velocity. The subtask can then be independently pointed by Android and added to the sprint in place of this task.
Change 274848 had a related patch set uploaded (by BearND):
Change RB remote config key
So the 2.1.143 beta release went out last week and (per T128887 and this diff) should be sending requests with "x-analytics: preview=1". However, the webrequest table still does not seem to see them (see below). Is there something else still missing, or is this the wrong query?
hive (default)> select is_pageview, x_analytics_map["preview"] from wmf.webrequest where year=2016 and month=3 AND day = 22 and x_analytics_map["preview"] is not NULL limit 10; ... OK is_pageview _c1 Time taken: 320.668 seconds
should be sending requests with "x-analytics: preview=1".
Can android devs confirm (via wireshark or similar) that app is sending the header?
Please use x-analytics header to tag whether a request is a pageview
I don't understand, does this mean that now anyone can make any request to the WMF servers be counted as a pageview simply by adding this header?
I don't understand, does this mean that now anyone can make any request to the WMF servers be counted as a pageview simply by adding this header?
There is a bug on this code though (mea culpa) and that is that we need to first look at filters such us http response code before looking at the tagging. Duh. Thanks for catching this.
The idea behind this change is that for domains that are being counted as "knowledge pageviews" (do not know how else to call them) rather than inspecting urls via regexes on what constitutes a pageviev we are moving towards tagging requests that should be consider pageviews as such.
Change 279447 had a related patch set uploaded (by Nuria):
Look at filters before looking at the Pageview tagging on x-analytics