Page MenuHomePhabricator

Include HTTP Referer header when navigating through internal links
Closed, ResolvedPublic1 Story Points

Description

It would be useful for Analytics if we include the HTTP Referer header when requesting article content. When navigating to another article from a link, this can simply be the URL of the previous page that the user was browsing.

Event Timeline

Dbrant created this task.Apr 23 2018, 1:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 23 2018, 1:18 PM
LGoto set the point value for this task to 1.Apr 23 2018, 4:03 PM

Change 431682 had a related patch set uploaded (by Cooltey; owner: Cooltey):
[apps/android/wikipedia@master] Include HTTP Referer header when navigating through internal links

https://gerrit.wikimedia.org/r/431682

Great! Adding Analytics so that the Analytics Engineering team is aware and can double-check that this works for ingestion in the referer field of the webrequest table.

fdans moved this task from Incoming to Radar on the Analytics board.May 14 2018, 3:47 PM

Change 436918 had a related patch set uploaded (by Dbrant; owner: Dbrant):
[apps/android/wikipedia@master] Follow-up: Refine and extend our support for the Referer header.

https://gerrit.wikimedia.org/r/436918

Change 431682 merged by jenkins-bot:
[apps/android/wikipedia@master] Include HTTP Referer header when navigating through internal links

https://gerrit.wikimedia.org/r/431682

Change 436918 merged by jenkins-bot:
[apps/android/wikipedia@master] Follow-up: Refine and extend our support for the Referer header.

https://gerrit.wikimedia.org/r/436918

Dbrant closed this task as Resolved.Jun 27 2018, 8:14 PM

Great, thanks! From a quick look at the data, this seems to have rolled out around June 21?

SELECT year, month, day, CONCAT(year,'-',LPAD(month,2,'0'),'-',LPAD(day,2,'0')) AS date,
referer_class,
SUM(view_count) AS views
FROM wmf.pageview_hourly
WHERE year = 2018 AND month =6
AND access_method = 'mobile app'
AND user_agent_map['os_family'] = 'Android'
AND agent_type = 'user'
GROUP BY year, month, day, referer_class
ORDER BY year, month, day, referer_class LIMIT 10000;

year	month	day	date	referer_class	views
2018	6	1	2018-06-01	internal	6
2018	6	1	2018-06-01	none	4906463
2018	6	2	2018-06-02	internal	29
2018	6	2	2018-06-02	none	5558288
2018	6	3	2018-06-03	internal	33
2018	6	3	2018-06-03	none	6115054
2018	6	4	2018-06-04	external (search engine)	1
2018	6	4	2018-06-04	internal	6
2018	6	4	2018-06-04	none	5138083
2018	6	5	2018-06-05	internal	12
2018	6	5	2018-06-05	none	4994861
2018	6	6	2018-06-06	internal	35
2018	6	6	2018-06-06	none	4930961
2018	6	7	2018-06-07	external (search engine)	2
2018	6	7	2018-06-07	internal	54
2018	6	7	2018-06-07	none	4867617
2018	6	8	2018-06-08	internal	38
2018	6	8	2018-06-08	none	4966747
2018	6	9	2018-06-09	internal	31
2018	6	9	2018-06-09	none	5544654
2018	6	10	2018-06-10	internal	45
2018	6	10	2018-06-10	none	6167614
2018	6	11	2018-06-11	external (search engine)	2
2018	6	11	2018-06-11	internal	25
2018	6	11	2018-06-11	none	5246565
2018	6	12	2018-06-12	internal	66
2018	6	12	2018-06-12	none	5020015
2018	6	13	2018-06-13	external (search engine)	1
2018	6	13	2018-06-13	internal	37
2018	6	13	2018-06-13	none	4916525
2018	6	14	2018-06-14	external (search engine)	1
2018	6	14	2018-06-14	internal	55
2018	6	14	2018-06-14	none	4889998
2018	6	15	2018-06-15	internal	31
2018	6	15	2018-06-15	none	4869373
2018	6	16	2018-06-16	external (search engine)	1
2018	6	16	2018-06-16	internal	34
2018	6	16	2018-06-16	none	5554345
2018	6	17	2018-06-17	external (search engine)	1
2018	6	17	2018-06-17	internal	33
2018	6	17	2018-06-17	none	6165212
2018	6	18	2018-06-18	internal	10
2018	6	18	2018-06-18	none	5178965
2018	6	19	2018-06-19	internal	30
2018	6	19	2018-06-19	none	5022436
2018	6	20	2018-06-20	internal	48
2018	6	20	2018-06-20	none	4862532
2018	6	21	2018-06-21	external	5
2018	6	21	2018-06-21	external (search engine)	280
2018	6	21	2018-06-21	internal	469
2018	6	21	2018-06-21	none	4854108
2018	6	22	2018-06-22	external	95
2018	6	22	2018-06-22	external (search engine)	14098
2018	6	22	2018-06-22	internal	19185
2018	6	22	2018-06-22	none	4913261
2018	6	22	2018-06-22	unknown	16
2018	6	23	2018-06-23	external	206
2018	6	23	2018-06-23	external (search engine)	25981
2018	6	23	2018-06-23	internal	36683
2018	6	23	2018-06-23	none	5453626
2018	6	23	2018-06-23	unknown	34
2018	6	24	2018-06-24	external	321
2018	6	24	2018-06-24	external (search engine)	30286
2018	6	24	2018-06-24	internal	46635
2018	6	24	2018-06-24	none	6078344
2018	6	24	2018-06-24	unknown	45
2018	6	25	2018-06-25	external	330
2018	6	25	2018-06-25	external (search engine)	27446
2018	6	25	2018-06-25	internal	43016
2018	6	25	2018-06-25	none	4989313
2018	6	25	2018-06-25	unknown	22
2018	6	26	2018-06-26	external	2882
2018	6	26	2018-06-26	external (search engine)	369914
2018	6	26	2018-06-26	internal	497632
2018	6	26	2018-06-26	none	4048209
2018	6	26	2018-06-26	unknown	310
2018	6	27	2018-06-27	external	5170
2018	6	27	2018-06-27	external (search engine)	657959
2018	6	27	2018-06-27	internal	857853
2018	6	27	2018-06-27	none	3303737
2018	6	27	2018-06-27	unknown	454
2018	6	28	2018-06-28	external	5647
2018	6	28	2018-06-28	external (search engine)	731858
2018	6	28	2018-06-28	internal	963577
2018	6	28	2018-06-28	none	3179193
2018	6	28	2018-06-28	unknown	477
2018	6	29	2018-06-29	external	3589
2018	6	29	2018-06-29	external (search engine)	423151
2018	6	29	2018-06-29	internal	575169
2018	6	29	2018-06-29	none	1668626
2018	6	29	2018-06-29	unknown	324
91 rows selected (158.475 seconds)