HomePhabricator

move netmapper processing to common VCL

Authored by BBlack.

Description

move netmapper processing to common VCL

(Note, this was originally these 3 commits
f092ee39eb89caf71fd34bd3a1f49e89b7a07e71
941b3e6e97f0b09954f6eb218b358f237196ebdb
e73eab0d80eeca86a764b8e689a8cba2e943edea
Then was reverted along with later stuff in:
e37e79857a6457578348451216d8efdfa465a814
, now re-applied)

This moves the netmapper "carriers" and "proxies" database lookups
to common VCL across all clusters, and integrates them with the
early vcl_recv IP processing done there to generate a standard set
of headers related to client IPs, trusted proxies, carriers, etc.

Note that new header names were invented here for the netmapper
results: "X-Carrier" contains MCC-MNC, and "X-Carrier-Meta"
contains the zeronet metadata like "wap". Combined, these are
what what Zero calls "X-CS2". "X-Trusted-Proxy" also contains
what Zero has traditionally calls "X-Forwarded-By".

This allows for an easier transition of the zero.inc.vcl code to
use the headers supplied from the common VCL, which copies the
data to its legacy headers and then possibly further
manipulates/uses/deletes those legacy headers.

Doing that also removes any fears about the sudden appearance of
previously mobile/zero -specific headers in text/upload requests
causing issues at the MediaWiki layer.

Bug: T89177
Bug: T96847
Bug: T109286
Change-Id: I8479f147a9c92cdba2733fcb8f1f92dec0bfaa47

Details

Committed
BBlackOct 6 2015, 8:57 PM
Parents
rOPUP7fe3129fa6b8: ishmael: remove module, decom service
Branches
Unknown
Tags
Unknown
Reverted By
rOPUPbd50d6249ca1: Revert "move netmapper processing to common VCL"
rOPUP20211fc71026: X-Client-IP 10/12 - switch zero.inc to using XC + XCM
rOPUP243003f43ac2: X-Client-IP 8/12 - Set X-CIP
rOPUPc84ab72da31d: X-Client-IP 11/12 - remove outdated 404-01b zero case
rOPUPe134cf747d97: X-Client-IP 9/12 - Set X-C + X-C-M
rOPUP61f989811145: X-Client-IP 12/12 - switch zero analytics to use XC/XCM
rOPUP9fb60f877d21: X-Client-IP 5/12 - recv_fe_ip_proc frontend-only
rOPUPcfd939df4ad9: X-Client-IP 7/12 - Set X-T-P
rOPUPa41131f12bdc: X-Client-IP 6/12 - unset the 4x new headers
rOPUP9a48587705eb: X-Client-IP 4/12 - move XFF-setter out of recv_fe_ip_processing
rOPUP1bdc39b5923c: X-Client-IP 3/12 - remove fe default on be guard
rOPUPcface5c9d01b: X-Client-IP 2/12 - rename ip_proc sub, move req.restarts guard
rOPUPab076e4f6a85: X-Client-IP 1/12 - just move netmapper import + init
rOPUP37c509052159: X-Client-IP 12/12 - switch zero analytics to use XC/XCM
rOPUP3edd989e4dd2: X-Client-IP 11/12 - remove outdated 404-01b zero case
rOPUP19f9f3d63a0c: X-Client-IP 10/12 - switch zero.inc to using XC + XCM
rOPUPae5745ce17bb: X-Client-IP 9/12 - Set X-C + X-C-M
rOPUP1198ceee51a2: X-Client-IP 8/12 - Set X-CIP
rOPUPc29ebea7b6ee: X-Client-IP 7/12 - Set X-T-P
rOPUPfe47ba5e1089: X-Client-IP 6/12 - unset the 4x new headers
rOPUP238a5cf4bf1b: X-Client-IP 5/12 - recv_fe_ip_proc frontend-only
rOPUP2982dcd22caf: X-Client-IP 4/12 - move XFF-setter out of recv_fe_ip_processing
rOPUPfcfce07a2aca: X-Client-IP 3/12 - remove fe default on be guard
rOPUP61855e421164: X-Client-IP 2/12 - rename ip_proc sub, move req.restarts guard
rOPUP633dcb401388: X-Client-IP 1/12 - just move netmapper import + init
rOPUPbda95f5571e2: Revert "move netmapper processing to common VCL"
ChangeId
I8479f147a9c92cdba2733fcb8f1f92dec0bfaa47

Event Timeline