Page MenuHomePhabricator

Switch Varnish's GeoIP code to libmaxminddb/GeoIP2
Closed, ResolvedPublic

Description

We currently have GeoIP/libgeoipip code in our VCL for the GeoIP cookies & geoiplookup. We should replace it with code that uses libmaxminddb & the GeoIP2 databases. This would allow us proper IPv6 lookups as well and, depending on their accuracy and our accuracy requirements, might just let us fully decommission the separate geoiplookup.wikimedia.org domain.

Related Objects

Event Timeline

faidon raised the priority of this task from to Needs Triage.
faidon updated the task description. (Show Details)
faidon added projects: acl*sre-team, Traffic.
faidon added a subscriber: faidon.
BBlack triaged this task as Medium priority.May 20 2015, 8:17 AM

Change 253619 had a related patch set uploaded (by Faidon Liambotis):
varnish: switch from libGeoIP to libmaxminddb

https://gerrit.wikimedia.org/r/253619

How to test accuracy? The original accuracy test for MaxMind was https://meta.wikimedia.org/wiki/MaxMindCityTesting

The vendor for both databases is the same, so the accuracy of the IPv4 data would be the same. The IPv6 switch means that we'd geolocate IPv6 directly now instead of falling back to IPv4; this would possibly be less accurate but I think we should just do it anyway (but not during the fundraiser period).

ori set Security to None.
ori added a subscriber: Jgreen.

looks like this was waited for until the fundraising was over, can it be resumed now? also would the migration to varnish4 have any impact on it?

It's mostly been blocked on lack of anyone having time to work on it, too. At this point, it's probably wiser to delay this to post-varnish4 instead of doing work under varnish3 that will have to be re-evaluated/ported to varnish4 shortly after. Will update blockers.

I'm planning to merge up the new version of the Varnish GeoIP code from rOPUP371d7cc737d0 in the next couple of days. If anyone has the time to give it some review for stupid mistakes, it would be appreciated!

Change 305419 had a related patch set uploaded (by BBlack):
GeoIP VCL: re-set old IPv6 no-data cookies

https://gerrit.wikimedia.org/r/305419

Change 305647 had a related patch set uploaded (by BBlack):
Prep varnishd for libmaxminddb-based VCL

https://gerrit.wikimedia.org/r/305647

Change 305648 had a related patch set uploaded (by BBlack):
varnish: remove libgeoip from text VCL compilation

https://gerrit.wikimedia.org/r/305648

Change 305647 merged by BBlack:
Prep varnishd for libmaxminddb-based VCL

https://gerrit.wikimedia.org/r/305647

Change 253619 merged by BBlack:
varnish: switch from libGeoIP to libmaxminddb

https://gerrit.wikimedia.org/r/253619

Change 305419 merged by BBlack:
GeoIP VCL: re-set old IPv6 no-data cookies

https://gerrit.wikimedia.org/r/305419

BBlack claimed this task.

Change 305648 merged by BBlack:
varnish: remove libgeoip from text VCL compilation

https://gerrit.wikimedia.org/r/305648