Page MenuHomePhabricator

No maps tile requests in webrequests table as of 1 June 2017
Closed, ResolvedPublic


Our counts of map tiles served disappeared as of 1 June 2017 (query we use). In fact, running the following simple query shows that there are no tile requests after 31 May 2017:

  TO_DATE(ts) AS `date`,
  COUNT(1) AS requests
FROM wmf.webrequest
  webrequest_source = 'maps'
  AND year = 2017 AND (
    (month = 5 AND day = 31)
    OR (month = 6 AND day = 1)
  AND http_status = '200'
  AND uri_path RLIKE '^/([^/]+)/([0-9]{1,2})/(-?[0-9]+)/(-?[0-9]+)(@([0-9]\.?[0-9]?)x)?\.([a-z]+)$'
  AND uri_query <> '?loadtesting'

Screen Shot 2017-06-05 at 6.21.51 PM.png (405×1 px, 78 KB)

Event Timeline

This looks like it might be related to T164608#3248269. It looks like the webrequest_source has changed to "upload" and that some differentiation has to be done on the hostname ( I'm not familiar with how the data is stored, but hopefully, this will point you in the correct direction. If not, we can dig into this together...

Correct, webrequest_source="maps" no longer exists.

@Nuria - how can we get this fixed? Is the data for the last 6-ish days lost? @MaxSem - is this something you can help us with?

My guess is that @mpopov probably knows enough to fix that request (now that he knows what has changed). If it is not the case, I'm pretty sure that with a few minutes between @mpopov and myself, we should find a way...

Change 357440 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[wikimedia/discovery/golden@master] Update partition info for maps tiles

Change 357440 merged by Bearloga:
[wikimedia/discovery/golden@master] Update partition info for maps tiles

mpopov claimed this task.

Thanks for finding the source of the problem, @Gehel!

@debt: the tile counts for june will be backfilled

Change 373628 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[wikimedia/discovery/golden@master] metrics::maps::users_by_country: Fix wmf.webrequest partition

Change 373628 merged by Bearloga:
[wikimedia/discovery/golden@master] metrics::maps::users_by_country: Fix wmf.webrequest partition