Hive claims it doesn't have any Maps data on Oct 31st
Closed, ResolvedPublic

Description

If we run the tileset script, it chokes and dies on 30 October - Hive doesn't return any data that meets the query. Wat?

Query:

SELECT style, zoom, scale, format, user_id, cache, COUNT(1) AS n
FROM (
   SELECT
     REGEXP_EXTRACT(uri_path, '^/([^/]+)/([0-9]{1,2})/(-?[0-9]+)/(-?[0-9]+)(@([0-9]\\.?[0-9]?)x)?\\.([a-z]+)$', 1) AS style,
     REGEXP_EXTRACT(uri_path, '^/([^/]+)/([0-9]{1,2})/(-?[0-9]+)/(-?[0-9]+)(@([0-9]\\.?[0-9]?)x)?\\.([a-z]+)$', 2) AS zoom,
     COALESCE(REGEXP_EXTRACT(uri_path, '^/([^/]+)/([0-9]{1,2})/(-?[0-9]+)/(-?[0-9]+)(@([0-9]\\.?[0-9]?)x)?\\.([a-z]+)$', 6), '1') AS scale,
     REGEXP_EXTRACT(uri_path, '^/([^/]+)/([0-9]{1,2})/(-?[0-9]+)/(-?[0-9]+)(@([0-9]\\.?[0-9]?)x)?\\.([a-z]+)$', 7) AS format,
     CONCAT(user_agent, client_ip) AS user_id,
     cache_status AS cache
   FROM wmf.webrequest
   WHERE
     year=2015 AND month=10 and day=30
     AND webrequest_source = 'maps'
     AND http_status IN('200','304')
     AND uri_path RLIKE '^/([^/]+)/([0-9]{1,2})/(-?[0-9]+)/(-?[0-9]+)(@([0-9]\\.?[0-9]?)x)?\\.([a-z]+)$'
) prepared
GROUP BY style, zoom, scale, format, user_id, cache;
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 4 2015, 9:33 PM
Yurik edited the task description. (Show Details)Nov 4 2015, 10:16 PM
Yurik set Security to None.
Yurik edited the task description. (Show Details)Nov 4 2015, 10:20 PM
Yurik added a comment.Nov 4 2015, 11:03 PM

Update: The query runs fine in Hive, so it must be something else

Yes, the query /runs/, it just returns no data.

Yurik added a comment.Nov 4 2015, 11:10 PM

@Ironholds, i tried running it - it returned data for me. Could you paste your exact query?

Aklapper changed the title from "Hive claims it doesn't have any Maps data on the 31st" to "Hive claims it doesn't have any Maps data on Oct 31st".Nov 5 2015, 10:33 AM
Ironholds closed this task as "Resolved".Nov 5 2015, 5:15 PM
Ironholds claimed this task.

And now it works. WTF? Alright, resolved.