Webrequest logs get auto-purged at 60 days. We need to make a copy of them for the period of June 21 to 30 (Note that survey ran from 2017-06-22 (13:09 UTC) to 2017-06-29 (23:19 UTC).). We will keep this copy until the 90-day period allowed by the privacy policy and anonymize/aggregate PII right before that point.
Note that this research involves de-biasing of the results based on webrequest logs and we need to keep this data as long as possible until the research is finished.
The following extraction is considered:
CREATE TABLE motivations.all_requests AS SELECT client_ip, user_agent, geocoded_data, user_agent_map, ts, referer, title, uri_path, uri_host, uri_query, http_status, is_pageview, access_method, referer_class, normalized_host, pageview_info, year, month, day, hour, agent_type FROM wmf.webrequest WHERE year = 2017 AND month = 6 AND day in (21,22,23,24,25,26,27,28,29,30) AND webrequest_source = 'text' AND access_method != 'mobile app' AND agent_type = 'user';