Page MenuHomePhabricator

Data Lake - mediarequests - adapt HQL query to include counting mediarequests from webrequest_text
Closed, ResolvedPublic

Description

This https://github.com/wikimedia/analytics-refinery/blob/7d8dc71fdf5c92baeab08d0bc91d4f30dbd0ff21/hql/mediarequest/hourly.hql#L92-L97 (and possibly others?) need to include webrequest text as a source for mediafiles.

See parent task.

The switchover will begin slowly happening in late July 2026. If we don't switch before then, mediarequest counts for thumbnails will start to drop.

Details

Related Changes in Gerrit:

Event Timeline

I have spoken with @Ladsgroup . For our mediarequest and mediacount jobs to continue to work, we need to:

  • Have them read rows from both upload and text webrequest sources
  • Have their request uri_host be one of (upload.wikimedia.org, thumb.wikimedia.org)

Change #1318667 had a related patch set uploaded (by Joal; author: Joal):

[analytics/refinery@master] Update media jobs for thumb change

https://gerrit.wikimedia.org/r/1318667

Change #1318667 merged by Joal:

[analytics/refinery@master] Update media jobs for thumb change

https://gerrit.wikimedia.org/r/1318667