Page MenuHomePhabricator

Contact wikidata bot owners & script runners not passing maxlag
Closed, DuplicatePublic

Description

For the last few weeks wikidata has included dispatch lag in maxlag.

The tracked lag can be seen @ https://grafana.wikimedia.org/dashboard/db/wikidata-dispatch?refresh=1m&panelId=22&fullscreen&orgId=1&from=now-7d&to=now

Some bots & scripts are not passing sensible values to maxlag, either NULL or a very large number.

This ticket tracks the contacting of those people in sub tickets with NDA access (no idea if including UAs etc need to be hidden).

A hive query that can be used to get the information needed to follow up here (once you modify the dates) can be found below:

SELECT COUNT(*) AS requests,
       useragent,
       params["maxlag"] AS maxlag,
       array_contains(errorcodes, "maxlag") AS hitMaxLagError
FROM wmf_raw.apiaction
WHERE wiki = "wikidatawiki"
  AND params["action"] RLIKE '^wbl?(create|edit|set|add|remove|link|merge)'
  AND (array_contains(errorcodes, "maxlag") OR haderror = 0)
  AND useragent != '127.0.0.1'
  AND YEAR = 2018
  AND MONTH = 07
  AND DAY = 15
GROUP BY useragent,
         params["maxlag"],
         array_contains(errorcodes, "maxlag")
ORDER BY requests DESC LIMIT 1000;

Related Objects

StatusSubtypeAssignedTask
DuplicateAddshore
ResolvedAddshore
ResolvedNone

Event Timeline

Addshore triaged this task as Medium priority.Aug 22 2018, 3:21 PM
Addshore lowered the priority of this task from Medium to Low.Oct 16 2018, 12:57 PM

Low, as dispatching has sped up dramatically now :D

Addshore closed subtask Restricted Task as Resolved.Jan 17 2019, 5:31 PM