Page MenuHomePhabricator

Debug blank datafiles generated by generate.py [8 pts] {lamb]
Closed, ResolvedPublic

Description

These graphs are getting blank datafiles: http://mobile-reportcard.wmflabs.org/#edits_daily-graphs-tab

Event Timeline

kevinator raised the priority of this task from to Normal.
kevinator updated the task description. (Show Details)
kevinator added a subscriber: kevinator.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 22 2015, 5:35 PM
Jdlrobson set Security to None.
mforns claimed this task.Jun 24 2015, 9:07 PM
mforns edited projects, added Analytics-Kanban; removed Analytics-Backlog.
mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.

This seems to me as related to Sean Pringle's comments on slow queries (as Dan Andreescu had foreseen :]).
These 3 queries hit a view (defined in config.yaml) named "edits_web".
This view gathers all data from all MobileWebEditing schema revisions.
Currently this sums up to ~100M records.
After that, some filters are applied, which do not take advantage of the table's indexes.
So the query is extremely slow, and getting slower with each new event being inserted.
This report is calculated with timeboxed=false option, so it is calculated from scratch every day.
My guess is that some days it is taking too long and the generate.py can not compute it.

mforns moved this task from In Progress to Done on the Analytics-Kanban board.Jun 25 2015, 4:07 PM
kevinator closed this task as Resolved.Jun 29 2015, 3:28 PM