Page MenuHomePhabricator

Check and potentially timebox limn-flow-data reports {tick} [5 pts]
Closed, ResolvedPublic

Description

http://flow-reportcard.wmflabs.org/
In the context of EventLogging audit, we should make sure that this project's queries do not use data that will be auto-purged.
Optionally, fix any problems with the dashboard and migrate to reportupdater if necessary.

Event Timeline

mforns created this task.Jul 30 2015, 9:33 PM
mforns raised the priority of this task from to Needs Triage.
mforns updated the task description. (Show Details)
mforns added a project: Analytics-Backlog.
mforns added a subscriber: mforns.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 30 2015, 9:33 PM
Milimetric renamed this task from Check and potentially timebox limn-flow-data reports {Tick} to Check and potentially timebox limn-flow-data reports {tick}.Aug 7 2015, 4:36 PM
Milimetric set Security to None.
kevinator triaged this task as High priority.Aug 10 2015, 4:52 PM
Milimetric renamed this task from Check and potentially timebox limn-flow-data reports {tick} to Check and potentially timebox limn-flow-data reports {tick} [5 pts].Aug 10 2015, 4:59 PM
mforns claimed this task.Aug 11 2015, 7:36 PM
mforns edited projects, added Analytics-Kanban; removed Analytics-Backlog.
mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.Aug 12 2015, 2:53 PM

Change 231438 had a related patch set uploaded (by Mforns):
Permit custom db param in report config

https://gerrit.wikimedia.org/r/231438

Change 231446 had a related patch set uploaded (by Mforns):
Timebox reports and move to reportupdater

https://gerrit.wikimedia.org/r/231446

The refactor seems to work, however there are 2 things to mention:

  1. I had to make a small change in reportupdater, and this must be deployed first. See the first gerrit patch.
  2. There is an ongoing conversion from LQT to FLOW, that is adding data to the tables for dates in the past. So if the metrics are rerun from the beginning of time, they will change including these newly added values.
    • Hence, I'll transform the current CSVs into TSVs, so that reportupdater will only update the reports from now on, and keep the old values as they were.
    • If, in the future, the collaboration team wants to have only the new data from the beginning of time, we just delete all TSVs and the reports will be regenerated complete.
    • We can, in this case, leave the old CSV files as a historical record.

Change 231438 merged by Milimetric:
Permit custom db param in report config

https://gerrit.wikimedia.org/r/231438

Change 231446 merged by Milimetric:
Timebox reports and move to reportupdater

https://gerrit.wikimedia.org/r/231446

mforns moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Aug 14 2015, 8:11 PM
kevinator closed this task as Resolved.Aug 18 2015, 2:47 PM
kevinator added a subscriber: kevinator.