Page MenuHomePhabricator

Clean up mobile-reportcard dashboards {frog} [8 pts]
Closed, ResolvedPublic

Description

Summary: we are cleaning up http://mobile-reportcard.wmflabs.org

  • We will remove all editing related graphs and ask the edit team about them
  • We will fix up and timebox the remaining mobile graphs

Keep and Clean Up:

Disable and Check With Edit Team as part of a different task:

Event Timeline

Milimetric raised the priority of this task from to Needs Triage.
Milimetric updated the task description. (Show Details)
Milimetric added a project: Analytics-Backlog.
Milimetric added a subscriber: Milimetric.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 30 2015, 8:02 PM
Milimetric renamed this task from Clean up mobile-reportcard dashboards to Clean up mobile-reportcard dashboards {frog}.Jun 30 2015, 8:04 PM
Milimetric set Security to None.
kevinator triaged this task as Unbreak Now! priority.Jul 2 2015, 4:41 PM
kevinator moved this task from Incoming to Low on the Analytics-Backlog board.
Milimetric moved this task from Low to Prioritized on the Analytics-Backlog board.Jul 6 2015, 4:10 PM
Milimetric updated the task description. (Show Details)Jul 6 2015, 4:17 PM
Milimetric renamed this task from Clean up mobile-reportcard dashboards {frog} to Clean up mobile-reportcard dashboards {frog} [13 pts].Jul 6 2015, 4:19 PM
Milimetric moved this task from Prioritized to Tasked on the Analytics-Backlog board.

Any news here? This has been "Unbreak now" priority for three weeks, has no assignee set, and is blocked by T103385 which only has "normal" priority.

@Milimetric: This task has been "Unbreak Now!" priority for four weeks. Please set an assignee and/or correct the priority now.

The task this is blocked on is now resolved (we'll update shortly). We'll get to this soon, but I'll change it from Unbreak Now to High.

Milimetric lowered the priority of this task from Unbreak Now! to High.Jul 27 2015, 4:06 PM
mforns claimed this task.Jul 27 2015, 4:24 PM
mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.Jul 28 2015, 8:43 PM

http://datasets.wikimedia.org/limn-public-data/mobile/datafiles/ui-daily-historic.csv uses MobileWebClickTracking schema.
This schema has seen a dramatic drop in volume of events since mid December 2014, and today it hardly receives any event.
I will ask Adam Baso about this schema in our next meeting.

mforns added a comment.EditedJul 28 2015, 10:13 PM

Same for http://datasets.wikimedia.org/limn-public-data/mobile/datafiles/main-menu-daily.csv, it uses MobileWebMainMenuClickTracking and this schema suffered two drops in the reception of events: mid February and mid May.
Will include this in the conversation with Adam.
UPDATE: Actually, this schema is receiving events again since Jul 8!
Moved to reportupdater, should work.

mforns added a comment.EditedJul 29 2015, 2:57 PM

http://datasets.wikimedia.org/limn-public-data/mobile/datafiles/page-ui-daily.csv seems to be receiving events normally.
However, mid February MobileWebUIClickTracking saw a big drop (will ask that as well), and also the report hasn't been updated since Jun 25.

The problem was that queries for certain days returned unexpected values, and made generate.py crash. In generate.py, when a date crashes, all following dates are skipped. I moved the report to reportupdater and it should work now when deployed.

http://datasets.wikimedia.org/limn-public-data/mobile/datafiles/watchlist-activity.csv is the same case as the latter. Moved it to reportupdater and it should work. Also shifted the start date 1 day to 2014-01-02 to avoid the peak.

http://datasets.wikimedia.org/limn-public-data/mobile/datafiles/mobile-options.csv seems to be working fine in reportupdater.
I created a non-timeboxed version of it called mobile-options-last-3-months; it will roll over the last 90 days for this metric.
Non-timeboxed works here, because the schema used (MobileOptionsTracking) contain no sensitive data and can be kept indefinitely.

Change 227911 had a related patch set uploaded (by Mforns):
Clean up mobile-reportcard reports and dashboards

https://gerrit.wikimedia.org/r/227911

Change 227911 merged by Milimetric:
Clean up mobile-reportcard reports and dashboards

https://gerrit.wikimedia.org/r/227911

mforns moved this task from In Code Review to Done on the Analytics-Kanban board.Jul 31 2015, 10:21 PM

There's one thing missing!
We need to enable the reportcard in puppet.
Will do.

Change 228420 had a related patch set uploaded (by Mforns):
Reenable reporting for mobile-reportcard

https://gerrit.wikimedia.org/r/228420

So we should wait for this to be merged and shall see the mobile-reportcard graphs again :]

Change 228840 had a related patch set uploaded (by Milimetric):
Enable the mobile data crunching job

https://gerrit.wikimedia.org/r/228840

Change 228840 merged by Ottomata:
Enable the mobile data crunching job

https://gerrit.wikimedia.org/r/228840

This cleanup is not 100% done, because some post-deployment cleanup still has to happen next week. But in the meantime, the new cleaner dashboard is running again and producing data: http://mobile-reportcard.wmflabs.org/

mforns moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Aug 10 2015, 4:06 PM
mforns renamed this task from Clean up mobile-reportcard dashboards {frog} [13 pts] to Clean up mobile-reportcard dashboards {frog} [8 pts].Aug 12 2015, 3:48 PM

Change 228420 abandoned by Mforns:
Reenable reporting for mobile-reportcard

Reason:
Done through another patch

https://gerrit.wikimedia.org/r/228420

kevinator closed this task as Resolved.Aug 14 2015, 3:19 PM
kevinator added a subscriber: kevinator.