Page MenuHomePhabricator

Display info about import time,MBs moved, no of file revisions and no of text revisions
Closed, ResolvedPublic5 Estimate Story Points

Description

Motivation
The FileImporter grafana dashboard currently displays faulty data around import time, MBs moved, no of file revisions and no of text revisions. But we still want to access that data

Acceptance Criteria

  • Build a display so that one can access import time, MBs moved, no of file revisions and no of text revisions, either really on a per-move basis, or on a somehow sense making aggregation (ideally daily)

Notes

  • Grafana and not strict time series data is tricky. Tipps and notes can be found here

Event Timeline

Restricted Application added a project: TCB-Team. · View Herald TranscriptJun 12 2019, 11:21 AM
Lea_WMDE set the point value for this task to 5.Jun 12 2019, 1:06 PM
Jan_Dittrich added a subscriber: Jan_Dittrich.EditedJun 12 2019, 1:08 PM

If Graphana can create them reliably, last-[n]-weeks histograms would be nice for time, MB, revisions. (if people want something really fancy: Kernel density overlayed)

awight added a subscriber: awight.Jun 19 2019, 3:09 PM

These specific graphs might not be faulty. Part of this task should be to build trustworthy metrics to compare against the graphs.

awight claimed this task.Jun 24 2019, 11:30 AM
awight moved this task from Sprint Backlog to Doing on the WMDE-QWERTY-Sprint-2019-06-12 board.

If Graphana can create them reliably, last-[n]-weeks histograms would be nice for time, MB, revisions. (if people want something really fancy: Kernel density overlayed)

I added a couple of heatmaps for us to iterate on, let me know whether you prefer these to the scatter & daily average graph, for example.

@Lea_WMDE btw, this is review for you, maybe I should put in the "Demo" column?

@awight, oh I see, yes then demo column is great :)

import time and MBs moved look awesome!
What I am still looking for is no of file revisions and no of text revisions :)

What I am still looking for is no of file revisions and no of text revisions :)

Those are in the purple boxes at the bottom. I rearranged the color scheme there so that they get darker as we go up the percentiles, let me know if it doesn't look right to you.

(not moving task)

can we trust those numbers, though? Aren't percentiles for events that don't trigger at least every minute by default wrong on grafana? That's why I'm super happy with the heat map solution you used for the other cases.

can we trust those numbers, though? Aren't percentiles for events that don't trigger at least every minute by default wrong on grafana? That's why I'm super happy with the heat map solution you used for the other cases.

Thanks, I think you're totally right about that. If more than one event per <time unit> gets aggregated, the percentiles will be fake. Happy to replace with a heat map!

Replaced with heat maps.

Lea_WMDE closed this task as Resolved.Jun 25 2019, 5:39 PM
Lea_WMDE moved this task from Demo to Done on the WMDE-QWERTY-Sprint-2019-06-12 board.