Create a Grafana board for TemplateWizard metrics described in T258920
Use new edit count bucketing described in T273475
Dimensions: byEditCount, byWiki
Metrics:
| open | save | abort | |
|---|---|---|---|
| description | number of times the TemplateWizard is opened per day | a change was made and saved successfully | the window was closed without saving |
| normalization | by wikitext editor 2010 edit sessions | by open | by open |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T271291 Follow-up changes to metrics | |||
| Resolved | awight | T269986 Add edit count bucketing to all metrics | |||
| Resolved | awight | T273475 Adjust edit count bucketing for TemplateWizard, segment all metrics | |||
| Resolved | Lena_WMDE | T270239 Grafana board for TemplateWizard | |||
| Resolved | None | T287578 Generate wikitext_2010 edit sessions for normalization in TemplateWizard |
Moving this to the backlog because it's still blocked until we can deploy the aggregation job.
This is unblocked now. Moving into the sprint because it's related to the other Grafana board changes.
@Andrew-WMDE there's something wired with edit buckets on this board. Lets have a look at this together. Otherwise it seems adjusted properly.
Moving this back to doing, because we want to add a normalization by edit sessions to the "open by ..." metrics. https://phabricator.wikimedia.org/T271902
We can not use the existing metric for wikitext_2010 edit sessions from codemirror, because the codemirror extension might not be installed everywhere. Creating a new ticket to generate the needed aggregation for template wizard and moving this back to stalled.
... the codemirror extension might not be installed everywhere.
I think it is installed everywhere on Wikimedia clusters, but there are other potential sources of error such as the codemirror number coming from an aggregate of multiple events, grouped by session ID. An independent metric gives us some certainty that the session count is comparable to the wizard launch count, I guess.
As follow-up, we could create an new graph to verify that CodeMirror and TemplateWizard counts for wikitext-2010 and -2017 editing sessions are nearly identical.
The graphs should be updated, but I can't tell until we have live data. Setting as blocked.
If we run into problems with matching sparse series lists, this panel has some tricks to reuse.
Minimal dashboard shows raw opens, open rate, and success rate, over edit count bucket and wiki.
Maybe this was discussed already, but does this also mean, that we should clean this up? Meaning: Remove the board, remove the metrics gathering in the code and the aggregation?