Page MenuHomePhabricator

Grafana board for TemplateWizard
Closed, ResolvedPublic0 Estimated Story Points

Description

Create a Grafana board for TemplateWizard metrics described in T258920
Use new edit count bucketing described in T273475

Dimensions: byEditCount, byWiki

Metrics:

opensaveabort
descriptionnumber of times the TemplateWizard is opened per daya change was made and saved successfullythe window was closed without saving
normalizationby wikitext editor 2010 edit sessionsby openby open

Event Timeline

Lena_WMDE set the point value for this task to 2.Dec 16 2020, 9:24 AM
Lena_WMDE moved this task from Backlog to In sprint on the WMDE-Templates-FocusArea board.
awight changed the task status from Open to Stalled.Dec 18 2020, 2:44 PM
lilients_WMDE changed the task status from Stalled to Open.Jan 12 2021, 11:03 AM
lilients_WMDE changed the task status from Open to Stalled.Jan 12 2021, 3:28 PM
lilients_WMDE moved this task from Doing to Sprint Backlog on the WMDE-QWERTY-Sprint-2021-01-06 board.
awight moved this task from Doing to Sprint Backlog on the WMDE-QWERTY-Sprint-2021-01-06 board.
awight added subscribers: lilients_WMDE, awight.

Moving this to the backlog because it's still blocked until we can deploy the aggregation job.

Lena_WMDE changed the point value for this task from 2 to 3.Jan 20 2021, 9:31 AM

This is unblocked now. Moving into the sprint because it's related to the other Grafana board changes.

lilients_WMDE changed the task status from Stalled to Open.Feb 26 2021, 4:31 PM
lilients_WMDE updated the task description. (Show Details)

@Andrew-WMDE there's something wired with edit buckets on this board. Lets have a look at this together. Otherwise it seems adjusted properly.

lilients_WMDE updated the task description. (Show Details)
lilients_WMDE moved this task from Demo to Doing on the WMDE-TechWish-Sprint-2021-07-21 board.
lilients_WMDE added a subscriber: WMDE-Fisch.

Moving this back to doing, because we want to add a normalization by edit sessions to the "open by ..." metrics. https://phabricator.wikimedia.org/T271902

We can not use the existing metric for wikitext_2010 edit sessions from codemirror, because the codemirror extension might not be installed everywhere. Creating a new ticket to generate the needed aggregation for template wizard and moving this back to stalled.

... the codemirror extension might not be installed everywhere.

I think it is installed everywhere on Wikimedia clusters, but there are other potential sources of error such as the codemirror number coming from an aggregate of multiple events, grouped by session ID. An independent metric gives us some certainty that the session count is comparable to the wizard launch count, I guess.

As follow-up, we could create an new graph to verify that CodeMirror and TemplateWizard counts for wikitext-2010 and -2017 editing sessions are nearly identical.

The graphs should be updated, but I can't tell until we have live data. Setting as blocked.

If we run into problems with matching sparse series lists, this panel has some tricks to reuse.

Minimal dashboard shows raw opens, open rate, and success rate, over edit count bucket and wiki.

awight changed the point value for this task from 3 to 1.Aug 18 2021, 8:12 AM
awight changed the point value for this task from 1 to 0.Sep 1 2021, 8:07 AM
awight moved this task from Sprint Backlog to Demo on the WMDE-TechWish-Sprint-2021-09-01 board.
Lena_WMDE claimed this task.
Lena_WMDE added a subscriber: Lena_WMDE.

We no longer need this board.

We no longer need this board.

Maybe this was discussed already, but does this also mean, that we should clean this up? Meaning: Remove the board, remove the metrics gathering in the code and the aggregation?