This is the code exporting metrics to Graphite/Grafana: https://gerrit.wikimedia.org/r/c/analytics/reportupdater-queries/+/645345/

To be able to show the percentage of users that have syntax highlighting (CodeMirror) enabled, we need the total number of users. We would like to add new variable that only checks if the respective editor is enabled:

MediaWiki.CodeMirror.preferences.byPreference.wikitext_2010.byEnabled.true
MediaWiki.CodeMirror.preferences.byPreference.wikitext_2017.byEnabled.true

With this we can compare to the existing combination of editor and codeMirror and get the percentage:

MediaWiki.CodeMirror.preferences.byPreference.wikitext_2010_and_CodeMirror.byEnabled.true
MediaWiki.CodeMirror.preferences.byPreference.wikitext_2017_and_CodeMirror.byEnabled.true

@lilients_WMDE Small note on the graphana intro text:

"Users can activate it in the editor itself (toggle) or in their user settings (preferences). " Users can only toggle in the editor itself. There is not setting under preferences. We should update here for clarity.

I also think it would be useful to add something to the intro text about terms, because it's not clear at the moment which graphs are tracking toggling (can be multiple times per day for a single user) and which are tracking a single preference setting per day for a single user (I think this is only the enabled graph?). I think you are using 'usages' for toggling but it would be great to have a definition included of what exactly we mean by that vs. activations. Also, in the activations section what is meant by true / false?

Lastly, we talked about segmenting by editor count. Is this still planned for the future? Maybe this is in another ticket, but if it is would be good to link here.

@ECohen_WMDE Thanks for your feedback! I updated the intro text and the titles of the graphs. Hope it is clearer now.

For the segmentation by edit count we need to update the data logging first and add the option byEditCount. I guess it would make sense to put this in a separate ticket.

@lilients_WMDE Thanks! I think it's much clearer and the table you made above is very helpful. I have a few follow-up comments and questions:

General

• I think it would be useful to have the definitions on the grafana board. Since the ticket will be resolved and not referenced much, but the board will continue to be used and referenced in the future (and possibly also viewed by people outside the team). Is it possible to pull in the full table?
• When a graph is by editor type (not broken down by wiki) do these numbers represent all editing sessions across all wikis? Or is it just en wiki?

Preferences

• Seems like there is a duplicate graph under preferences, both showing the codemirror numbers
• Are the bar graphs with blue numbers representing the current state, updating each day? Would be also nice to have the line graph (like in sessions where the same data is presented both ways). It would be helpful to show these numbers over time to see if they remain relatively stable (are the same number of people consistently turning it on/off over time). Then we can get a good idea of how many people use it consistently. This will also be an important number for us after we deploy, to see if there are any changes over time there.

Toggles

• Is the line representing 'true' equal to clicks to enable the highlighting? Then 'false' is tracking clicks which turn it off? If yes, then I would rename true to "clicks to activate" and false to "clicks to deactivate" in the legend below the graph.

For the segmentation by edit count we need to update the data logging first and add the option byEditCount. I guess it would make sense to put this in a separate ticket.

I think this is it? T271291: Follow-up changes to metrics I can never remember which should be the parent, but would make sense if they're linked.

@ECohen_WMDE Thanks for your feedback and questions!

General

• Adding the definitions to the board is a very good idea. I will try what works best.
• All graphs display data for the selected variables (editor types or wikis). You can also select a subset in the dropdowns at the top left.

Preferences

• The duplicate graph was a leftover. :) I removed it.
• The blue number panels display a mean number for all days with a small overview of the time line. I added additional line graphs for the preferences data.

Toggles

• Yes, true shows the enable toggle and false the disable toggles. I also added this in the info text of the chart (hover the i in the top left corner). But sure renaming would be better, I will look for a way to do this.

I linked the follow-up ticket both ways.

Thanks!

Because the extension CodeEditor only is being used for pages where none of our editor types are used and this way there should not be a conflict with CodeMirror we removed the metric about the usage of CodeEditor .

Some late review of the Grafana board,

• Needs normalized numbers (separate ticket). Some of the board is not usable without this, for example the graphs of how many users have enabled wikEd and syntax highlighting are basically just comparing the relative sizes of the wikis.
• Toggle counts are per day? It's hard to tell, the info text says "total".
• "Activation toggles by editor type" doesn't really make sense, now that we see there are no toggles for the 2017 editor. Maybe we can get rid of that panel?
• The toggle graphs with + and - look good, this presentation works for me.
• "User sessions" panels should say "User sessions with syntax highlighting enabled", or something more concise. I would otherwise confuse with the "Total user sessions" panels. In fact, I am confused, apparently "total" is also only counting the sessions with highlighting enabled?
• We should also have a graph of the proportion of user sessions with syntax highlighting enabled over time, so we can see if adoption is increasing.
• Filtering works flawlessly!

I think I may have preemptively closed this because in the sprint review it seemed like everyone was saying it was done. @awight @Lena_WMDE @lilients_WMDE Should we re-open?

In T270265#6791214, @ECohen_WMDE wrote:

I think I may have preemptively closed this because in the sprint review it seemed like everyone was saying it was done. @awight @Lena_WMDE @lilients_WMDE Should we re-open?

All good. If any of my comments are relevant, I think it makes sense to address in a follow-up task.

@awight Thanks for your review.

• I added a note for the needed normalization (and also a warning for sampled data). And also marked all not normalized data with a blue background.
• I added the "per day" info everywhere it makes sense.
• Not sure, if we want to remove the data for the 2017 editor. I guess there might be more data in the future?
• I updated the sessions graphs to "Enabled sessions" to make clear, that they are different from "normal" user sessions.
• I added proportional graphs for the user sessions. Good catch!

Not sure, if we want to remove the data for the 2017 editor. I guess there might be more data in the future?

Yes, please leave in. We want to see if behavior changes in the future as well.