Last week I made this patch: https://gerrit.wikimedia.org/r/c/VisualEditor/VisualEditor/+/1125258

It logs to VisualEditorFeatureUse with an action of tool-used and toolbar-group-active, and the feature as the name of the toolGroup or the command associated with the tool.

The one quirk it has is that you'll see notice a fair bit, because that's the one that's automatically activated when someone edits an article which has page notices.

@DLynch - Can you clarify again when these new events would fire/what they mean?

@MNeisler Sure!

tool-used would fire anytime someone interacted (by keyboard or mouse) with a tool in a toolbar. A tool is a thing in a toolbar that runs a command when you use it. These will mostly already be triggering other events (the window-open-from-tool set, or various other command-specific variants. The benefit of this event existing is that there were a few tools that didn't trigger any other event (undo/redo), and you don't need to build some big and comprehensive list of all the tool-related events to track usage.

toolbar-group-active would fire anytime a toolbar group became active, meaning that it was toggled to show its contents. A toolbar group is a container for a number of tools-or-other-things. In general, if interacting with something in a toolbar triggers a menu popup to appear, it's a toolbar group. (I said "other things" because notices is a toolbar group, and all it shows is the notices popup which doesn't contain any tools you could use.) This wasn't currently trackable, unless you took the aforementioned tool usage events as a proxy indicating that the group must have been activated -- and that would't get you the people who opened a menu and then closed it without using anything.

Thanks @DLynch!

you don't need to build some big and comprehensive list of all the tool-related events to track usage.

^ This change will make the queries much more straightforward.

Just one clarifying question:

If someone clicks the new proposed + menu on the Mobile VE toolbar, it seems like the toolbar-group-active would fire. What feature would be associated with this large tool group.

@MNeisler Based on the implementing patch it looks like it should be insert. Same as the equivalent menu on desktop, just different contents.

I've confirmed that the instrumentation implemented in this task will meet the requirements needed to evaluate the use of the Mobile VE tool use menu and the new tools we're adding within it.

I also updated the VEFU Data dictionary to document these new actions.

cc @ppelberg

Change #1125258 had a related patch set uploaded (by DLynch; author: DLynch):

[VisualEditor/VisualEditor@master] Instrument tool usage

https://gerrit.wikimedia.org/r/1125258

Change #1125258 merged by jenkins-bot:

[VisualEditor/VisualEditor@master] Instrument tool usage

https://gerrit.wikimedia.org/r/1125258

Change #1130996 had a related patch set uploaded (by Esanders; author: Esanders):

[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (3ca28e0da)

https://gerrit.wikimedia.org/r/1130996

Change #1130996 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (3ca28e0da)

https://gerrit.wikimedia.org/r/1130996

I checked the events for all the tools on both desktop and mobile:

1. For special characters on desktop action:tool-used is getting fired twice, both when opening it and also when closing it (along with action:dialog-abort).
2. Is it a correct behavior that, for special characters the action associated is tool-used instead of toolbar-group-active. Even though, for Special Characters it opens up a container for a number of tools-or-other-things such as different characters?

That's all technically-correct.

The way special-character works is that the "tool" in the toolbar is actually just running a command that toggles a different toolbar on and off. It looks very similar to the "I clicked a button and a popup menu appeared" situation of e.g. the "insert" group, but it actually works completely differently. (The find/replace dialog uses the same system, but for whatever reason we have its tool only ever open it, rather than letting it toggle. 🤷🏻)

Thanks @DLynch for clarifying the two issues mentioned. I think we can close this task now.