Page MenuHomePhabricator

Add instrumentation for Wikifuntions VE events
Closed, ResolvedPublic

Description

This task involves the work of adding instrumentation to visualeditorfeatureuse

The need for this instrumentation emerged in response to the Abstract Wikipedia Team to check the usage of search and other features in Wikifunctions in VisualEditor.

Requirements

  • Add the following events to track
Event to be trackedevent.featureevent.action
The user changes or starts the search queryWikifunctionsCallsearch-change-query
The user selects an function from the search resultsWikifunctionsCallsearch-choose-function
The user changes or starts the query in previewWikifunctionsCallpreview-change-query
The user clicks the link to Wikifunctions while reading a function outputWikifunctionsCallfunction-link-click

Notes and Useful Links

Event Timeline

cchen triaged this task as Medium priority.Aug 23 2025, 8:14 AM
cchen added a project: Abstract Wikipedia team.
cchen updated the task description. (Show Details)
cchen renamed this task from Add instrumentation for Wikifuntions VE searches to Add instrumentation for Wikifuntions VE events.Aug 23 2025, 8:44 AM
cchen updated the task description. (Show Details)
DSmit-WMF changed the task status from Open to In Progress.Sep 9 2025, 8:47 AM
DSmit-WMF claimed this task.

I am a little bit unsure what variables to use
I see in our current code base we use 'mw.eventLog.submitInteraction', but these variables are for wikifunctions ui.

mw.eventLog.submitInteraction(
    'mediawiki.product_metrics.wikifunctions_ui',
    '/analytics/mediawiki/product_metrics/wikilambda/ui_actions/1.0.0',
    actionParam,
    this.removeNullUndefined( interactionData ) );

What would the variables be for the events mentioned above?
params are: streamName, schemaID, action, interactionData

The questions are listed below as 1-4.

mw.eventLog.submitInteraction(
    // 1. streamName: what should we use here? 'eventlogging_VisualEditorFeatureUse'?
    'mediawiki.product_metrics.wikifunctions_ui', 

    // 2. schemaID: what should we use here? '/analytics/legacy/visualeditorfeatureuse/1.2.0'? or 1.0.0?
    '/analytics/mediawiki/product_metrics/wikilambda/ui_actions/1.0.0', 

    // 3. action: assuming this is the action from the table, eg; 'search-change-query' etc
    action,        
    
    // 4. interactionData: What to send here? where do we add the event.feature 'WikifunctionsCall'?
    // interactionData

Or can we just use this?

mw.eventLog.submit( 'visualeditorfeatureuse', {
    feature: 'WikifunctionsCall',
    action: 'search-change-query' // any action
} );

Also it mentions in the ticket to update the schema: 'Update visualeditorfeatureuse schema with new values'.
But I am wondering what to update in the yaml? should I just bump the version? should I just add some documentation? (see below)

//latest.yaml:
[..]
 feature:
            description: >-
              The user-facing capability that is being used, such as 'link',
              'autosave', 'clipboard' or 'textStyle/bold'. This is not an enum
              so that the visual editor's maintainers have flexibility in which
              of its many features to implement and because extensions can add
              arbitrary new features which could pass through
              already-instrumented code paths.

              New values (T402711):
              - WikifunctionsCall
            type: string
          action:
            description: >-
              The specific, technical action that has been trigged, written as a
              hyphen-separated phrase consisting of the object and the action
              taken. For example, 'dialog-open' or
              'annotation-toggle-selection'. This is not an enum so that the
              visual editor's maintainers have flexibility in which of its many
              features to implement and because extensions can add arbitrary new
              features which could pass through already-instrumented code paths.

              New values (T402711):
              - search-change-query
              - search-choose-function
              - preview-change-query
              - function-link-click

Maybe this is a super common task, but new to this its a bit unclear 😅

Last questions:

  1. do we not want to track what the user exactly searches for, so the search query itself send as data to the event? or which function the user selects? which preview the user starts? which links the user clicks?
  1. 'the user starts the query in preview' -> what is exactly ment by this? when the preview is loaded for a function when the user is filling in the input fields for function? or something else? (see below)

Screenshot 2025-09-09 at 12.32.03.png (676×1 px, 58 KB)

Change #1186468 had a related patch set uploaded (by Daphne Smit; author: Daphne Smit):

[mediawiki/extensions/WikiLambda@master] VisualEditor: Add measurement utilities

https://gerrit.wikimedia.org/r/1186468

Or can we just use this?

All the actual logging is handled by the WikimediaEvents extension. You shouldn't use the mw.eventLog methods directly, because there's a lot of adjustments made to the data (adding session IDs, etc) before the logging happens.

Instead you should either do:

ve.track('activity.WikifunctionsCall', {
    action: 'search-change-query' // any action
} );

or

mw.track('visualEditorFeatureUse', {
    feature: 'WikifunctionsCall',
    action: 'search-change-query' // any action
} );

I know nothing at all about how you're using this, but make sure that you're within an editAttemptStep session (i.e. VisualEditor or some other editor is active) when you call those, or nothing will get logged.

do we not want to track what the user exactly searches for, so the search query itself send as data to the event? or which function the user selects? which preview the user starts? which links the user clicks?

VisualEditorFeatureUse is intended for broad feature-usage checks, not any sort of user data, so logging the search query itself would be outside what it's expected to do. It's for "the user searched, and selected something", not "the user searched for FOO and selected BAR".

Also it mentions in the ticket to update the schema: 'Update visualeditorfeatureuse schema with new values'.

You don't need to change anything in that schema -- it's an extremely open-ended one that's not expected to enumerate allowed values, because it just has all sorts of VE internal names piped directly into it. The closest to a list of possible values you might want to update is https://www.mediawiki.org/wiki/VisualEditor/FeatureUse_data_dictionary but that's not mandatory.

This helps a lot. Thank you so much!

  • I didnt know of the existence of ve.track() so I will use that.
  • The events will be fired from a Vue app rendered inside a ve.ui.WikifunctionsCallDialog or ve.ui.WikifunctionsCallContextItem when the user is editing an article. so this should be fine.
  • VisualEditorFeatureUse usage is clear now
  • seems useful to update that page dictionary indeed

Change #1186468 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] VisualEditor: Add measurement utilities for visual editor

https://gerrit.wikimedia.org/r/1186468

Instrumentation has landed and will ship to production in the wmf.20 train next week.

Tested and confirmed that all the events are logged in visualeditorfeatureuse schema.

Tested and confirmed that all the events are logged in visualeditorfeatureuse schema.

Excellent, thank you!