I've been thinking about the feedback that we've received on the draft API proposal thus far. AIUI the main sticking point has been about ClickThroughRateInstrument. The feedback about it has been:
- It's atypical: Instrument developers either embed instruments in their feature code or they embed hooks in their feature code which instruments embedded in other modules can listen to. ClickThroughRateInstrument takes a different take altogether, listening to DOM events directly. Kosta called this "outside in" instrumenting (paraphrasing) whereas instrument developers typically take an "inside out" approach
- It's (currently) incompatible with Vue apps: ClickThroughRateInstrument requires a stable DOM element to function. While Vue does an excellent job at maintaining DOM elements during the mount/patch render phase, there's no guarantee that a DOM element is stable during the lifetime of the Vue app
So how do we reconcile The Vision™ – pre-canned, specified, well-documented instruments that can be used as Product Health Metrics as well as primary, secondary, and guardrail metrics – with the status quo? We offer the components of the instrument as well as the instrument itself.
An instrument has three responsibilities:
- Manage state
- Create analytics events
- Respond to UX events
We can offer a low-level component for each of the above that instrument developers can use as well as a high-, and mid-level components that uses those components.
Example 1
Consider a feature that shows a dialog with two buttons labelled "Accept" and "Reject." The experiment owner wants to run an experiment on the dialog and wants to monitor the click-through rate of the buttons.
// NOTE: Object and method names are all subject to intense bikeshedding const ACCEPT = { friendly_name: E1_FRIENDLY_NAME, selector: E1_SELECTOR }; const DECLINE = { friendly_name: E2_FRIENDLY_NAME, selector: E2_SELECTOR }; const eventStream = mw.xLab.newEventStream( STREAM_ID, SCHEMA_ID ); // High Level // ========== const { Instrument } = require( 'ext.wikimediaEvents.xLab' ).ClickThroughRate; Instrument.start( eventStream, ACCEPT, DECLINE ); // Or... eventStream.include( Instrument, ACCEPT, DECLINE ); // Mid Level // ========= const { EventSubmitter } = require( 'ext.wikimediaEvents.xLab' ).ClickThroughRate; const eventSubmitter = new EventSubmitter( eventStream ); // When the feature code shows the dialog: eventSubmitter.submitImpression( ACCEPT, DECLINE /*, ... */ ); // When the feature code detects that the user has clicked the Accept button: eventSubmitter.submitClick( ACCEPT ); // Low Level // ========= const { StateManager, EventFactory } = require( 'ext.wikimediaEvents.xLab' ).ClickThroughRate; let state = StateManager.newState(); // When the feature code shows the dialog: state = StateManager.impression( state, ACCEPT ); eventStream.submitInteraction( ...EventFactory.impression( state, ACCEPT ) ); state = StateManager.impression( state, DECLINE ); eventStream.submitInteraction( ...EventFactory.impression( state, DECLINE ) ); // When the feature code detects that the user has clicked the Accept button: state = StateManager.click( state, ACCEPT ); eventStream.submitInteraction( ...EventFactory.click( state, ACCEPT ) );