@ppelberg and @DLynch I've completed a QA of the aggregate data logged since deployment of the edit check at partner wikis on 11 October in T347908. All data appears to be logging as expected. See a summary of checks and some initial data below. Let me know if you have any questions or if any checks would be helpful.

I just had a few clarifying questions noted below.

• I noticed that we started logging edit check events prior to the 11-Oct deployment. We recorded about 32 editing sessions with edit check events from 14 September through 10 October. Were these instances where users enabled Edit Check via the URL parameter?
• Some editing sessions have multiple context-show events with different timestamps. I'm trying to understand when this would occur. Could this happen if a user selects "<" in the citoid window or rejection reason window and sees the context item check again?

Summary of checks and initial results:

• Edit checks have been shown at 36 distinct editing sessions since the deployment at the initial partner wikis on 11-October.
• Edit checks have been triggered at 4 of the 9 partner wikis. See table of number of edit check sessions by wiki below:

Number of editing sessions where edit check was shown by wiki since 11 Oct

• All actions are recorded with the expected feature in VEFU and numbers appear as expected. For example, there are an equal number of edit-check-reject events and subsequent rejection reason window events (action = window-open-from-check feature = editCheckReferencesInspector).

Number of events and editing sessions by edit check event type since 11 Oct

• Multiple edit check events per session ✅. Confirmed that there are multiple edit check events associated with each session where edit check was triggered (e.g. context-show, edit-check-confirm, etc). This is expected since we track events throughout the edit check workflow and edit-check dialogs are non-dismissable. The majority of these sessions have at least 5 edit check events with some having as high as 15 edit check events within a single session.

• Order of workflow checks: ✅
  • Confirmed that all events within a session appear in the expected order based on recorded timestamps for each event. The context-show is the first event logged as expected for all these sessions with subsequent confirm and reject workflow events appearing within the expected order
  • Confirmed citoid window events are logged following the edit-check-confirm event so we can track user's interaction with the citoid feature.
• Edit check has been shown in 35 editing sessions by 8 distinct registered users and 1 editing session by an unregistered user ✅
• Edit check events have only been logged for users with <100 cumulative edits ✅
• All edits check events only occur where editor_interace = 'visualeditor'
• Edit check events logged on both desktop and mobile platforms

Number of edit checks shown by platform since 11 Oct

Updated the data dictionary with new actions and features added to track engagement with the first iteration of Edit Check.

@MNeisler:

I noticed that we started logging edit check events prior to the 11-Oct deployment. We recorded about 32 editing sessions with edit check events from 14 September through 10 October. Were these instances where users enabled Edit Check via the URL parameter?

Anyone who had enabled it would still be generating events regardless of how they accessed it. This was probably us testing it, or people in the community who we gave the link to so they could try it out before agreeing to enable it.

Some editing sessions have multiple context-show events with different timestamps. I'm trying to understand when this would occur. Could this happen if a user selects "<" in the citoid window or rejection reason window and sees the context item check again?

You get the context-show any time it shows, not just once per session. Backing out of the citoid or rejection dialogs would cause it, as would any other way of leaving the flow (continuing to the "save" dialog then going back to editing from there, or pressing the "<" in the toolbar next to "Before publishing") then returning to it with references still missing.