I've completed the Tone Check AB test analysis results. Please see a summary of some key findings below. Additional metrics and data results are available in the full report.

The instrumentation spec looks good to me and I agree with logging the visible event for both checks and suggestions.

@ppelberg
I've completed investigating the decrease in edit completion rate we observed for logged-in users in the Mobile Web section editing dead end experiment. See key findings and results below.

@Kgraessle @DMburugu - Have the events needed for this dashboard been instrumented? Or is that still in progress?
Once the instrumentation is complete, I can start to set up the dashboard with the metrics identified in the task description.

Quick clarifying question: The user criteria specified in T402647 states:

To be opted in to the experiment, registered accounts must have made 100 edits. We have decided not to include an upper limit on edit count or tenure.

Source of Discrepancy
The high constructive edit rates currently reported in the section editing dead-end experiment appear to be due to the LEFT JOIN from t_cohort to t_experiment.

@Samwalton9-WMF Some initial guidance about sample size needed below:

Do we have a new end date in mind?

Thanks for checking on this @Trizek-WMF.

For the purposes of WE1.1 KR scoring, a 20% increase in revert rate on mobile web means an x% decrease in constructive edit rate on mobile web, right? So what would x be in this case?

@ppelberg
I've confirmed these events are logging in VisualEditorFeatureUse as expected. See summary of checks and some initial data below:

@mpopov ok I pushed some proposed changes to the MR today. The MR should now include all the metrics needed for testing but let me know if you have any questions/suggested changes.

Sure! No problem. I'll plan to make the needed additions and updates and push changes to the MR on Monday.

I reviewed and added some comments to the current editing metrics MR, primarily around ensuring that we have constructive edit and completion rate metrics limited to the following conditions for the section editing experiment:

• registered users with 100 or fewer edits and unregistered users (the target audience for the experiment)
• main page namespace edits
• section-initiated
• mobile web edits

I've completed an analysis reviewing the impact of Paste Check based on the results of AB test collected from 9 October 2025 through 28 November 2025. See summary of results below and the full report for additional details on methodology and metrics by various dimensions (including number of checks shown, platform, user experience, partner Wiki).

I think the goal should be to differentiate where we believe the user experience is different enough to potentially impact the identified metrics (e.g. edit abandonment rate).

Per discussions with @mpopov , documenting the next steps for automated analytics:

• I'll review the definitions drafted by @mpopov and documented in this patch and provide comments.
• Once all definitions are specified and the experiment begins, Mikhail will help verify that the metrics are working as expected using experiment data.
• Mikhail will then merge the merge requests to officially add the definitions to the metrics catalog for automated analytics
• We can then register the experiment for automated analytics. Mikhail or I can register.

@ppelberg: a couple of clarifying questions to help inform bucketing requirements:

It's not technically mobile-exclusive -- it's anywhere that this section editing mode is enabled. Which is everywhere on mobile, and also wikitech and officewiki.

I'm wondering if we should add mobile to the feature name to help clarify that this event does not occur on desktop. Maybe feature = section-switch-mobile?

Confirming that the defined Editing metrics in the task description look good to me.

Next Step: We will also review metrics related to suppressed Paste Checks (pastes that would have caused Paste Check had the text not been copied from a known source) once we obtain sufficient events from the instrumentation added in T407302. This will be used to understand how many edits Paste Check is being suppressed from showing within and how often these types of edits are reverted and may be problematic.

@ppelberg I've verified the buckets are balanced and confirmed that AB test data is logging as expected based on the bucketing requirements defined in the task description.

@ppelberg I've completed an analysis of Paste Check Leading indicators as described in the task description.

I've re-run the analysis with the following criteria:

@KStoller-WMF
I completed an analysis reviewing changes in constructive edits on English Wikipedia for ~1 month before and ~1 month after the September 3, 2025 100% rollout of the Add a Link structured task.

I've confirmed that Paste Check AB test data is logging as expected based on the bucketing requirements defined in T405422.

Summarizing some initial results from an analysis of Paste Check eligible events logged to date:

I've confirmed that all paste check events identified in the task description are logging as expected in VisualEditorFeatureUse.

I've verified this event (feature: editCheck-paste, action: relevant-paste) is logging as expected in VisualEditorFeatureUse.

The above is a quote from the report. Might you have intended to say, "...(125 of 1,377 published edits in the test group)..." ?

I've completed an initial analysis of Leading Indicators 1-5. See the summary of some key findings below and full report for details on other breakdowns and methodology.

@Samwalton9-WMF
I reviewed the number of distinct editors that met the following criteria at id.wiki, tr.wiki, simple.wiki, bn.wiki, az.wiki.:

• Made at least one edit in the last 4 weeks. Note: In this analysis, this included any edits completed from 1 August through 29 August 2025.
• Have made 100 - 1000 edits
• Registered their account no more than 2 years ago.

We set a target of 4% relative increase in mobile web constructive edits based on the analysis summarized in T389403#10960480. This was documented on the Annual Plan page.