Documenting some notes to help answer the open questions and goals described in the task based on my review of available resources.

I've finished retrieving all citation data that accompanied new content edits made in the last 90 days. Per @Isaac's suggestion, I modified the notebook and resulting dataset to retain full URLs in addition to the URL. This helped provide more specific source type details for urls where a Citoid response was available.

I've completed an initial draft plan. Next steps are to share with the team for review and comment.

Thank you @Isaac! This was very useful. I was able to modify the notebook to include all the identified focus wikis and some additional parameters from mediawiki. I also expanded the notebook to include some earlier analysis I did to retrieve metadata on the source type and source title where available from the Citoid API.

Note: This is currently blocked on resolution of T349121

Are these numbers for all projects and languages combined, minus those that use ENOM?

@AlexisJazz Yes, that's correct

@ppelberg I've checked and confirmed that events are being logged as expected in VisualEditorFeatureUse. See summary of checks and some initial data counts below:

While working on these updates, I found that the total number of publish events was higher than the number of attempts started for both edits of existing implementation and edits of existing tests. The difference is especially high for edit existing test events.

Thanks @DLynch. That helps clarify.

Suggested approach:

1. Use Mediawiki action API to retrieve a random list of articles from identified wikipedias (including page id and title). The sample pull with be limited to articles in the main namespace (ns = 0).
2. Query Mediawiki action APIL again to retrieve article data including URL and article content.
3. Query Citoid APl to find citation metadata for the list of URLs found on the previous step.
4. Parse content to retrieve citations and associated metadata within the article
5. Use the list of revisions ids to identify which ones that meet the following requirements:
   • Made with Visual Editor
   • Identified as adding new content (editcheck-newcontent ) that include a reference (editcheck-newreference)
   • Published within the last 90 days

Reopening as there was an error with the initial patch when computing sanitization

Updated the data dictionary with new actions and features added to track engagement with the first iteration of Edit Check.

@ppelberg and @DLynch I've completed a QA of the aggregate data logged since deployment of the edit check at partner wikis on 11 October in T347908. All data appears to be logging as expected. See a summary of checks and some initial data below. Let me know if you have any questions or if any checks would be helpful.

Resolving this as topic subscriptions has been instrumented and the analysis this work refers to was completed.

Updated the task description with some potential approaches.

@ppelberg - Please see the analysis results summarized below and let me know if you have any questions or if there is any additional info that would be helpful in understanding when to show the fallback prompt.

@mforns - I've added you as a reviewer on this patch to add wikifunctions_ui to the allowlist. Here's a link to the current documentation about the wikifunctions_ui custom fields. Please let me know if you have any questions or suggested revisions. Thank you!

I've completed my review of 3 schemas (EditAttemptStep, VisualEditorFeatureUse, and wikifunctions_ui ) against the proposed event types and updated the spreadsheet with my feedback and comments.

Going to mark this resolved as the dashboard now includes charts to reflect all primary metrics. I will continue to monitor the dashboard and make refinements to address for any load time issues and visualize more time series data as well as collect more data. I will create a separate ticket if any more significant updates are needed.

I've completed the additional updates described in T343015#9196124 to the dashboard.

The initial version of the wikifunctions dashboard has been completed including the addition of data from wmf_product.active_editors and additional launch metrics.

Scheduled meeting on 27-September to discuss next steps

@DMartin-WMF
Thank you for clarifying. I'm going to go ahead and resolve this ticket as I've confirmed all instrumented events are logging as expected and don't see any further data discrepancies.

I'm currently working on updating the task description with proposed bucketing requirements:

Here are two proposed events to be logged in VisualEditorFeatureUse to track engagement with this feature

As a temporary fix, I've adjusted the default time filter to 2022-01-01 <= col < 2023-09-01. This will allow all the time series charts to load correctly and provide details on recent trends. However, as mentioned in T346636#9179412, the error will still reappear for these charts if the time range is extended to display larger time ranges.

Some additional information on the error that's appearing for all line charts: