This task is about enhancing the tool @matmarex built in T260393#6384098 for detecting when edits made with DiscussionTools make changes to the page other than adding content.

Background

A key principle of the Talk pages project (an outcome of the 2019 Talk page consultation) is making it easier and more efficient for people, across experience levels, to communicate with others on Wikipedia. To deliver on this, we are introducing new communication tools...communication tools that are required to be backwards compatible with existing Wikitext talk pages.

Core to this "backwards compatibility," is ensuring these new communication tools do not affect existing content in undesirable ways. The script Bartosz wrote enables us to detect suspicious diffs at scale and thus validate whether these new tool are in fact delivering on the agreement established in the Talk page consultation.

Trouble is, as the Reply Tool and New Discussion Tool is made more widely available, the number of suspicious diffs this script identifies will likely grow beyond a single person's ability to review them, thus this task.

Meta

Objectives

• The Editing Team can more accurately prioritize issues knowing how frequently/infrequently they occur
• The Editing Team can identify issues earlier and decrease the likelihood edits produced by DiscussionTools disrupt editors

Target user(s)

• Members of the Editing-team

Requirements

Meta

• "Script" [i] can be configured to run at a to-be-defined interval without any individual needing to manually start it
• The script's output can be read in the browser without needing to install additional libraries
• The script should look review all edits made with DiscussionTools, regardless of how DiscussionTools have been deployed at that project. Read: the script should identify edits whether they were made on projects where DT are available as opt-out preference, as a Beta Feature or not at all (someone used DT by appending ?dtenable=1 to the URL of a discussion page).

Consuming script's output

• The page should include metadata to help the Editing Team know:
  • How many total DiscussionTool edits have been analyzed to-date
  • Of the DiscussionTool edits the script has "analyzed" to-date, the number of diffs the script has tagged as "suspicious"
• When the table of suspicious edits was last updated (read: when the script was last run)
• The page should contain a list of diffs the script has identified as suspicious
  • Each suspicious diff should have the following values associated with it:
    • The URL of the project on which the diff was made
    • The URL of the diff
    • The time the edit producing the diff was made

Open questions

• Is it necessary to tag suspicious edits as they happen vs. identifying them en masse at run-time, as is currently being done?
• Is it necessary to implement any kind of time out handling?

Done

• All "Open questions" are implemented
• All "Requirements" are met

---

i. "Script" is used here as a general term; it may not be the best way to describe the code that will need to be written to deliver on the required functionality

---

Ideas for the future

• A way to relate diffs to each other to help categorize issues
• A way to triage suspicious diffs. E.g. mark "Archive" or "Investigate"
• A way to "tune" the script such that categories of diffs deemed not to be harmful can be ignored in the future