1. How will we assume the reference check will evolve to become more accurate and exhaustive over time? E.g. will the reference check get triggered in a relatively narrow set of cases to start and grow to be more expansive over time? Will the reference check get triggered in a broad set of cases to start and grow to be more precise over time? Something else?

DECIDED: To start, we will pursue an approach with Edit Check that minimizes the likelihood of false positives and is implemented in ways (T327959) that empower volunteers, on a per-project basis, to evolve the heuristic (T324730) to become more robust/complex (read: minimize false negatives).

We arrived at the decision above in service of the "Objectives", and in line with the "Assumptions" documented below...

Objectives

1. Increase the likelihood that newcomers and Junior Contributors find the guidance Edit Check is presenting them with, and the editing experience more broadly, to be intuitive and straightforward so that they feel encourage to return to edit again
2. Decrease the likelihood that Edit Check is creating more work for experienced volunteers by prompting newcomers and Junior Contributors to add sources when they are not needed

Assumptions

• Following an approach that minimizes false positives will increase the likelihood that newcomers and Junior Contributors will intuitively see the prompts Edit Check is presenting them with as relevant/applicable to the change they're wanting to make. As a result, we assume these people will be:
  • More likely to trust/consider/engage with the guidance/prompt Edit Check is presenting them with
  • Not be further discouraged from returning to edit again because they found the experience to be straightforward
• Newcomers and Junior Contributors are going to be less likely to detect false positives and therefore, more likely to become confused by them (read: Edit Check being triggered unnecessarily) [i]
• Experienced volunteers will be more motivated to improve upon/contribute to a simple system that works in expected ways most of the time rather than a complex/robust system that works some of the time.
  • See: Gall's Law: "A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system."
• The Editing Team, in partnership with volunteers, will feel empowered to make Edit Check more robust over time (and therefore minimize false negatives) because doing so will feel relatively fast and lightweight. Where "fast and lightweight" in this context mean the Edit Check heuristic (T324730) will be:
  • Represented in natural language (as well as in code) to increase the range of volunteers who can audit and iterate upon the rules that comprise it
  • Configurable on a per-project basis so that volunteers can ensure Edit Check conforms with local conventions and policies (T327563) without needing consider how these decisions could impact other projects
  • Auditable on a per-project basis (T327959) so that volunteers can evaluate how Edit Check is performing "in production" (T324733) and adapt the heuristics in ways that increase the likelihood that it is causing people who are new to accompany the new content they are attempting to add with a reference (T325713).

i. via User:Phlsph7 at en.wiki