Page MenuHomePhabricator

[SPIKE] Decide how/where we'll store reference check decline responses
Closed, ResolvedPublic

Description

People who decide not to add a reference when Edit Check prompts them to are prompted to offer a reason as to why they made this choice:

Screenshot 2023-07-10 at 3.44.51 PM.png (1×734 px, 92 KB)

This task involves the work with determining how/where the software will log these responses.

Story

As an experienced volunteer who is reviewing an edit wherein they notice someone added new content without including a corresponding reference, I'd like to know A) if the person who made this edit explicitly decided to excluded a reference and B) what caused them to make such a decision so that I can make a judgement call about whether this person was acting in good or bad faith and decide what – if anything – I'll do in response (e.g. Revert, Thank, look for a source myself, write on the person who made the edit's talk page, etc.)

Open question(s)

  • 1. How will we log when people engage with this view in some way (read: when they take an action to explicitly acknowledge that they've decided not to include a reference)? This information will affect how we go about evaluating the impact of Edit Check. Reason: we are currently considering someone explicitly declining to add a reference and also explaining why as a positive outcome.

Done

  • Answers to all Open questions are documented

Event Timeline

The edit summary would seem to be the logical place for this to me, since this is ultimately information about the edit they're making, and the edit summary is where we store meta-information about edits.

There are drawbacks to using the summary -- the editor could remove it, it's limited to 800 characters (on one line), and it's not particularly searchable.

Open question(s)

  1. How will we log when people engage with this view in some way (read: when they take an action to explicitly acknowledge that they've decided not to include a reference)? This information will affect how we go about evaluating the impact of Edit Check. Reason: we are currently considering someone explicitly declining to add a reference and also explaining why as a positive outcome.

We met on Wednesday, 18 July to talk about the above and we converged on the following.

In the near-term...

  • The edit summary will be automatically populated with the reason people provide for why they decided to add a reference when the reference check invites them to do so, as @Sdkb proposed in T341533#9009500. Implementation will happen in T329593.
    • Note: this approach comes with some drawbacks which @DLynch named in T341533#9009599 and I've expanded below. [i]
  • The reason people provide for why they decided to decline adding a reference will also be stored via event logging. Implementation will happen in T324735.

Longer-term, we're not yet clear what approach could supersede this current one.


i. Drawbacks of using edit summary to make decline reason publicly visible:

  • The person making the edit could delete the "decline reason"
  • The person making the edit could alter the "decline reason" in ways that make it unhelpful to the people encountering it in the process of reviewing the edit
  • Seeing an aggregated view of the decline reasons people provide on-wiki will be somewhat laborious
  • The edit summary is limited to 800 characters which could unhelpfully limit peoples' ability to write an edit summary of their choosing while also including decline reasons in a future where someone dismisses multiple invitations Edit Check presents them with.
ppelberg claimed this task.

Thanks for the update! Looking at the listed drawbacks, the first two seem valid, although given that we're already planning to allow the editor to choose the decline reason, having it be editable in the summary isn't really giving them control over anything they weren't already able to control themselves. Regarding the edit summary limit, that could be seen as a feature rather than a bug. We don't want super long edit summaries in anyone's watchlist, and the automatically added decline reason would count toward the optimal limit just as other info would.

Another potential drawback is that autopopulating the summary might lead editors to think that they don't have to add anything else to it, when in actuality they should be added a description/reason for why they added content, not just why they declined to add a reference with it.

Another potential drawback is that autopopulating the summary might lead editors to think that they don't have to add anything else to it, when in actuality they should be added a description/reason for why they added content, not just why they declined to add a reference with it.

@Sdkb: great spot.

We see the drawback you're naming above as meaningful enough for us to deviate from the plan I shared in T341533#9035540...

Rather than using the edit summary to make the decline reason visible to people who are reviewing edits made with edit check, we're instead going to surface the decline response using edit tags.

This will mean that in cases where people decline to add a reference when Edit Check invites them to do so, an edit tag that corresponds to the "decline response" they provided will be appended to said edit. The work to implement this will happen in T329593.

We think this revised approach will:

  1. Make it relatively easy and intuitive for volunteers reviewing edits made when Edit Check was activated to find the reason why someone declined to add a reference when prompted. Non-hidden edit tags are exposed by default on history pages, just as edit summaries are.
  2. Avert the risk of newcomers perceiving the edit summary as something they need not pay attention to because it's usually populated for them.

Note: the Editing Team continues to think that a future wherein A) multiple reference checks can be activated within a single edit or B) multiple checks of different types can be activated within a single edit will necessitate a more scalable approach. An initial investigation into what a "more scalable approach" could look like will happen in T342460.