Page MenuHomePhabricator

Prompt people to contextualize why they are declining to reference new content they are adding
Closed, ResolvedPublic

Description

The initial reference check user experience we've been defining in T325711 includes an option for people to forego adding a reference to the content they are attempting to add:

image.png (782×440 px, 62 KB)

This task involves the work with finalizing the responses people will be presented within the moment immediately after they decline to add a reference.

NOTE: the work involved with deciding if and how the reason people who decline to add a reference select is shared publicly is happening in T341533.

Stories

  • As a newcomer/Junior Contributor who has chosen to forego adding a reference to the content they are planning to publish to Wikipedia, I would like to know what information/content I can offer to the experience volunteers who are likely to review/patrol this soon-to-be published edit, so that I can increase the likelihood that the contribution I consider to be useful and worthwhile remains on the wiki.
  • As an experienced volunteer who is reviewing an edit in which someone has a) added new content and b) decided NOT to verify that new content with a reference, I would like to know why this person made this decision so that I can decide what – if any – action (e.g. revert the edit, post a message on the person who made the edit's talk page, etc.) to take in response

Requirements

In-edit decline responses

  • People who decide not to add a reference when Edit Check prompts them to consider doing so ought to see a dialog that asks them to articulate "why" they've made this decision.
    • This dialog ought to contain a question that reads: Why are you not adding a citation?
    • This dialog ought to contain the following choices for people to select from:
Decline responseEdit tag nameQQQ
I'm not sure what citation to add.editcheck-reference-decline-uncertainA way for people to express they do not know what citation to add.
The information I'm adding is widely known.editcheck-reference-decline-common-knowledgeA way for people to indicate how they came to know the information they're adding. [ii]
I don't think citations are relevant to the change I'm making.editcheck-reference-decline-irrelevantA way for people to indicate that they think the Edit Check prompt appeared in error. [iii]
Othereditcheck-reference-decline-otherA catch-all response people can provide if none of the other options presented reflect why they decided not to add a reference.
  • People need to be made aware that the response they give will be public for other people to see and how other editors are likely to use the response people provide

Showing decline response post-edit

  • The response people provide by way of the view this ticket is asking us to introduce will cause an edit tag to be appended to said edit. This edit tag should:
    • Be visible in history pages by default. Read: tag should not be hidden.
    • Correspond to the specific decline response someone provided
    • Be linked to mw:Edit check/Tags so that people who are interested can learn more about what the tag they're encountering means
  • The edit tags should be named as is described in the cart above. See column titled Edit tag name.

Open questions

  • 1. What additional information/context might Senior Contributors find helpful in determining what action to take in response to someone who published an edit that they explicitly declined to accompany with a reference?
  • 2. How will we log when people engage with this view in some way (read: when they take an action to explicitly acknowledge that they've decided not to include a reference)? This information will affect how we go about evaluating the impact of Edit Check. Reason: we are currently considering someone explicitly declining to add a reference and also explaining why as a positive outcome.
    • We'll address this question in T341533
  • 3. To what extent will the decline responses this ticket introduces be configurable on-wiki, on a per project basis? [i]

Per the Editing Team's 9 August offline discussion, to start, projects will be able to configure the decline responses using the existing Translatewiki interface/workflow. If this level of configurability proves insufficient, we'll consider a more "robust"/"expressive" approach as part of T327959.

Done

  • Answers to all open questions are documented
  • Requirements are implemented
  • mw:Edit check/Tags is updated to document the tags added as part of this task

Loose

  • Per what @Whatamidoing-WMF raised in the Needs Discussion meeting the Editing Team convened on 8 February, we came to agree that the moment/experience this task is asking us to design ought to feel necessary and important. Said another way: ideally, people would not experience this moment like they do other standard feedback forms that sometimes accompany new features.

References


i. See: https://www.mediawiki.org/wiki/Talk:Edit_check#c-Joe_Roe-20230630144700-Feedback:_Joe_Roe
ii. See T329593#8635552
iii. See T324735#8847900

Related Objects

Event Timeline

[new exploration] more conversation-like:

Dismiss_Discussion-exploration.png (1×1 px, 252 KB)

What additional information/context might you find helpful in assessing the quality of an edit and the intentions of the person making it so that you can decide how best to respond to it/them?

Well, to assess the quality of the edit, all I really need is the edit itself, plus access to the source it uses. From there, I can look up the reliability of the source (via WP:RSN) or judge it for myself and check whether the information added is actually in the source (good sources with e.g. page numbers are helpful here, so it's notable that VE doesn't facilitate those yet per T216817). I can look at the nearby content of the article and the talk page to see if there's any prior consensus in the area (e.g. an RfC that decided not to include this piece of info).

For the intentions of the person making it, I certainly want to know if they have any paid or unpaid conflict of interest. I want to see their edit history and user talk page to help determine if they're a WP:SPA (single-purpose account) or a point-of-view pusher. The operating question is "What made you want to edit this page?"

This context is helpful – thank you for sharing it, @Sdkb.

I'd like to explore this last point you raised about the intentions of the person making the edit in the context of the case of someone explicitly declining to add a reference when edit check prompts them to consider doing so as F36824591 shows...

Specifically, I wonder the following...

If you imagine yourself in the position of someone who:

  1. Is reviewing an edit that Newcomer A made that involves them adding new content without a reference
  2. Knows that Edit Check prompted Newcomer A to consider adding a reference and they explicitly declined to do so

...what – if any – information/rationale do you think Newcomer A could provide about why they made the choice that they did that you could imagine being useful to you?

Note: Based on what you articulated in T327330#8566413, it seems like (among potentially other pieces of information) you would find it helpful to see evidence that this person: A) understands, or made an attempt to understand, why references are generally needed and B) why the guidance in this particular case is difficult for this person to apply.

Ah, that makes sense!

I'll break this down into two sub-scenarios. In the first, it's clear from context why they declined to add the reference. Maybe the edit check system suggested adding one, but both the new user and I can see that it's not needed. Maybe they're inserting their own opinion or a piece of vandalism and know that they're probably going to get reverted anyways. Maybe they're struggling with how to format their reference, so they just dumped a plain URL into the text (which edit check doesn't recognize as a reference but which I can clean up).

In all these scenarios, perhaps the user can provide some info that helps me figure out what's going on slightly faster, but normally the edit itself will speak louder than whatever the user says about it. (I do want to highlight vandalism/other bad faith edits as a use case here. I know you're developing edit check with the goal of assisting good-faith editors, but there are a also a lot of bad-faith editors, and they're going to disproportionally trigger the check, so we want to think about that user flow. If the edit check makes it harder for them to hide what they're doing, or adds friction that discourages them, that could be a positive.)

In the second sub-scenario, it's not clear to me why they declined to provide a reference. This would be where they added something that looks to me like they should have added a reference, but they didn't. In that case, whether or not I revert them depends on a combination of (a) how sensitive/controversial the information is, and (b) my intuition about whether or not the information is true. They can't say much to affect (a), but perhaps some answers to (b) might make me more or less confident that they're acting in good faith and providing true info. The question I would want to ask someone in that situation is, "How do you know the information you are adding?" If they say, "I read it somewhere," then that's an opportunity to ask them again to please provide the source. If they can't remember it, that's plausibly good faith, and ideally we'd like them to find it, but if they can't we can try to find it or another source ourselves later. If they say, "It's from my own personal experience," then it's possibly original research. If they say, "It's obvious," then I'll want to check to see if it's truly WP:BLUESKY.

From the questions in the mockup, I notice that one is "citation is not needed here". This assumes that the new editor knows when citations are needed or not needed, which is a...bad assumption. I think it might be better to have something like "I did not add new information" and "The information I added does not need a citation". If the user selects the second option, the software could return something like "Wikipedia requires encyclopedic content to be verifiable through citations to reliable sources. Content that is not cited is often reverted. Would you like to add a citation? [Yes] [No]".

I'm not terribly sure where to chime in here, I hope this is appropriate.
I'm from SSA (now emigrated) and have encountered many issues along the way that I know have discouraged many. One of those is the issue of "Own research". I have read over a lot of the discussion around the Edit Assist, and everywhere I see the assumption that the Wp policies are graven in stone and all that needs to happen is to educate users.
Then I saw this question about asking users WHY they decline to reference. While I understand the probable motivation, for a lot of users being confronted with this for the first time, they will see that as seriously passive-aggressive and hostile. I'm hopeful that the further discussions on how to phrase the question will address this.
The real issue for me is this concept on WP that "if it's not referenced ''to our standards'' then it doesn't belong". Then I see that part of the stated intention of this project is to "fill in the gaps" from people's knowledge. Which is true? Does something only get onto WP if it has been '''widely''' reported on in the media or academic journals? That clashes somewhat with the stated intention. Is there not a way of allowing actual local user knowledge? As an example, I deployed to the DRC with the Force Intervention Brigade. I spent a year there and was intimately involved up to the Brigade level, and a little at the Division level. I saw ALL the documentation of all the activities inter alia combat, planning, staff meetings at the highest levels, interaction with the UN dignitaries, Post Battle damage assessments, Pre-battle plans, Indirect fire plans, Air Movement orders and debriefs, post combat debriefs and a plethora of other such stuff. I have an archive of a lot of that documentation on my computer. NONE of it is allowed to influence anything I write on WP! I cannot even challenge false information which is reliably resourced (to WP standards) but is absolutely false. I am very good friends with the guy who made the 212m Sniper kill shot outside Munigi. I have the medal citation, the post-battle debrief document etc. Him and I just laugh at the inaccuracies of the accounts of what happened which we cannot (may not?) correct.

I hope this contributes somehow to the group-think on this topic.

Re: the tags, I just want to make sure we call out in the task that this is going to cause us some amount of technical/design debt that'll need to be addressed as soon as we have multiple edit checks available in a single edit. (Because "someone declined an edit check" is a non-specific signal that won't help reviewers if the edit might have needed multiple citations, or if there was a citation and a questionable link and a npov issue.)

First off, thank you for being patient with me. I know it's taken me a while to get back to you, @BoonDock.

With this in mind, responses to the points you raised below. Please know: I recognize the follow up questions I'm posing may be difficult to engage with considering the time that's elapsed between when now and when you first commented...

I have read over a lot of the discussion around the Edit Assist, and everywhere I see the assumption that the Wp policies are graven in stone and all that needs to happen is to educate users.

Can you please say a bit more about this? What have you seen/read that is contributing you to perceiving Edit Check as perpetuating the idea that Wikipedia has firm, inflexible rules that must be followed in all cases?

...I ask this because a core design principle this project is attempting to embody is the pillar of No firm rules, as documented here.

So, if there are facets of the experience that you think are not fulfilling this intention, we'd value knowing so that we can consider improving them!

Then I saw this question about asking users WHY they decline to reference. While I understand the probable motivation, for a lot of users being confronted with this for the first time, they will see that as seriously passive-aggressive and hostile. I'm hopeful that the further discussions on how to phrase the question will address this.

People experiencing the prompt that invites them to share why they decided not to add a reference as hostile is definitely not desirable!

Can you please say a bit more about how you personally experienced the prompt? What did you think/feel upon seeing it?

The real issue for me is this concept on WP that "if it's not referenced ''to our standards'' then it doesn't belong". Then I see that part of the stated intention of this project is to "fill in the gaps" from people's knowledge. Which is true? Does something only get onto WP if it has been '''widely''' reported on in the media or academic journals? That clashes somewhat with the stated intention. Is there not a way of allowing actual local user knowledge.

I personally agree with you in thinking that if we see Wikipedia rules as fixed and apply them without considering the broader context a specific edit exists within, we are likely to continue perpetuating the kinds of systemic biases that contribute to the knowledge gaps that exist on Wikipedia.

With the above said, the Edit Check project, if done well, is meant to:

  1. Increase peoples' awareness of the policies that exist and will influence how the contributions they're making are received + reviewed
  2. Reinforce the no firm rules pillar (as I noted above)

Said in another way: evolving policy may very well be needed to address the knowledge gaps you're referring to and the Editing Team sees Edit Check's role in this broader conversation as making more people aware of and clear about the policies as they currently exist.

See more in: Explicitness is Important.

I'm not terribly sure where to chime in here, I hope this is appropriate.

I'm glad you decided to act on your instinct to share this feedback with us!

While we welcome constructive feedback wherever/however it happens to arrive, in the future I think feedback of this sort is best suited for discussion on the Mediawiki project page (mw:Edit check) as we tend to use Phabricator (this space) for more technical discussions about how a given piece of functionality will/could be implemented...

@ppelberg the above responses look great to me with the single caveat that "common knowledge" might be a) paradoxically not universal and b) hard to translate into other languages (it would be slightly wordy in French for instance: "de notoriété publique"). In light of that, what do you think of changing that to "widely known" / "commonly accepted" or something else along those lines?

@ppelberg the above responses look great to me with the single caveat that "common knowledge" might be a) paradoxically not universal and b) hard to translate into other languages (it would be slightly wordy in French for instance: "de notoriété publique"). In light of that, what do you think of changing that to "widely known" / "commonly accepted" or something else along those lines?

Great spot, @nayoub – let's try "widely known." I've updated the task description to reflect this adjustment.

Change 952229 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Update edit check decline reasons and record tags for them

https://gerrit.wikimedia.org/r/952229

This probably needs some iteration over what the tags should be called. Right now they're all in that patch as NOREF: <short label>

image.png (142×1 px, 58 KB)

Change 952229 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Update edit check decline reasons and record tags for them

https://gerrit.wikimedia.org/r/952229

Change 956476 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Update edit check tag labels

https://gerrit.wikimedia.org/r/956476

Change 956476 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Update edit check tag labels

https://gerrit.wikimedia.org/r/956476