Page MenuHomePhabricator

Define user experience for prompting people to review and replace unreliable sources
Open, Needs TriagePublic

Description

This task involves the work with converging on what people will experience when they attempt to reference a source that a project has determined to be unreliable while editing using the visual editor. A parallel effort for the 2010 wikitext editor is is happening in T347435.

Where "unreliable" in the scope of this ticket refers to a source that exists on a given project's Special:BlockedExternalDomains page.

Note: we can see a future where Edit Checks presents people with feedback about source beyond those that projects block from being published. Tho, this work will happen in T348060 once, at a minimum, T346849 is resolved. [i]

Story

As a person who is unaware of Wikipedia's reliability policy and who is attempting to reference a source that the project I'm editing has deemed to be unreliable and unfit for publishing on-wiki, I'd value being made aware of this information and presented with an easy and timely way to act on it, so that I can publish the change(s) I'm making and increase the likelihood that they remain on the wiki.

User experience

This section will eventually contain the proposed user experience for, what we're calling, the Reference Reliability Edit Check.

Learning objectives

This ticket is scoped with the goal of helping us to arrive at initial answers to the following questions:

  1. When and how is the feedback shown initially?
  2. What call(s) to action is presented along with that feedback?
  3. When you engage with the call to action, what does the workflow look like?

Open questions

  • 1. What - if anything – will we need to do to ensure the UX this ticket will introduce does, at a minimum, not conflict with what people will see when they attempt to an invalid domain. E.g. google.con
    • Screenshot 2023-10-30 at 12.20.28 PM.png (518×914 px, 87 KB)
  • 2. What default message will people see within Citoid when they attempt to add a source that is listed on MediaWiki:Spam-blacklist and/or MediaWiki:BlockedExternalDomains.json?

Done

  • Answers to all "Open questions" are documented
  • Mockups are added to the "User experience" section above
  • "Requirements" are documented in "User experience" section above

i. Please see T346849#9217888 for more context about how/why the Editing Team has arrived at this point of view.

Related Objects

Event Timeline

ppelberg updated the task description. (Show Details)

Per what we discussed during today's offline conversation, next steps are to:

  • Iterate on the Citoid first-run experience. Read: the new message that we're planning to show the first time someone opens Citoid.
    • Note: in exploring the above, we'll need to consider the existing Citoid onboarding experience (pulsing dot + instructional dialog).
  • Iterate on how we present the reliability feedback within Citoid so as to clearly relate the system feedback to the reference people are in the midst of generating a citation for.

Update: 1 Nov 2023

Outcomes from today's team meeting are below.

Next steps

  1. Prepare designs for mobile
  2. Prepare artifacts for sharing designs/flows (desktop + mobile) with volunteers. This sharing will happen in T350314.
  3. Generate designs for a "first-run" Citoid experience. This work will happen in the newly-created T350322.

Decided

  • To start, the feedback people receive when they attempt to generate a citation for an invalid source (e.g. google.con) will remain as it currently is.
    • In deciding the above, we accept that, in the short-term, feedback will be presented to people within Citoid in two different ways.
      • Thinking: people will receive the types of feedback to be distinct enough (error vs. qualitative feedback) to expect/not be surprised by varying visual presentations.

New questions

  • What – if any – feedback should the interface provide people when they attempt to cite Wikipedia as a source (T95390)? E.g. offer people the ability to generate a link (rather than a citation).
  • How might we enable volunteers to define source-specific feedback? We'll explore this question in T349261#9299961.
  • What edge cases do we need to consider in the context of offering people feedback about how reliable? @Mvolz is thinking about this in T350317.

Update: 8 Nov 2023

Outcomes from today's team meeting are below.

Next steps

  • DETERMINE the contents of the default message people will see within Citoid when they attempt to add a source that is listed on MediaWiki:Spam-blacklist and/or MediaWiki:BlockedExternalDomains.json
  • @ppelberg to update === Requirements to reflect what we "DECIDED" below

Decided

  1. Feedback message presented within Citoid
    1. In the time between now and when a machine-readable format is introduced to store how reliable projects consider a given source to be (T346849), any time someone tries to cite a domain that is listed on MediaWiki:Spam-blacklist and/or MediaWiki:BlockedExternalDomains.json, they will see one standard message. Note: this "message" will be configurable on-wiki using a soon-to-be created interface message and ought to include a way for people to "learn more" about how/why a project has come to classify a given domain as spam.
      1. Thinking: it's important that the feedback people are provided reflect the actual policy/convention the change they're trying to make is relevant to. Said another way: it could be confusing, for both newcomers and the experienced volunteers maintaining artifacts like MediaWiki:Spam-blacklist and MediaWiki:BlockedExternalDomains.json, if Citoid led people to think they're being blocked from adding a source because it is unreliable when in fact this is happening because a project has deemed the source to be spam.
  2. When we expand the reference reliability check to include feedback about source reliability [i], it'll be important that experience NOT outright block people from adding a source.
    1. Reason: there are cases where someone might want to cite an unreliable source to substantiate a fact about said unreliable source.
  3. Per "1.A.", the API T349261 will introduce will no longer include a field for volunteers to define the message that will be presented within Citoid on a per-domain basis. [ii]
  4. The API T349261 will introduce will need to "call" / check against both MediaWiki:Spam-blacklist and MediaWiki:BlockedExternalDomains.json.
  5. In the background, Citoid will attempt to request the citation and the reliability of the URL someone has entered in parallel. In TICKET (@ppelberg to create), we'll improve upon this approach by re-checking the reliability of the URL after Citoid returns the citation.
    1. Note: this would cover cases when the URL someone initially enters into Citoid is "transformed" in the process of the citation being generated. Think: URL shorteners.

i. At present, the feedback we're introducing here is scoped towards sources/domains projects have deemed to be spam. Which, in effect, are unreliable and distinct from sources/domains projects have explicitly converged on some reliability-related consensus around.
ii. T349261 has been updated (via T349261#9318196) to reflect this requirement.

Re: 1.A, this message is probably going to be very similar to the existing spamprotectiontext message, but we can't just use that one because (a) its default wording expects to be talking about a whole attempted-edit and so is a bit too vague ("probably caused by a link to a forbidden external site"), and (b) some wikis (looking at you, enwiki) have customized it to be huge and so it would cause display issues squeezing it into the citoid dialog.

Re: 2, we could say there's six grades of referenced-website, which mostly relates to the categories in the legend in the perennial sources page. In rough order of "goodness":

  1. Generally reliable.
  2. Unknown; nobody has made a rule against it, but could be unreliable, so we shouldn't say it's "good". This is going to cover a lot of citations to information on minor websites.
  3. Situational ("no consensus"); sources for which you need to read the warning. (To pick an early example from the list: "Arab News is reliable unless the article is about the Saudi Arabian government".)
  4. Generally unreliable; you probably shouldn't use this, but it's not actually forbidden.
  5. Deprecated; you can only use this for self-descriptions, i.e. referencing the fact of the content existing on the site
  6. Blocked; you literally cannot add this to the wiki.

The last one is the only one we should actually block people from adding, because there's occasional valid reasons to use all of the others.

ppelberg renamed this task from Generate design concepts for prompting people to review and replace unreliable sources to Define user experience for prompting people to review and replace unreliable sources.Nov 28 2023, 12:42 AM