Page MenuHomePhabricator

Add a link: ambassador evaluation of "wrong target"
Closed, ResolvedPublic

Description

In T301884: Add a link: rejection reasons, we identified that about a quarter of rejection responses are "wrong target", meaning the user thinks the suggested link target is wrong and should actually be a different article. If this is really happening frequently, we may want to build this capability: T269648: Add a link: changing the link target (not for initial release)

But before building that, we want to understand qualitatively whether newcomers are giving this response accurately and with good judgment, or whether there is more nuance.

Therefore, we will use this spreadsheet for the ambassadors to evaluate about 20 of these rejections each, and note when the user made the judgment correctly or incorrectly, and to generate general impressions on what is happening with the response.

These videos contain a brief explanation of the task (the first one got cut off, so please watch both in sequence):

  1. https://drive.google.com/file/d/1wguwdgzccOQtUUJ2WRo-gquE5OXSG4mE/view
  2. https://drive.google.com/file/d/1_GUrclB0PdQpCoEqS0a3A5edc1EtX5Lm/view

Event Timeline

The ambassadors have finished their evaluation in the spreadsheet. Here are the most important notes:

  • Newcomers giving the "wrong target" response are usually right. In other words, they are generally assessing the rejection reason correctly.
    • arwiki: 42%
    • bnwiki: 77%
    • cswiki: 58%
    • eswiki: 79%
    • frwiki: 70%
  • It does not seem like the users actually mean to be giving the "more or fewer words" response instead. They are accurately assessing that the link target is the issue, not the link anchor.
  • So in thinking about T269648: Add a link: changing the link target (not for initial release), it seems like this could be a useful improvement. But with a couple of caveats:
    • Without also giving users the ability to change the anchor, it has limited value.
    • We may want to prevent users from linking to disambiguation pages, which do come up in the link menu by default.
    • Perhaps instead of adding flexibility and functionality to the "add a link" task, it's an opportunity to nudge users to the visual editor. Perhaps they select that rejection reason and we say, "Do you want to change the link? Here's how you can in Visual Editor!"
  • In doing this evaluation, we noticed that there are a good number of suggestions for which we do not offer an appropriate rejection reason. They are phrases that should not have any link at all. For instance, a phrase like "younger children" may be suggested to link to a movie called "Younger Children". It's not that this is the wrong link suggestion -- it's that the phrase is just occurring in a sentence and shouldn't have a link. Perhaps we should add a rejection reason for this case.

We will talk on the team about what next steps we should do.