Page MenuHomePhabricator

Help panel: Review similar past projects (MoodBar, Article Feedback Tool) & summarize relevant learnings for in-context help design
Closed, ResolvedPublic

Description

Background

The idea we're pursuing has similarities with two previous Mediawiki projects that we might be able to learn from.

  • Article Feedback: readers would rate an article on its quality and leave unstructured comments.
  • MoodBar: new editors would give feedback on their editing experience after saving an edit.

@nettrom_WMF has read the papers on these projects, and @RHo might learn valuable information from them to aid design of the in-context help feature.

Summary of learnings from both tools relevant to In-context help

A. Article feedback tool
  • Potential for account creation and editor activation - not logged in users who start an edit may be prompted to create an account when asking for help so that they can get feedback from the “Help team”.
  • Provide more upfront help options prior to allowing submission of free-text comments and questions - Per findings from the Article Feedback Tool, making it too easy to submit unstructured and out-of-context comments about articles leads to a high degree of noise for moderators. Whilst our hypothesis is that users who are in the editing context and seeking help for a specific task may be less likely to submit unuseful questions, providing more self-directed help alternatives upfront may further reduce volume.
  • Mechanisms to provide context to comments – The lack of context to comments in AFT was a main point of dissatisfaction for editors. For in-context help, it would be ideal if users submitting a question or comment would be able to denote where in the article or UI they would like assistance – for example being able to submit a screenshot along with their question, or being able to highlight the section related to their comment.
B. MoodBar
  • Capturing Email for responses is an effective option – As shown in the MoodBar experiment, email notifications is an effective means of engaging new editors -
  • Clear, prominent call to action for new editors to ask for feedback/help - per high level findings, the addition of a tooltip drawing attention to the Moodbar significantly increased its use.
  • Tracking the perceived helpfulness of responses is important - It would be useful for us to track similar new user perceptions of feedback, since a poor experience from receiving unhelpful advice has been shown to negatively impact editing.

Full review doc

https://docs.google.com/document/d/1sggnhz2pgRfLhF1Pgz-VJgnOuIkTejJCOk8dICKMSc4

Event Timeline

Some ressources concerning AFT on fr.wp. An RfC has decided to add AFT on the top 100 articles for a 6 month trial and 100% of Help pages. A consultations was scheduled to know what would be the newt steps:

A post experiment survey with the following conclusions:

  • The evaluation tool in its current form is not satisfactory due to the large number of comments which are of no value.
  • The fact that the comments are disconnected from the articles does not allow the usual contributors of the article to have easy knowledge of those comments.
  • The large number of comments waiting for processing shows that deployment on all articles is a priori premature.

Given the fact that the WMF has no plan to move forward, after that vote on the Bistro, the team in charge has decided to remove the tool.

RHo renamed this task from Help desk: learning from MoodBar and Article Feedback Tool to Review similar past projects (MoodBar, Article Feedback Tool) & summarize relevant learnings for in-context help design.Oct 30 2018, 7:25 PM
RHo removed RHo as the assignee of this task.Nov 2 2018, 12:25 PM
RHo updated the task description. (Show Details)

hi @MMiller_WMF, @nettrom_WMF - I wrote up some high-level learnings to consider in the ICH design, please feel free to add and comment on the full doc:
https://docs.google.com/document/d/1sggnhz2pgRfLhF1Pgz-VJgnOuIkTejJCOk8dICKMSc4/

MMiller_WMF added a comment.EditedNov 2 2018, 10:41 PM

@RHo -- thank you for doing such a thorough job. I checked out some of the wiki pages related to the two projects, and it is pretty difficult to find useful information, designs, and mockups, so that was probably a difficult part of this work. I left some comments in your document for us to discuss.

I think some of the most interesting parts are:

  • Considerations on how users find out that they have received a response.
  • The opportunity to encourage users to add their email addresses so it's easier for them to receive replies.
  • The fact that having a "special queue" where experienced editors triage comments did not work out.

Awesome job, @RHo! I think you did a great analysis on these. One point I would like to add about ArticleFeedbackTool:

One of the reasons the ArticleFeedbackTool was so hated by the community is that it didn't just create one new queue of work for the community to process, but two. Because users were allowed to enter free text and because it's the internet, they sometimes entered things that needed to be completely deleted from existence, such as private phone numbers and email addresses, death threats, racial slurs, etc. But because the ArticleFeedback input lived in its own interface rather than being posted to the Talk page or some other wiki page, the normal processes for moderating and deleting such content couldn't be used. Instead, entirely new processes and interfaces for flagging and deleting the content had to be built and learned by the community. This was seen as a waste of scarce moderator resources.

High Level Finding: Don't allow users to post free text unless you're going to publish it to a wiki page. Otherwise, you have to completely reinvent the wheel.

hi @MMiller_WMF - I've added everyone's comments on this task into doc and used them to inform the initial designs explorations on T206715, so it should be ready to close.

MMiller_WMF renamed this task from Review similar past projects (MoodBar, Article Feedback Tool) & summarize relevant learnings for in-context help design to Help pane: Review similar past projects (MoodBar, Article Feedback Tool) & summarize relevant learnings for in-context help design.Nov 13 2018, 10:49 PM
MMiller_WMF closed this task as Resolved.
MMiller_WMF claimed this task.

Great -- resolving now.

MMiller_WMF renamed this task from Help pane: Review similar past projects (MoodBar, Article Feedback Tool) & summarize relevant learnings for in-context help design to Help panel: Review similar past projects (MoodBar, Article Feedback Tool) & summarize relevant learnings for in-context help design.Nov 29 2018, 7:11 PM