Background
The Moderator Tools team is currently working on a project to build an 'automoderator' tool for Wikimedia projects. It will allow moderators to configure automated prevention or reversion of bad edits based on scoring from a machine learning model.
We are running usability tests to get feedback on the new edit summary. More specifically on the copy and the false positive report link.
Target audience
The audiences for the survey are admins and experienced editors on Chinese, German, Italian, Arabic and Japanese wikis.
Platform: Desktop
Languages: Simplified Chinese, German, Italian, Arabic and Japanese.
Research Goals | Questions
Evaluate usability
Does the placement of the link interfere with existing workflows?
Does this make the False positive page easily accessible?
Expectations
Did admins expect to find this link in the edit summary?
Survey questions
Results
Feedback on the edit summary in general:
- 3 people were interested in seeing a link to why something was reverted (this was also frequently requested when testing with newcomers).
- And/or an explanation of what 'unconstructive' meant.
- 2 people wanted to add tags.
- 3 people wanted to see the diff/summary of the change made.
- 1 person requested a setting to hide the username.
Edit history:
- Similarly to the edit summary, the people said they expected to see, the reason for the revert and diff/summary.
People thought that clicking on 'Report a false positive' would:
- 5 x Undo a revert
- 1 x report directly
- 1 x bring you to form
False positive page:
- 50% expected the link to open the false positive page, while 50% didn't.
- 5 x expected that the edit would be automatically undone.
- 2 x wanted to see a button to undo revert on the page.
Would this bot be helpful for your wiki?
- 3x depends on accuracy and definition of unconstructive edit.
- 1x Thought that the bot should detect but shouldn't perform reverts.