This task involves the work of reconsidering what happens when people elect to decline taking action on a Peacock Check that is presented to them.
The need for this exploration emerged during a 19 March Editing Team discussion wherein we came to align on the following disconnect:
- Publishing an edit that introduces text written in – what experienced volunteers are likely to consider – a non-neutral tone is relatively disruptive
- Dismissing a Peacock Check, at present, is relatively lightweight/seamless
The disconnect between "1." and "2.", we think, will send people encountering Peacock Check a conflicting message.
Stories
- As a newcomer/Junior Contributor who has explicitly chosen to forego revising the tone of what they've written, I would like to know what information/content I can offer to the experienced volunteers who are likely to review/patrol this soon-to-be published edit, so that I can increase the likelihood that the contribution I consider to be useful and worthwhile remains on the wiki.
- As an experienced volunteer who is reviewing an edit in which someone has a) added new text written in a tone considered to be non-neutral and b) decided NOT to revise the tone of said text, I would like to know why this person made this decision so that I can decide what – if any – action (e.g. revert the edit, post a message on the person who made the edit's talk page, etc.) to take in response
Requirements
User experience
- When people choose to Decline revising the tone of the new text they are writing when Tone Check invites them to consider doing so...
- Present the following "survey" (see mockup below) that prompts people to indicate why they've declined to revise the tone of what they've written by selecting one of the following three options:
- The tone is appropriate
- I’m not sure how to revise the tone
- Other (for now, we will only include the "Other" option without TextInput, and we will include it in T397889)
- On desktop, the decline survey ought to be presented in the sidebar, per T381610
- The Decline response someone submits needs to be logged so that we can aggregate these responses
- Based on what the Editing Team discussed offline on 12 May, this logging should happen automatically via VEFU.
- Present the following "survey" (see mockup below) that prompts people to indicate why they've declined to revise the tone of what they've written by selecting one of the following three options:
Mockup
NOTE: UX adjustments (e.g. button treatment, card title, etc.) should also be applied to the Reference Check decline survey
Decision(s) to be made
- 1) What - if any – modifications will we make to the Peacock Check's "decline path" to ensure it reinforces the importance of considering tone when adding new content to Wikipedia and the potential impact of not doing so?
- See ===Requirements
-
2) How/if might we use the signal the decline path ideally captures/elicits could be useful for the model retraining the ML Team is planning in T393103- We're deferring this question for now and will, instead, revisit this when we prioritize work on T393103.
-
3) Related to "2)," how might we surface the signal the decline path [ideally] captures to volunteers so they can use it as an input to improve moderation processes? E.g. maintain an allow-list of sorts to avoid false positives.- One idea, might it be possible to look at the SHAP values the model returns for edits where some declines to revise what they've written and log this in some place?
- Note: we're going to defer this question for now and instead revisit it when we prioritize work on T395175.
- 4) How – if at all – will we pass on the responses volunteers offer to the model/ML Team?
- TBD; see T393103.
Done
- Potential "Approaches" are documented
- An "Approach" is chosen
- Approach is implemented
- ⭐️ Editing QA to verify events are being emitted client-side as expected
- @MNeisler to verify decline responses are being logged in VEFU as expected
