This task involves the work with evaluating the extent to which the combination of the Topic Container, Clear Affordances/Visual Cues, and Page Frame interventions impacts Junior Contributors' ability to intuitively recognize and use talk pages to communicate with other people and Senior Contributors' ability to scan the page for the information they need to evaluate the state of a conversation.
Planning Document
Decisions to be made
What – if any – adjustments to the Topic Container, Clear Affordances/Visual Cues, and Page Frame designs will we prioritize making prior to making the feature available to partner wikis as an opt-in beta feature.
Test requirements
- Platform: Desktop + Mobile
- Languages: English, French, Japanese and Arabic
- Target populations: junior and senior contributors who are not "early adopters" to talk page improvements
Test goals
- Validate that the improvements help to make the page more recognizable as an interactive discussion space to junior contributors and a more scannable/useful experience for senior contributors
- Identify any pain points or bugs
Research questions
Primary Questions
- Do the changes increase the speed with which junior contributors can accurately describe what talk pages can be used for? (Do they recognize these as interactive spaces? Do they view topics as discussions? Do they see that “talk” pages are distinct from “article” pages?
- Do the changes increase the confidence with which junior contributors can accurately describe what talk pages can be used for?
- Do the changes increase the speed and confidence with which senior contributors can identify the state of a conversation?
- Are junior and senior contributors able to successfully reply to and post new topics on discussion pages?
Supplemental Questions
- Do Junior Contributors understand and find value in the two {Reply} button treatments: one version of the button that appears beneath comments posted without any indentation; a second version of the button that appears beneath comments posted with at least one level of indentation?
To do before study can begin
- Have a stable beta build of the complete Talk Page experience on desktop
- Have a stable beta build of the complete Talk Page experience on mobile
- Write scripts for all of the tests
- Create recruitment screeners for junior and senior contributors
- Co-ordinate non-english tests with Research Team (this will need to include translation services)
- Create all test pages on the beta build
- Create all user accounts for testing purposes
- Create document for data logging and synthesis
To do after the study has begun
- Watch and log the tests
- Synthesize and summarize the test findings
- File new tickets for issues that surfaced during the tests (if applicable)
Findings
This section will be populated with a summary of the test findings. This section should also contain links to tickets filed to address usability issues that surfaced during the tests.