The Growth team (supported by Design Strategy) aims to conduct user testing on unregistered editors' understanding of temporary accounts, and their attitudes/inclination to account creation.
Testing aims to begin in English, followed by at least one subsequent round with Japanese-using users.
Test format
Testers on an unmoderated testing platform (likely usertesting.com) will be asked to interact with a series of screens (currently planning for static mocks) and posed a series of closed- and open-ended questions as they navigate a version of the planned IP masking workflow.
Research questions
- What do testers think of the different proposed name formats?
- Do they want more information in the name itself (e.g., "anon-"; "temp-", etc., or in other places?
- Should the main Call To Action come when users are invited to create an account?
- Should we also add a CTA to log in?
- How do users react when invited to create an account?
- What about a CTA to “End session/log out”?
- What do people understand about the relationship between editing, account, and temporary account?
- Do people understand their ability to edit?
- How much would their interest be in Growth-type features? Creating an account? (Bonus)
Participant profile questions
- Age range
- Gender
- WP access frequency
- WP visit duration
- Editor?
- Have you ever added, changed, or edited anything in a Wikipedia article?
- Account holder?
- Have you ever created a Wikipedia account, either for editing or reading?
draft temporary account name formats to test
- *23-8.712
- *unregistered23-8.712
- *temp23.8.712
- *anonymous23-8.712
- *anon23-8.712
Protocal steps (draft pending finalized materials)
- Please move this usertesting instruction panel somewhere where it does not obstruct your browser window.
- You will be asked to explore some features on Wikipedia. As you follow the prompts, please continue to talk, saying out loud what is running through your mind. We will sometimes remind you to please KEEP TALKING.
- Before we begin, we’d like to ask you a few questions about Wikipedia.
- Can you explain how information gets added to Wikipedia? Where do the articles come from?
- How much do you trust the information that you read on Wikipedia?
- Have you ever thought about creating a Wikipedia account? Why or why not?
- Let's get started. Imagine that you are reading this Wikipedia article.
- Have you ever read this Wikipedia article before?
- Can you find the spot on the page where it says "not logged in"? What does “not logged in” mean?
- Now, imagine that you see something in the article that you want to change.
- Does anything stick out to you that needs changing?
- Where would you go first to start changing content on this page? Please walk us through what you would do to begin.
- If you are not there already, please press the "Edit" tab on the top right of the page. Now, let's do some editing! Remember to please KEEP TALKING.
- Have you ever seen a page like this before on Wikipedia?
- What do you think about this page? What is going on here?
- Does anything stand out to you here?
- Please read the message in the yellow box at the top of the page.
- What is this message trying to say?
- Have you ever seen a message like this before?
- Is there anything in this message that you find confusing?
- Go ahead and click on the typo in the article text to “fix” it. Now, click on “Publish changes” at the bottom left of the screen. Remember to please KEEP TALKING.
- You’re back where you started! But, something has happened...
- What has changed about this page since the first time you saw it?
- Is there anything on this page that is confusing to you at this point?
- There is now a username at the top left of the screen. What is it?
- Congratulations! You’ve fixed a typo in a Wikipedia article.
- Beginning from the beginning, please explain exactly what happened during this process and what you did to complete it.
- Questions
- On the editing page, what did you think of the warning message in the yellow box?
- If this had been a "real" and you really had corrected the typo, would you have a Wikipedia account at this point?
- What did you think when you saw “*unregistered23-8.712"?
- What would happen if you closed Wikipedia today, but came back to edit the same article tomorrow. When you try to edit the same article tomorrow, what would happen?
- Would you have the same ~Unregistered8712~ name, or would you be given a new one?
- Now, we’d like to show you a few options for your temporary username. Please read them out loud. Remember to KEEP TALKING.
- *23-8.712
- *unregistered23-8.712
- *temp23.8.712
- *anonymous23-8.712
- *anon23-8.712
- Do any of these name options stand out to you in any way? Why?
- Are any of them confusing or unclear? Why?
- Which of these options would you prefer to see if you corrected a typo in a Wikipedia article? Why?
- Is there anything else you would like us to know about your experience with Wikipedia? or is something you wish you could do in Wikipedia but currently can not do?
Acceptance criteria
- Research findings: https://docs.google.com/presentation/d/1h54h-3byyRvoCYLQMB5GM14u38Mj3i1tURuWlegEC8w/edit#slide=id.g1dfa5f6a716_0_89 (note: updated as ES, JA, AR results come through)
- UX recommendations based on research: https://docs.google.com/presentation/d/1A11bOqlPL4mhkMKJeFe8JG-q3Rclu7rjGrUC0AVHLg0/edit#slide=id.g229a3345d18_0_289
- Replicate tests in other languages (see T328616)
The final report is here:
IP Masking: Usability tests in English, Spanish, Japanese, and Arabic