Page MenuHomePhabricator

[Design] Design a two-step+confirm reporting process for logged in and logged out readers reporting dark mode bugs.
Closed, ResolvedPublic

Description

The usability study in T370825 found that readers expect to fill in a short form in a separate window or popup when they click the "Report an issue with dark mode" link in the Appearance menu.

As a Wikipedia reader, I want to quickly explain the issue I'm reporting in dark mode.

Open questions:

  • What is the best technical mechanism to achieve our goal?
    • Can we use the link to launch a quick survey?
    • Can we use the link to launch a Codex Dialogue?
  • If we had a single, simple multiple choice question that let the readers describe their issue in a bit more detail, would we be able to use that data?
  • What would be helpful to maintainers fixing issues without adding additional coordination cost to the issue triage process on our end?

Event Timeline

JScherer-WMF created this task.

More questions:

  1. Where will the reports GO? Follow on, WHO will be able to read the reports?
  2. What should be told to the reporter beside an 'input accepted' feedback? (i.e. Can they follow up on "their report"?) From a UX perspective: "If I report something, what should I expect to happen next?"

More questions:

  1. Where will the reports GO? Follow on, WHO will be able to read the reports?

These are great questions. I'm going to do the annoying designer thing and answer your question with a question: From your perspective as a technical contributor, where should the reports go, what format would you like, and what should they include that would be helpful for making fixes?

From a RACI perspective - who is Responsible for acting upon the reports? Because they are a key stakeholder. Is WMF staff taking this responsibility? If this is something that per-project volunteers are expected to manage, they likely need to end up on-project.

From a RACI perspective - who is Responsible for acting upon the reports? Because they are a key stakeholder. Is WMF staff taking this responsibility? If this is something that per-project volunteers are expected to manage, they likely need to end up on-project.

That's right. It needs to be per-project volunteers because the issues are primarily created by hardcoded colours in volunteer-maintained templates which we don't maintain. On the projects that you work on, where would you want to see these reports, and what would you like them to include?

I've been working on the design for this today a little bit. Based on the usability testing and a conversation with @dchen, I had the idea that we might be able to collect structured data to help volunteers find issues and fix them. This approach has a few benefits:

  • Volunteers might be better served if they know what kinds of elements they're looking for on a flagged page
  • The reporters have their expected form to submit
  • Volunteers no longer need to sift through free text reports to interpret where an issue might be.
  • We could theoretically package these up and send them to communities in an automated way.
  • We could use the same reporting mechanism for both logged out and logged in users.
  • UI bug reports wouldn't get mixed in with general user feedback.

Drawbacks:

  • We would need to build a way to automatically package these reports up and put them where volunteers can see/use them in a format that is useful. This is a complex design task, and I doubt all volunteer communities would find the same format useful.
  • We might miss some elements in the list and prevent reports
  • It's unclear if knowing that, for example, Article X has a dark mode issue with a table somewhere reduces the coordination cost of finding the bugs compared to the current reports we're getting.
  • There's still no issue tracking system in place to check out tasks,
  • Volunteers still manually need to disambiguate whether the fix or a bug report belongs on a template page or on an article.

I'm imagining an experience like the short multiple selection survey I see when I unsubscribe from an email newsletter.

As a first pass on what we would need to have in the selection list, I gave the existing reports from EN Wiki to Chat GPT and ran through several analysis prompts to land on a list of elements that readers are having a difficult time seeing based on the reports we have received to date. The list definitely needs refining/checking still.

This is a quick, non-final mockup of what this approach might look like:

image.png (1×2 px, 993 KB)

I'll bring this to design review tomorrow and get some feedback from the design team on it.

From a RACI perspective - who is Responsible for acting upon the reports? Because they are a key stakeholder. Is WMF staff taking this responsibility? If this is something that per-project volunteers are expected to manage, they likely need to end up on-project.

That's right. It needs to be per-project volunteers because the issues are primarily created by hardcoded colours in volunteer-maintained templates which we don't maintain. On the projects that you work on, where would you want to see these reports, and what would you like them to include?

I suppose something like:
"Publish the report information to a new section on the page {{int:dark-mode-problem-reports}}" Defaulting [[Mediawiki:dark-mode-problem-reports]] to something like [[Project talk:Dark mode]]. Potentially using the user session to do so (ensure that the report has some sort of disclaimer about publishing a revision, especially if it will allow free form text).

From a RACI perspective - who is Responsible for acting upon the reports? Because they are a key stakeholder. Is WMF staff taking this responsibility? If this is something that per-project volunteers are expected to manage, they likely need to end up on-project.

That's right. It needs to be per-project volunteers because the issues are primarily created by hardcoded colours in volunteer-maintained templates which we don't maintain. On the projects that you work on, where would you want to see these reports, and what would you like them to include?

I suppose something like:
"Publish the report information to a new section on the page {{int:dark-mode-problem-reports}}" Defaulting [[Mediawiki:dark-mode-problem-reports]] to something like [[Project talk:Dark mode]]. Potentially using the user session to do so (ensure that the report has some sort of disclaimer about publishing a revision, especially if it will allow free form text).

Thanks for this! Do you know of any other volunteers I should talk to about this?

Moving to blocked on others because there are several plates spinning related to this. What would this look like on a technical level? How would this affect roadmap?

Jdlrobson subscribed.

We need some input from Olga on this ticket so moving to next sprint.

Moving to blocked while we consider and discuss options for next steps

Given that we're removing the logged out bug reporting links for now, I will put this in sign off.