Some more ideas/questions that we floated about requirements/scope:

• How might we make the system distributed, asynchronous, and independent of WMF staff involvement?
• How might we flag a dark mode issue in a place where someone who can fix it will see it? (e.g. not phabricator)
  • maybe a "this page has issues with dark mode" category
  • maybe a semi-automated talk page post
    • If so, then which talk page? Article talk? Template talk? Which template?
  • A post to the local technical village pump + allow the village pump admins to choose a different destination for reports locally.
    • This is a nice idea because different communities can customize where the reports go.
• add it to https://en.wikipedia.org/wiki/Wikipedia:Community_portal/Open_tasks

• How might we ensure the quality of the dark mode bug reports so that future maintainers have the correct context for making a fix? What should be included in a bug report? (screen shots? URLs? browser info? etc.)
• How might we inform potential bug reporters what counts as a "dark mode" bug?
• How might we inform readers that there are known dark mode issues on a page after a bug has been verified?
  • Do we need to do this?
• How might we create an affordance for verifying bugs when they are raised?
• How might we prevent an influx of low-quality bug reports/spam by bots and bad actors?
  • open to logged in only?

The main question for me that is holding up a design on this is where the reports go and why. We have two options so far: categories and talk page posts. What other reporting mechanisms do we want to explore? What can we control programmatically?

@sgrabarczuk I think we need feedback from some veteran contributors on where these reports could/should go. Where do you think these discussions should happen?

In T367325#9885753, @JScherer-WMF wrote:

The main question for me that is holding up a design on this is where the reports go and why. We have two options so far: categories and talk page posts. What other reporting mechanisms do we want to explore? What can we control programmatically?

I think these two make sense to start with, and potentially could even work in sequence. I could see us doing something like:

v1 (deployment blocker): scope is link to report out shown to logged-in users + redirecting to talk page
v2 (post-deployment): scope is adding a category, potentially marking pages with that category and displaying a message that the page might have issues, etc

Discussed with @JScherer-WMF and we think we want to:

1. Show link in appearance menu: "Report issue with dark mode" for logged-in users only
2. Link goes to pre-populated form on talk page of same page that asks for details, screenshots, etc
3. User posts form as new section on Talk page

From here, community members review and start fixing.

Wonder if there is any opportunity to also (at some point) link to guidelines for more people to attempt fixes in the same place the report? Could this be an opportunity to increase people who volunteer as template editors?

It would be nice to link to the recommendations somewhere. We would need to simplify them first, though, to make them regular editor-friendly.

I said that off-Phab, so I'll add it here for visibility:

Instead of inviting people to report issues on the respective talk pages, which will be many and which are not watched by many Wikipedians (let's call it the decentralized model) we should centralize feedback: in "Report issue with dark mode", give a link to one page per wiki, for example the main page talk page (which is the only certainly existing talk page on any wiki), and allow communities to change the link to redirect feedback to the page of their choice (Village Pump, VPT, WP talk:Dark mode, whatever).

Pros: 1. much better visibility of reports, 2. editors could learn about solutions to repeated issues by checking the previously reported and solved cases.

In T367325#9896201, @sgrabarczuk wrote:

It would be nice to link to the recommendations somewhere. We would need to simplify them first, though, to make them regular editor-friendly.

I said that off-Phab, so I'll add it here for visibility:

Instead of inviting people to report issues on the respective talk pages, which will be many and which are not watched by many Wikipedians (let's call it the decentralized model) we should centralize feedback: in "Report issue with dark mode", give a link to one page per wiki, for example the main page talk page (which is the only certainly existing talk page on any wiki), and allow communities to change the link to redirect feedback to the page of their choice (Village Pump, VPT, WP talk:Dark mode, whatever).

Pros: 1. much better visibility of reports, 2. editors could learn about solutions to repeated issues by checking the previously reported and solved cases.

+1 for considering this alternative. Another aspect is whether Talk pages is able to and a good place to support an influx of screenshots that would be likely for this report? It is not obvious and cumbersome how these would be stored, and also potential issues with T&S(?). Perhaps the alternative single page on wiki is linking to a a new phab board that better supports image uploads with reporting bugs?

In T367325#9897691, @RHo wrote:

In T367325#9896201, @sgrabarczuk wrote:

It would be nice to link to the recommendations somewhere. We would need to simplify them first, though, to make them regular editor-friendly.

I said that off-Phab, so I'll add it here for visibility:

Instead of inviting people to report issues on the respective talk pages, which will be many and which are not watched by many Wikipedians (let's call it the decentralized model) we should centralize feedback: in "Report issue with dark mode", give a link to one page per wiki, for example the main page talk page (which is the only certainly existing talk page on any wiki), and allow communities to change the link to redirect feedback to the page of their choice (Village Pump, VPT, WP talk:Dark mode, whatever).

Pros: 1. much better visibility of reports, 2. editors could learn about solutions to repeated issues by checking the previously reported and solved cases.

+1 for considering this alternative. Another aspect is whether Talk pages is able to and a good place to support an influx of screenshots that would be likely for this report? It is not obvious and cumbersome how these would be stored, and also potential issues with T&S(?). Perhaps the alternative single page on wiki is linking to a a new phab board that better supports image uploads with reporting bugs?

I agree that the need to upload screenshots to commons would probably reduce the quality of the bugs being reported. From our conversations: the main concern with a phab board is that it might create an expectation that Web or another team at WMF would be responsible for fixing the issues that get reported. I also like the option of centralizing bug reports. It seems that we're playing with two models for information flow here:

I put together a flow chart of how this might look for Talk pages (black arrows) and some kind of centralized, technical community space like TVP (blue arrows).

Posting in a centralized place like TVP has the benefit of going more directly to people who can fix the issues. It also lets the technical editors benefit from one another, whereas each talk page is an island. The biggest issue with the Talk page plan is that someone has to see the bug, and somehow find someone who can fix it.

Created a design task for this interface in T367795. It's blocked until we agree on the information flow on this.