@Krinkle after a conversation with @NBaca-WMF we realised that the affected teams might want to do their own QA analysis and this will be an umbrella task for their work.

However, to kickstart this work stream they need to know 2 things:

• What exactly needs to be tested, for both backwards compatibility and the new structure
• The roadmap timelines so they can plan the work effectively

Could you help me fill out these info in the task description? Does that make sense to you or do you have any questions or concerns?

We're deploying next week. There is no further need for dedicated testing afaik. We've prepared this back in March already. The rollout is gradual enough that anything we have test coverage and QA procedures for is naturally going to be testing it without needing to be told to do so, or planned separately.

Is there a specific feature you're worried about, and does that feature not have test coverage or QA regression testing on a regular basis on e.g. Beta Cluster or group0/group1 wikis?

In T403638#11144746, @Krinkle wrote:

Is there a specific feature you're worried about, and does that feature not have test coverage or QA regression testing on a regular basis on e.g. Beta Cluster or group0/group1 wikis?

There are a few teams learning this for the first time and I figured we should just give enough context about how they can validate our assumptions to be more confident with the roll-out.

In T403638#11144733, @MSantos wrote:

However, to kickstart this work stream they need to know 2 things:

• What exactly needs to be tested, for both backwards compatibility and the new structure
• The roadmap timelines so they can plan the work effectively

There is nothing remainig for dedicated testing right now. The change is following a rollout very similar to our weekly train deployments.

• For the past ten years, dev environments and CI have already worked without a mobile domain, so assuming developers generally work on their features locally and use CI, that should give teams confidence that their features work fine.
• No stable or otherwise supported APIs are changing or being removed. We even performed a widescale audit last quarter to identify places in our codebases where people have have written undocumented/internal/unsupported logic that bypasses our APIs. We did indeed find some and addressed them in T390923. There is no policy, requirement, or standing expectation that we do such proactive work, but I figured it will help smoothen the rollout.
• We've already rolled out to the Beta Cluster last month, and to group0 this week (test.wikipedia.org). Details in T401595.
• The remaining schedule is outlined in T403510: [Main Rollout] Enable unified mobile routing on remaining wikis.
• Tech News and Wikitech-l will announce this week the details, things to look out for, and where to report issues.

I expect general weekly QA practices and testing procedures to cover and uncover things naturally without need for dedicated testing at this time. Note that we don't have such an extra dedicated test procedure for the PHP 8.3, either. Our general work is setup such that every week changes happen anywhere in our stack and rollout gradually. Do we trust this process? Is there something specific that warrants additional testing? If so, for which features?

After the Tech News and Wikitech-l message, people with specific fears are free to try it on testwiki or Beta and verify or validate as they see fit.

Hm, I'm not sure that general-purpose QA flows will necessarily uncover potential breaking changes here. I also think teams and team-embedded QTE folks have more context on their features and can ensure that the appropriate user flows are being tested and validated.

I think teams would benefit here from expected high-level area-specific test cases that they should ensure still work, eg.

1. Attempt to open an m. domain, ensure expected device experience loads
2. Attempt to open a non-m. domain on a desktop device, ensure that the device is detected appropriately / a mobile view is served to mobile devices and desktop to desktop devices
3. Double check with team engineers that any mobile-specific logic still applies

Are there other use cases we should make sure to test for?

I mention this because even with extensive up-front work, and general purpose QA flows, without a targeted testing session focused on specific workflows that will be impacted by this change, and without testing informed by team-specific knowledge and background, we are less likely to catch these kinds of issues. I suspect this can be accomplished with relatively little effort, and indeed the most likely outcome is that no changes need to be made -- and likewise that this can be accomplished without substantially impacting the timeline for deliverables -- but it is worth each team doing a check against their own codebases and workflows as one final check here.

I also mention this because I have heard from teams that they are not aware that this change is coming - knowing that it is not only coming but that it will impact their workflows in ways XYZ will be helpful in crowd-sourcing the broader and more holistic work necessary to catch breakages that require deeper team-specific knowledge to test.

Is it possible to provide such a basic user-focused workflow so that teams have a heads up?

Hey @Krinkle!

In general, I'm fairly confident you have thought of everything but my time here has shown me there is always something unorthodox that is prone to breakage, and I'd like to avoid the situation where teams complain that they weren't consulted. FWIW I think you should feel free to set a clear deadline on how long the testing should be to ensure that it doesn't impact your timeline (I can help you proactively remind people that their window is closing).

In terms of specific things I think QA would help with:

• Having everyone look at it will generate questions for answers I am pretty sure you already have and we can create a FAQ to save us time down the road.
  • I know many of my QA engineers use the mobile domain manually in the address bar to test on and I'll want to work with them to understand if the two modes are truly identical and explore whether they should rely on browser emulation going forward.
  • I've been fielding a lot of questions recently about "how will this work" so having some dedicated time for teams to pause and look at test.wikipedia.org will allow them to answer their own questions
  • We have different behaviour on https://en.wikipedia.org/w/load.php?modules=site and https://en.m.wikipedia.org/w/load.php?modules=site - and apps team were worrying about that impacting their stuff (especially if at some point the mobile domain becomes a redirect either now or very much later in the timeline - I am not sure if this will be answered by the QA phase but I suspect there will be some follow up questions).
• We're suspecting some disruption to performance dashboards. Peter already made a task about that, but I think we can take the time to get familiar with how it works on test.wikipedia.org so we are confident about switching over and make sure there is less burden on Peter. The move of the beta cluster for example caught us off guard as we didn't have a dedicated QA for that and we've not had Selenium coverage for Popups since because of T402206: HyperSwitch/errors/not found (404) on beta cluster: There was an issue displaying this preview
• I'd like to reassure my analyst that the changeover is not going to impact their analysis this quarter by letting them look at data on test.wikipedia.org - and give them confidence that they have the fields they need to analyze success

PS. Do you have a FAQ on wikitech yet that I can point people to? Apologies if I missed this while on sabbatical but I haven't seen it.
PPS. Are you planning a wikitech-l thread?

Hey folks! I was made aware of this a couple of weeks ago and will help keep things organized and reported here (since I manage the QS team). I've flagged it with them and am following up individually to ensure that as teams get on top of this we make sure to track and provide updates here.

Currently this is arriving as an unplanned task so right now we're organizing and discussing. I'm going to make sure that we hit specific workflows that might be impacted by this.

From Native Apps: They're researching the impact this may have on Native Apps currently. @ABorbaWMF will be producing a test plan in the near future around any necessary testing based on that investigation.

I made a note of this when testing the campaign events extension T404244#11177992, but also wanted to bring this up here because it isn't specific to the extension.

When testing on tests wiki on a laptop / desktop in full screen, there is a difference in resolution, or one is more zoomed in / stretched than the other:

https://test.wikipedia.org/wiki/Main_Page and click on mobile view	https://test.m.wikipedia.org/wiki/Main_Page

Clicking back and forth between the two screenshots above, the size of the content is displayed differently. I'm not sure if this is expected, but because is a discrepancy between the two I wanted to call it out. It doesn't affect functionality.

In T403638#11178012, @vaughnwalters wrote:

I made a note of this when testing the campaign events extension T404244#11177992, but also wanted to bring this up here because it isn't specific to the extension.

When testing on tests wiki on a laptop / desktop in full screen, there is a difference in resolution, or one is more zoomed in / stretched than the other:

https://test.wikipedia.org/wiki/Main_Page and click on mobile view	https://test.m.wikipedia.org/wiki/Main_Page

Clicking back and forth between the two screenshots above, the size of the content is displayed differently. I'm not sure if this is expected, but because is a discrepancy between the two I wanted to call it out. It doesn't affect functionality.

This was an error on my side! You can disregard this comment.

Requirement

Scope: Mobile domain sunsetting QA (desktop + mobile).

• Both standard and mobile subdomain URLs must correctly render Minerva for mobile.
• Redirects from m. subdomains must resolve without breaking bookmarks or links.
• User-critical flows (navigation, search, editing, login/logout, account settings) must work seamlessly on the unified domain, logged in and logged out.
• Extensions, gadgets, and scripts must not rely on m. domain string checks; instead, they must use supported feature flags (MobileContext->shouldDisplayMobile(), mw.config.get('wgMFMode'), .skin-minerva).
• Redirection and toggling between mobile/desktop views must work properly on unified URLs.
• Analytics/events may be impacted if they previously depended on m. domain detection; these should be validated once clarified.

BDD

Feature: Unified domain support for mobile and desktop

  Scenario: Accessing via standard domain
    Given I open a wiki page on en.wikipedia.org
    When the page loads on a mobile device
    Then the Minerva skin is applied
    And navigation and content render correctly

  Scenario: Accessing via mobile subdomain
    Given I open a wiki page on en.m.wikipedia.org
    When the page loads
    Then I am redirected or resolved to the unified domain
    And the Minerva skin is applied

  Scenario: Switching between mobile and desktop modes
    Given I am on a wiki page
    When I toggle to desktop view
    Then the Vector skin is applied
    When I toggle back to mobile view
    Then the Minerva skin is reapplied

  Scenario: User flows work seamlessly
    Given I am logged in
    When I search, navigate, edit, or log out
    Then the actions complete without error
    And the correct skin is maintained

  Scenario: Event instrumentation validation (pending clarification)
    Given I am using Minerva on the unified domain
    When I trigger a mobile-specific action (e.g., menu click, search, navigation)
    Then the appropriate event schema fires
    And event payloads identify mobile mode via feature flags (not `m.` domain)

Test Result - Prod

Status: ✅ PASS
Environment: testwiki
OS: macOS Sequoia 15.5
Browser: Chrome Canary (latest as of test date)
Device: MS
Emulated Device: iPhone 14 Pro (Minerva)

Test Case 1: Standard domain loads Minerva on mobile

1. Open https://test.wikipedia.org on a mobile device.
2. ✅ AC1: The Minerva skin is applied (verify .skin-minerva in DOM).
3. ✅ AC2: Page content and navigation are functional.

Test Case 2: Mobile subdomain redirects correctly

4. Open https://test.m.wikipedia.org on a mobile device.
5. ✅ AC3: URL resolves to https://test.wikipedia.org.
6. ✅ AC4: Minerva skin is applied.

Test Case 3: Switching between modes

7. On https://test.wikipedia.org, toggle desktop view.
8. ✅ AC5: Vector skin is applied (verify .skin-vector-2022 or .skin-vector).
9. Toggle back to mobile view.
10. ✅ AC6: Minerva skin reapplies.

Test Case 4: Logged-in user flows

11. Log in on https://test.wikipedia.org.
12. Perform search, navigation, edit, and logout.
13. ✅ AC7: Each action completes without errors.
14. ✅ AC8: Skin remains correct for the mode.

Test Case 5: Event instrumentation validation (pending clarification)

22. Trigger mobile-specific interactions (menu click, search, navigation).
23. Inspect network tab for eventlogging requests.
24. ❓ AC13: Events fire correctly on unified domain.
25. ❓ AC14: Event payloads identify Minerva via feature flags (wgMFMode, .skin-minerva), not via m. domain.

@ovasileva This test case is the only one I need assistance with. 1) is this in our scope? and 2) What current events are we currently relying on for metrics?

This evidently causes some issues for editors trying to get around Mainland China censorship: https://en.wiktionary.org/wiki/Wiktionary:Feedback#What_happened_to_en.m.wiktionary.org?

@Lhy7889678

Thanks. I replied there.