Assumptions

The context for this problem:

• T297758: the goal of Vector 22 supporting pre-existing vector.js scripts.
• T291099: declined alternative implementation.
• T300070: (other) bug in current implementation, loading common.js twice.
• T305594: (this) bug in current implementation, not waiting for the site module.

As I understand it, the general goal here is for a wiki's community to generally be satisfied when we enable vector-2022. And, by the time it becomes the default, the majority of gadgets and scripts "just work". And, for those that don't, there's a page to consult with notable differences and how to accommodate them. And, editors would by then that and where such page exists (e.g. mentioned in Tech-News once early on and again after the switch, on Wikitech-l, and a small mention in a more general non-technical blog post).

As I understand it, the ambition for Vector 22 (as neatly summarised by @Jdrewniak in T234907) was "not disrupting the workflows of our most engaged community members" such that "gadgets, user scripts all continue to function." and all the while having "[no] negative performance impact".

Past

We've done this before in 2010, when we deployed the Vector skin. It later became the default over Monobook. Opt-in at first, then by default for unregistered and new accounts. And eventually for existing accounts via a one-off nudge.

Development of Vector did not include making breaking changes to Monobook. Not via Monobook code, nor via MediaWiki core in ways that affect user scripts. This was key to maintaining credibility and trust. To do differently would've nullified the appealing risk profile associated with creating Vector. Namely, that it can't damage the long tail of rich ecosystems around each Wikimedia-hosted and third-party MediaWiki site.

Anyone could explore and see how we're doing by occasionally opting in and out. One might port over a gadget or script if interested that time. Add vector to a gadgets skins filter, or accommodate Vector in any skin-specific CSS rule, or apply the same skinStyles file to both skins, or fork the skinStyles file if needed, or by copying some or all import statements from their personal monobook.js to vector.js, etc. Likewise, these opt-in actions and modifications by early adopters did not and could not affect those on Monobook, including the person switching. E.g. switch back for a while, and Monobook is still the same as when you left it.

Observations

These considerations were either not researched, not found, or chosen differently. That's fine of course, though if the latter, I suppose documenting the reasons for choosing so would give others a chance to appreciate why.

Despite that, I do notice a trend in these differences reversing.

• (2020) Re-use preference key and Skin class, through a non-standard global skinversion option.
  • Outcome: Skin class and preference are split. Version override removed.
  • Known issues: Leaks through caches and rendering layers (T288113, T300278, T302627, T303265, T305232, T305966).
  • Note: The known issues cover but a small portion of the long tail that awaited someone able and willing to report. It relied on a service class that violated Dependency Injection principles (T242835#6064656), and actually still does today, continuing to produce a long tail of bugs that I suspect are being patched at the edge instead of at the source.
• (Aug 2021) Re-use skinStyles entry, through fake skin keys in ResourceLoader.
  • Outcome: The skinStyles keys remained split and intuitive to use.
  • Known issues: Indefinite cost of maintaining, documenting, and teaching an unneeded system with additional required concepts to learn for every developer. Future bugs from unintended sharing when the two diverge with unequal QA coverage. Added risk and weeks of unscheduled work from building/testing something new. (T288857)
• (Dec 2021) Re-use mw.config skin key, through overriding the value in vector-2022 to be vector.
  • Outcome: The config key is split. Config override removed.
  • Known issue: Breaks the standard API in JS for recognising the current skin. Corrupted edit previews and any other API-based features that vary by skin (T297758, T300814). Would also have affected EventLogging and CentralNotice reporting. Possibly incorrect load.php request context. And compromise the standard API to deal with differences between skins, such as required for gadget authors to support Vector 22 in the first place.
• (Jan 2022) Re-use Skin.js and Skin.css, through appending to the site and site.styles modules, and replacing the user module. (T297758)
  • Outcome: ... perhaps scripts split and override removed?

The dates above suggest these were reactionary proposals — not a general plan to make the skins identify as one.

Known issues

The re-using of skin.js/skin.css seems problematic to me. I'll share my own experience here, but I imagine more compromises exist for other people:

• First impression. This set me up for a bad first impression. I observe styles and integrated tools not working well, thus starting an immediate and mandatory chase to disable or fix stuff, only to arrive at a then-incomplete experience. Continued exposure uncovers more half-working tools. If you start out with a clean and unpersonalised state, with some gadgets that are skin-agnostic or already updated, that might leave a better first impression.
• Cut dead wood. It's easier to append then to clean up. We can encourage communities to take inventory and update based on demand and ability. We've gone for a decade with little to no mandatory transition. Taking this opportunity to identify unwanted code, by proxy of choosing what to copy, allows imported scripts that are broken or unwanted, to become unreferenced. This improves performance, and also reduces security risk over the long run. It is incumbent to our ecosystem that this approach is paired with: an adequate timeline with months of consented opt-in, and the option to opt-out.
• Incoherent. User scripts are now the only place in the platform where one applies something to Vector 19, but then affects Vector 22 as well. This is especially surprising for site-wide scripts. We lack an intuitive way to avoid or undo this effect.
• Analysis, global search. If script names no longer correlate to a skin, this makes it difficult for maintainers to know in which skin their script is imported.
• Deprecation, automation. So long as vector.js remains ambiguous in meaning, it's hard to know how and when something requires changes by an interface editor. And impossible to do en-mass with automation.

Besides the above principal issues, there are also technical issues:

• Extension support. The user module is reserved name that extensions use to await the registering of hooks and plugins. Such as VisualEditor, WikiLove, and others. These don't use dependencies, as VE doesn't require user scripts. And user scripts may contain snippets that throw an error, which should not lead prevent features from loading. This doesn't work reliably in Vector 22 today.
• Internal load order. The established load order is common.js -> skin.js, and likewise for common.css -> skin.css. This doesn't work reliably in Vector 22 today.
• External load order. The established load order is site -> user. Similar to extension support, user scripts run regardless of site-wide scripts throwing an exception, hence no dependency but a graceful await (like Promise.finally).
• Global scope. User scripts run in the global scope. This is applied to the user module, but in Vector 22 the variables in vector.js are undefined. E.g. put var x_foo = 123; console.log('x_foo', window.x_foo); in vector.js and you'll get foo 123 in Vector, but foo undefined in Vector 22.

Recommendation

• Remove VectorResourceLoaderModule overrides.
• (Maybe) Run a maintenance script to one-time copy over pages.

If we treat script loading like we treat skins everywhere else in the platform — as separate — then its straight-forward and self-explanatory where site-wide and personal customisations go.

Are we worried that those who have personal customisations can't find their script or know to copy it over? I suppose we could offer a button or nudge of sorts to one-time copy it over. E.g. a little gadget that if vector.js is non-empty and vector-2022.js doesn't exist, offer a button that copies it with mw.Api, and a "No thanks" that creates it an empty page, or remember in localStorage and ask again in a month.

If the automatic and silent way of today is preferred, we could run a little maintenance script that performs the copy once for all vector.css/js. That would be easy to do. And it leaves a paper trail of what and why. That ticks all the benefit boxes with none of the downside boxes!

This solves T297758 in that pre-existing vector.css/js site-wide and personal files are loaded. Yet, the architecture would be coherent, and editors can easily improve or remove pieces as-needed. Though, I think it'd be even better to not bother a copy, and give users agency. Let them choose this themselves, after a good first impression, with the opportunity to copy things over as they see fit, and naturally know what if anything needs further attention.