Page MenuHomePhabricator

Remove document.write support (deprecated since 1.26)
Open, LowPublic

Description

It's been deprecated for a while. I attempted to remove it last week, which led to T193191. Let's continue here to form a proper plan.

Root cause

The root cause for T193191 was use of document.write(). In general, document.write() is only safe to use for the purpose of adding arbitrary HTML tokens into the HTML stream that the browser parses from the server, and only from an inline or synchronous script referenced in the HTML source.

Using it from an asynchronous script has never worked reliably in any browser. Trying to write from an async script, always behaves the same: It cannot add to the page because the page is already "closed" (e.g. reached </html>). Ideally, a browser would ignore this impossible instruction.

Instead, for compatibility with web sites that predate the existence of Wikipedia, browsers must assume the script intended to create a new document but forgot to call document.open() before document.write(), where document.open() will create a new page (replacing the current one). This causes the infamous blank page. To emphasise, this is not something MediaWiki does. Browsers do this intentionally for compatibility with very old web sites that are even older than our legacy scripts from 2010.

Untill 2013, MediaWiki loaded scripts synchronously, which meant document.write() just worked. After that, we've started to load scripts asynchronously. The good thing is, our users don't use document.write() for adding visible page content in a specific place. Instead, they typically used it to add invisible elements, such as <style>, <script>, or model dialogs. This meant that in 2015, after numerous reports about old scripts not working on various wikis, I decided to do something the browser cannot do: Magically make document.write() work, by removing the browser method, and replacing it with our own. Our version would invisibly parse the HTML and add it to the end of the page. (T108139, d861c6593a).

This has various maintainability issues and cannot be kept forever. For example, our "magic way" to make document.write work for our old scripts, has the down-side of also breaking other ways to use the method, for example, it caused site notices to sometimes appear at the bottom of the page (depending on how fast the page loaded) - see T125323 for details. This was caused by us, and would not have happened had we kept document.write unmagicified. Basically: Either we break modern content, or we break old scripts. There is no way to have both.

Our version of document.write has been deprecated with console warnings for three years, since 2015. There are still lots of search results (pasted at T193191#4163158), but the good news is that most are not active issues. Many are unused scripts (like unused templates), or technically loaded/used but do nothing or break due to unrelated issues before reaching the call to document-write. The number of actual visible issues that cause blank pages, should be small.

Last week, I realised that browsers have a built-in mechanism to automatically ignore document.write() for asynchronous scripts. This seemed like a good alternative to our current "magical" legacy version because that means it still works for good code that uses the method synchronously (like DismissableSiteNotice), and doesn't blank pages when called from async scripts, but still gets a console warning (like now, but from the browser instead of us), and all that – without having to maintain our own JavaScript code to make it work.

Unfortunately, as we saw today, this mechanism does not work the way I thought it would. While all browsers support it (and have for many years), it only affects top-level script execution. The trick does not work for indirect execution via timeouts, events, callbacks etc. And we load scripts in MediaWiki indirectly (usually via "Promise", or "requestIdleCallback", or "requestAnimationFrame"). After some research, I found browsers behave this way intentionally for compatibility reasons (https://www.w3.org/Bugs/Public/show_bug.cgi?id=9767) and to keep predictable behaviour for code that runs indirectly, impacting only top-level execution (sync vs async). I have an isolated test case (https://codepen.io/Krinkle/pen/pVjORL/?editors=0010), but the short version is: upstream browsers cannot help us.

Past

  • 2010 (MediaWiki 1.17):
    • Script were synchronous, slow, and HTML display was interruptible by scripts.
    • No ResourceLoader.
    • And at this time, most browsers (except IE) allowed document.write() always, possibly blanking the page in certain cases.
  • 2011 (MediaWiki 1.18):
    • ResourceLoader introduced. Scripts were still synchronous and behave the same way as before.
    • Wikibits deprecated.
    • Meanwhile, Firefox and Safari started to ignore document.write() on async scripts. Other browsers followed.
  • 2013 (MediaWiki 1.23):
    • Wikibits deprecation warnings added to the browser console.
  • 2015:
    • ResourceLoader scripts made asynchronous.
    • Some legacy scripts using document.write() broke.
    • Created a magic replacement for document.write() that works by parsing and appending to the page (T108139, d861c6593a).
    • Deprecated document.write() with console warnings.

Future

For the short-term we have no choice but to revert and keep our magical version of document.write(), because the version that browsers provide is not compatible with our legacy scripts. To avoid this problem in the future I propose a deadline to permanently remove this feature. After that, we will disable document.write() within MediaWiki.

Proposal:

  1. Announce in Tech News that document.write has been deprecated with console warnings since 2015, reach out to JavaScript/Gadget developers in the community to look for console warnings related to this and decide to either remove the problematic code (if not used), or migrate to more modern methods. For support there is https://www.mediawiki.org/wiki/ResourceLoader/Migration_guide_(users) and its talk page.
  2. Update our document.write() to do "nothing" (except console warning, like now). This will match the behaviour of browsers for document.write in async scripts.

During this "phase 2", anything that relies on document.write() will fail but in an isolated way (That means, no blank pages, the page will work but without the script). These can then be found and addressed. The start of Phase 2 will also be in Tech News with the same links for support. This time reaching to look for local functionality that is missing or different.

  1. Scan for remaining uses and remove them, and remove our document.write().

Event Timeline

Krinkle created this task.Apr 28 2018, 2:16 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 28 2018, 2:16 AM
Krinkle triaged this task as Low priority.Apr 28 2018, 2:18 AM
Krinkle moved this task from Inbox to Backlog on the MediaWiki-ResourceLoader board.
TheDJ added a comment.Apr 30 2018, 7:38 AM

The user area is very hard to cleanup, because it's unclear who is running a userscript from another user, and who isn't. Which user is even active, and which one isn't.

From experience, i think you will need to be a lot more direct/invasive/persuasive to handle these last cases. I rarely find document.write on en.wp any longer, but every once in a while, there's a user, or a returning user who I encounter, and they still have it in a userscript, or monobook.js. Unless these people are told they have a problem, these people are basically invisible (some have been so used to all their scripting being broken, they don't even realise they are missing functionality) and I'm sure this is much much worse on the many other wiki's.

Krinkle updated the task description. (Show Details)Jun 11 2018, 2:14 PM
Od1n awarded a token.Jun 24 2018, 12:31 AM
Vvjjkkii renamed this task from Remove document.write support (deprecated since 1.26) to h1daaaaaaa.Jul 1 2018, 1:14 AM
Vvjjkkii raised the priority of this task from Low to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
Mainframe98 renamed this task from h1daaaaaaa to Remove document.write support (deprecated since 1.26).Jul 1 2018, 8:08 AM
Mainframe98 lowered the priority of this task from High to Low.
Mainframe98 updated the task description. (Show Details)
Mainframe98 added a subscriber: Aklapper.
Krinkle edited projects, added Technical-Debt (Deprecation); removed Technical-Debt.
Krinkle moved this task from Not yet to <= 2015 / MW 1.26 on the Technical-Debt (Deprecation) board.

The user area is very hard to cleanup [..]

Yeah, agreed. And the breakage, while minimal quantity, is very disruptive due to the browser blanking the page (instead of e.g. an error message).

Hence I recommend that when we "remove" it, we actually replace it with a no-op that warns in the console. That way, for those that actively use the scripts they are loading and notice their absence, can seek to upgrade them to supported load methods. And for scripts that don't work or have been forgotten, automatically unload without further visible disruption.

What I haven't yet figured out is where to do the replacement, and whether we can/should do it in a way that e.g. inline scripts part of an extension can still use document.write() in the way that browsers support and we could support without issue.