Page MenuHomePhabricator

Wikimedia wikis should use https:// in $wgServer
Open, MediumPublic

Description

Currently most Wikimedia wikis use protocol-relative URLs in $wgServer, which means that wfExpandUrl with PROTO_RELATIVE will expand to a protocol-relative URL. That does not help Wikimedia Foundation wikis, since we don't actually use HTTP in any URLs anymore.

When third parties use an HTML snippet from a MediaWiki API that uses PROTO_RELATIVE (which is arguably the most correct choice for our API), and displayed that on a third-party site over plain HTTP, this will also connect to us over HTTP (exposing users to MITM), and incur a performance hit (because of the extra redirect).

Use cases

[…]

Below are are examples of ossified code in gadgets and user scripts that have come to rely on a protocol-relative value in mw.config wgServer:

  • Extract origin hardcoded:
    • Example: https://tools.wmflabs.org/pageviews#…&project=' + mw.config.get('wgServer').substring(2)
    • Problematic part: mw.config.get('wgServer').substring(2)
    • Recommended: mw.config.get('wgServerName')
  • Link to a wiki page
    • Example: '<a href="https:' + mw.config.get('wgServer') + mw.util.getUrl('Special:Contributions/Blankpage') + '">blank</a>'
    • Comment: There is no need for a full URL here, and thus no need to involve wgServer in the first place. Calling mw.util.getUrl() suffices, because HTML like <a href>, and CSS, and JS features like anchor.href = , location.href = , document.location = assignments, and XHR/jQuery.ajax, all naturally accept relative URLs. MediaWiki has also done this for decades. View source on any Wikipedia article and find a blue link, which are rendered as <a href="/wiki/Foo">`.
    • Recommended: '<a href="' + mw.util.getUrl('Special:Contributions/Blankpage') + '">blank</a>'
  • Obtain full URL for external use
    • Example: 'https://tools.wmflabs.org/wikilint/cgi-bin/wikilint?…&url=http:' + mw.config.get( 'wgServer' ) + mw.util.getUrl( mw.config.get( 'wgPageName' ) )
    • Comment: This is ossified in two ways, it assumes a protocol-relative wgServer, and assumes that getUrl returns a path-only value, and redundantly passes wgPageName to mw.util.getUrl, which is its default already. Note that in most most cases, a full URL is not needed and mw.util.getUrl() suffices. If a full URL is needed, expand it with via new URL(…, location).toString() instead.
    • Problematic part: 'http:' + mw.config.get( 'wgServer' ) + mw.util.getUrl( mw.config.get( 'wgPageName' ) )
    • Recommended: new URL( mw.util.getUrl(), location ).toString()

[…]

Outline

  • Audit usage of mw.config wgServer in JavaScript pages on Wikimedia wikis
    • Invite Wikitech ambassadors via Tech News to address hardcoded assumptions based on results in Global Search link at T118413#10973765.
    • Fix remaining prominent gadgets to migrate hardcoded assumptions about protocol-relativity, as outlined above.
    • Triage remaining results to look for new use cases that we don't have a solution for yet, and report them on this task.
    • Address those new use cases, if any.
  • Switch WMF wgServer to https.

Details

Event Timeline

Tgr raised the priority of this task from to Needs Triage.
Tgr updated the task description. (Show Details)
Tgr added a project: Wikimedia-Site-requests.
Tgr added subscribers: Tgr, BBlack, ori.

Change 253053 had a related patch set uploaded (by Gergő Tisza):
Improve the documentation of wfExpandUrl

https://gerrit.wikimedia.org/r/253053

Luke081515 triaged this task as Medium priority.Dec 9 2015, 8:43 AM

This task description says that "we don't actually support HTTP anymore" and @Bawolff said the same on IRC this evening. This statement feels misleading since we do support HTTP and will continue to do so indefinitely. Keeping the HTTP --> HTTPS redirects working means supporting HTTP, as far as I'm concerned.

See also T54253.

Right, T54253 is the task where Tim lays out why protocol-relative URLs aren't really that great. Adding explicit protocols and folding this task into T54253 seems reasonable to me.

This task description says that "we don't actually support HTTP anymore" and @Bawolff said the same on IRC this evening. This statement feels misleading since we do support HTTP and will continue to do so indefinitely. Keeping the HTTP --> HTTPS redirects working means supporting HTTP, as far as I'm concerned.

To be pedantic, I mean we no longer support serving mediawiki traffic as HTTP, so we would not want to generate any urls using anything other than https, which is what $wgServer is controlling.

When the task was written, we still allowed HTTP POST to the API (since redirects don't work there) and that could cause problems with relative URLs. Now that HTTP is completely disallowed (not from the user's POV but in the sense that our MediaWiki servers will never receive one) relative URLs are less problematic, but they still require some work to turn into absolute URLs so replacing the relative URLs in the configuration would be a performance (micro)optimization, at least.

Just wondering if it's better to continue with T207073: Test wg(Canonical)Server as defined currently (it is based on current standard) or block it on this task?

I suppor this change. Note however, that a great many user scripts and gadgets rely on the fact that mw.config.get('wgServer') is protocol-relative. Using constructs such as location.protocol + wgServer to form URLs.

Before we can fliip the switch, we'll need to survey the land (using mwgrep or https://global-search.toolforge.org/), identify the common patterns, figure out how they can be done instead in a way that works both today and after the change, then document this migration guide, and announce it through Tech News.

I suppor this change. Note however, that a great many user scripts and gadgets rely on the fact that mw.config.get('wgServer') is protocol-relative. Using constructs such as location.protocol + wgServer to form URLs.

Good catch!

What would be the sane way to support gadgets without them having to know whether wgServer is protocol-relative? Having something like wgOrigin? Or just expect them to do a manual check?

Tgr removed Tgr as the assignee of this task.Oct 28 2020, 12:33 AM
Tgr added a project: User-Tgr.

Moving from assignee to user tag to indicate that I'd like to fix this but have no timeline whatsoever.

What would be the sane way to support gadgets without them having to know whether wgServer is protocol-relative? Having something like wgOrigin? Or just expect them to do a manual check?

If we could do this over again, I would've made wgServer always expanded and just not support client-side awareness protocol-relativity, at least not within core. The base module could have contained an expression like location.protocol === 'http:' ? 'http:' : 'https:' in front of the mw.config assignment if the server-side is configured in a protocol-relative manner. Note that before protocol-relative support was added wgServer in JS was already always expanded and its this change that first broke lots of JS code that was not expecting that. A decade later, the opposite compat issue has emerged.

Change 253053 merged by jenkins-bot:

[mediawiki/core@master] Improve the documentation of wfExpandUrl

https://gerrit.wikimedia.org/r/253053

I've audited Global Search for wgServer usage.

Below are are examples of ossified code in gadgets and user scripts that have come to rely on a protocol-relative value in mw.config wgServer:

  • Extract origin hardcoded:
    • Example: https://tools.wmflabs.org/pageviews#…&project=' + mw.config.get('wgServer').substring(2)
    • Problematic part: mw.config.get('wgServer').substring(2)
    • Recommended: mw.config.get('wgServerName')
  • Link to a wiki page
    • Example: '<a href="https:' + mw.config.get('wgServer') + mw.util.getUrl('Special:Contributions/Blankpage') + '">blank</a>'
    • Comment: There is no need for a full URL here, and thus no need to involve wgServer in the first place. Calling mw.util.getUrl() suffices, because HTML like <a href>, and CSS, and JS features like anchor.href = , location.href = , document.location = assignments, and XHR/jQuery.ajax, all naturally accept relative URLs. MediaWiki has also done this for decades. View source on any Wikipedia article and find a blue link, which are rendered as <a href="/wiki/Foo">`.
    • Recommended: '<a href="' + mw.util.getUrl('Special:Contributions/Blankpage') + '">blank</a>'
  • Obtain full URL for external use
    • Example: 'https://tools.wmflabs.org/wikilint/cgi-bin/wikilint?…&url=http:' + mw.config.get( 'wgServer' ) + mw.util.getUrl( mw.config.get( 'wgPageName' ) )
    • Comment: This is ossified in two ways, it assumes a protocol-relative wgServer, and assumes that getUrl returns a path-only value, and redundantly passes wgPageName to mw.util.getUrl, which is its default already. Note that in most most cases, a full URL is not needed and mw.util.getUrl() suffices. If a full URL is needed, expand it with via new URL(…, location).toString() instead.
    • Problematic part: 'http:' + mw.config.get( 'wgServer' ) + mw.util.getUrl( mw.config.get( 'wgPageName' ) )
    • Recommended: new URL( mw.util.getUrl(), location ).toString()

The following are unaffected patterns which worked before 2014 when wgServer was absolute, and worked after it became protocol-relative, and will continue to work it changes back to absolute.

  • Expand article path
    • Example: url = wgServer + '/wiki/Special:Contributions/'+encodeURIComponent(user)
    • Comment: This is almost certainly redundant, because /wiki/Foo works fine by itself. But, it is harmless in this state and works either way because https://en.wikipedia.org/wiki/Foo and //en.wikipedia.org/wiki/Foo also work fine.
  • Extract protocol prefix slash-aware
    • Example: imagePath = (wgServer.substring(0, 2) != '//' ? location.protocol : "") + '//upload.wikimedia.org'
    • Comment: Again, this is redundant since //upload.wikimedia.org works fine by itself, but, it is harmless and works fine when done in a slash-aware way. A lot of code started doing this around 2014 to try to handle both ways. It seems relevant authors failed to realize 1) that // alone suffices to accomplish that, and 2) that 'location.protocol' + // also accomplishes that. There is never a situation in which this ternary adds value. But... it is harmless as-is and won't be impacted by this task.
  • Detect current URL
    • Example: if (href.indexOf (script) == 0 || href.indexOf (wgServer + script) == 0 || wgServer.indexOf('//') == 0 && href.indexOf (document.location.protocol + wgServer + script) == 0) {}
    • Comment: This pattern was introduced by HotCat.js, and is still around on may wikis. It works fine as-is since it handles both forms.
  • Extract origin slash-aware
    • Example A: domain = mw.config.get( 'wgServer' ).substring( mw.config.get( 'wgServer' ).lastIndexOf("//")+2);
    • Example B: domain = mw.config.get( 'wgServer' ).substring( mw.config.get( 'wgServer' ).lastIndexOf("//")+2);