Page MenuHomePhabricator

mw.util.getUrl using params should use short urls if they exist
Closed, DeclinedPublic

Description

When using mw.util.getUrl with params, it seems that it creates an index.php url even if the wiki has short urls set up, which is what it did until it was changed in rMW687dc7cf57bdc74a80b2ae3176cda1facd535ee9.

var page = mw.config.get('wgPageName');
mw.util.getUrl(page);
mw.util.getUrl(page, {
    action: 'delete'
};

Event Timeline

This is causing a minor inconvenience for us at the Undertale/Deltarune Wiki, since our analytics software is tracking page visits by URL, and all article visits through RelatedArticles get segmented separately from the rest of the visits because RelatedArticles adds wprov=rarw1 (which I assume stands for "Wikimedia provenance") to the page URL, turning it from a pretty URL back into an index.php URL.

I get the reasoning for reverting to $wgScript rather than $wgArticlePath here was that it's similar to how Title::getLocalURL() works. But Title::getLocalURL() fires the GetLocalURL hook which can be used to turn the URL back into a pretty URL, which is what we do, and it works pretty well. There is no such mechanism for mw.util.getUrl() -- perhaps mw.hook could be fired in a similar manner to what GetLocalURL does?

// Proposed behavior for mw.util.getUrl
function betterMwUtilGetUrl(pageName, params) {
    // Just as regular
    const url = mw.util.getUrl(pageName, params);
    const query = $.param(params);
    // Calls a hook
    const output = {pageName, url, query};
    mw.hook('util.getUrl').fire(output);
    // Returns the hook output
    return output.url;
}

// ... In a potential extension or user JS that wants to use pretty URLs
mw.hook('util.getUrl').add(input => {
    const {wgArticlePath, wgScript} = mw.config.get([
        'wgArticlePath',
        'wgScript',
    ]);
    const encodedPageName = mw.util.wikiUrlencode(input.pageName);
    if (input.url === `${wgScript}?title=${encodedPageName}&${input.query}`) {
        input.url = `${wgArticlePath.replace('$1', encodedPageName)}?${input.query}`;
    }
});

// ... When used
console.log(betterMwUtilGetUrl('Page name', {a: 1})); // /wiki/Page_name?a=1

But then again, I'm not sure why neither Title::getLocalURL() nor mw.util.getUrl() respect $wgArticlePath in the first place.

@KockaAdmiralac Four reasons:

  • The server-side PHP code and client-side JS code should behave the same way.
  • $wgArticlePath is for page views, not for other actions. The canonical form for action=history, action=edit, action=raw, etc is through index.php. And this is expected in various places such as when purging the CDN cache for action=history or action=raw. This does not pose any conflict with pageview URL or analytics, because these are not pageviews.
  • There are no parameters in MediaWiki for the page view action (diff/oldid are historically part of action=historysubmit), which means the presence of query parameters is a good proxy for the URL not being a pageview. The wprov parameter is not part of the MediaWiki software.
  • The distinction between /wiki/ and /w/ allows for scalable and cheap distinction between URLs that robots should and should not crawl through robots.txt. It would be prohibitively expensive to have only /wiki/, because that would require us to allow everything in robots.txt and force bots to rely on expensive and un-cachable URLs to be served by our backend servers merely to say robots=noindex to the majority of them. This would cost a ton money and resources.

I'll note that relying on exact URL identity as page identity is not stable because there are effectively infinitely many variations on URL encoding. E.g. /wiki/Banana == /wiki/B%61n%61n%61. What you can do, and what WMF does in its edge layer is:

  1. strip parameteters like utm_, ref, and wprov. VCL example
  2. normalize (most) URL encoding. https://wikitech.wikimedia.org/wiki/URL_path_normalization
  3. extract X from /wiki/X or /w/index.php?title=X (if no other parameters) and use that for analytics. Or, if you must use a URL-like value, rewrite one to the other in a translation step specific to analytics ingestion (avoid real user-facing redirects because this will inevitably cause redirect loops).

As for RelatedArticles specifically, that extension should not be constructing its own URLs, which violates separation of concerns and is unsupported. It should get the URL from the MediaWiki API and strictly append wprov=. This is how other users of wprov operate as well, and is a bug specific to that one extension. Fixing this will not directly solve your problem, because it would still be a "different" URL, but at least you can then strip the query parameter and be left with the same URL after that (instead of still being left with index.php).

Happy to help further with your specific question, but the task as-written is requesting behaviour we explicitly don't want. It would make the JS code yield URLs that in 99% of cases are non-canonical, inconsistent, and threatens scalablity.

The specific case of appending wprov to a canonical URL is unrelated, because appending a parameter (e.g. through new URL() + URL.toString) would not switch away from pretty URLs. That can only happen if code were to construct its own unofficial pageview URLs from scratch by calling mw.util.getUrl, instead of amending an official URL.

Makes sense. We deal with robots by specifically excluding action, diff, oldid and similar parameters in robots.txt, but I get that you would have to track whenever a new parameter appears and (remember to) exclude it from robots.txt.

I'll probably open a new task/submit a patch for RelatedArticles. You might want to copy your response to T305797: WMF wikis don't redirect to short url, and instead shows bare /w/index.php as well.