Page MenuHomePhabricator

Remove legacy javascript globals
Open, LowPublic

Description

For readers of Tech News (October 2019):

Gadgets and user scripts can access variables about the current page in JavaScript. In 2015, this information was moved from global variables named wg* to mw.config. The old way via wg* global variables will be removed later this year.

To find out if a gadget or user script that you use is affected by this change, open the JavaScript console in your browser. If you see warnings like the below, then you are using a gadget or script that will stop working if the issue is not fixed:

Use of "wgPageName" is deprecated. Use mw.config instead.

The following may help with finding the problem script or with fixing the issue once once found:

If you want your wiki to be one of the first to try the new way that no longer has these global variables, leave a comment below. This can help find new issues that we may have missed. If your wiki choose to enable the new way, you can let us know to undo it if a problem is found.


Timeline
  • Add option to disable legacy globals: wgLegacyJavaScriptGlobals. – T30916, 49ce5de7d94 (MediaWiki 1.17 in 2011)
  • Add deprecation notices for accessing mw.config.get() when wgLegacyJavaScriptGlobals is enabled. – T58550, 24f84b08cf91ef (MediaWiki 1.25 in 2014)
  • Migrate usage and gradually disable on Wikimedia wikis:
    • Turn it off on test2.wikipedia.org. – T67011, bbfba08d31 (2014)
    • Fix the Collection extension to not depend on these. T177259
    • Turn it off on group0 wikis. (test.wikipedia.org, mediawiki.org, and read-only wikis)
    • Turn it off for developers and in Continuous Integration for code merges. (DevelopmentSettings)
    • Turn it off on Beta Cluster for all wikis.
    • Turn it off on group1 wikis. (mostly non-Wikipedia)
    • Turn it off everywhere. (+group2: Wikipedia)
  • Turn it off by default in MediaWiki core. – T35837 (in release 1.35)
  • Remove the feature. – (in release 1.36 or later)

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 541581 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[operations/mediawiki-config@master] [Beta Cluster] Disable wgLegacyJavaScriptGlobals

https://gerrit.wikimedia.org/r/541581

Change 541581 merged by jenkins-bot:
[operations/mediawiki-config@master] [Beta Cluster] Disable wgLegacyJavaScriptGlobals

https://gerrit.wikimedia.org/r/541581

For Tech News:

Gadgets and user scripts can access variables about the current page in JavaScript. In 2015, this information was moved from global variables named wg* to mw.config. The old global variables will be removed later this year. You can tell the developers if you want to try this out on your wiki first, at T72470.

I've added more details to the task description as well.

For Tech News:

Gadgets and user scripts can access variables about the current page in JavaScript. In 2015, this information was moved from global variables named wg* to mw.config. The old global variables will be removed later this year. You can tell the developers if you want to try this out on your wiki first, at T72470.

I've added more details to the task description as well.

I'm catching up on this: has it been done, or announced?

It has been done for DevelopmentSettings, Beta Cluster, group 0 and test2wiki. It has not been done for group 1 or group 2.

@Trizek-WMF The announcement still needs to be made yes. The change has not yet been deployed to wikis beyond beta and testing.

Thank you for your replies. I'm adding the call for testers on Tech News.

@Krinkle and @Nirmos, is there a schedule for the next groups?

The next group will happen some weeks after we have a few early adopters, which is what I'm hoping to get through Tech News.

@Krinkle: I'm available if you want to deploy to svwiki.

Change 558621 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[operations/mediawiki-config@master] Disable wgLegacyJavaScriptGlobals on svwiki

https://gerrit.wikimedia.org/r/558621

Change 558621 merged by jenkins-bot:
[operations/mediawiki-config@master] Disable wgLegacyJavaScriptGlobals on svwiki

https://gerrit.wikimedia.org/r/558621

@Nirmos Sorry about the delay! The change has now been applied.

Before I applied, it I browsed around the svwiki's main page and several special pages and random articles and did not see any site-side scripts or gadgets that rely on the legacy feature. Let me know if you find any issues (either here, or on IRC or via wiki email, whichever you prefer).

Mentioned in SAL (#wikimedia-operations) [2020-02-12T20:38:55Z] <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T72470 - Disable wgLegacyJavaScriptGlobals on svwiki (duration: 01m 08s)

Some before-and-after stats with regard to the number of global variables. This assumes Object.keys( window ).length and https://sv.wikipedia.org/wiki/Portal:Huvudsida

The old values were recorded on November 16 2019 and the new values just now. The odd thing is that logged out users now have more global variables than logged in users. I don't know how that's possible.

BeforeAfterDecreaseDecrease (%)
Logged in39123915238.9
Logged out37924113836.4

Found it. Redux and ReduxThunk exist while logged out, but not while logged in. No idea what that is.

Edit: Seems to be a dependency for hovercards, which makes sense.

Redux is this: https://en.wikipedia.org/wiki/Redux_(JavaScript_library)

It's apparently used by the Popups extension, which is apparently loaded for anons but not for logged in users.

I have been trying to see if we can deploy this in wikidata and Persian Wikipedia and I couldn't find any deprecation warnings when I opened several pages but is there a better way? For example sending a report to logstash in these cases (similar to CSP violations) or a codesearch in mwgrep?

@Ladsgroup I don't think there are any more uses in core or extensions. The main concern is gadgets and user scripts, so we need (technical) users to each look out for their own personal scripts and deprecation warnings they get.

Using an insource-search I see about 100 results on wikidata.org. But it's possible many of those are unused files (e.g. an import or skin that they no longer use).

@Ladsgroup I don't think there are any more uses in core or extensions. The main concern is gadgets and user scripts, so we need (technical) users to each look out for their own personal scripts and deprecation warnings they get.

Using an insource-search I see about 100 results on wikidata.org. But it's possible many of those are unused files (e.g. an import or skin that they no longer use).

I started a discussion in WD:PC and I will drop it in Monday. It should be okay given that it's deprecated six years ago and javascript is not part of stable interface policy.

Change 596180 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Disable wgLegacyJavaScriptGlobals on fawiki and wikidatawiki

https://gerrit.wikimedia.org/r/596180

Change 596180 merged by jenkins-bot:
[operations/mediawiki-config@master] Disable wgLegacyJavaScriptGlobals on fawiki and wikidatawiki

https://gerrit.wikimedia.org/r/596180

Mentioned in SAL (#wikimedia-operations) [2020-05-13T11:14:41Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:596180|Disable wgLegacyJavaScriptGlobals on fawiki and wikidatawiki (T72470)]] (duration: 01m 06s)

Are we going to turn this off by default, at least, for 1.35? Or should we re-tag to 1.36?

I think flipping the default for the 1.35 LTS should be fine now, yeah, let's do it?

Change 617193 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/core@master] Disable wgLegacyJavaScriptGlobals by default

https://gerrit.wikimedia.org/r/617193

Change 618030 had a related patch set uploaded (by Reedy; owner: Jforrester):
[mediawiki/core@REL1_35] Disable wgLegacyJavaScriptGlobals by default

https://gerrit.wikimedia.org/r/618030

Change 618030 merged by jenkins-bot:
[mediawiki/core@REL1_35] Disable wgLegacyJavaScriptGlobals by default

https://gerrit.wikimedia.org/r/618030

Change 617193 merged by jenkins-bot:
[mediawiki/core@master] Disable wgLegacyJavaScriptGlobals by default

https://gerrit.wikimedia.org/r/617193

Can we also enable this on meta.wikimedia.org given it can contain global JS ?
What is the plan and the blockers for rolling this out further?

Can we also enable this on meta.wikimedia.org given it can contain global JS ?

The config applies to when/where the script executes, not where it is stored. Can you elaborate on why you'd like to do Meta-Wiki next?

What is the plan and the blockers for rolling this out further?

Lower numbers at https://grafana.wikimedia.org/d/000000037/mw-js-deprecate?panelId=7&fullscreen. Note that those numbers are 1/100 of the actual numbers, and also doesn't count anons.

The config applies to when/where the script executes, not where it is stored. Can you elaborate on why you'd like to do Meta-Wiki next?

If a user is using global JS on meta wiki they have a chance of seeing the client side error when they next edit and realize it doesn't work before it appears on mediawiki.org =.

Lower numbers

Yes but what kind of outreach (e.g. user notice) has been done and is planned to get lower numbers?

Has it been considered using mw.notify to notify users of these problems e.g. "There is a problem with one of the gadgets you have installed. This error will continue to show until the associated gadgets have been removed or updated." ?

Given production code is not using these globals I think it's fair to use mw.notify in this way and makes the error visible to the users using them and the maintainers.

I'm particularly concerned with these errors going unfixed as now that I'm reviewing logstash for client side errors, on many occasions I've been able to identify users and the pages they are visiting through the pages they edit.

@Krinkle any objections to enabling this on meta wiki? After enabling client error logging on meta wikipedia I've managed to squash a huge bug that was impacting lots of people loading the script in foreign wikis via global scripts by observing a single occurrance of an error there that happened on meta wiki. I truly believe having it enabled on meta wiki, where global scripts are managed will help with upgrading this code.

Using mw.notify has been considered in the past. It is used on some wikis already for users in the interface-admin or sysop group. Beyond that, I don't think it's something we should use for end-users as for the most part these errors are inconsequential to them (whatever is broken isn't something they want, need, or know about), and not in their realm to understand or know how to do anything about.

Using Meta-Wiki as the next step seems fine. I do think it would be fairly disruptive though, and I'm not sure about most global scrips being maintained there. That's not been in my experience anyway. User's personal global.js is there of course, but I generally don't actually utilise the scripts there, and the config of meta-wiki doesn't apply to scripts imported at run-time from Meta-Wiki to local wikis, where the errors are generally seen.

Want to write the blurb for the next Tech-News?

We have client side error logging on meta wikipedia now, so I'm happy to jump in and fix the most prominent errors and/or generate a list of scripts that are still active and broken.

Beyond that, I don't think it's something we should use for end-users as for the most part these errors are inconsequential to them (whatever is broken isn't something they want, need, or know about), and not in their realm to understand or know how to do anything about.

A quick search on the English Wikipedia for deprecated globals shows that they're being used on about 5,000 js pages, most of which are in the userspaces of editors that are not admins or iadmins: link.

Almost half of those are used directly in users' common.js, vector.js, or monobook.js: link

Can TourBot be used to fix uses? As an interface admin on meta I'd be willing to help replace the uses if I understand how to use tourbot

(I don't know what Tourbot is)

A tool by Krinkle for mass code migration. I am pretty sure Krinkle runs it on his account, though the framing of the documentation makes it seem that he is optimistic that other people would run it ;). See also Le Tour de Wikí.

A quick search on the English Wikipedia for deprecated globals shows that they're being used on about 5,000 js pages, most of which are in the userspaces of editors that are not admins or iadmins: link.

My experience shows many of these are inactive. What will be more reliable is seeing the spikes on production error logs.

My recomemndation would be to speed up the deprecation here. For all wikis where we see a large amount of errors in production logs I can patch MediaWiki:Common.js as appropriate using mw.log.deprecate.

I think getting this code out of the repositories should be our priority here. We have mechanisms in place for reducing disruption to editors and I'm happy to be responsible for those changes.

Lower numbers at https://grafana.wikimedia.org/d/000000037/mw-js-deprecate?panelId=7&fullscreen. Note that those numbers are 1/100 of the actual numbers, and also doesn't count anons.

Is there a graph for this filtered by different wikis?

A quick search on the English Wikipedia for deprecated globals shows that they're being used on about 5,000 js pages, most of which are in the userspaces of editors that are not admins or iadmins: link.

My experience shows many of these are inactive. What will be more reliable is seeing the spikes on production error logs.

Indeed. Code search should not inform deprecation timelines for user-generated JavaScript/CSS. There is far too much baggage, copies, sandboxes and unused/archived scripts. This is fine, and often desired/intentional. Not to mention that there's also lots of potential for conflict or accidental naming of things in ways that might match something we're looking for.

We already have live tracking of actual hits to the deprecated code path, which seems preferred:
https://grafana.wikimedia.org/d/000000037/mw-js-deprecate

My recomemndation would be to speed up the deprecation here. For all wikis where we see a large amount of errors in production logs I can patch MediaWiki:Common.js as appropriate using mw.log.deprecate.

I'm not sure what you mean by that. Apart from one module, the user, for {common,skin}.js, there is generally no expectation for core, gadgets, global user scripts, or other site scripts, that site would run first, or even that the site module is in working order.

I think getting this code out of the repositories should be our priority here. We have mechanisms in place for reducing disruption to editors and I'm happy to be responsible for those changes.

Which code? This task is specifically about wg mw.config variables. This involves about three lines of code, that as maintainer I excercise responsibility for and don't think our users are ready for us to drop. I'm not sasying that it needs to take a long time, or that we need to wait. It just takes effort to work with individual communities on this, and I simply don't see it benefiting my team or MW in general to work on this task right now.

Are there other on-going JS deprecations you think are due for removal and/or that would benefit from being resourced for a spring to finish soon?

We've deprecated, announced, helped migrate, and removed dozens of JavaScript method and modules over the past two years. These generally can and do come and go quickly, in the order of days, weeks or a few months. This task is only about the wg variables.

I will help with this if/when I can get global interface editor right (requested a couple of minutes ago, will take two weeks).

I will help with this if/when I can get global interface editor right (requested a couple of minutes ago, will take two weeks).

Got it now, will clean up the wikis soon.

So I cleaned only wgServer and only in mediawiki namespace across all wikis. I wait to see how it gets reflected in the numbers and then move on to the next one.

So I cleaned only wgServer and only in mediawiki namespace across all wikis. I wait to see how it gets reflected in the numbers and then move on to the next one.

Out of interest, are you having to do this manually or do you have a bot of some kind doing find/replace?

I wrote a python bot (based on pywikibot) to find and replace them but it's semi-automated, meaning I approve each one of them (and it has more features, like showing a bigger diff, editing it in browser or editor) so it empowered me to do hundreds in a couple of hours. I can share the code with you if you want to. One note though: I don't feel comfortable having a bot password that can edit site-wide js so I made one that's bound to my IP, do that it you want to get a bot password for it.

Number-wise, I don't see a huge drop for wgServer while I removed it from mediawiki namespace from basically everywhere (including several MediaWiki:Common.js ones). It might be that it needs to propagate through caches *shrugs*

Most of the errors I've seen so far in this form are typically coming from user scripts.

hmm, can you somehow give me a list of top used user scripts? I can feed it to my bot.

hmm, can you somehow give me a list of top used user scripts? I can feed it to my bot.

Sadly not. The errors I see are on the wikis where we've already disabled the config flag

My recomemndation would be to speed up the deprecation here. For all wikis where we see a large amount of errors in production logs I can patch MediaWiki:Common.js as appropriate using mw.log.deprecate.

This is what I was getting at here. Perhaps a better approach is to switch the flag on a wiki, use logstash to work out where the errors are coming from and then run the script to fix them. Once the errors are back to zero, repeat on a different wiki.

hmm, can you somehow give me a list of top used user scripts? I can feed it to my bot.

Sadly not. The errors I see are on the wikis where we've already disabled the config flag

That has survivor bias. We first clean the mediawiki namespace of those wikis before disabling global variables.

My recomemndation would be to speed up the deprecation here. For all wikis where we see a large amount of errors in production logs I can patch MediaWiki:Common.js as appropriate using mw.log.deprecate.

This is what I was getting at here. Perhaps a better approach is to switch the flag on a wiki, use logstash to work out where the errors are coming from and then run the script to fix them. Once the errors are back to zero, repeat on a different wiki.

I'm not 100% sure, at least I think we must clean mw namespace first before moving forward. I have been fixing cases in Common.js, we can't break a whole wiki because they are too small to have tech savvy admins.

My suggested course of action:

  • Clean up mediawiki namespace of wikis of these globals
  • Turning it on for a set of wikis
  • See the errors (coming from user scripts) fix the top offending ones, leave the long tail.
  • Move on to the next batch.

It's basically similar to your suggestion but with fixing mw first. Even admins in English Wikipedia decided they won't clean up all user scripts: https://en.wikipedia.org/wiki/Wikipedia:Interface_administrators%27_noticeboard/Archive_2#Preparing_for_removal_of_Javascript_globals

FWIW on enwiki (which I presume is the bulk of it), I intend on reviving that this weekend since I do have a list to go off of.

In 2015, this information was moved from global variables named wg* to mw.config.

As a side note, MW also declares a few globals without the wg* prefix, namely: debug, skin, stylepath.

I'm not 100% sure, at least I think we must clean mw namespace first before moving forward. I have been fixing cases in Common.js, we can't break a whole wiki because they are too small to have tech savvy admins.

Please keep in mind that needs interface-admin permissions, which are even sparser.

An update: wgNamespaceNumber is now done on mw namespace of all wikis (including at least ten wikis on common.js). Will see how it'll show itself in the next week.

It's basically similar to your suggestion but with fixing mw first.

That was pretty much my suggestion? :) Although I don't think. On some wikis, I assure you there have been both gadget and user errors coming in thick and fast. The gadget ones obviously got fixed right away, but the user ones keep coming.

Even admins in English Wikipedia decided they won't clean up all user scripts: https://en.wikipedia.org/wiki/Wikipedia:Interface_administrators%27_noticeboard/Archive_2#Preparing_for_removal_of_Javascript_globals

FWIW I think we should be deleting user scripts that haven't been edited in over 2 years. The software should take care of that IMO, given code from 10 years ago, can't be expected to still run today :) It might also be worth to point out the privacy concerns of allowing these to stay broken. Given we now log errors, a user script generated error - particularly one run on every page - does leak information unavoidably (albeit to people that have signed NDAs) about browser history across Wikipedia.

wgNamespaceNumber shows immediate drop, will see if it continues to drop or not:

Wouldn't it be easier to just replace all variables at once?

Wouldn't it be easier to just replace all variables at once?

And how do you propose to do that?

Well, whereever in Ladsgroups current code we have wgNamespaceNumber we could just have a list of all globals which need replacement. In particular, this means we do not have to visit wikis multiple times.

There's a discrete (if lengthy) number of items, so one could list 'em all. Alternatively, regex makes it fairly doable (this is what I was doing).

One word of caution, esp. re: automation: I found that occasionally someone would define something like wgPageName = mw.config.get('wgPageName'), making wgPageName quite usable, and possibly overwritten, so some reading of the page in question is necessary.

Wouldn't it be easier to just replace all variables at once?

I considered it an the code can handle it too but I decided against it: - Since it's semi-automated I want to make the diff small as as possible to make it less prone to error (specially since it's interface and not a random vandalism in articles) - It would split the work to manageable chunks.

There's a discrete (if lengthy) number of items, so one could list 'em all. Alternatively, regex makes it fairly doable (this is what I was doing).

One word of caution, esp. re: automation: I found that occasionally someone would define something like wgPageName = mw.config.get('wgPageName'), making wgPageName quite usable, and possibly overwritten, so some reading of the page in question is necessary.

Yes, the code has a part to avoid changing scripts like that.

One word of caution, esp. re: automation:

In case it's relevant, I'm also quite used to the following pattern:

var config = mw.config.get( [ 'wgAction', 'wgNamespaceNumber' /* , ... */ ] );

if ( config.wgAction === '...' ) {}

wgPageName is also cleaned from mediawiki namespace now.

I'm going to include a mention of this in the next issue of my user scripts newsletter on enwiki (https://en.wikipedia.org/wiki/Wikipedia:Scripts%2B%2B/Next) - any copy edits to my blurb would be appreciated.

So I built a list of the most used user scripts (based on the number of requests loading that script) and have been cleaning those too. Some variables are GDFR:

But since it's user scripts, it's a bit controversial.

Just by means of an enwiki update, I wanted to note this discussion on the int-admin's board: https://en.wikipedia.org/wiki/Wikipedia:Interface_administrators%27_noticeboard#Removing_legacy_javascript_globals_from_skin_pages

Basically, the imported user scripts have either been fixed or, if their "owners" are active, they've been notified; if there's no response or action in a few days or a week or so, I'll fix the remainders. The majority of cases, however, are presumably coming from user's skin pages; I've got that list, sorted by active users and skin. After some discussion, I've posted a note at the technical noticeboard for editors to see and be aware of the process; barring major opposition, I'll go ahead in a few days. Presumably that should be a chunk of these, and I'll report back here when done.

legacy variables are slowly decreasing:

My suggestion now. Given that I have cleaned as much as possible for now. I suggest sending a message to tech news and disabling it for group1 one week afterwards. Does that sound good to you @Krinkle and @Jdlrobson ?

Group1 is fine, assuming that any spikes in errors on logstash will be dealt with in follow-ups (or the change rolled back after making that data available). This is important to me given the error rates are currently pretty low and we have error alerting setup. Post-change we'll want to make sure

I'd suggest for that reason deploying on a Monday and making sure at the end of the week we are below the current threshold here: https://grafana.wikimedia.org/d/000000566/overview?viewPanel=16&orgId=1.