Page MenuHomePhabricator

Deprecate and remove mediawiki.RegExp
Closed, ResolvedPublic

Description

In 2015 I created this module with the idea that it'd be part of a small group of modules that would host lots of related utilities. The idea being that there would be a small number of modules bundling a larger group of utilities.

The naming scheme was modelled after YUI, which treated its Y constant similar to the browser scope, so you'd have utilities like Y.Object.mixin, and Y.String.includes, etc. I have no idea why I did that, and we never used this style for anything else.

In terms of cost, it made sense at the time because it was a replacement for an existing module, rather than something new. The old module was jquery.mwExtension, which included among other functions, $.escapeRE. That module was deprecated, except for this function. It needed a new home, and we created mw.RegExp for it.

Looking back, this tiny module costs more to describe in startup metadata, dependency relations, and closure wrapping, than the actual contents itself. For an example of how that cost adds up, see T213426#5009603.

Hence, even if it was not loaded on every page, it would save bytes by merging into a module that is. In actually, it is loaded on all pages, and would save even more bytes by merging into a module that also is.

The deprecation for this is low priority because it's only a single module, and it's been around for several years. It's imho not worth the disruption to try and phase this out in a single release cycle.

Plan:

  • MediaWiki 1.34 Move mw.RegExp.escape to mw.util.escapeRE, and make mediawiki.RegExp a deprecated alias for mediawiki.util.
  • MediaWiki 1.35 (next LTS): Remove mediawiki.RegExp.

Details

Related Gerrit Patches:

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Nirmos added a subscriber: Nirmos.Mar 15 2019, 11:22 AM
Krinkle edited projects, added patch-welcome; removed JavaScript.Apr 12 2019, 2:09 AM
Krinkle triaged this task as Medium priority.May 9 2019, 4:46 PM
Krinkle edited projects, added MW-1.34-release; removed MW-1.33-release.
Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)
aaron lowered the priority of this task from Medium to Lowest.Jun 6 2019, 10:41 AM

Hey there, should this be moved to 1.35? The cut is a couple of weeks away. If it needs to go out in 1.34, is there anything I can do to help get it out of the door?

Yep, next step is actionable:

Move mw.RegExp.escape to mw.util, and make mediawiki.RegExp a deprecated alias for mediawiki.util.

I just haven't gotten around to it.

Change 534695 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/core@master] Deprecate mediawiki.RegExp; move only function to mw.util

https://gerrit.wikimedia.org/r/534695

Change 534829 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] mediawiki.util: Merge 'jquery.accessKeyLabel' into this module

https://gerrit.wikimedia.org/r/534829

Change 534835 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/FlaggedRevs@master] Update jquery.accessKeyLabel -> mediawiki.util

https://gerrit.wikimedia.org/r/534835

Change 534836 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/Translate@master] Update jquery.accessKeyLabel (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/534836

Change 534829 merged by jenkins-bot:
[mediawiki/core@master] mediawiki.util: Merge 'jquery.accessKeyLabel' into this module

https://gerrit.wikimedia.org/r/534829

Change 534835 merged by jenkins-bot:
[mediawiki/extensions/FlaggedRevs@master] Update jquery.accessKeyLabel -> mediawiki.util

https://gerrit.wikimedia.org/r/534835

Change 534836 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Update jquery.accessKeyLabel (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/534836

Change 534695 merged by jenkins-bot:
[mediawiki/core@master] Deprecate mediawiki.RegExp; move only function to mw.util

https://gerrit.wikimedia.org/r/534695

Izno added a subscriber: Izno.Sep 7 2019, 4:29 PM

Minor note, there is a duplicate listed dependency at 1702 and 1705 of Resources.php.

Minor note, there is a duplicate listed dependency at 1702 and 1705 of Resources.php.

Fixed in I1ac1cbe20bc66092ce15d426d0d6a8e0c8c8d815.

Jdforrester-WMF edited projects, added MW-1.35-release; removed MW-1.34-release.

Deprecation done; removal will be next release, or later.

Change 535231 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/ContentTranslation@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535231

Change 535232 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/UploadWizard@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535232

Change 535236 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] mediawiki.util: Move mw.RegExp over as well to simplify migration

https://gerrit.wikimedia.org/r/535236

Change 535240 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/Translate@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535240

Change 535242 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/WikiEditor@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535242

Change 535244 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/skins/MinervaNeue@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535244

(Moving to Blocked until the 1.35 cycle begins.)

Change 535240 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535240

Change 535232 merged by jenkins-bot:
[mediawiki/extensions/UploadWizard@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535232

Change 535236 merged by jenkins-bot:
[mediawiki/core@master] mediawiki.util: Move mw.RegExp over as well to simplify migration

https://gerrit.wikimedia.org/r/535236

Change 535231 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535231

Change 535244 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535244

Change 535242 merged by jenkins-bot:
[mediawiki/extensions/WikiEditor@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/535242

He7d3r added a subscriber: He7d3r.Sep 14 2019, 1:28 PM
Od1n added a subscriber: Od1n.Oct 8 2019, 12:35 AM

Just as a kind reminder, ResourceLoader/Core modules documentation would need updating.

Change 541409 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] mediawiki.util: Remove 'mediawiki.RegExp' module alias

https://gerrit.wikimedia.org/r/541409

Krinkle claimed this task.Oct 8 2019, 3:49 AM

Change 541409 merged by jenkins-bot:
[mediawiki/core@master] mediawiki.util: Remove 'mediawiki.RegExp' module alias

https://gerrit.wikimedia.org/r/541409

Krinkle closed this task as Resolved.Oct 8 2019, 7:00 PM
Krinkle removed a project: Patch-For-Review.
Krinkle updated the task description. (Show Details)

This function was moved around multiple times now. I'm not sure since when $.escapeRE existed, but I remember when it stopped working without a hint I was able to notice in time. All user scripts I wrote broke, leaving all my users puzzled. Just a few months later the same scripts broke again because the function stopped being available without requesting it first. That was the point I got sick of running after a single line (!) of code and copy pasted it into my code.

I'm thankful for this ticket and the lengthy explanation and can follow all the arguments. What I would like to ask for is this: Please, please don't remove the mw.RegExp.escape alias just because our code search makes it look like it's not used any more. As of now, enwiki alone contains more than 200 usages, dewiki about 20. I believe it's safe to assume a large portion of the about 1000 wikis in our cluster copy-pasted one script that contains the old signature. Not even mentioning documentation and help pages written by volunteers.

TL;DR: Removing the old signature will break gadgets and user scripts.

As of now, enwiki alone contains more than 200 usages, dewiki about 20.

If you search correctly, the numbers are 52 and 9 Fun fact: 2 of the scripts in the dewiki search are by me. I deprecated them 3 years ago, I just don't care if they break. There are probably more such scripts by other authors.

@thiemowmde I understand. Removal will not happen right away, but I don't plan to keep the alias around beyond 1.35. We do migrations like these several times a year. It does not make sense to me to keep this one just because it has had more previous incarnations than others.

There were 10-20X more search results before Krenair and I ran our bots. What remains tends to mostly be old user scripts and MediaWiki-namespace pages that have been unreferenced or already-broken for several years. The bot is programmed to skip replacements on pages that contain invalid syntax before or after the string replacement. I also notice quite often when I run this semi-automated bot that the diffs context contains references to hookEvent() which was deprecated in 2011 and removed two years ago in 2017. That is dead code being updated just to omit them from search results...

For what it's worth, mw.RegExp.escape is really simple to find/replace with a very low risk of false positives, so I will likely do that in both user- and mediawiki- namespaces and thus leave no references behind. The only exception being pages that aren't actually valid JS (e.g. common.css containing JS code, not unusual to find), or pages that use it in an odd way (e.g. x=mw, y=x.RegExp, y.escape(), or something like that).

Widely-used methods won't rapidly disappear. I held back Sajax for 4 years, wikibits for 6 years. We upgraded jQuery 3 over the course of a year (T124742). All tracked via mw-js-deprecate to monitor actual usage, regardless of search results (which often have false positives, and don't highlight a random user script that might be loaded cross-wiki by a default-on gadget). If at the time of planned removal there are more than a handful of known non-dead references left out in the wild, it should go through Tech News and be announced ahead of time with an easy and documented migration path.

Whatever method you guys use to gauge the impact of a change like this seems to have failed in this case: there are a number of highly used scripts that are being reported broken today. I think it's a bit naive to simply say, "there are only 52 scripts that use this module on en-wiki, it's no big deal." If even one of those 52 scripts is used by thousands of editors, there will be a large impact. Is there a mechanism whereby these kinds of changes can be communicated to editors who maintain scripts ahead of time, so that these changes don't result in mass confusion and chaos?

https://en.wikipedia.org/w/index.php?title=Special:Search&limit=500&offset=0&ns2=1&search=%22mediawiki.RegExp%22&advancedSearch-current={} search is showing possibly hundreds of uses on this project alone (I have not validated the results at all)

@Xaosflux It must be possible to migrate to a non-deprecated module.

@Masumrezarock100 I agree, was just noting based off of @Scottywong 's comment

It looks what I expected to happen and what actually happens now is very different:

  • The module mediawiki.RegExp doesn't exist any more. Code requesting it is already broken. I'm not sure if this is correct, but it looks like the deprecation phase for this was a few weeks only on the wmf cluster (deprecated, removed), and not announced anywhere. I scanned TechNews, but couldn't find it being mentioned.
  • Code not requesting the module might continue to work thanks to the deprecated function alias. However, it's more likely such code was broken before. For 3 years there was no guarantee any more the function would be available without requesting the module.

In other words: It appears like all user code using this one-liner is now broken, and having only the function mw.RegExp.escape() as a deprecated alias is pointless.

We do migrations like these several times a year.

If my analysis above is correct, this might turn out to not be one of these migrations where users do have enough time to even notice the deprecation.

Possible ways forward, I can see:

  • Partly revert the change, possibly by providing an empty, deprecated mediawiki.RegExp module for a longer time, and backport that.
  • Run a bot that not only scans for mw.RegExp.escape() calls, but for the module name mediawiki.RegExp as well.
  • Just remove the alias mw.RegExp.escape() as well and live with the consequences, including the social aspects.
  • Run a bot that not only scans for mw.RegExp.escape() calls, but for the module name mediawiki.RegExp as well.

That is exactly what I've done. (I haven't even started migration on mw.RegExp.escape yet.)

I have left no uses of this module name in WMF-maintained Git (Code search).
I have left no uses of this module name in all gadgets scripts, gadget definitions and all project-space scripts on all public WMF wikis (Global search), except for 2 pages on foundationwiki (fishbowl config).

I did not migrate personal script files and we've never done so in the past. However I have started to do those as well. Although there is significantly more noise there due to there being much more syntax errors and unused pages, so it seems wasteful especially with the non-zero risk of breaking scripts.

I'm afraid my main point got a little lost: It appears like the deprecation phase for users on the wmf cluster was way to short. Can we do something about this?

I'm afraid my main point got a little lost: It appears like the deprecation phase for users on the wmf cluster was way to short. Can we do something about this?

Deprecation phases exist to migrate. The migration was done. The change was announced. Can you explain where you think there is a problem?

The question is who needs to follow such a deprecation. We can't expect our volunteer on-wiki developers to be around all the time. With something that is known to be used in user scripts, I expect an announcement in our TechNews and related channels. Something like "the ResourceLoader module mediawiki.RegExp will be removed in four weeks, please replace it with mediawiki.util". Not only was such an announcement not done, it appears the deprecation phase was much shorter than what the official MediaWiki release cycle suggests.

This is – if true – an issue.

What are we going to do about this?

JavaScript code does not follow the MediaWiki PHP Deprecation policy.

When something is used in the way you described, it should indeed be announced. As we will do in a few weeks for mw.RegExp.escape, after edit interface volunteers have done the mass replacement (as I don't want to announce it beforehand and incur a lot of pointless manual labour).

In retrospect, this could have been done for the module removal as well. I didn't because there were only 2 references left on all WMF wikis, on foundationwiki. I did not include personal user scripts in that count. In the future, I will consider the count for personal scripts as well. Although take into account that any numbers we get from search are amplified by false positives and unused/broken/invalid scripts. Past experience also tells me that Tech News rarely results in updates to personal user scripts. We have done this in the past and waited for weeks, months even years, and it rarely makes a difference, including with little to no complains after removal, confirming that those scripts were indeed unused.

Bottom line: It will be announced.

Xover added a subscriber: Xover.Oct 25 2019, 7:58 PM

Is it expected behaviour that mw.loader.using('mediawiki.RegExp', function() {…}) will fail silently?

I would have expected mw.loader.using() to complain, loudly, in my console if I tried to feed it a non-existent (in this case, a removed) module—like mw.loader.load() does when fed something unresolvable—but that's not what I'm seeing. This (loaded through local Common.js) leaves no trace anywhere.

And if that is indeed the expected behaviour… should it be?

Yes. In asynchronous JavaScript code, errors are not thrown synchronously as exceptions but provided to an error callback via Promise or jQuery.Deferred. Here, then() is the success handler, and catch() is the error handler.

Read more about this at:

mw.loader.using('something-strange')
  .then(function () {
    // do something with a strange module
    return $.ajax('/foo.json');
  })
  .then(function () {
     // do something with foo.json
     return unknown.function();
  })
  .catch(console.warn); // any issue with anything along the way, gets caught here much like try-catch
Shortcut
mw.loader.using('something-strange', console.log, console.warn);
⚠️ Error: Unknown module: something-strange
TheDJ added a subscriber: TheDJ.Oct 25 2019, 8:14 PM

Small note. Generally we had much longer periods with deprecation messages being emitted and the admins would clean up the scripts they used themselves. This limited impact.

And sometimes I as an en.wp admin would go through many of the user scripts that I suspected (based on experience) were in use and I would proactively fix them in the few days before an actual removal (because that was less work than dealing with complaining users afterwards). I haven't done so in this case as i'm not currently active in English Wikipedia.

@Krinkle, is it possible to make deprecation "noisy"? Even when script owners are personally informed about upcoming changes with messages on their primary user talk pages, or messages are posted in their native language to the official page for that wiki's gadgets, scripts don't always get fixed until they break. So instead of trusting the deprecation notices from the web browser, could we have a little box, similar to the "Your edit was published" toast, that says "You're running a script that will break soon"?

Change 547002 had a related patch set uploaded (by Zoranzoki21; owner: Zoranzoki21):
[mediawiki/extensions/InlineCategorizer@master] mediawiki.RegExp (deprecated) -> mediawiki.util

https://gerrit.wikimedia.org/r/547002

@Krinkle, is it possible to make deprecation "noisy"? Even when script owners are personally informed about upcoming changes with messages on their primary user talk pages, or messages are posted in their native language to the official page for that wiki's gadgets, scripts don't always get fixed until they break. So instead of trusting the deprecation notices from the web browser, could we have a little box, similar to the "Your edit was published" toast, that says "You're running a script that will break soon"?

We've intentionally never done this, because in some cases it'd be much worse than noisy, it'd be catastrophically disruptive. For example, a script that runs on each row of RecentChanges, etc..