Load .json configuration files via ResourceLoaderWikiModule
Open, Stalled, LowPublic

Description

We are trying to limit JS editing to as few accounts as possible to reduce the likelihood of XSS compromises (T190015); configuration should be moved to JSON files to remain widely available for editing. There is no way currently to load a JSON page with ResourceLoader though; loading with AJAX has suboptimal performance. There should be a more efficient way to load such files.

One concrete use case is the geonotice config on enwiki; this should be moved to JSON but loading it blocks banner display so performance is key.

Tgr created this task.Jul 3 2018, 8:20 PM
Restricted Application added a project: Performance-Team. · View Herald TranscriptJul 3 2018, 8:20 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
MusikAnimal added a subscriber: MusikAnimal.
stjn added a subscriber: stjn.Jul 20 2018, 5:26 PM

As another example where this would be helpful: in Russian Wikipedia sysops and bureaucrats frequently edit a gadget for showing users with advanced groups so that it would stay up-to-date. If there won’t be a more comfortable solution such as JSON, quite a lot people might argue either for workarounds that undermine security (such as content model change?) or for giving those rights to them by default when they don’t really edit any other pages.

Krinkle added a subscriber: Krinkle.EditedJul 20 2018, 5:48 PM

Two quick thoughts:

  • Interface: We'll need a find a transport method for the JSON file, and a way for the JS code to read its contents. It's probably best to build this on top of T133462, which would provide a built-in way for multiple files to be exported by a module (rather than concatenated). The read method could be something like require('./foo.json').
  • Edit rights: While inclusion of JSON files seems useful, I do not think we should have separate edit rights for the JS/CSS file of a gadget and the JSON files.

As another example where this would be helpful: in Russian Wikipedia sysops and bureaucrats frequently edit MediaWiki:Gadget-markadmins.js.

You might be interested in the bot that Wikimedia Commons uses for MediaWiki:Gadget-markAdmins-data.js.

stjn added a comment.Jul 20 2018, 5:54 PM

You might be interested in the bot that Wikimedia Commons uses for MediaWiki:Gadget-markAdmins-data.js.

It is a viable solution, but there are some user groups for us that are present only in the name, like ArbCom members and clerks.

Krinkle triaged this task as Low priority.Jul 20 2018, 6:52 PM
Krinkle moved this task from Inbox to Accepted: Enhancement on the MediaWiki-ResourceLoader board.
Krinkle moved this task from Backlog: Small / Maintenance to Blocked on the Performance-Team board.
Tgr added a comment.Jul 27 2018, 2:12 PM
  • Edit rights: While inclusion of JSON files seems useful, I do not think we should have separate edit rights for the JS/CSS file of a gadget and the JSON files.

JS/CSS is dangerous in the hands of a malicious user, JSON not really. For Gadgets 2.0, not sure if there is any use case for different people editing the JSON files and the rest; current permissions are just how MediaWiki: pages are handled, and I think it's reasonable to make JSON editing available to more users there, as it tends to be used for bot control and such (not necessarily in gadgets).

Translation might be a use case for wider access, although T156210: Support translation of JSON blobs in Translate is the nicer solution there.

SerDIDG added a subscriber: SerDIDG.Aug 1 2018, 9:04 PM

Right now many people who ask for interface-admin (see T190015) permissions in ruwiki are worried about the ability to edit gadgets data. But with the current state where the integration of JSON with gadgets is poor, the ability to load external JSON for gadgets is very limited. Hence, the efforts to integrate JSON with ResourceLoader as soon as possible are essential from the perspective of the interface-admin flag purpose.

Tgr added a comment.Aug 2 2018, 12:36 PM

For 99% of gadgets ResourceLoader integration makes little difference. geonotice is unusually performance-sensitive but something like the admin coloring gadget is not. Just loading it via a plain API query with a short smaxage should be fine.

Right now we have at least 3 widely used gadgets that would need such query, probably there will be more. Having 5–10 additional requests doesn't seem to make negligable difference. Many such gadgets also add elements that shift content on pages which in turn causes misclicks, so the earlier they run the better. Anyway, is it so difficult to implement? I thought splitting large JS files into modules according to the semantics of different code parts and loading them, together with other modules, in one package is exactly what ResourceLoader is for.

Krinkle changed the task status from Open to Stalled.Aug 2 2018, 4:19 PM

Please remember that I have accepted the feature request, but this will not be implemented in the short-term while other projects take priority.

Regarding the efficiency issues, nobody is being forced to use a JSON page fetched over AJAX. In fact, I would recommend against it for exactly the reasons you state. Fetching it would be inefficient, makes the code harder to maintain, and would render more slowly.

So, for today - use a JS page instead. That is the current best practice, and is supported by ResourceLoader. Place the data on a JS page (not a JSON page). They work the same way, in the same namespace, with the same protection level. The JSON would be prepended by a simple statement, such as MyGadget.data = { }, MyGadget.setData({ }) or mw.hook('myGadget.data').fire({ }). ResourceLoader, through Gadgets, can already combine these into a single bundle - together with the rest of the gadget - in a single request.

Anyway, is it so difficult to implement?

Concatenating the plain text as saved onto a .js page and a .json page together would be easy, but the result would be invalid syntax. It will not work the way you want. To do this, we need to build a communication channel between load.php (server-side) and mw.loader (client-side) for sending this data, and then we need to build a way for your gadget JS code to find the data.

Task T133462 is about building that channel (for multiple chunks of JS code). Once that exists, we can also use that channel for JSON data. To find the data, we will build a require() function that will work similar to what many web developers have seen in other environments (Node.js, AMD, UMD, etc.). This makes it easy to learn from tutorials that explain JavaScript in general (instead of new things to learn just for Gadgets on Wikipedia).

the efforts to integrate JSON with ResourceLoader as soon as possible are essential from the perspective of the interface-admin flag purpose.

Sorry, but I do not understand the link between JSON integration and interface-admin.

We currently have a single group ("sysop") that grants the ability to protect pages, block users, and modify and deploy software (Gadgets) saved to JS and CSS pages. This has many downsides: Communities have to entrust a user with all of it, or nothing. Hackers compromising an admin account will be capable of all the above, even if the admin only intended to use one or the other.

Separating these groups, has benifits:

  • Reduce security risk: Users will only ask the rights they wanted. Community stays in control of granting the rights.
  • Communities can give sysop rights without worrying about cybersecurity.
  • Communities can give interface-admin rights without worrying about deletion/blocking policy and trusted activity in that area. I have seen several times that very good volunteer developers was active as software developer, but not as general editor/content reviewer. This makes it difficult to become sysop.
Tgr added a comment.Aug 2 2018, 4:36 PM

They work the same way, in the same namespace, with the same protection level.

JS requires editsitejs which is soon going to be restricted to interface-admin. JSON requires editsitejson which sysops will keep (and it has a separate grant for OAuth that does not involve sitewide JS editing permissions).

@Krinkle Thanks for the detailed explanation.

@Tgr Yes, that's the link between JSON integration and interface-admin that I'm talking about. Right now we have 1) gadgets data on JS pages, 2) sysops actively editing some of this data. So, people ask for interface-admin flag that they could do without if the gadgets data was on JSON pages.

Tgr added a comment.Aug 2 2018, 5:05 PM

I guess you could attempt a horrible hack where you change the content model of a JS page to wikitext and transclude the JSON into it (and wrap with a variable assignment). No idea if it would actually work though...

KTC added a subscriber: KTC.Aug 8 2018, 2:40 PM
TheDJ added a subscriber: TheDJ.Aug 14 2018, 11:05 AM
Pharos added a subscriber: Pharos.Aug 27 2018, 5:52 PM
RP88 added a subscriber: RP88.Aug 30 2018, 10:03 PM
Izno added a subscriber: Izno.Sep 22 2018, 8:19 PM

en.wp has proposed to have a bot sync from an admin-protected page. Obviously, we need to make sure there is consensus, but once we establish that, I suspect that we may be able to run it on other wikis.

ToBeFree added a subscriber: ToBeFree.

@Izno Please note, we think making a bot to sync json pages to js pages is far from ideal, and having this be able to work server-side in a way that "data" can be edited outside of "code" would be much preferred. For anyone following you can check on the bot build progress here: https://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/MusikBot_II_2