RFC: Shadow namespaces
Open, HighPublic

Tokens
"Like" token, awarded by Liuxinyu970226."Like" token, awarded by MGChecker."Love" token, awarded by Oetterer."Like" token, awarded by Vedmaka."Like" token, awarded by Micru."Love" token, awarded by cscott."Like" token, awarded by Qgil."Like" token, awarded by Jdlrobson.
Assigned To
Authored By
daniel, Feb 28 2015

Description

This is a request for comments regarding implementing shadow namespaces, which refers to the concept where if a local page doesn't exist, it will be transparently fetched from a remote wiki, like how InstantCommons and foreign file repos currently work.

For more details see https://www.mediawiki.org/wiki/Requests_for_comment/Shadow_namespaces

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes
cscott added a subscriber: cscott.Feb 8 2016, 6:21 PM

Is there any alpha-quality code for this, by any chance? Off in T120345 we want to set up read-only mirrors of the main WMF wikis to test various parser fixes/changes. One way of implementing that would be to configure basically *every* namespace of our test wiki as a shadow of (say) enwiki, much like you could point instantcommons at enwiki to shadow all the media files. This would be an interesting torture test of this RFC, so it might be an interesting first testbed for shadow namespace patches?

cscott awarded a token.Feb 8 2016, 7:30 PM

Nope, nothing yet outside of the GlobalUserPage extension...you could try and hack that up if you're brave ;)

Micru awarded a token.Feb 28 2016, 1:02 AM
Micru added a subscriber: Micru.
DStrine assigned this task to brion.Mar 2 2016, 10:15 PM
DStrine moved this task from Needs shepherd to Under discussion on the TechCom-RfC board.
brion added a comment.Mar 2 2016, 10:17 PM

Quick note from archcom irc triage session:

  • @Legoktm will pick up the RfC ownership & work sides of this
  • @brion will be shepherd
  • @Qgil needs this for some other stuff, so keep in loop for use cases?

Is this task still stalled?

RobLa-WMF mentioned this in Unknown Object (Event).Apr 13 2016, 10:03 PM

fwiw, I commented on https://www.mediawiki.org/wiki/Talk:Requests_for_comment/Shadow_namespaces about "Centralized, but locally-appendable, documentation pages"

Legoktm changed the task status from Stalled to Open.Apr 20 2016, 10:35 PM
GWicke added a subscriber: GWicke.Apr 27 2016, 8:16 PM

How is change propagation / purging going to be handled?

Kghbln added a subscriber: Kghbln.May 4 2016, 3:44 PM
Vedmaka added a subscriber: Vedmaka.
Oetterer added a subscriber: Oetterer.
RobLa-WMF mentioned this in Unknown Object (Event).May 11 2016, 12:09 AM
RobLa-WMF triaged this task as High priority.May 18 2016, 5:18 PM

The RFC on mw.org currently says the following:

Shadow namespaces could be used to solve the following use-cases:

  • Global Scribunto/Lua modules; Module namespace; T41610
  • Global templates; Template namespace; T6547
  • Global user pages; User namespace; T16759
  • Global help pages; Help namespace; T14306
  • would allow sharing of global styles for T90914, T112991, and T105845 in general

It also references T66474: Global namespace (looks local, lives centrally) in the "Proposal" section.

@brion: thanks for taking on shepherd responsibilities for this! Seeing all of the moving pieces, I'm having a hard time keeping this in my head.

Given the importance and urgency of this one, I'm setting the priority to "High".

The task T135845#2376958 is enought explicit, complete and stable to need that some LUA coders begin to try centre and tools libraries, to change them in their own places for changes and to talk about how to adapt and insert them between other parts of mediawiki.

Yurik added a subscriber: Yurik.Jun 26 2016, 10:38 PM

Despite being a huge proponent of a shared code and template repository, I think this proposal has one fundamental flaw, and might need to be rethought before implementing. It presumes and encourages two types of content: shared for everyone and used by a single wiki. Yet, I believe the single-wiki override will not be much different than the current situation, and will only result in more fragmentation. Instead, I think we should implement a shared repository without any options to override. This way if a subset of wikis need different content, they will simply create a copy with a slightly different name. We will still encourage cross-wiki content reuse, and each wiki can either go with version A or version B (or C, ...) of the content, yet all will be easily accessible from the same location.
If the only reason shadow namespaces were introduced was to not create new namespaces, I think we should at least prohibit creation of templates or modules if identically named page already exists in the shared repo. This will be similar to how we merged user accounts. Eventually more and more pages will be merged in, making wikis more consistent.

This feature will really help a lot to the ones who maintain wiki-farms or have wikis on separate subdomains as multi-language solution.

@Yurik, do you propose to restrict creating of particular templates/modules in wikis, but fetch them from shared repository? This sounds like a very strange idea.

A potential problem I foresee is different languages wanting to use the same page name. If wikis A, B and C all want to use page name X, then which wiki's content is used?

Yurik added a comment.Jun 27 2016, 1:04 AM

@tom29739 nah, doubt its a major issue - if the same title is needed by multiple languages (which by definition is unlikely), they can use longer titles qualifying the difference, e.g. Parameters and Parameters RTL
@Vedmaka, I don't argue with the how great it would be to share cross-wiki. I am concerned about allowing per-wiki override.

Shadow namespaces attempt to create OOP-style inheritance model, allowing selective override of features. But I feel this is a mistake in this case because one would have to override the entire module or template, not a specific piece of it. Thus it is still a per-wiki copy, only with an extra unknown. If we keep all same modules/templates on the same wiki, searchable and usage-trackable, it would be greatly simplify things.

The only time where we really do need per-language customization is when we do translations. But that should not be done as shadows, instead I think we should use mw.loadData() from subpages - allowing translate wiki to be setup to update it all in one place. Especially because sometimes we shouldn't localize based on the source wiki, but rather based on the user's preferences.

Rical added a comment.Jun 27 2016, 8:54 AM

@Yurik, I agree and changed my own version in Module:Central.fr.ws and my last short description of task.

Shadow namespaces attempt to create OOP-style inheritance model, allowing selective override of features. But I feel this is a mistake in this case because one would have to override the entire module or template, not a specific piece of it. Thus it is still a per-wiki copy, only with an extra unknown. If we keep all same modules/templates on the same wiki, searchable and usage-trackable, it would be greatly simplify things.

I don't think you have to override the entire module or template. We could use labeled section transclusion for overriding specific parts of pages. or have a central lua module that exposes functions, and local wikis can augment or hook into those functions. The reason for using page as the as the "unit" of cross-wiki content copying is because it is generally the most reasonable unit that already exists...

... We could use labeled section transclusion for overriding specific parts of pages. or have a central lua module that exposes functions, and local wikis can augment or hook into those functions. The reason for using page as the as the "unit" of cross-wiki content copying is because it is generally the most reasonable unit that already exists...

I agree that page is ideal as a unit, but this will also mean that most of the time this entire unit will be copied. Could you give an example of a usecase where a subsections would be used? Also, I'm worried about cross-wiki managing all such pieces - I would much rather have them all together.

Rical added a comment.Aug 31 2016, 1:09 AM

In my Module:Central which defines the basic libraries and functions of central modules I already "augment or hook some functions" in modules with different names without ambiguity.
In cases of identical modules names that could disturb the process if different cascades of modules change functions definitions in unpredictible ways, like in well known multiple inheriting in C++.

@Legoktm, would this be a good topic for this week's IRC meeting (E269)?

I don't think there's anything much to talk about that we didn't discuss last time? I'm still working on implementing what was discussed at the last meeting.

Yurik added a comment.Sep 6 2016, 4:49 PM

@Legoktm, is there a summary of what exactly you are implementing in this step?

A potential problem I foresee is different languages wanting to use the same page name. If wikis A, B and C all want to use page name X, then which wiki's content is used?

I believe the goal currently is to replicate/emulate the behavior of global user pages and foreign file repositories: if the local page/file exists, prefer that, otherwise fall back to the global/centralized version.

Instead, I think we should implement a shared repository without any options to override. This way if a subset of wikis need different content, they will simply create a copy with a slightly different name. We will still encourage cross-wiki content reuse, and each wiki can either go with version A or version B (or C, ...) of the content, yet all will be easily accessible from the same location.

How do you envision your proposed implementation being deployed? It sounds like you'd want to flip a switch and suddenly pages would start inheriting from a central wiki, even if the local page exists. This sounds like a very difficult migration path, it would be inconsistent with existing behavior for global user pages and files, and it would probably be a pretty confusing user experience for everyone.

If the only reason shadow namespaces were introduced was to not create new namespaces, I think we should at least prohibit creation of templates or modules if identically named page already exists in the shared repo.

Users want flexibility, of course. I don't need to convince you of that. :-) For global user pages, we already have some warnings in the user interface when editing a local user page of the same name. Same with local file description pages, at least on the English Wikipedia. We also have a Special page or some other database report that lists shadowed file names (i.e., file names that exist both locally and on the central wiki).

As a compromise, one could encourage people to delete the local versions once they have been imported into the central repository.

Rical added a comment.Oct 14 2016, 8:19 PM

In any case, local users need to know if the available object is local or central. Perhaps from the style of the title and from a dedicated sign.

If the central version has priority, the local user cannot delete or rename this local object, and cannot choose which object use. Also the local user cannot rename and re-use it with another version name. Also that need an option to manage all cases.

In any case, local users need to know if the available object is local or central. Perhaps from the style of the title and from a dedicated sign.

Sure. We already have user-visible notices for files that are coming from a foreign file repository such as Wikimedia Commons. We also already have user-visible notices for global user pages coming from Meta-Wiki.

If the central version has priority, the local user cannot delete or rename this local object, and cannot choose which object use. Also the local user cannot rename and re-use it with another version name. Also that need an option to manage all cases.

While I agree with what you're saying, I think this is essentially the cost of having a centralized repository. You give up some local control over an item in exchange for having fewer versions/editions of a similar global item. (I'm reminded of Brexit!) We make this trade-off in a couple of places already, such as in the File and User namespaces. We want to generalize the functionality to make it easier to extend to other namespaces such as Help, as I understand it.

Rical added a comment.Nov 6 2016, 12:20 AM

OK for central modules which have the priority and replace local ones.
I see 2 ways to retreave local versions:

  • The replacement is registered in the history of the object page. Then any user can find and reuse the object before the central replace.
  • Admins could have a right for this case.
Abbe98 added a subscriber: Abbe98.Dec 25 2016, 10:12 PM

@ahroni has an un conference session scheduled for today. Please sign up.

Amire80 added a subscriber: Amire80.Jan 9 2017, 6:45 PM

I added a session to the 2017 Dev Summit, which is closely related to this task. Hope to see some of this task's subscribers there. Sorry about the super-short notice, I'll do my best to make it useful nevertheless.

I started an Etherpad for the session: https://etherpad.wikimedia.org/p/devsummit17-xwiki-templates

People who understand the technology inside Shadow namespaces are welcome to add it.

On the enwiki, templates are created, put in use, merged and deleted all the time. It may be more complex to manage them on a shared repository for other projects, but "no real way" is simply wrong.

On the enwiki, templates are created, put in use, merged and deleted all the time.

Of course, and the same is true for each single wiki.

It may be more complex to manage them on a shared repository for other projects, but "no real way" is simply wrong.

"No real way" refers to sharing templates between wikis. And there is no real way for that. There is currently no shared repository, and there's no way to write a template on one wiki and use it on another, except importing it and adapting it manually, which creates a fork and doesn't get automatic updates from the source template.

Note-taker(s) of this session: Follow the instructions here: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Session_Guidelines#NOTE-TAKER.28S.29 After the session, DO NOT FORGET to copy the relevant notes and summary into a new wiki page following the template here: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Your_Session and also link this from the All Session Notes page: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/All_Session_Notes. The EtherPad links are also now linked from the Schedule page (https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Schedule) for you!

Thanks a lot for this, @Legoktm. I plan to follow up further.

Does it have to be "templates"? What if I want to transclude a poem from Wikisource into a Wikipedia article about the poem? Or a quotation into a Wiktionary entry, to show how a word gets used?

More information: https://www.mediawiki.org/w/index.php?title=Topic:T6ukmc98uuun7ydc (especially @Billinghurst's comment)

This comment was removed by Billinghurst.

What's the current status of the Shadow namespace work? Has it gotten stalled?