Page MenuHomePhabricator

The Great Namespaceization Effort
Open, LowPublic

Description

Objective

  1. MediaWiki should use namespaces in a consistent way with low maintenance overhead.
  1. Extension registration should be fast.

Problem statement

  • The use of namespaces in MediaWiki is inconsistent. Directory names are not necessarily related to namespace names.
  • The convention of listing the path of all classes in autoload.php has become unwieldy.
  • Extension registration is slow, due to the need to merge enormous class maps. Class names in the top level use generic names which risk conflicting with the PHP core.

Proposal

I propose moving all MW core classes to the MediaWiki\ namespace, with class_alias() for backwards compatibility. Autoloading would be done using PSR-4, which maps namespaces to directory names.

To work around bugs in PHP, class_alias() calls will be inserted in the file scope at the end of the file defining the new class. The autoloader will have an array mapping each legacy class name to the path of the new file. This is how class_alias() is already used in MW core. Whether this legacy autoloader is integrated with the PSR-4 autoloader or is registered separately is an implementation detail.

A notable difference between the current layout and that described by PSR-4 is that the directory names must match the namespace names, including capitalisation. We already have a MediaWiki\Linker namespace, but the files would be moved from includes/linker to MediaWiki/Linker.

Another difference is that there must only be one class per file.

MediaWiki has more directories than namespaces, and this makes sense given how directories are used in editors. Additional namespaces only really become essential when there is a risk of class name conflicts. To use PSR-4, we will have to trade off between our desire to limit the number of namespaces for usability, while still making files easy to find. The plan I detail below generally leans towards introducing additional namespaces.

There is the question of plural names. We (inconsistently) use plurals to indicate that a directory contains multiple things of a certain type, e.g. each member of includes/ is an include, each member of includes/specials is a special. This makes more sense for directories than for namespaces, given the way each is accessed: the class name MediaWiki\Specials\Activeusers would be strange to see in calling code or an error message. The namespace hierarchy is more of an ontology than a collection. I propose to use singular namespace names where the members of the namespace are are singular instance of the parent concept. So: specials -> Special, jobs -> Job, skins -> Skin. But perhaps jobqueue/utils -> JobQueue\Utils since each class also contains utils, not a single util. And perhaps services -> Services, since it has container classes, the members are not themselves services.

There is the question of whether to retain class name prefixes and suffixes. For example, we have 116 special pages, certainly enough to deserve a separate directory. But should we have MediaWiki\Special\SpecialActiveusers or MediaWiki\Special\Activeusers? I think redundant prefixes should be removed, except when the resulting name becomes very ambiguous. We should keep in mind that text editors often only display the base name.

Some directories do not make sense as modules or namespaces, and exist only out of a desire to reduce the number of PHP files in includes/. These should probably be moved into their respective parent directories. Specifically:

  • cache
  • clientpool
  • compat
  • debug
  • exception
  • json

But in the other direction, I would propose:

  • Sanitizer, MagicWord, MagicWordArray -> Parser\
  • ForkController -> Maintenance\
  • MessageCache -> Language\
  • Message, RawMessage -> Language
  • cache/localisation -> Language\LocalisationCache
  • A directory for non-class code: WebStart.php, Setup.php, DefaultSettings.php, Defines.php, GlobalFunctions.php, OutputHandler.php, NoLocalSettings.php. Namespacing of global functions and constants might be easier with PHP 7 group use, I'm not proposing that at the moment.
    • Also shell scripts, config files?
  • A namespace for the web app setup, request routing and response (Request?) MediaWiki, PathRouter, AjaxDispatcher, AjaxResponse, WebRequest, WebRequestUpload, FauxRequest, DerivativeRequest, WebResponse, OutputPage, HeaderCallback, exception/*
  • Watchlist: WatchedItem, WatchedItemQueryService, WatchedItemQueryServiceExtension
  • StubObject: StubObject, DeprecatedGlobal
  • A namespace for revision, page storage (ArticleStore?): Revision, RevisionList, MergeHistory, MovePage, HistoryBlob, LinkBatch, LinkCache,
  • A namespace for links storage: BacklinkCache, LinksUpdate, LinksDeletionUpdate
  • UserCache -> User\UserCache
  • Feed: FeedItem, ChannelFeed, RSSFeed, AtomFeed, FeedUtils

The classes in libs and utils would be better placed under the Wikimedia\ namespace, corresponding to the Composer vendor namespace, reflecting our aspirations for them to be separate from MediaWiki.

Other special cases:

  • jobqueue/jobs -> JobQueue\Job
  • includes/libs/rdbms/defines.php: migrate to namespaced constants.
  • specials -> Special
  • specials/helpers -> Special\Helper
  • specials/pagers -> Special\Pager
  • languages/classes -> MediaWiki\Language
  • languages/data/ZhConversion -> MediaWiki\Languages\Data\ZhConversion
  • maintenance: we can map MediaWiki\Maintenance to this directory for now
  • tests:
    • Some test classes are currently in the namespace they cover, I don't think that works with PSR-4. They should be in MediaWiki\Test\PHPUnit instead.
    • Test classes in MediaWiki\Test\PHPUnit, parser test runner in MediaWiki\Test\Parser
    • Classes under libs/ will be under the Wikimedia namespace, so associated tests should also be in the Wikimedia namespace
    • Map namespace MediaWiki\Test to directory tests\MediaWiki

New directory names will match namespace names, except for maintenance and tests. So for example, the language class files will not stay in languages/.

Existing namespaced extensions mostly use the top level, which I think is fine as long as the name is a distinctive product name. For generic descriptive names, I would prefer MediaWiki\Extension over MediaWiki\Extensions.

The transition should be fully scripted so that it can be used to rebase open changesets in Gerrit.

Details

SubjectRepoBranchLines +/-
mediawiki/coremaster+125 -43
mediawiki/coremaster+136 -42
mediawiki/tools/phan/SecurityCheckPluginmaster+134 -6
mediawiki/coremaster+333 -60
mediawiki/coremaster+247 -237
mediawiki/coremaster+99 -37
mediawiki/coremaster+337 -129
mediawiki/coremaster+215 -73
mediawiki/coremaster+533 -303
mediawiki/coremaster+31 -16
mediawiki/coremaster+176 -38
mediawiki/coremaster+61 -28
mediawiki/coremaster+105 -57
mediawiki/coremaster+497 -178
mediawiki/coremaster+107 -36
mediawiki/coremaster+255 -45
mediawiki/coremaster+24 -4
mediawiki/coremaster+94 -0
mediawiki/coreREL1_40+58 -2
mediawiki/coremaster+58 -2
mediawiki/coremaster+876 -235
mediawiki/coremaster+2 -1
mediawiki/coremaster+307 -511
mediawiki/coremaster+511 -307
mediawiki/coremaster+113 -43
mediawiki/coremaster+87 -38
mediawiki/coremaster+31 -22
mediawiki/coremaster+18 -7
mediawiki/coremaster+24 -148
mediawiki/coremaster+39 -14
mediawiki/tools/namespaceizermaster+646 -0
Show related patches Customize query in gerrit

Related Objects

StatusSubtypeAssignedTask
OpenNone
ResolvedLegoktm
ResolvedLegoktm
Resolvedtstarling
ResolvedLucas_Werkmeister_WMDE
ResolvedTheresNoTime
ResolvedJdforrester-WMF
ResolvedJoe
ResolvedDzahn
Resolvedhashar
ResolvedJdforrester-WMF
ResolvedLadsgroup
ResolvedMoritzMuehlenhoff
Resolvedjijiki
ResolvedMoritzMuehlenhoff
ResolvedTrizek-WMF
ResolvedDzahn
Resolved Gilles
ResolvedDzahn
ResolvedRequestPapaul
Resolvedjijiki
DeclinedNone
ResolvedDzahn
ResolvedDzahn
ResolvedPapaul
Resolved Cmjohnson
ResolvedRequest Cmjohnson
ResolvedRequestPapaul
ResolvedAndrew
ResolvedArielGlenn
ResolvedDzahn
ResolvedLegoktm
ResolvedPapaul
ResolvedDzahn
Declined Gilles
ResolvedVolans
ResolvedDzahn
ResolvedLegoktm
ResolvedPleaseStand
ResolvedJoe
Resolvedtstarling
ResolvedArielGlenn
ResolvedJoe
Resolvedtstarling
ResolvedJdforrester-WMF
ResolvedJdforrester-WMF
ResolvedLegoktm
ResolvedJdforrester-WMF
ResolvedDaimona
ResolvedDaimona
ResolvedJdforrester-WMF
ResolvedJoe
ResolvedJMeybohm
ResolvedJoe
ResolvedJoe
ResolvedJoe
ResolvedJoe
ResolvedKrinkle
ResolvedJoe
ResolvedClement_Goubert
ResolvedClement_Goubert
ResolvedClement_Goubert
ResolvedMainframe98
ResolvedJoe
ResolvedZabe
ResolvedLadsgroup
ResolvedSecurityDaimona
OpenDaimona
OpenNone

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 849126 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] Reorg: Move StubObject classes in includes to its own directory

https://gerrit.wikimedia.org/r/849126

Change 849126 merged by jenkins-bot:

[mediawiki/core@master] Reorg: Move StubObject classes in includes to its own directory

https://gerrit.wikimedia.org/r/849126

Change 849554 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] Reorg: Move some of request related classes to MediaWiki/Request

https://gerrit.wikimedia.org/r/849554

Change 849554 merged by jenkins-bot:

[mediawiki/core@master] Reorg: Move some of request related classes to MediaWiki/Request

https://gerrit.wikimedia.org/r/849554

From https://integration.wikimedia.org/ci/job/mwext-php74-phan-docker/6764/console:

includes/View/AbuseFilterViewList.php:164 UnusedPluginSuppression Plugin BuiltinSuppressionPlugin suppresses issue SecurityCheck-ReDoS on this line but this suppression is unused or suppressed elsewhere

(for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/AbuseFilter/+/849574)

This just made me realize that taint-check does not read class aliases when determining the taintedness of hardcoded methods. In practice, if WebRequest is namespaced, taint-check no longer knows that getVal() etc. return an unsafe value. I think the patch above should be reverted immediately, then we can fix this, and then re-do the patch.

Change 850081 had a related patch set uploaded (by Zabe; author: Zabe):

[mediawiki/core@master] Revert "Reorg: Move some of request related classes to MediaWiki/Request"

https://gerrit.wikimedia.org/r/850081

This just made me realize that taint-check does not read class aliases when determining the taintedness of hardcoded methods. In practice, if WebRequest is namespaced, taint-check no longer knows that getVal() etc. return an unsafe value. I think the patch above should be reverted immediately, then we can fix this, and then re-do the patch.

So, thinking about this... In the long run, I believe it would be better to have those annotations directly on the method, rather than hardcoded in the plugin. That way, the annotations are much easier to find, change, read, and document. This should be done carefully though, to make sure that we don't break anything. I think this could be a possible game plan:

  • Copy annotations from taint-check to WebRequest (and potentially other classes, but not all of them for now)
    • Backport said change to all supported releases, just to be sure.
  • Add a test in core to ensure that said annotations are working (i.e., write a test file that uses the unsafe methods, make sure that it's analyzed by phan, make sure it reports issues). Ideally, this would be done for all methods hardcoded in taint-check, not just the ones we're migrating now.
  • Resolve T291743 so that caused-by lines are easier to read when annotations are involved
  • In taint-check, remove hardcoded taintedness from methods that are now annotated in core
  • Release new version of taint-check and mw-phan, upgrade to it in core
  • Finally, move the WebRequest class and its friends.

If it sounds complex, it's because it is complex. However, some of the steps would make it easier to rename other classes in the future.

Just quick q: What classes are hard-coded like this? While you change the phan rules, I move the other classes.

Change 850081 merged by jenkins-bot:

[mediawiki/core@master] Revert "Reorg: Move some of request related classes to MediaWiki/Request"

https://gerrit.wikimedia.org/r/850081

Just quick q: What classes are hard-coded like this? While you change the phan rules, I move the other classes.

Eh... See https://github.com/wikimedia/phan-taint-check-plugin/blob/master/MediaWikiSecurityCheckPlugin.php#L57 :)

I started working on T321806 to unblock this task, but it could take some time.

Change 850424 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] Reorg: Move some of request related classes to MediaWiki/Request

https://gerrit.wikimedia.org/r/850424

I have been creating a lot of patches here which can be spammy for everyone subscribed to this ticket. I'm moving them to T321882: Reduce number of files directly under includes/

Change 842538 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] title: Move Title.php to includes/title/ where related classes are

https://gerrit.wikimedia.org/r/842538

Change 842538 merged by jenkins-bot:

[mediawiki/core@master] title: Move Title.php to includes/title/ where related classes are

https://gerrit.wikimedia.org/r/842538

I have been creating a lot of patches here which can be spammy for everyone subscribed to this ticket. I'm moving them to T321882: Reduce number of files directly under includes/

Can I suggest we also do something like that for maintenance/? That folder has a bunch of partially related files that could be organized. Things like

  • update scripts that are part of update.php or manual updating, but unlikely to be run independently (eg migrateRevisionActorTemp.php, updateRestrictions.php, populateChangeTagDef,php)
  • scripts that are for people developing in core, like generateConfigSchema.php, formatInstallDoc.php, manageForeignResources.php, findClasses.php, findDeprecated.php
  • scripts relating to changing server configuration (eg migrateUserGroup.php, namespaceDupes.php, renameRestrictions.php
  • scripts to get some information shown on the command line (eg showSiteStats.php, version.php, showJobs.php, parse.php, lag.php)

and other potential groups. If there were some folders organizing these then it would be easier to find things in my view. You could have folders like maintenance/updates/, maintenance/configuration/, maintenance/developers/, etc.

I totally agree we need to clean up maintenance/. I have removed numerous scripts after the clean up of T259771: RFC: Drop support for older database upgrades but the biggest complexity here is that they are being used by being called directly and thus depend on the path and there is not an easy way to make a "class alias" for a path, possibly you can do symlinks but they won't work cross platforms and all sorts of mess around that.

The good news is yesterday I closed T99268: RfC: Create a proper command-line runner for MediaWiki maintenance tasks making maint scripts runnable via class names rather than path. The only thing left is to actually deprecate calling maint scripts directly (via a deprecating warning, I don't know how yet tbh?) switch all of the calls to use run.php instead and then drop the support for direct call in 1.41 and then let the party begin. Does that sound good to you?

Change 893535 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Reorg: Namespace the Title class

https://gerrit.wikimedia.org/r/893535

Change 893535 merged by jenkins-bot:

[mediawiki/core@master] Reorg: Namespace the Title class

https://gerrit.wikimedia.org/r/893535

Change 906054 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] Add namespaced classes to 1.40 release notes

https://gerrit.wikimedia.org/r/906054

Change 906054 merged by jenkins-bot:

[mediawiki/core@master] Add namespaced classes to 1.40 release notes

https://gerrit.wikimedia.org/r/906054

Change 905984 had a related patch set uploaded (by Jforrester; author: Amir Sarabadani):

[mediawiki/core@REL1_40] Add namespaced classes to 1.40 release notes

https://gerrit.wikimedia.org/r/905984

Change 905984 merged by jenkins-bot:

[mediawiki/core@REL1_40] Add namespaced classes to 1.40 release notes

https://gerrit.wikimedia.org/r/905984

Krinkle renamed this task from The Great Namespaceization and Reorg to The Great Namespaceization Effort.Apr 19 2023, 8:29 PM

We finished moving around 70 files out of includes/ to a directory inside it. They all also got namespaced. I think now we can move forward to namespacing files that are inside directories (e..g parser/, etc.)

Change 957723 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace core Pagers under \MediaWiki\Pagers

https://gerrit.wikimedia.org/r/957723

Change 957888 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace HistoryPager under \MediaWiki\Pagers

https://gerrit.wikimedia.org/r/957888

Change 957889 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace 'special' Pagers under \MediaWiki\Pagers

https://gerrit.wikimedia.org/r/957889

Change 957890 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace remaining 'specialpage' files under \MediaWiki\SpecialPage

https://gerrit.wikimedia.org/r/957890

Change 958500 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace TitleValue under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958500

Change 958501 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace NamespaceInfo under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958501

Change 958502 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace TitleFormatter under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958502

Change 958503 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace TitleParser under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958503

Change 958504 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace remaining Title-related classes under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958504

Change 957723 merged by jenkins-bot:

[mediawiki/core@master] Namespace core Pagers under \MediaWiki\Pager

https://gerrit.wikimedia.org/r/957723

Change 957888 merged by jenkins-bot:

[mediawiki/core@master] Namespace HistoryPager under \MediaWiki\Pager

https://gerrit.wikimedia.org/r/957888

Change 957889 merged by jenkins-bot:

[mediawiki/core@master] Namespace 'special' Pagers under \MediaWiki\Pager

https://gerrit.wikimedia.org/r/957889

Change 957890 merged by jenkins-bot:

[mediawiki/core@master] Namespace remaining 'specialpage' files under \MediaWiki\SpecialPage

https://gerrit.wikimedia.org/r/957890

Change 958500 merged by jenkins-bot:

[mediawiki/core@master] Namespace TitleValue under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958500

Change 958501 merged by jenkins-bot:

[mediawiki/core@master] Namespace NamespaceInfo under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958501

Change 958502 merged by jenkins-bot:

[mediawiki/core@master] Namespace TitleFormatter under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958502

Change 958503 merged by jenkins-bot:

[mediawiki/core@master] Namespace TitleParser under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958503

Change 958504 merged by jenkins-bot:

[mediawiki/core@master] Namespace remaining Title-related classes under \MediaWiki\Title

https://gerrit.wikimedia.org/r/958504

Change 958919 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace User under \MediaWiki\User

https://gerrit.wikimedia.org/r/958919

Change 959000 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace remaining User-related classes under \MediaWiki\User

https://gerrit.wikimedia.org/r/959000

Change 959001 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace Sanitizer under \MediaWiki\Parser

https://gerrit.wikimedia.org/r/959001

Change 958919 merged by jenkins-bot:

[mediawiki/core@master] Namespace User under \MediaWiki\User

https://gerrit.wikimedia.org/r/958919

Change 959152 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace Config-related classes under \MediaWiki\Config

https://gerrit.wikimedia.org/r/959152

Change 959153 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] RELEASE-NOTES-1.41: Consolidate PSR-4 deprecation efforts into one list

https://gerrit.wikimedia.org/r/959153

Change 959000 merged by jenkins-bot:

[mediawiki/core@master] Namespace remaining User-related classes under \MediaWiki\User

https://gerrit.wikimedia.org/r/959000

Change 959001 merged by jenkins-bot:

[mediawiki/core@master] Namespace Sanitizer under \MediaWiki\Parser

https://gerrit.wikimedia.org/r/959001

Change 959152 merged by jenkins-bot:

[mediawiki/core@master] Namespace Config-related classes under \MediaWiki\Config

https://gerrit.wikimedia.org/r/959152

Change 959153 merged by jenkins-bot:

[mediawiki/core@master] RELEASE-NOTES-1.41: Consolidate PSR-4 deprecation efforts into one list

https://gerrit.wikimedia.org/r/959153

Change 976311 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] Namespace remaining files under includes/deferred

https://gerrit.wikimedia.org/r/976311

Change 976311 merged by jenkins-bot:

[mediawiki/core@master] Namespace remaining files under includes/deferred

https://gerrit.wikimedia.org/r/976311

Change 982929 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/core@master] WIP: Move Parser to Mediawiki\Parser namespace.

https://gerrit.wikimedia.org/r/982929

Change 995123 had a related patch set uploaded (by Daimona Eaytoy; author: Daimona Eaytoy):

[mediawiki/tools/phan/SecurityCheckPlugin@master] Add compatibility for Parser namespacing

https://gerrit.wikimedia.org/r/995123

Change 995123 merged by jenkins-bot:

[mediawiki/tools/phan/SecurityCheckPlugin@master] Add compatibility for Parser namespacing

https://gerrit.wikimedia.org/r/995123

Change 997927 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@master] Move hooks used by OutputPage into includes/Output/Hook

https://gerrit.wikimedia.org/r/997927

Change 997927 merged by jenkins-bot:

[mediawiki/core@master] Move hooks used by OutputPage into includes/Output/Hook

https://gerrit.wikimedia.org/r/997927

Change 982929 merged by jenkins-bot:

[mediawiki/core@master] Move Parser to Mediawiki\Parser namespace

https://gerrit.wikimedia.org/r/982929

After deploy of next week, I will run lsc 😈