Page MenuHomePhabricator

Define stable interface policy for Parsoid
Open, MediumPublic

Description

Review of https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/900380 renewed some debate about proxying utils for extensions.

@ssastry summarized,

TLDR of discussion: we want to move PHPUtils and DOMUtils to the src/Core/Utils namespace and make them directly usable. Everything else in src/Utils should be proxied through src/Ext.

@cscott adds,

Core and Ext are the only two namespaces that should be used outside parsoid, and everything in them is @stable unless otherwise marked; everything outside those two namespaces is @internal (and shouldn't be marked otherwise).

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
mediawiki/extensions/ImageMapmaster+3 -3
mediawiki/extensions/CheckUsermaster+32 -33
mediawiki/extensions/MobileFrontendmaster+12 -12
mediawiki/extensions/SecurePollmaster+11 -11
mediawiki/extensions/ProofreadPagemaster+3 -3
mediawiki/extensions/MobileFrontendContentProvidermaster+2 -2
mediawiki/extensions/DiscussionToolsmaster+29 -29
mediawiki/extensions/Citemaster+11 -11
mediawiki/extensions/ConfirmEditmaster+2 -2
mediawiki/extensions/Poemmaster+1 -1
mediawiki/extensions/IPInfomaster+10 -10
mediawiki/extensions/GlobalBlockingmaster+2 -2
mediawiki/extensions/TextExtractsmaster+4 -4
mediawiki/extensions/AbuseFiltermaster+6 -6
mediawiki/extensions/CentralAuthmaster+2 -2
mediawiki/extensions/LinkTargetmaster+1 -1
mediawiki/extensions/MediaSpoilermaster+2 -2
mediawiki/extensions/Kartographermaster+1 -1
mediawiki/vendormaster+1 K -1 K
mediawiki/services/parsoidmaster+40 -0
mediawiki/services/parsoidmaster+754 -734
mediawiki/services/parsoidmaster+36 -0
mediawiki/extensions/WikiLambdamaster+3 -3
mediawiki/vendormaster+304 -80
mediawiki/services/parsoidmaster+8 -5
Show related patches Customize query in gerrit

Event Timeline

Arlolra triaged this task as Medium priority.Mar 17 2023, 10:10 PM
Arlolra moved this task from Needs Triage to Tech Debt / Big changes on the Parsoid board.

Core and Ext are the only two namespaces that should be used outside parsoid, and everything in them is @stable unless otherwise marked; everything outside those two namespaces is @internal (and shouldn't be marked otherwise)."

I can understand this guideline for extensions but I will note that this might be hard guideline to follow for core because a lot of things in Parsoid are used in core. Here is a list of all Parsoid classes already used in Core along with use counts:

21 use Wikimedia\Parsoid\Parsoid;

10 use Wikimedia\Parsoid\ParserTests\TestMode as ParserTestMode;
10 use Wikimedia\Parsoid\ParserTests\Test as ParserTest;
 4 use Wikimedia\Parsoid\ParserTests\TestFileReader;
 1 use Wikimedia\Parsoid\ParserTests\StyleTag as ParsoidStyleTag;
 1 use Wikimedia\Parsoid\ParserTests\RawHTML as ParsoidRawHTML;
 1 use Wikimedia\Parsoid\ParserTests\ParserHook as ParsoidParserHook;
 1 use Wikimedia\Parsoid\ParserTests\Article as ParserTestArticle;

16 use Wikimedia\Parsoid\Core\PageBundle;
11 use Wikimedia\Parsoid\Core\ClientError;
10 use Wikimedia\Parsoid\Core\TOCData;
 9 use Wikimedia\Parsoid\Core\ResourceLimitExceededException;
 6 use Wikimedia\Parsoid\Core\SectionMetadata;
 5 use Wikimedia\Parsoid\Core\SelserData;
 4 use Wikimedia\Parsoid\Core\ContentMetadataCollector;
 1 use Wikimedia\Parsoid\Core\ContentMetadataCollectorCompat;

 9 use Wikimedia\Parsoid\Config\SiteConfig;
 7 use Wikimedia\Parsoid\Config\PageConfig;
 5 use Wikimedia\Parsoid\Config\DataAccess;
 3 use Wikimedia\Parsoid\Config\PageContent as IPageContent;
 2 use Wikimedia\Parsoid\Config\PageConfig as IPageConfig;
 1 use Wikimedia\Parsoid\Config\SiteConfig as ISiteConfig;
 1 use Wikimedia\Parsoid\Config\PageConfigFactory;
 1 use Wikimedia\Parsoid\Config\DataAccess as IDataAccess;
 1 use Wikimedia\Parsoid\Config\Api\SiteConfig as ApiSiteConfig;
 1 use Wikimedia\Parsoid\Config\Api\PageConfig as ApiPageConfig;
 1 use Wikimedia\Parsoid\Config\Api\DataAccess as ApiDataAccess;

 9 use Wikimedia\Parsoid\Utils\ContentUtils;
 6 use Wikimedia\Parsoid\Utils\DOMUtils;
 4 use Wikimedia\Parsoid\Utils\DOMCompat;
 2 use Wikimedia\Parsoid\Utils\Timing;
 1 use Wikimedia\Parsoid\Utils\Utils;
 1 use Wikimedia\Parsoid\Utils\ScriptUtils;
 1 use Wikimedia\Parsoid\Utils\DOMDataUtils;

 5 use Wikimedia\Parsoid\DOM\Document;
 1 use Wikimedia\Parsoid\DOM\Element;

 1 use Wikimedia\Parsoid\Mocks\MockMetrics;

 1 use Wikimedia\Parsoid\Ext\ExtensionModule;

One answer is to say many of those namespaces should be embedded in Core and that other utils should all get proxied through Ext/ and that is possible, but let us think this through a bit. We can definitely mandate namespaces as entirely internal-Parsoid that core shouldn't have any access to, for example (Wt2html, Html2wt, Tokens, NodeData, Logger). But, I expect (DOM, Config, Wikitext, Language, ParserTests, Utils) will have classes that will be needed in m/w core in some form.

My interpretation would be that @internal is acceptable to use in core, but maybe we should be a little more precise, because it's not clear that we want to use *all* of this stuff indiscriminately in core -- at the very least we ought to mark the things used in core so that we don't 'accidentally' break core by changing them in Parsoid.

Might be worth distinguishing three categories of Parsoid APIs; I'm going to make up @tags for them:

  • For use by Extensions: @stable-ext
  • For use by core: @internal-core
  • For use by parsertests @internal-tests
  • For use by the REST API @internal-rest

The first category is closest to @stable in core's Stable Interface Policy, since there are third parties involved so deprecation/waiting a release cycle/etc is probably warranted.

The second and third categories are more like @internal in core's Stable Interface Policy -- they can be changed without extensive deprecation etc, but I think we still want a little bit of control/enforcement over what exactly is used here to reduce the possibility of unintentional breakage in production.

The last category is perhaps the hardest, since historically the REST API was part of Parsoid, not core, and might want more internal access to Parsoid that otherwise warranted. Daniel has talked about pulling this API into an extension, though. It seems like this *might* warrant a separate marking, since methods exported for use by the REST API may not be appropriate for general third party use.

Going through subbu's list:

The *\Config\* stuff is obviously meant to be used by core, but probably not meant to be used by extensions except possibly via the MediaWiki service which supplies it. (But maybe not even then.) @internal-core

The *\DOM\* stuff is similarly meant to be used, so should probably just be added to the stable interface policy. @stable

There's a bunch of stuff in ParserTests which is intended to be used *in dev mode only* (ie for running tests) but not for production code. I'm not sure how to enforce that, exactly. We used to keep the parser test stuff in the composer -dev path, but that didn't work with how CI was running tests for some reason. Needs investigation. @internal-tests

The troublesome bits are the stuff in *\Utils\*. I can't tell if that's more incidental use in parser tests (which is /probably/ ok?) or more subtle dependencies between core and Parsoid (perhaps in the REST API?). I think we'd already talked about moving DOMUtils/DOMCompat to Core (or to a separate library). But the other stuff probably needs closer examination.

Just as a quick add-on -- I think the real blocking child task here is to configure CI (phan?) to enforce some of these guidelines. If we want to have fine-grained stability markers ("some methods in Util are @stable and some are @internal") instead of the big-hammer "everything in this namespace is @stable") then we need some tooling support.

ABreault-WMF renamed this task from Adopt a stable interface policy refactor to Define stable interface policy for Parsoid.Sep 29 2025, 7:59 PM
ABreault-WMF subscribed.

From @cscott in T405967

We should have a deprecation/stability policy for Parsoid. Recently we've inherited parts of core's policy, but implementing SiteConfig::deprecated()/PHPUtils::deprecated() method which ties in to core's similar deprecation warning mechanism. And we've informally said things like "code outside parsoid shouldn't use anything but classes in Wikimedia\Parsoid\Ext and WIkimedia\Parsoid\Core" but that has been sparsely enforced. We should at some point write down a proper policy, probably borrowing heavily from core's, to have it available to folks outside our team.

Change #1192207 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] Add links to HISTORY.md to where releases are now tracked

https://gerrit.wikimedia.org/r/1192207

We should probably add Wikimedia\Parsoid\DOM* to the list of "things we expect others to use". DOMCompat/DOMUtils live in Wikimedia\Parsoid\Utils (and are mirrored in Wikimedia\Parsoid\Ext) but could probably be moved to Wikimedia\Parsoid\DOM as logically part of the "DOM compatibility package".

Change #1192207 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Add links to HISTORY.md to where releases are now tracked

https://gerrit.wikimedia.org/r/1192207

Change #1193889 had a related patch set uploaded (by OSleger; author: OSleger):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.22.0-a26

https://gerrit.wikimedia.org/r/1193889

Change #1193889 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.22.0-a26

https://gerrit.wikimedia.org/r/1193889

Change #1234116 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/services/parsoid@master] Move Utils/DOMCompat to Ext/DOMCompat, since it is stable to use

https://gerrit.wikimedia.org/r/1234116

Change #1234119 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/DiscussionTools@master] Only code in Wikimedia\Parsoid\{Ext,DOM} is stable to use

https://gerrit.wikimedia.org/r/1234119

Change #1234121 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/MobileFrontend@master] Only code in Wikimedia\Parsoid\{Ext,DOM} is stable to use

https://gerrit.wikimedia.org/r/1234121

Change #1234122 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/CheckUser@master] Only code in Wikimedia\Parsoid\{Ext,DOM} is stable to use

https://gerrit.wikimedia.org/r/1234122

Change #1234123 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/Kartographer@master] Only code in Wikimedia\Parsoid\{Ext,DOM} is stable to use

https://gerrit.wikimedia.org/r/1234123

Change #1234124 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/Cite@master] Only code in Wikimedia\Parsoid\{Ext,DOM} is stable to use

https://gerrit.wikimedia.org/r/1234124

Change #1234125 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/AbuseFilter@master] Only code in Wikimedia\Parsoid\{Ext,DOM} is stable to use

https://gerrit.wikimedia.org/r/1234125

Change #1234127 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/ProofreadPage@master] Only code in Wikimedia\Parsoid\{Ext,DOM} is stable to use

https://gerrit.wikimedia.org/r/1234127

Change #1234129 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/ImageMap@master] Only code in Wikimedia\Parsoid\{Ext,DOM} is stable to use

https://gerrit.wikimedia.org/r/1234129

Change #1234130 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/WikiLambda@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234130

Change #1234132 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/SecurePoll@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234132

Change #1234133 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/CentralAuth@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234133

Change #1234134 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/ConfirmEdit@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234134

Change #1234135 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/GlobalBlocking@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234135

Change #1234136 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/IPInfo@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234136

Change #1234137 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/LinkTarget@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234137

Change #1234138 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/MediaSpoiler@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234138

Change #1234139 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/MobileFrontendContentProvider@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234139

Change #1234140 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/TextExtracts@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234140

Change #1234141 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/services/parsoid@master] Add DOMUtils::parseHTML to Ext\DOMUtils

https://gerrit.wikimedia.org/r/1234141

Change #1234149 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/services/parsoid@master] Add ContentUtils::createAndLoad* to Ext\ContentUtils

https://gerrit.wikimedia.org/r/1234149

Change #1234130 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234130

Change #1234141 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Add DOMUtils::parseHTML/::parseHTMLToFragment to Ext\DOMUtils

https://gerrit.wikimedia.org/r/1234141

Change #1234116 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Move Utils/DOMCompat to Core/DOMCompat, since it is stable to use

https://gerrit.wikimedia.org/r/1234116

Change #1234149 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Add ContentUtils::createAndLoad* to Ext\ContentUtils

https://gerrit.wikimedia.org/r/1234149

Change #1235865 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a14

https://gerrit.wikimedia.org/r/1235865

Change #1235865 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.23.0-a14

https://gerrit.wikimedia.org/r/1235865

Change #1234123 abandoned by C. Scott Ananian:

[mediawiki/extensions/Kartographer@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

Reason:

Unnecessary after Ie733fc4d8384f5154dab66336e7673b5bb5b5b68

https://gerrit.wikimedia.org/r/1234123

Change #1235896 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/Poem@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1235896

Change #1234125 merged by jenkins-bot:

[mediawiki/extensions/AbuseFilter@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234125

Change #1234133 merged by jenkins-bot:

[mediawiki/extensions/CentralAuth@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234133

Change #1234138 merged by jenkins-bot:

[mediawiki/extensions/MediaSpoiler@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234138

Change #1234137 merged by jenkins-bot:

[mediawiki/extensions/LinkTarget@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234137

Change #1234140 merged by jenkins-bot:

[mediawiki/extensions/TextExtracts@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234140

Change #1234135 merged by jenkins-bot:

[mediawiki/extensions/GlobalBlocking@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234135

Change #1234136 merged by jenkins-bot:

[mediawiki/extensions/IPInfo@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234136

Change #1234134 merged by jenkins-bot:

[mediawiki/extensions/ConfirmEdit@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234134

Change #1235896 merged by jenkins-bot:

[mediawiki/extensions/Poem@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1235896

Change #1234119 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234119

Change #1234124 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234124

Change #1234132 merged by jenkins-bot:

[mediawiki/extensions/SecurePoll@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234132

Change #1234127 merged by jenkins-bot:

[mediawiki/extensions/ProofreadPage@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234127

Change #1234139 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontendContentProvider@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234139

Change #1234122 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234122

Change #1234121 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234121

Change #1234129 merged by jenkins-bot:

[mediawiki/extensions/ImageMap@master] Only code in Wikimedia\Parsoid\{Core,DOM,Ext} is stable to use

https://gerrit.wikimedia.org/r/1234129