Page MenuHomePhabricator
Create Task
Maniphest T255586

Replace HTMLFormatter by Remex
Closed, ResolvedPublic
Actions

Assigned To
matmarex
Authored By
cscott
Jun 16 2020, 4:53 PM
Tags
Referenced Files
None
Subscribers
Aklapper
cscott
Jdforrester-WMF
Jdlrobson-WMF
Krinkle
matmarex
TehKittyCat
Tgr

Description

wikimedia/html-formatter uses loadHTML. It shouldn't. It should use Remex.

Maybe we should just remove the HTMLFormatter framework into Remex and remove uses of the library entirely from core.

Review of current usage: https://codesearch.wmcloud.org/search/?q=\bHtmlFormatter\\HtmlFormatter\b&files=php

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
WikiTextStructure: Replace HtmlFormatter with Parsoid DOMUtilsmediawiki/coremaster+23 -15
ExtractFormatter: Replace HtmlFormatter with Parsoid DOMUtilsmediawiki/extensions/TextExtractsmaster+84 -58
SpecialIPInfoTest: Replace HtmlFormatter with Parsoid's Remex-based helpersmediawiki/extensions/IPInfomaster+7 -17
Use Remex-based helpers provided by Parsoid for tests that parse HTMLmediawiki/coremaster+14 -15
Replace HtmlFormatter with Remex-based helpers provided by Parsoidmediawiki/extensions/LinkTargetmaster+5 -8
MobileFormatter: Use RemexHtmlmediawiki/extensions/MobileFrontendmaster+111 -106
Convert the RemovableClasses feature into a Transformmediawiki/extensions/MobileFrontendmaster+78 -95
ApiHelp: Replace use of HtmlFormatter in fixHelpLinks()mediawiki/coremaster+33 -28
Customize query in gerrit

Related Objects
Search...

Event Timeline

cscott created this task.Jun 16 2020, 4:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 16 2020, 4:53 PM
cscott added a subtask: T217850: Remex could use some helper/utility classes.Jun 16 2020, 4:53 PM
Aklapper added a project: MediaWiki-Vendor.Jun 16 2020, 5:12 PM
Jdlrobson added a project: HtmlFormatter.Jul 21 2020, 3:22 PM
Jdlrobson mentioned this in T258964: HtmlFormatter library: Code stewardship review.Jul 27 2020, 4:45 PM
Jdforrester-WMF subscribed.Jul 27 2020, 6:31 PM
Tgr subscribed.Jul 27 2020, 6:43 PM

Sort of duplicates T217360: Replace libxml/xpath in HtmlFormatter with Remex/zest.

cscott added a comment.Aug 5 2021, 11:16 PM

We probably want to use the dodo library here, too.

cscott mentioned this in T296648: Replace use of HtmlFormatter in Linker::expandLocalLinks() with Remex.Nov 29 2021, 4:35 PM
Tgr mentioned this in T217705: RemexHTML should be able to parse HTML into an existing DOM node.Dec 23 2022, 4:11 AM
matmarex closed subtask T217850: Remex could use some helper/utility classes as Resolved.Dec 29 2022, 8:25 PM
gerritbot added a comment.Feb 5 2023, 6:12 PM

Change 873766 had a related patch set uploaded (by Umherirrender; author: Bartosz Dziewoński):

[mediawiki/core@master] ApiHelp: Replace use of HtmlFormatter in fixHelpLinks()

https://gerrit.wikimedia.org/r/873766

gerritbot added a project: Patch-For-Review.Feb 5 2023, 6:12 PM
gerritbot added a comment.Feb 5 2023, 6:27 PM

Change 873766 merged by jenkins-bot:

[mediawiki/core@master] ApiHelp: Replace use of HtmlFormatter in fixHelpLinks()

https://gerrit.wikimedia.org/r/873766

Maintenance_bot removed a project: Patch-For-Review.Feb 5 2023, 6:30 PM
ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.22; 2023-02-06).Feb 5 2023, 7:00 PM
matmarex added a subtask: T296648: Replace use of HtmlFormatter in Linker::expandLocalLinks() with Remex.Feb 7 2023, 8:46 PM
matmarex closed subtask T296648: Replace use of HtmlFormatter in Linker::expandLocalLinks() with Remex as Resolved.
Krinkle subscribed.Feb 7 2023, 9:41 PM
matmarex mentioned this in T345319: TypeError: Argument 1 passed to HtmlFormatter\HtmlFormatter::onHtmlReady() must be of the type string, null given, called in /srv/mediawiki/php-1.41.0-wmf.24/vendor/wikimedia/html-formatter/src/HtmlFormatter.php on line 314.Sep 4 2023, 4:26 PM
cscott mentioned this in T348251: Move ContentHolder from core to independent library.Oct 5 2023, 1:49 PM
TehKittyCat subscribed.Dec 20 2023, 2:41 AM
gerritbot added a comment.May 12 2024, 3:55 PM

Change #1030548 had a related patch set uploaded (by Func; author: Func):

[mediawiki/extensions/MobileFrontend@master] Convert the RemovableClasses feature into a Transform

https://gerrit.wikimedia.org/r/1030548

gerritbot added a project: Patch-For-Review.May 12 2024, 3:55 PM
gerritbot added a comment.May 12 2024, 5:00 PM

Change #1030551 had a related patch set uploaded (by Func; author: Func):

[mediawiki/extensions/MobileFrontend@master] MobileFormatter: Use RemexHtml

https://gerrit.wikimedia.org/r/1030551

gerritbot added a comment.Aug 28 2024, 11:11 PM

Change #1030548 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] Convert the RemovableClasses feature into a Transform

https://gerrit.wikimedia.org/r/1030548

ReleaseTaggerBot added a project: MW-1.43-notes (1.43.0-wmf.21; 2024-09-03).Aug 29 2024, 12:00 AM
gerritbot added a comment.Jul 15 2025, 2:37 PM

Change #1030551 abandoned by Bartosz Dziewoński:

[mediawiki/extensions/MobileFrontend@master] MobileFormatter: Use RemexHtml

Reason:

Superseded by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MobileFrontend/+/1156787

https://gerrit.wikimedia.org/r/1030551

matmarex mentioned this in T217360: Replace libxml/xpath in HtmlFormatter with Remex/zest.Jul 15 2025, 2:38 PM
matmarex edited projects, added MobileFrontend, CirrusSearch, IP Info, TextExtracts, MediaWiki-extensions-LinkTarget, MediaWiki-General; removed Patch-For-Review, MediaWiki-Vendor.
matmarex subscribed.

Review of current usage: https://codesearch.wmcloud.org/search/?q=\bHtmlFormatter\\HtmlFormatter\b&files=php

Restricted Application added projects: Discovery-Search, Trust and Safety Product Team. · View Herald TranscriptJul 15 2025, 2:38 PM
Dreamy_Jazz subscribed.EditedJul 15 2025, 2:43 PM

Can we get an explanation as to why HtmlFormatter should not be used? Is HtmlFormatter deprecated?

The task description is not clear on why this is needed. Knowing that will help Trust and Safety Product Team prioritise this.

matmarex added a comment.Jul 15 2025, 9:56 PM

HtmlFormatter relies on the HTML parsing and serialization provided by PHP's built-in DOM library, which is from the HTML 4 era, and known to cause a variety of bugs, e.g. T348928, T251434, T396695.

I believe that only reason it hasn't been deprecated is because there is no maintainer (and no one wants to touch if out of fear of becoming the maintainer), see T258964: HtmlFormatter library: Code stewardship review.

It doesn't do much anyway, and what it does is better done using Remex-based helpers provided by Parsoid, so this shouldn't be too difficult, if it's not as deeply integrated into your code as it was in MobileFrontend. I'll do one or two of these as an example.

gerritbot added a comment.Jul 15 2025, 10:18 PM

Change #1169784 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/core@master] Use Remex-based helpers provided by Parsoid for tests that parse HTML

https://gerrit.wikimedia.org/r/1169784

gerritbot added a project: Patch-For-Review.Jul 15 2025, 10:18 PM
matmarex added a comment.Jul 15 2025, 10:55 PM

I picked the LinkTarget for my second patch, and funnily enough, it also had a recent bug caused by interactions with HtmlFormatter: T397603: TOC not being shown when using LinkTarget

gerritbot added a comment.Jul 15 2025, 11:00 PM

Change #1169787 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/LinkTarget@master] Replace HtmlFormatter with Remex-based helpers provided by Parsoid

https://gerrit.wikimedia.org/r/1169787

Dreamy_Jazz added a project: Trust and Safety Product Sprint (Sprint Rum baba (July 28 - August 15)).Jul 16 2025, 10:20 AM
gerritbot added a comment.Jul 16 2025, 12:47 PM

Change #1169787 merged by jenkins-bot:

[mediawiki/extensions/LinkTarget@master] Replace HtmlFormatter with Remex-based helpers provided by Parsoid

https://gerrit.wikimedia.org/r/1169787

gerritbot added a comment.Jul 16 2025, 2:22 PM

Change #1169784 merged by jenkins-bot:

[mediawiki/core@master] Use Remex-based helpers provided by Parsoid for tests that parse HTML

https://gerrit.wikimedia.org/r/1169784

ReleaseTaggerBot added a project: MW-1.45-notes (1.45.0-wmf.11; 2025-07-22).Jul 16 2025, 3:01 PM
Tchanders edited projects, added Trust and Safety Product Sprint; removed Trust and Safety Product Sprint (Sprint Rum baba (July 28 - August 15)).Jul 16 2025, 3:26 PM
Restricted Application removed a project: Trust and Safety Product Team. · View Herald TranscriptJul 16 2025, 3:26 PM
Tchanders moved this task from Backlog to Maintenance to triage on the Trust and Safety Product Sprint board.Jul 16 2025, 3:26 PM
matmarex updated the task description. (Show Details)Jul 16 2025, 4:02 PM
matmarex removed a project: Patch-For-Review.Jul 17 2025, 9:29 PM
gerritbot added a comment.Jul 18 2025, 12:34 PM

Change #1170539 had a related patch set uploaded (by Esanders; author: Esanders):

[mediawiki/extensions/IPInfo@master] SpecialIPInfoTest: Replace HtmlFormatter with Parsoid's Remex-based helpers

https://gerrit.wikimedia.org/r/1170539

gerritbot added a project: Patch-For-Review.Jul 18 2025, 12:34 PM
gerritbot added a comment.Jul 18 2025, 1:07 PM

Change #1170539 merged by jenkins-bot:

[mediawiki/extensions/IPInfo@master] SpecialIPInfoTest: Replace HtmlFormatter with Parsoid's Remex-based helpers

https://gerrit.wikimedia.org/r/1170539

Dreamy_Jazz updated the task description. (Show Details)Jul 18 2025, 1:07 PM
Dreamy_Jazz edited projects, added Trust and Safety Product Sprint (Sprint Cannoli (July 7 - July 25)); removed Trust and Safety Product Sprint.
Dreamy_Jazz moved this task from Priority Backlog to Done on the Trust and Safety Product Sprint (Sprint Cannoli (July 7 - July 25)) board.
gerritbot added a comment.Jul 18 2025, 1:40 PM

Change #1170561 had a related patch set uploaded (by Esanders; author: Esanders):

[mediawiki/extensions/TextExtracts@master] ExtractFormatter: Replace HtmlFormatter with Parsoid DOMUtils

https://gerrit.wikimedia.org/r/1170561

gerritbot added a comment.Jul 22 2025, 12:53 AM

Change #1170561 merged by jenkins-bot:

[mediawiki/extensions/TextExtracts@master] ExtractFormatter: Replace HtmlFormatter with Parsoid DOMUtils

https://gerrit.wikimedia.org/r/1170561

ReleaseTaggerBot edited projects, added MW-1.45-notes (1.45.0-wmf.12; 2025-07-29); removed MW-1.45-notes (1.45.0-wmf.11; 2025-07-22).Jul 22 2025, 1:00 AM
matmarex added a comment.Jul 22 2025, 2:11 AM

I dropped a note for the Citizen skin here: https://github.com/StarCitizenTools/mediawiki-skins-Citizen/issues/1094

matmarex updated the task description. (Show Details)Jul 22 2025, 2:12 AM
alistair3149 updated the task description. (Show Details)Jul 30 2025, 10:54 PM
gerritbot added a comment.Jul 31 2025, 8:25 PM

Change #1174812 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/core@master] WikiTextStructure: Replace HtmlFormatter with Parsoid DOMUtils

https://gerrit.wikimedia.org/r/1174812

Dreamy_Jazz unsubscribed.Jul 31 2025, 8:49 PM
Jdforrester-WMF added a comment.Jul 31 2025, 9:28 PM

There's now no uses left in code search, beyond vendor/core's dependency. Is there a practical way in which we can emit deprecation warnings on access to HtmlFormatter, other than doing a "don't use me" release of the library? Otherwise, we could just declare it deprecated now and drop it in 1.46?

Jdforrester-WMF updated the task description. (Show Details)Jul 31 2025, 9:29 PM
gerritbot added a comment.Jul 31 2025, 9:46 PM

Change #1174812 merged by jenkins-bot:

[mediawiki/core@master] WikiTextStructure: Replace HtmlFormatter with Parsoid DOMUtils

https://gerrit.wikimedia.org/r/1174812

ReleaseTaggerBot edited projects, added MW-1.45-notes (1.45.0-wmf.13; 2025-08-05); removed MW-1.45-notes (1.45.0-wmf.12; 2025-07-29).Jul 31 2025, 10:00 PM
matmarex updated the task description. (Show Details)Jul 31 2025, 10:50 PM
matmarex added a comment.Jul 31 2025, 11:14 PM
In T255586#11052147, @Jdforrester-WMF wrote:

Is there a practical way in which we can emit deprecation warnings on access to HtmlFormatter, other than doing a "don't use me" release of the library?

We can use trigger_error( ..., E_USER_DEPRECATED ); in the constructor: https://codesearch.wmcloud.org/libraries/?q=E_USER_DEPRECATED&files=php

And we can mark the package as abandoned: https://getcomposer.org/doc/04-schema.md#abandoned (wikimedia/remex-html could be the suggested replacement)

Otherwise, we could just declare it deprecated now and drop it in 1.46?

Certainly. In fact, I don't see why we couldn't drop it in 1.45, but we can go slower if you prefer. We should just add a release note in MediaWiki when dropping it.

matmarex closed this task as Resolved.Aug 2 2025, 12:14 AM
matmarex claimed this task.
matmarex mentioned this in T401028: Deprecate the HtmlFormatter library and drop it from MediaWiki core.

I think we can resolve this, since we've replaced all of its uses. Thanks for contributing, everyone.

I filed a separate task for actually deprecating/removing the library itself: T401028: Deprecate the HtmlFormatter library and drop it from MediaWiki core

EBernhardson removed a project: Discovery-Search.Aug 11 2025, 3:17 PM
Jdlrobson-WMF added a project: Reader Growth Team.Aug 28 2025, 4:31 PM
Jdlrobson-WMF subscribed.

Addressed by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MobileFrontend/+/1156787

Jdlrobson-WMF mentioned this in T400401: TypeError in MobileFrontend RemovableClassesTransform (PHP 8.4).Aug 28 2025, 4:31 PM
Jdforrester-WMF added a comment.Aug 28 2025, 4:38 PM
In T255586#11129422, @Jdlrobson-WMF wrote:

Addressed by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MobileFrontend/+/1156787

One of the seven pieces of this work was, yes. See the description.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits