Page MenuHomePhabricator

Deprecate and remove non-remex Tidy modes of the core parser
Closed, ResolvedPublic

Description

Set the stage for eventual migration of the core parser to Parsoid by deprecating and removing unsupported non-tidy and non-remex-tidy modes of the core parser.

Details

ProjectBranchLines +/-Subject
mediawiki/extensions/ParserMigrationmaster+0 -2
mediawiki/coremaster+3 -11
mediawiki/coremaster+2 -12
mediawiki/services/parsoidmaster+3 -5
mediawiki/extensions/InputBoxmaster+23 -23
mediawiki/extensions/SyntaxHighlight_GeSHimaster+4 -9
mediawiki/extensions/ChessBrowsermaster+2 -2
mediawiki/extensions/WikiSEOmaster+9 -9
mediawiki/services/parsoidmaster+291 -290
mediawiki/services/parsoidmaster+4 -235
mediawiki/coremaster+57 -2
mediawiki/extensions/TinyMCEmaster+5 -5
mediawiki/extensions/FundraisingEmailUnsubscribemaster+1 -1
mediawiki/coremaster+9 -75
mediawiki/extensions/BlueSpiceFoundationmaster+0 -1
mediawiki/extensions/BlueSpiceFoundationREL1_35+0 -1
mediawiki/coremaster+8 -15
mediawiki/extensions/UploadWizardmaster+0 -1
mediawiki/extensions/AbuseFiltermaster+0 -1
mediawiki/coremaster+1 -36
mediawiki/extensions/Echomaster+0 -1
mediawiki/extensions/Flowmaster+0 -2
mediawiki/extensions/VEForAllmaster+1 -2
mediawiki/extensions/DocBookExportmaster+3 -1
mediawiki/extensions/Commentsmaster+1 -2
mediawiki/extensions/TextExtractsmaster+20 -29
mediawiki/coremaster+7 -3
mediawiki/extensions/BlueSpiceSocialmaster+1 -31
mediawiki/extensions/BlueSpiceFoundationmaster+1 -2
mediawiki/coremaster+45 -327
mediawiki/coremaster+5 -7
mediawiki/coremaster+403 -436
mediawiki/coremaster+81 -43
mediawiki/skins/Nimbusmaster+5 -2
mediawiki/extensions/WikiArticleFeedsmaster+1 -1
mediawiki/coremaster+8 -171
mediawiki/extensions/WikiForummaster+2 -2
mediawiki/extensions/TranslateSvgmaster+1 -1
mediawiki/extensions/TestLanguageNameGrammarmaster+4 -1
mediawiki/extensions/QuizGamemaster+2 -2
mediawiki/extensions/PollNYmaster+2 -2
mediawiki/extensions/PictureGamemaster+3 -3
mediawiki/extensions/Athenamaster+5 -5
mediawiki/extensions/DataTransfermaster+5 -2
mediawiki/extensions/FanBoxesmaster+3 -3
mediawiki/extensions/DidYouKnowmaster+1 -1
mediawiki/extensions/Commentsmaster+2 -2
mediawiki/extensions/PageFormsmaster+6 -6
mediawiki/extensions/SportsTeamsmaster+3 -3
mediawiki/extensions/SocialProfilemaster+4 -4
mediawiki/extensions/ConfirmAccountmaster+4 -4
mediawiki/extensions/Formmaster+6 -3
mediawiki/extensions/PageFormsmaster+6 -4
mediawiki/extensions/Videomaster+13 -11
mediawiki/coremaster+33 -562
mediawiki/coremaster+13 -0
mediawiki/extensions/SecurePollmaster+3 -4
mediawiki/extensions/Translatemaster+74 -8
mediawiki/coreREL1_32+192 -28
mediawiki/extensions/Collectionmaster+1 -1
mediawiki/coremaster+1 -1
mediawiki/extensions/Collectionmaster+5 -5
mediawiki/coremaster+192 -28
mediawiki/coremaster+42 -20
mediawiki/extensions/ConfirmEditmaster+1 -1
mediawiki/extensions/OAuthmaster+4 -4
mediawiki/extensions/AbuseFiltermaster+5 -7
mediawiki/extensions/CodeReviewmaster+3 -3
mediawiki/extensions/ConfirmEditREL1_32+2 -2
mediawiki/extensions/CentralAuthmaster+6 -8
mediawiki/extensions/TemplateSandboxmaster+10 -3
mediawiki/extensions/VipsScalermaster+4 -1
mediawiki/extensions/FlaggedRevsmaster+2 -2
mediawiki/coremaster+9 -7
mediawiki/extensions/LiquidThreadsmaster+12 -7
mediawiki/extensions/DonationInterfacemaster+11 -1
mediawiki/extensions/Translatemaster+47 -13
mediawiki/coremaster+9 -0
mediawiki/coremaster+5 -3
mediawiki/extensions/SecurePollmaster+9 -9
mediawiki/extensions/Cargomaster+23 -8
mediawiki/extensions/AdManagermaster+13 -13
mediawiki/extensions/SemanticGenealogymaster+16 -16
mediawiki/extensions/MOOCmaster+16 -13
mediawiki/extensions/examplesmaster+8 -5
mediawiki/extensions/CheckUsermaster+4 -5
mediawiki/extensions/WikiLexicalDatamaster+24 -24
mediawiki/coremaster+58 -49
mediawiki/extensions/ConfirmEditmaster+2 -2
mediawiki/extensions/CollaborationKitmaster+2 -2
mediawiki/extensions/FundraiserLandingPagemaster+4 -1
mediawiki/extensions/MassMessagemaster+4 -2
mediawiki/extensions/MapSourcesmaster+9 -3
mediawiki/extensions/FlaggedRevsmaster+2 -2
mediawiki/extensions/ZeroPortalmaster+3 -3
mediawiki/extensions/Collectionmaster+4 -4
mediawiki/extensions/Mathmaster+5 -2
mediawiki/extensions/CentralAuthmaster+1 -1
mediawiki/extensions/CodeReviewmaster+5 -5
mediawiki/extensions/ArticlePlaceholdermaster+4 -4
mediawiki/extensions/CodeReviewmaster+5 -5
mediawiki/extensions/Wikibasemaster+1 -1
mediawiki/coremaster+4 -0
mediawiki/coremaster+3 -2
mediawiki/coremaster+30 -6
mediawiki/extensions/ProofreadPagemaster+1 -1
mediawiki/coremaster+172 -18
mediawiki/extensions/Translatemaster+4 -5
mediawiki/coremaster+1 -1
mediawiki/coremaster+9 -5
mediawiki/coremaster+10 -4
mediawiki/extensions/ProofreadPagemaster+1 -1
mediawiki/coremaster+10 -2
mediawiki/coremaster+22 -3
mediawiki/coremaster+20 -11
Show related patches Customize query in gerrit

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 565146 merged by jenkins-bot:
[mediawiki/core@master] Update parserTests to v2 format (default html/php tests to tidy)

https://gerrit.wikimedia.org/r/565146

ssastry triaged this task as Medium priority.Feb 27 2020, 1:11 AM

Change 585330 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/core@master] Remove codepaths which ran parser in 'untidy' mode

https://gerrit.wikimedia.org/r/585330

Change 585331 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/core@master] Deprecate ParserOptions::getTidy() and ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/585331

Change 585330 merged by jenkins-bot:
[mediawiki/core@master] Remove codepaths which ran parser in 'untidy' mode

https://gerrit.wikimedia.org/r/585330

Change 585331 merged by jenkins-bot:
[mediawiki/core@master] Deprecate ParserOptions::getTidy() and ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/585331

Change 593825 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/BlueSpiceSocial@master] Always tidy the parser output

https://gerrit.wikimedia.org/r/593825

Change 593820 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/core@master] Deprecate a few more tidy-related methods

https://gerrit.wikimedia.org/r/593820

Change 593827 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/BlueSpiceFoundation@master] Use MWTidy::tidy() directly

https://gerrit.wikimedia.org/r/593827

Change 593879 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/TextExtracts@master] Tidy is no longer configurable in MW 1.35

https://gerrit.wikimedia.org/r/593879

Change 593827 merged by jenkins-bot:
[mediawiki/extensions/BlueSpiceFoundation@master] Use MWTidy::tidy() directly

https://gerrit.wikimedia.org/r/593827

Change 593825 merged by jenkins-bot:
[mediawiki/extensions/BlueSpiceSocial@master] Always tidy the parser output

https://gerrit.wikimedia.org/r/593825

Change 593820 merged by jenkins-bot:
[mediawiki/core@master] Deprecate a few more tidy-related methods

https://gerrit.wikimedia.org/r/593820

Change 593879 merged by jenkins-bot:
[mediawiki/extensions/TextExtracts@master] Tidy is no longer configurable in MW 1.35

https://gerrit.wikimedia.org/r/593879

Well, includes/parser/MWTidy.php still exists, along with a couple more of the deprecated-but-not-quite-removed things. I think as soon as 1.35 is branched (well, let's say "next week") we could actually remove those hard-deprecated bits in 1.36 and close out this task.

Here's a short list:

  • ParserBeforeTidy hook
  • includes/parser/MWTidy::singleton() (but pretty much the entire class)
  • ParserOptions::getTidy() / setTidy()
  • maintenance/benchmarks/benchmarkTidy.php (probably)
  • tests/parser/TestFileReader.php support for +untidy
  • tests/parser/editTests.php seems to have mentions of tidy as well
  • tests/parser/parserTests.php has some +tidy clauses which can be cleaned up since that's the default.
  • tests/phpunit/includes/OutputPageTest.php seems to have some untidy tests?

Change 619861 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/core@master] Remove ParserBeforeTidy hook, deprecated in 1.35

https://gerrit.wikimedia.org/r/619861

Change 619862 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/core@master] Remove 'tidy' from ParserOptions

https://gerrit.wikimedia.org/r/619862

Change 619863 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/core@master] Remove all methods of MWTidy except for MWTidy::tidy()

https://gerrit.wikimedia.org/r/619863

Change 619891 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/Flow@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619891

Change 619892 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/AbuseFilter@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619892

Change 619893 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/Echo@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619893

Change 619894 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/UploadWizard@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619894

Change 619895 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/BlueSpiceFoundation@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619895

Change 619896 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/Comments@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619896

Change 619897 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/DocBookExport@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619897

Change 619898 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/extensions/VEForAll@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619898

Change 619861 merged by jenkins-bot:
[mediawiki/core@master] Remove ParserBeforeTidy hook, deprecated in 1.35

https://gerrit.wikimedia.org/r/619861

Change 619891 merged by jenkins-bot:
[mediawiki/extensions/Flow@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619891

Change 619892 merged by jenkins-bot:
[mediawiki/extensions/AbuseFilter@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619892

Change 619893 merged by jenkins-bot:
[mediawiki/extensions/Echo@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619893

Change 619894 merged by jenkins-bot:
[mediawiki/extensions/UploadWizard@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619894

Change 619862 merged by jenkins-bot:
[mediawiki/core@master] Remove 'tidy' from ParserOptions

https://gerrit.wikimedia.org/r/619862

Change 619896 merged by jenkins-bot:
[mediawiki/extensions/Comments@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619896

Change 619897 merged by jenkins-bot:
[mediawiki/extensions/DocBookExport@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619897

Change 619898 merged by jenkins-bot:
[mediawiki/extensions/VEForAll@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619898

Change 619895 merged by Robert Vogel:
[mediawiki/extensions/BlueSpiceFoundation@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/619895

Change 620457 had a related patch set uploaded (by Robert Vogel; owner: C. Scott Ananian):
[mediawiki/extensions/BlueSpiceFoundation@REL1_35] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/620457

Change 620457 merged by Robert Vogel:
[mediawiki/extensions/BlueSpiceFoundation@REL1_35] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/620457

Change 619863 merged by jenkins-bot:
[mediawiki/core@master] Remove all methods of MWTidy except for MWTidy::tidy()

https://gerrit.wikimedia.org/r/619863

ParserMigration is still using \MWTidy::factory and now failing

ParserMigration is still using \MWTidy::factory and now failing

Yeah, if that code ever gets un-abandoned there'll be a bunch of fixes we'll need to do.

Still left:

  • ParserOptions::setTidy() (deprecated in 1.36, needs to be removed in 1.37)
  • maintenance/benchmarks/benchmarkTidy.php (probably)
  • tests/parser/editTests.php seems to have mentions of tidy as well
  • tests/parser/parserTests.php has some +tidy clauses (and +untidy) which can be cleaned up

Moving to "things to do before 1.36" task.

Change 565116 merged by jenkins-bot:
[mediawiki/extensions/FundraisingEmailUnsubscribe@master] Replace deprecated OutputPage::parse(), which will be removed from MW

https://gerrit.wikimedia.org/r/565116

Change 565123 abandoned by Duncancrane:
[mediawiki/extensions/TinyMCE@master] Replace deprecated OutputPage::parse(), which will be removed from MW

Reason:
This has been achieved by another patch

https://gerrit.wikimedia.org/r/565123

Change 665128 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/core@master] Deprecate MWTidy and TidyDriverBase::supportsValidate()

https://gerrit.wikimedia.org/r/665128

Change 665128 merged by jenkins-bot:
[mediawiki/core@master] Deprecate MWTidy and TidyDriverBase::supportsValidate()

https://gerrit.wikimedia.org/r/665128

Hey there, should this be moved to 1.37? The cut for 1.36 has happened, and 1.36.0-rc.0 will be cut in a fortnight or so, after which feature changes shouldn't be landed and back-ported.

Anything else left here? Looks like all dependent subtasks are resolved.

I think the last thing to do is actually to remove some of the methods which were deprecated in 1.35/1.36, like ParserOptions::setTidy().

I think this is still the list, and yes @Jdforrester-WMF we should do this in 1.37.

Still left:

  • ParserOptions::setTidy() (deprecated in 1.36, needs to be removed in 1.37)
  • maintenance/benchmarks/benchmarkTidy.php (probably)
  • tests/parser/editTests.php seems to have mentions of tidy as well
  • tests/parser/parserTests.php has some +tidy clauses (and +untidy) which can be cleaned up

Moving to "things to do before 1.36" task.

Change 725077 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/services/parsoid@master] Get rid of +untidy tests

https://gerrit.wikimedia.org/r/725077

Change 725079 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/services/parsoid@master] Get rid of +tidy reference in parser tests

https://gerrit.wikimedia.org/r/725079

Change 725082 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/extensions/SyntaxHighlight_GeSHi@master] Get rid of +tidy, +untidy references from parser tests

https://gerrit.wikimedia.org/r/725082

Change 725083 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/extensions/InputBox@master] Get rid of +tidy references from parser tests

https://gerrit.wikimedia.org/r/725083

Change 725087 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/extensions/WikiSEO@master] Get rid of +tidy references from parser tests

https://gerrit.wikimedia.org/r/725087

Change 725081 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/extensions/ChessBrowser@master] Get rid of +tidy references from parser tests

https://gerrit.wikimedia.org/r/725081

Change 725077 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Get rid of +untidy tests

https://gerrit.wikimedia.org/r/725077

Change 725079 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Get rid of +tidy reference in parser tests

https://gerrit.wikimedia.org/r/725079

Change 725087 merged by jenkins-bot:

[mediawiki/extensions/WikiSEO@master] Get rid of +tidy references from parser tests

https://gerrit.wikimedia.org/r/725087

Change 725082 merged by jenkins-bot:

[mediawiki/extensions/SyntaxHighlight_GeSHi@master] Get rid of +tidy, +untidy references from parser tests

https://gerrit.wikimedia.org/r/725082

Change 725081 merged by jenkins-bot:

[mediawiki/extensions/ChessBrowser@master] Get rid of +tidy references from parser tests

https://gerrit.wikimedia.org/r/725081

Change 725083 merged by jenkins-bot:

[mediawiki/extensions/InputBox@master] Get rid of +tidy references from parser tests

https://gerrit.wikimedia.org/r/725083

Change 725116 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/services/parsoid@master] Start hard deprecating +tidy uses in parser test files

https://gerrit.wikimedia.org/r/725116

Change 725116 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Start hard deprecating +tidy uses in parser test files

https://gerrit.wikimedia.org/r/725116

Change 727513 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/core@master] Get rid of ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/727513

Change 727516 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/core@master] Tweak Tidy handling in editTests.php

https://gerrit.wikimedia.org/r/727516

I think once those patches merge, we can bid this task goodbye.

Change 727513 merged by jenkins-bot:

[mediawiki/core@master] Get rid of ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/727513

Change 727516 merged by jenkins-bot:

[mediawiki/core@master] Tweak Tidy handling in editTests.php

https://gerrit.wikimedia.org/r/727516

Change 733748 had a related patch set uploaded (by Zabe; author: Zabe):

[mediawiki/extensions/ParserMigration@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/733748

Change 733748 merged by jenkins-bot:

[mediawiki/extensions/ParserMigration@master] Remove calls to ParserOptions::setTidy()

https://gerrit.wikimedia.org/r/733748