Page MenuHomePhabricator

Ensure all the features of parse.js are covered by parse.php
Open, NormalPublic

Description

Here's what's currently available in parse.js,

  • --help, -h Show this help message [boolean] [default: false]
  • --version Show version number [boolean]
  • --wt2html Wikitext -> HTML [boolean] [default: false]
  • --html2wt HTML -> Wikitext [boolean] [default: false]
  • --wt2wt Wikitext -> HTML -> Wikitext
  • --html2html HTML -> Wikitext -> HTML
  • --selser Use the selective serializer to go from HTML to
  • --normalize Normalize the output as parserTests would do. Use
  • --body_only Just return the body, without any normalizations
  • --config Path to a config.yaml file. Use --config w/ no
  • --oldtext The old page text for a selective-serialization
  • --oldtextfile File containing the old page text for a
  • --oldhtmlfile File containing the old HTML for a
  • --domdiff File containing the diff-marked HTML for used
  • --inputfile File containing input as an alternative to stdin
  • --pbin Input pagebundle JSON [default: ""]
  • --pbinfile Input pagebundle JSON file [default: ""]
  • --pboutfile Output pagebundle JSON to file [default: false]
  • --offline Shortcut to turn off various network fetches
  • --record Record http requests for later replay
  • --replay Replay recorded http requests for later replay
  • --verbose Log at level "info" as well
  • --useWorker Use a worker farm [boolean] [default: false] [ NOT RELEVANT ]
  • --linting Parse with linter enabled
  • --loadWMF Use WMF mediawiki API config [ WON'T SUPPORT ]
  • --prefix Which wiki prefix to use; e.g. "enwiki" for [ WON'T SUPPORT ]
  • --domain Which wiki to use; e.g. "en.wikipedia.org" for
  • --oldid Oldid of the given page. [default: null]
  • --outputContentVersion The acceptable content version.[default: "2.1.0"]
  • --pageName The page name, returned for {{PAGENAME}}. If no
  • --pageBundle Output pagebundle JSON [boolean] [default: false]
  • --scrubWikitext Apply wikitext scrubbing while serializing. This
  • --contentmodel The content model of the input. Defaults to
  • --wrapSections Output <section> tags (default false)
  • --debug Provide optional flags. Use --debug=help for
  • --trace Use --trace=help for supported options
  • --dump Dump state. Use --dump=help for supported options
  • --genTest Generates token transformer and DOM pass tests. [ WON'T SUPPORT ]
  • --genTestOut Output file to use for token transformer tests [ WON'T SUPPORT ]
  • --genDirectory Output directory to use for DOM tests [ WON'T SUPPORT ]
  • --genTestFragments Enable fragment generation in DOM genTest output [ WON'T SUPPORT ]
  • --fetchConfig Whether to fetch the wiki config from the server [ NOT RELEVANT ]
  • --fetchTemplates Whether to fetch included templates recursively [ NOT RELEVANT ]
  • --fetchImageInfo Whether to fetch image info via the API [ NOT RELEVANT ]
  • --expandExtensions Whether we should request extension tag [ NOT RELEVANT ]
  • --usePHPPreProcessor Whether to use the PHP preprocessor to expand [ NOT RELEVANT ]
  • --addHTMLTemplateParameters Parse template parameters to HTML and add them to
  • --maxdepth Maximum expansion depth [default: 40]
  • --apiURL http path to remote API, e.g.
  • --rtTestMode Test in rt test mode (changes some parse &
  • --useBatchAPI Turn on/off the API batching system [ NOT RELEVANT ]
  • --phpConfigFile PHP config to splice into the JS pipelines [ NOT RELEVANT ]
  • --color Enable color output Ex: --no-color

vs what's in parse.php,

> php bin/parse.php --help

Omnibus script to convert between wikitext and HTML, and roundtrip wikitext or
HTML. Supports a number of options pertaining to pointing at a specific wiki or
enabling various features during these transformations.

If no options are provided, --wt2html is enabled by default.
See --help for detailed usage help.

Usage: php parse.php [--body_only|--help|--html2wt|--oldhtmlfile|--oldtextfile|--quiet|--selser|--wt2html]

    --help (-1): Display this help message
    --quiet (-1): Whether to suppress non-error output

Script specific parameters:
    --body_only: Just return the body, without any normalizations as in
        --normalize
    --html2wt: HTML -> Wikitext
    --oldhtmlfile: File containing the old HTML for a
        selective-serialization (see --selser)
    --oldtextfile: File containing the old page text for a
        selective-serialization (see --selser)
    --selser: Use the selective serializer to go from HTML to Wikitext.
    --wt2html: Wikitext -> HTML

Event Timeline

Arlolra created this task.Jul 16 2019, 8:53 PM
Restricted Application added subscribers: Liuxinyu970226, Aklapper. · View Herald TranscriptJul 16 2019, 8:53 PM
Arlolra triaged this task as Normal priority.Jul 16 2019, 8:53 PM
Arlolra updated the task description. (Show Details)Jul 16 2019, 10:50 PM
Arlolra updated the task description. (Show Details)Jul 18 2019, 5:38 PM
Arlolra added a comment.EditedJul 18 2019, 5:49 PM

genTest is not relevant in the post-javascript world, so I'm crossing those off

Same for useWorker and phpConfigFile

Arlolra updated the task description. (Show Details)Jul 18 2019, 5:49 PM

genTest is not relevant in the post-javascript world, so I'm crossing those off

Ya, I removed those from ScriptUtils.php already.

Change 524369 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Add wt2wt/html2html modes to bin/parse.php

https://gerrit.wikimedia.org/r/524369

Change 524369 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add wt2wt/html2html modes to bin/parse.php

https://gerrit.wikimedia.org/r/524369

Arlolra updated the task description. (Show Details)Jul 19 2019, 4:01 AM

Change 524915 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Add option for inputfile to bin/parse.php

https://gerrit.wikimedia.org/r/524915

Change 524917 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Add pageName option to bin/parse.php

https://gerrit.wikimedia.org/r/524917

Trying to address T178914: parse.js should have standard arguments as part of this port might be something worth considering.

Trying to address T178914: parse.js should have standard arguments as part of this port might be something worth considering.

I could address the specific points mentioned there, but I would rather not be on the hook for coming up with a holistic vision

Trying to address T178914: parse.js should have standard arguments as part of this port might be something worth considering.

I could address the specific points mentioned there, but I would rather not be on the hook for coming up with a holistic vision

Ok ... since we might be running both parse.js and parse.php for a while during the port, I take back my recommendation (which was premised on the thought that maybe we don't need to backport the changes from the PHP version to the JS version). So, ok to just arrive at feature parity first.

Change 524915 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add option for inputfile to bin/parse.php

https://gerrit.wikimedia.org/r/524915

Arlolra updated the task description. (Show Details)Jul 22 2019, 10:08 PM
Arlolra updated the task description. (Show Details)Jul 23 2019, 4:18 PM
Arlolra updated the task description. (Show Details)

Change 525128 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Add scrubWikitext option to bin/parse.php

https://gerrit.wikimedia.org/r/525128

Change 524917 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add pageName option to bin/parse.php

https://gerrit.wikimedia.org/r/524917

Arlolra updated the task description. (Show Details)Jul 23 2019, 5:44 PM

Change 525128 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add scrubWikitext option to bin/parse.php

https://gerrit.wikimedia.org/r/525128

Arlolra updated the task description. (Show Details)Jul 23 2019, 6:16 PM

Change 525155 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Add wrapSection option to bin/parse.php

https://gerrit.wikimedia.org/r/525155

Change 525155 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add wrapSection option to bin/parse.php

https://gerrit.wikimedia.org/r/525155

Arlolra updated the task description. (Show Details)Jul 23 2019, 9:27 PM
Arlolra updated the task description. (Show Details)

Change 525166 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Add oldtextfile option to bin/parse.php

https://gerrit.wikimedia.org/r/525166

Arlolra updated the task description. (Show Details)Jul 23 2019, 10:32 PM
Arlolra added a comment.EditedJul 23 2019, 10:34 PM

There should probably be better test coverage of this bin or at least fixing this,
https://github.com/wikimedia/parsoid/blob/master/bin/toolcheck.sh#L19-L21

Edit: That part got done in T228217#5362937

Change 525166 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add oldtextfile option to bin/parse.php

https://gerrit.wikimedia.org/r/525166

Change 525312 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Run the toolcheck with bin/parse.php

https://gerrit.wikimedia.org/r/525312

Change 525333 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Add addHTMLTemplateParameters options to bin/parse.php

https://gerrit.wikimedia.org/r/525333

Change 525341 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Add rtTestMode option to bin/parse.php

https://gerrit.wikimedia.org/r/525341

Change 525312 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Run the toolcheck with bin/parse.php

https://gerrit.wikimedia.org/r/525312

Change 525333 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add addHTMLTemplateParameters options to bin/parse.php

https://gerrit.wikimedia.org/r/525333

Arlolra updated the task description. (Show Details)Jul 24 2019, 7:41 PM

Change 525341 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add rtTestMode option to bin/parse.php

https://gerrit.wikimedia.org/r/525341

Arlolra updated the task description. (Show Details)Jul 24 2019, 8:26 PM
ssastry moved this task from Backlog to Porting on the Parsoid-PHP board.Jul 26 2019, 12:50 AM

@Sbailey is going to be implementing the "--domain" functionality

Change 533616 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Added --linter true option to parse.php

https://gerrit.wikimedia.org/r/533616

Change 533610 had a related patch set uploaded (by Subramanya Sastry; owner: Sbailey):
[mediawiki/services/parsoid@master] Add --domain and --apiURL support to parse.php

https://gerrit.wikimedia.org/r/533610

Change 533610 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add --domain and --apiURL support to parse.php

https://gerrit.wikimedia.org/r/533610

Change 533616 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Added --lint option to parse.php

https://gerrit.wikimedia.org/r/533616

ssastry updated the task description. (Show Details)Sep 3 2019, 10:00 PM
ssastry updated the task description. (Show Details)

Change 539970 had a related patch set uploaded (by Subramanya Sastry; owner: Sbailey):
[mediawiki/services/parsoid@master] Add --trace --dump and --debug command line option support

https://gerrit.wikimedia.org/r/539970

Change 539970 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add --trace --dump and --debug command line option support

https://gerrit.wikimedia.org/r/539970

ssastry updated the task description. (Show Details)Thu, Oct 3, 7:29 PM