Page MenuHomePhabricator

Cite: Improve compatibility between Parsoid's port and the PHP extension
Closed, ResolvedPublic

Description

Error checks that the PHP Cite extension does are not performed in Parsoid's Cite implementation. Here is the list of i18n keys in the Cite extension that serve as proxies for error handling that Parsoid needs to support. As we add support, they will be X'd out.

  • "cite_error_ref_invalid_dir": "Invalid <code>dir=\"$1\"</code>, must be <code>ltr</code> or <code>rtl</code>",
  • "cite_error_ref_numeric_key": "Invalid <ref> tag; name cannot be a simple integer. Use a descriptive title",
  • "cite_error_ref_no_key": "The opening <ref> tag is malformed or has a bad name",
  • "cite_error_ref_too_many_keys": "Invalid <ref> tag; invalid names, e.g. too many",
  • "cite_error_ref_no_input": "Invalid <ref> tag; refs with no name must have content",
  • "cite_error_references_duplicate_key": "Invalid <ref> tag; name \"$1\" defined multiple times with different content",
  • "cite_error_references_invalid_parameters": "Invalid parameter in <references> tag",
  • "cite_error_references_no_text": "Invalid <ref> tag; no text was provided for refs named <code>$1</code>",
  • "cite_error_group_refs_without_references": "<ref> tags exist for a group named \"$1\", but no corresponding <references group=\"$1\"/> tag was found",
  • "cite_error_references_group_mismatch": "<ref> tag in <references> has conflicting group attribute \"$1\".",
  • "cite_error_references_missing_key": "<ref> tag with name \"$1\" defined in <references> is not used in prior text.",
  • "cite_error_references_no_key": "<ref> tag defined in <references> has no name attribute.",
  • "cite_error_empty_references_define": "<ref> tag defined in <references> with name \"$1\" has no content.",

The following error types won't be supported by Parsoid:

  • "cite_error_references_no_backlink_label": "Ran out of custom backlink labels. Define more in the [[MediaWiki:Cite references link many format backlink labels]] message.", [Parsoid uses CSS counters]
  • "cite_error_no_link_label_group": "Ran out of custom link labels for group \"$1\". Define more in the [[MediaWiki:$2]] message." [Parsoid uses CSS counters]
  • "cite_error_ref_nested_extends": "Extending <ref> tags more than one level deep is not allowed", [not supported currently by parsoid tokenizer]
  • "cite_error_included_ref": "Closing </ref> missing for <ref> tag", [not supported currently by parsoid tokenizer],
  • "cite_error_included_references": "Closing tag missing for <references>", [not supported currently by parsoid tokenizer],
  • "cite_error_references_missing_group": "<ref> tag defined in <references> has group attribute \"$1\" which does not appear in prior text.", [this error is unlikely to ever occur and not worth implementing],

Besides that, Parsoid also needs to address the following missing features.

  • Parsoid does not support the follow arg on ref tags
  • Parsoid doesn't know about special group names like "lower-alpha" etc. and treats them like other group names like "xyz", "books", etc.

The second issue has been split off into T272097.

One possible new error that might be worth reporting is a references section with group which has no preceding refs defined in that group. That is a similar but different error than 'cite_error_references_missing_group'.

Also, please check T221489, T221490, T221491 for additional changes to core Cite that needs to be reflected in Parsoid.


Version: unspecified
Severity: normal

Details

Reference
bz49538
ProjectBranchLines +/-Subject
mediawiki/vendormaster+1 K -527
mediawiki/services/parsoidmaster+13 -11
mediawiki/services/parsoidmaster+8 -3
mediawiki/services/parsoidmaster+3 -2
mediawiki/services/parsoidmaster+4 -3
mediawiki/services/parsoidmaster+11 -10
mediawiki/services/parsoidmaster+9 -5
mediawiki/vendorwmf/1.36.0-wmf.20+1 K -700
mediawiki/vendormaster+1 K -700
mediawiki/services/parsoidmaster+20 -9
mediawiki/services/parsoidmaster+9 -2
mediawiki/services/parsoidmaster+6 -5
mediawiki/vendormaster+813 -484
mediawiki/services/parsoidmaster+47 -2
mediawiki/services/parsoidmaster+5 -10
mediawiki/vendormaster+2 K -1 K
mediawiki/services/parsoidmaster+77 -33
mediawiki/vendormaster+1 K -974
mediawiki/services/parsoidmaster+8 -0
mediawiki/services/parsoidmaster+74 -7
mediawiki/services/parsoidmaster+15 -10
mediawiki/services/parsoidmaster+76 -6
mediawiki/services/parsoidmaster+52 -0
mediawiki/services/parsoidmaster+28 -0
mediawiki/vendormaster+203 -156
mediawiki/services/parsoidmaster+213 -40
mediawiki/services/parsoidmaster+131 -254
mediawiki/vendormaster+248 -404
mediawiki/services/parsoidmaster+28 -4
mediawiki/services/parsoidmaster+22 -18
mediawiki/vendormaster+414 -403
mediawiki/services/parsoidmaster+133 -27
mediawiki/vendormaster+226 -285
mediawiki/services/parsoidmaster+7 -67
mediawiki/vendormaster+1 K -641
mediawiki/services/parsoidmaster+34 -72
Show related patches Customize query in gerrit

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 608437 merged by jenkins-bot:
[mediawiki/vendor@master] Bump Parsoid to v0.12.0-a19

https://gerrit.wikimedia.org/r/c/mediawiki/vendor/ /608437

Change 608745 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Adding parsoid cite error for name defined but empty or white space

https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/ /608745

Change 608924 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Rename "cite_error_ref_no_text" to "cite_error_references_no_text"

https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/ /608924

Change 608924 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Rename "cite_error_ref_no_text" to "cite_error_references_no_text"

https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/ /608924

Change 608745 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Match core error key for self-closed ref without name

https://gerrit.wikimedia.org/r/608745

Change 609445 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump Parsoid to v0.12.0-a20

https://gerrit.wikimedia.org/r/609445

Change 609445 merged by jenkins-bot:
[mediawiki/vendor@master] Bump Parsoid to v0.12.0-a20

https://gerrit.wikimedia.org/r/609445

Change 610409 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP adding "follow" functionality to parsoid cite extension

https://gerrit.wikimedia.org/r/610409

Change 626238 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Follow up to "follow" functionality for Cite

https://gerrit.wikimedia.org/r/626238

Change 610409 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Adding "follow" functionality to the Cite extension

https://gerrit.wikimedia.org/r/610409

Change 626238 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Follow up to "follow" functionality for Cite

https://gerrit.wikimedia.org/r/626238

Change 627345 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/vendor@master] Bump parsoid to 0.13.0-a9

https://gerrit.wikimedia.org/r/627345

Change 627345 merged by jenkins-bot:
[mediawiki/vendor@master] Bump parsoid to 0.13.0-a9

https://gerrit.wikimedia.org/r/627345

Change 627392 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Additional cite test for multiple follow

https://gerrit.wikimedia.org/r/627392

Change 627392 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Additional cite test for multiple follow

https://gerrit.wikimedia.org/r/627392

Notes about errors that Parsoid Cite will not or may not generate and new errors that should be generated but have no key or error message yet in the en.json file:

At the moment, Parsoid Cite follows implementation can encounter errors and for the moment, are attached to existing messages, which are close but technically not accurate error messages for the error encountered. This is due to the core Cite en.json file not having any follow error keys and messages added. Whoever added follows functionality to core Cite failed to add error checking messages to the en.json file. For example:

"cite_error_ref_too_many_keys": "Invalid <code>&lt;ref&gt;</code> tag; invalid names, e.g. too many"

This is used when ref name and follow are defined simultaneously. We know that in code, yet the message generated fails to mention follow and name at the same time when it could. Admittedly, the error message while inexact, should point out to to the editor the nature of the error.

"cite_error_ref_nested_extends": "Extending <ref> tags more than one level deep is not allowed"

This error is not detectable in the Cite extension itself, as the tokenizer and related Parsoid code obfuscates this error in the tokenizing pass outside the scope of Parsoid Cite ability to be informed of its occurrence.

"cite-tracking-category-cite-error": "Pages with reference errors",
"cite-tracking-category-cite-error-desc": "Pages in this category have errors in the usage of references tags.",

These errors may not be generated at the same location of core Cite or may not be generated at all, if inline specific errors are deemed adequate as the utility of these in a VE session is questionable.

Change 628944 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a10

https://gerrit.wikimedia.org/r/628944

Change 628961 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] New Cite test that exercises groups and follows in mixed order

https://gerrit.wikimedia.org/r/628961

Change 629173 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP Adding more error checking to Cite extension

https://gerrit.wikimedia.org/r/629173

Change 628961 merged by jenkins-bot:
[mediawiki/services/parsoid@master] New Cite test that exercises groups and follows in mixed order

https://gerrit.wikimedia.org/r/628961

Change 632311 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Added cite error checks for invalid text direction

https://gerrit.wikimedia.org/r/632311

Change 632324 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Added cite error checks for numeric digit in name

https://gerrit.wikimedia.org/r/632324

Change 629173 abandoned by Sbailey:
[mediawiki/services/parsoid@master] This patch has been split into two patches for dir and numeric errors

Reason:
Split into two other patches, comments are still of interest

https://gerrit.wikimedia.org/r/629173

Change 632311 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Added cite error checks for invalid text direction

https://gerrit.wikimedia.org/r/632311

Parsoid does not tokenize unbalanced tags so the following two errors are not identifiable. A <ref ...> without closing </ref> and <referances ...> without a closing </references>. Self closing <ref ... /> and <references /> are handled properly.
"cite_error_included_ref": "Closing </ref> missing for <ref> tag",
"cite_error_included_references": "Closing tag missing for <references>",

Parsoid does not tokenize unbalanced tags so the following two errors are not identifiable.

Maybe the tokenizer needs to call a method of the extension api so they can return custom errors for this?

I am not convinced we need to support them. Those feel like they can be folded into the Linting project.

i.e. unclosed / unbalanced tags is not specific to Cite extension.

Change 632324 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Added cite error checks for numeric digit in name

https://gerrit.wikimedia.org/r/632324

Change 634665 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP Cite error Ran out of custom backlink labels

https://gerrit.wikimedia.org/r/634665

Change 634665 abandoned by Sbailey:
[mediawiki/services/parsoid@master] WIP Cite error Ran out of custom backlink labels

Reason:
Turns out that Parsoid Cite uses CSS for linkLable (renaming) and does not support custom cite_link_label_group aliases either.

https://gerrit.wikimedia.org/r/634665

Change 635100 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a12

https://gerrit.wikimedia.org/r/635100

Change 635100 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a12

https://gerrit.wikimedia.org/r/635100

Change 635667 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP Refs in group with no <references group="xxx" />

https://gerrit.wikimedia.org/r/635667

"cite_error_group_refs_without_references" is added on the refs, but there's a question about where the autogenerated references with group itself should also get an error,
https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/635667/10/tests/parser/citeParserTests.txt

Change 635667 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Refs in group with no <references group="xxx" />

https://gerrit.wikimedia.org/r/635667

Change 637772 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP Adding check for illegal attribute in references tag

https://gerrit.wikimedia.org/r/637772

Change 638161 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a14

https://gerrit.wikimedia.org/r/638161

Change 638161 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a14

https://gerrit.wikimedia.org/r/638161

Change 639572 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Fix adding 'cite_error_group_refs_without_references' to unnamed refs

https://gerrit.wikimedia.org/r/639572

Change 639572 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Fix adding 'cite_error_group_refs_without_references' to unnamed refs

https://gerrit.wikimedia.org/r/639572

Change 637772 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Adding check for illegal attributes in references tag

https://gerrit.wikimedia.org/r/637772

Change 641308 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a17

https://gerrit.wikimedia.org/r/641308

Change 641308 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a17

https://gerrit.wikimedia.org/r/641308

Change 641863 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP Adding cite error ref in reference no content defined

https://gerrit.wikimedia.org/r/641863

Change 642075 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP Adding cite error ref in reference with mismatched group

https://gerrit.wikimedia.org/r/642075

Change 642167 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP Cite error tag name defined in references not used before

https://gerrit.wikimedia.org/r/642167

Change 641863 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Adding cite error ref in reference no content defined

https://gerrit.wikimedia.org/r/641863

Change 642075 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Adding cite error ref in reference with mismatched group

https://gerrit.wikimedia.org/r/642075

Change 642167 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Cite error tag name defined in references not used before

https://gerrit.wikimedia.org/r/642167

Change 643565 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP Adding ...references_missing_group error to cite

https://gerrit.wikimedia.org/r/643565

Change 644393 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a18

https://gerrit.wikimedia.org/r/644393

Change 644221 had a related patch set uploaded (by C. Scott Ananian; owner: Subramanya Sastry):
[mediawiki/vendor@wmf/1.36.0-wmf.20] Bump wikimedia/parsoid to 0.13.0-a18

https://gerrit.wikimedia.org/r/644221

Change 644393 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a18

https://gerrit.wikimedia.org/r/644393

Change 644221 merged by jenkins-bot:
[mediawiki/vendor@wmf/1.36.0-wmf.20] Bump wikimedia/parsoid to 0.13.0-a18

https://gerrit.wikimedia.org/r/644221

Change 643565 abandoned by Sbailey:
[mediawiki/services/parsoid@master] WIP Adding ...references_missing_group error to cite

Reason:
After further analysis with Arlo, this error is a very unlikely edge case to occur and be worthy of testing for, determined to not be worthy of implementation.

https://gerrit.wikimedia.org/r/643565

Sbailey updated the task description. (Show Details)

Change 644609 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/extensions/Cite@master] Cite add to css for .mw-follow to add display: none;

https://gerrit.wikimedia.org/r/644609

Change 644614 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] WIP Parsoid Cite add class ms-follow for refs with follow

https://gerrit.wikimedia.org/r/644614

Change 649941 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Add group name parameter to cite_error_group_refs_without_references

https://gerrit.wikimedia.org/r/649941

Change 649935 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Add direction parameter to cite_error_ref_invalid_dir message

https://gerrit.wikimedia.org/r/649935

Change 649941 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add group name parameter to cite_error_group_refs_without_references

https://gerrit.wikimedia.org/r/649941

Change 649935 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add direction parameter to cite_error_ref_invalid_dir message

https://gerrit.wikimedia.org/r/649935

Change 650278 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Add group name to Cite error cite_error_references_group_mismatch

https://gerrit.wikimedia.org/r/650278

Change 650281 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Add ref/follow name to Cite error cite_error_references_missing_key

https://gerrit.wikimedia.org/r/650281

Change 650285 had a related patch set uploaded (by Sbailey; owner: Sbailey):
[mediawiki/services/parsoid@master] Add refname parameter to cite_error_empty_references_define error

https://gerrit.wikimedia.org/r/650285

Change 650278 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add group name to Cite error cite_error_references_group_mismatch

https://gerrit.wikimedia.org/r/650278

Change 650285 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add refname parameter to cite_error_empty_references_define error

https://gerrit.wikimedia.org/r/650285

Change 650281 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Add ref/follow name to Cite error cite_error_references_missing_key

https://gerrit.wikimedia.org/r/650281

Change 651570 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a21

https://gerrit.wikimedia.org/r/651570

Change 651570 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a21

https://gerrit.wikimedia.org/r/651570