Page MenuHomePhabricator
Create Task
Maniphest T389365

Suppress autogenerated reference list regardless of details subref glitches
Open, Needs TriagePublic
Actions

Description

Parser tests show that the <references /> tag is emitted during round-trip testing, even when it was autogenerated. This seems to be caused by currently unsupported versions of the details subref syntax, and should be carefully suppressed in all cases of good or bad subrefs.

Example:

Running test Subreferencing attribute blocked without feature flag [html2wt]... FAILED!                                                                                            [221/26590]
/srv/docker-dev/mediawiki/extensions/Cite/tests/parser/subReferencing.txt:12                   
--- /tmp/mwParser-expectedswwYId        2025-03-19 14:18:57.754374968 +0000                    
+++ /tmp/mwParser-actualpoiphq  2025-03-19 14:18:57.754374968 +0000                            
@@ -1 +1,2 @@                                                                                                                                                                                 
-<ref name="a" details="abc">def</ref>                                                         
+<ref name="" details="def"></ref>                                                                                                                                                            
+<references />

Look in ReferenceListTagHandler.php, where tests for $dataMw->autoGenerated result in empty text being returned from the function.

Use patch https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cite/+/1129278 to help expose code paths which continue to emit the references tag. Run tests as follows:

./modules/mediawiki/bin/mwscript tests/parser/parserTests.php --wiki=dev --file=/srv/docker-dev/mediawiki/extensions/Cite/tests/parser/subReferencing.txt  --html2wt

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Try to clarify comment about autogenerated <references> tagmediawiki/extensions/Citemaster+5 -6
Clean up references tag handler, step 1mediawiki/extensions/Citemaster+18 -24
Normalize input wikitext for better round-trip coveragemediawiki/extensions/Citemaster+86 -112
[WIP] Clean up references tag handlermediawiki/extensions/Citemaster+9 -7
[WIP] Skip autogenerated reference lists when serializing to wikitextmediawiki/extensions/Citemaster+3 -19
Don't html2wt subreferences when they aren't supportedmediawiki/extensions/Citemaster+16 -3
Customize query in gerrit

Related Objects
Search...

Event Timeline

awight created this task.Mar 19 2025, 2:26 PM
awight changed the subtype of this task from "Spike" to "Task".
awight claimed this task.Mar 25 2025, 12:23 PM
awight moved this task from Sprint Backlog to Doing on the WMDE-TechWish-Sprint-2025-03-19 board.
gerritbot added a comment.Mar 25 2025, 12:38 PM

Change #1131002 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] Don't html2wt subreferences when they aren't supported

https://gerrit.wikimedia.org/r/1131002

gerritbot added a project: Patch-For-Review.Mar 25 2025, 12:39 PM
awight added a comment.Mar 25 2025, 2:03 PM

Note to self: It seems that the autogeneration rules which should apply to wt2html were also applied in html2wt by mistake. Specifically, round-tripping to wikitext should not add an empty <references /> tag at the end of the document, if none existed before editing the page in VE. This is responsible for many of our parser tests not playing nice with wt2wt.

gerritbot added a comment.Mar 26 2025, 1:36 PM

Change #1131002 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Don't html2wt subreferences when they aren't supported

https://gerrit.wikimedia.org/r/1131002

ReleaseTaggerBot added a project: MW-1.44-notes (1.44.0-wmf.23; 2025-04-01).Mar 26 2025, 2:00 PM
Maintenance_bot removed a project: Patch-For-Review.Mar 26 2025, 2:30 PM
awight added a comment.Apr 1 2025, 11:00 AM

This turns out to be a very old behavior, probably unnoticed due to selective serialization. We'll fix it anyway, to support more round-trip testing.

gerritbot added a comment.Apr 1 2025, 11:29 AM

Change #1133107 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] [WIP] Skip autogenerated reference lists when serializing to wikitext

https://gerrit.wikimedia.org/r/1133107

gerritbot added a project: Patch-For-Review.Apr 1 2025, 11:29 AM
ABreault-WMF subscribed.Apr 1 2025, 3:14 PM
In T389365#10697966, @awight wrote:

This turns out to be a very old behavior, probably unnoticed due to selective serialization. We'll fix it anyway, to support more round-trip testing.

No, specifically,

Atogenerated references ... are not suppressed when serializing because apparently that's the behaviour Parsoid clients want.

gerritbot added a comment.Apr 1 2025, 4:46 PM

Change #1133107 abandoned by Awight:

[mediawiki/extensions/Cite@master] [WIP] Skip autogenerated reference lists when serializing to wikitext

Reason:

The tag should be included in wikitext, for some use cases such as Flow. These clients disable selser which makes it possible for the tag to appear in output.

https://gerrit.wikimedia.org/r/1133107

Maintenance_bot removed a project: Patch-For-Review.Apr 1 2025, 5:30 PM
awight added a project: WMDE-TechWish-Sprint-2025-04-02.Apr 2 2025, 9:04 AM
gerritbot added a comment.Apr 2 2025, 12:03 PM

Change #1133368 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] [WIP] Clean up references tag handler

https://gerrit.wikimedia.org/r/1133368

gerritbot added a project: Patch-For-Review.Apr 2 2025, 12:03 PM
MareikeHeuerWMDE moved this task from Sprint Backlog to Doing on the WMDE-TechWish-Sprint-2025-04-02 board.Apr 2 2025, 12:28 PM
gerritbot added a comment.Apr 2 2025, 1:48 PM

Change #1133408 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] Normalize input wikitext for better round-trip coverage

https://gerrit.wikimedia.org/r/1133408

awight added a comment.Apr 2 2025, 1:53 PM

After learning more about the problem: this is actually desirable behavior in some use cases such as Flow and CX, where the reference list tag should be added close to the context of any new ref tags. It's also harmless in the main editing case thanks to selective serialization.

So I've edited our parser tests to provide more normalized inputs, increasing the number which can be switched to full round-trip tests.

gerritbot added a comment.Apr 2 2025, 1:55 PM

Change #1133411 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] Try to clarify comment about autogenerated <references> tag

https://gerrit.wikimedia.org/r/1133411

awight removed awight as the assignee of this task.Apr 2 2025, 1:56 PM
awight moved this task from Doing to Tech Review on the WMDE-TechWish-Sprint-2025-04-02 board.
gerritbot added a comment.Apr 3 2025, 11:25 AM

Change #1133875 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/extensions/Cite@master] Clean up references tag handler, step 1

https://gerrit.wikimedia.org/r/1133875

gerritbot added a comment.Apr 3 2025, 11:56 AM

Change #1133408 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Normalize input wikitext for better round-trip coverage

https://gerrit.wikimedia.org/r/1133408

gerritbot added a comment.Apr 3 2025, 7:35 PM

Change #1133875 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Clean up references tag handler, step 1

https://gerrit.wikimedia.org/r/1133875

ReleaseTaggerBot edited projects, added MW-1.44-notes (1.44.0-wmf.24; 2025-04-08); removed MW-1.44-notes (1.44.0-wmf.23; 2025-04-01).Apr 3 2025, 8:00 PM
awight added a comment.Apr 9 2025, 11:55 AM

We found today that the autogenerated default reference list is inserted back into wikitext in some cases which were probably not intended. This seems to happen only if a non-default-group ref is present in the page?

Example:

<ref>Testing default group here</ref>
<ref group="g">Testing with a named group</ref>

We also found that an automatically generated group reference list is rendered in VE but renders empty (no refs) and displays as a generic tag:

image.png (76×193 px, 2 KB)

WMDE-Fisch added a project: Cite (Sub-referencing).Apr 16 2025, 2:08 PM
Lina_Farid_WMDE added a parent task: T397031: [Epic] Post-MVP: Safely save pages with existing sub-references VE .Jun 16 2025, 1:17 PM
Lina_Farid_WMDE removed a parent task: T387963: Spike: Look into wiring back sub-refs when saving in VE.Jun 17 2025, 8:34 AM
WMDE-Fisch moved this task from Incoming to Later on the Cite (Sub-referencing) board.Jul 4 2025, 10:42 AM
WMDE-Fisch added a comment.Sep 3 2025, 5:58 PM

@awight Just to understand the urgency here better: Could the issues described in the ticket be responsible for user facing dirty diffs or is it more about internal states that should never be user facing?

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits