Page MenuHomePhabricator
Create Task
Maniphest T262408

[TRACKING] Selser issues on talk pages
Open, MediumPublic
Actions

Description

Diffs taken from https://www.mediawiki.org/wiki/Extension:DiscussionTools/Edits_with_possible_corruption

Details

ProjectBranchLines +/-Subject
mediawiki/vendormaster+813 -484Bump wikimedia/parsoid to 0.13.0-a17
mediawiki/services/parsoidmaster+37 -0Selser: Skip over templates while adding selser-wrapper <span>s
mediawiki/corewmf/1.36.0-wmf.16+1 -1vendor: Bump wikimedia/parsoid to 0.13.0-a15
mediawiki/vendorwmf/1.36.0-wmf.16+44 -34Bump wikimedia/parsoid to 0.13.0-a15
mediawiki/vendormaster+44 -34Bump wikimedia/parsoid to 0.13.0-a15
mediawiki/services/parsoidmaster+31 -21Prevent selser corruption from unclosed tags in list items in some cases
mediawiki/vendormaster+2 K -1 KBump wikimedia/parsoid to 0.13.0-a14
mediawiki/services/parsoidmaster+436 -27SelSer: Preprocess DOMs to wrap text children of <li> in <span>s
mediawiki/vendormaster+1 K -974Bump wikimedia/parsoid to 0.13.0-a12
mediawiki/services/parsoidmaster+183 -66Record length of trimmed whitespace in additional DSR fields
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenRyasmeenT262408 [TRACKING] Selser issues on talk pages
 ResolvedssastryT262409 Space at end of another list item removed
 OpenNoneT262410 Space at start of parent list item (before template) removed
 OpenNoneT262411 <nowiki>s added to previous comment
 OpenNoneT262412 HTML normalised in parent tag
 OpenNoneT262413 Newlines added after templates to previous paragraphs when adding a comment
 DeclinedmatmarexT262414 {{safesubst:}} else where in doc is substituted when adding a comment
 ResolvedssastryT262448 Unmatched closing tags are removed
 ResolvedssastryT262449 Syntax for empty list item is removed
 ResolvedArlolraT264028 Whitespace in <nowiki>…</nowiki> normalized or something
 ResolvedppelbergT264027 <nowiki/> tag removed inside template parameter
 OpenssastryT265094 Line break in a link label was changed to a space
 ResolvedppelbergT266140 HTML entity replaced by the Unicode character in an edit
 OpenNoneT267157 Weird list corruption in a DiscussionTools edit (2020-09-11)
 DuplicateNoneT268035 Random space inserted in a list item in a DiscussionTools edit
 DuplicateNoneT268037 Template substituted in a DiscussionTools edit
 OpenNoneT268038 Parsoid removing unmatched closing tag causes unexpected indent-pre styling
 OpenRyasmeenT269017 Empty list item has whitespace removed
 ResolvedssastryT269901 Dirty diff: Single additional trailing whitespace added to several discussion tools edits in last 3+ weeks
 ResolvedssastryT270180 White space before top of page meta items is removed
 ResolvedssastryT277592 Parsoid added a newline before a space, resulting in indent-pre markup

Event Timeline

Esanders created this task.Sep 9 2020, 1:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 9 2020, 1:22 PM
Esanders mentioned this in T261321: Inspect Reply Tool diffs for instances of corruption.Sep 9 2020, 1:22 PM
Esanders updated the task description. (Show Details)Sep 9 2020, 1:43 PM
Esanders added a comment.Sep 9 2020, 2:04 PM

The two whitespace issues (T262409 & T262410) appear to account for the majority of corruption we are seeing, if you want to prioritise your focus.

ssastry added a subscriber: ssastry.Sep 9 2020, 2:48 PM

This below reproduces both the bugs.

[subbu@earth:~/work/wmf/parsoid] echo -e "*  a \n** b \n" > /tmp/wt
[subbu@earth:~/work/wmf/parsoid] php bin/parse.php < /tmp/wt > /tmp/old.html
[subbu@earth:~/work/wmf/parsoid] sed 's/b<\/li>/b<\/li>\n<li>b-new<\/li>/g;' < /tmp/old.html > /tmp/new.html
[subbu@earth:~/work/wmf/parsoid] php bin/parse.php --html2wt --selser --oldtextfile /tmp/wt --oldhtmlfile /tmp/old.html< /tmp/new.html > /tmp/edited.wt
[subbu@earth:~/work/wmf/parsoid] diff /tmp/wt /tmp/edited.wt
1,3c1,3
< *  a 
< ** b 
< 
---
> * a 
> ** b
> ** b-new
ppelberg added a project: Editing-team (Tracking).Sep 9 2020, 3:59 PM
ppelberg moved this task from Backlog to Blocked by others on the Editing-team (Tracking) board.
ppelberg added a subscriber: ppelberg.
ssastry triaged this task as Medium priority.Sep 10 2020, 5:26 PM
ssastry moved this task from Needs Triage to Current & Upcoming Work on the Parsoid board.
ssastry closed subtask T262409: Space at end of another list item removed as Resolved.Sep 17 2020, 6:00 PM
gerritbot added a comment.Sep 21 2020, 8:51 PM

Change 628937 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/services/parsoid@master] WIP: Selser: Preprocess doms to wrap li text nodes in wrapper

https://gerrit.wikimedia.org/r/628937

gerritbot added a project: Patch-For-Review.Sep 21 2020, 8:51 PM
ssastry claimed this task.Sep 23 2020, 10:59 PM
ssastry added a project: Parsing-Active-Work.
ssastry moved this task from Incoming Work (but not yet ready) to Doing (Other Work) on the Parsing-Active-Work board.
ppelberg mentioned this in T263709: [SPIKE] Why can the Reply Tool be opened on a page with broken/incomplete table syntax..Sep 24 2020, 12:34 AM
gerritbot added a comment.Sep 25 2020, 10:56 PM

Change 630219 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/services/parsoid@master] Record trimmed whitespace in additional DSR fields

https://gerrit.wikimedia.org/r/630219

matmarex added subtasks: T264028: Whitespace in <nowiki>…</nowiki> normalized or something, T264027: <nowiki/> tag removed inside template parameter.Sep 28 2020, 5:46 PM
ssastry raised the priority of this task from Medium to High.Oct 1 2020, 4:39 PM
ssastry moved this task from Doing (Other Work) to Epics / Tracking Tasks on the Parsing-Active-Work board.Oct 1 2020, 4:55 PM
gerritbot added a comment.Oct 13 2020, 4:55 AM

Change 630219 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Record length of trimmed whitespace in additional DSR fields

https://gerrit.wikimedia.org/r/630219

gerritbot added a comment.Oct 19 2020, 10:52 PM

Change 635100 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a12

https://gerrit.wikimedia.org/r/635100

gerritbot added a comment.Oct 20 2020, 1:28 AM

Change 635100 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a12

https://gerrit.wikimedia.org/r/635100

matmarex added a subtask: T266140: HTML entity replaced by the Unicode character in an edit.Oct 21 2020, 2:32 PM
cscott mentioned this in T266285: Deploy 6-element DSR to prod.Oct 22 2020, 9:50 PM
gerritbot added a comment.Oct 30 2020, 10:24 PM

Change 628937 merged by jenkins-bot:
[mediawiki/services/parsoid@master] SelSer: Preprocess DOMs to wrap text children of <li> in <span>s

https://gerrit.wikimedia.org/r/628937

Maintenance_bot removed a project: Patch-For-Review.Oct 30 2020, 11:10 PM
gerritbot added a comment.Nov 2 2020, 7:14 PM

Change 638161 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a14

https://gerrit.wikimedia.org/r/638161

gerritbot added a project: Patch-For-Review.Nov 2 2020, 7:14 PM
gerritbot added a comment.Nov 2 2020, 9:30 PM

Change 638161 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a14

https://gerrit.wikimedia.org/r/638161

Maintenance_bot removed a project: Patch-For-Review.Nov 2 2020, 10:10 PM
gerritbot added a comment.Nov 3 2020, 12:36 AM

Change 638210 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/services/parsoid@master] Prevent selser corruption from unclosed tags in list items in some cases

https://gerrit.wikimedia.org/r/638210

gerritbot added a project: Patch-For-Review.Nov 3 2020, 12:36 AM
gerritbot added a comment.Nov 3 2020, 5:24 AM

Change 638210 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Prevent selser corruption from unclosed tags in list items in some cases

https://gerrit.wikimedia.org/r/638210

matmarex added subtasks: T267158: Parsoid's Cleanup DOM pass incorrectly deletes empty nodes with transclusion info, T267157: Weird list corruption in a DiscussionTools edit (2020-09-11), T267156: Weird list corruption in a DiscussionTools edit (2020-10-29).Nov 3 2020, 7:31 PM
gerritbot added a comment.Nov 3 2020, 10:50 PM

Change 638735 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a15

https://gerrit.wikimedia.org/r/638735

gerritbot added a comment.Nov 3 2020, 11:30 PM

Change 638735 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a15

https://gerrit.wikimedia.org/r/638735

gerritbot added a comment.Nov 4 2020, 12:04 AM

Change 638520 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@wmf/1.36.0-wmf.16] Bump wikimedia/parsoid to 0.13.0-a15

https://gerrit.wikimedia.org/r/638520

gerritbot added a comment.Nov 4 2020, 4:27 PM

Change 638520 merged by jenkins-bot:
[mediawiki/vendor@wmf/1.36.0-wmf.16] Bump wikimedia/parsoid to 0.13.0-a15

https://gerrit.wikimedia.org/r/638520

gerritbot added a comment.Nov 4 2020, 5:31 PM

Change 639214 had a related patch set uploaded (by Brennen Bearnes; owner: Brennen Bearnes):
[mediawiki/core@wmf/1.36.0-wmf.16] vendor: Bump wikimedia/parsoid to 0.13.0-a15

https://gerrit.wikimedia.org/r/639214

gerritbot added a comment.Nov 4 2020, 6:06 PM

Change 639214 merged by jenkins-bot:
[mediawiki/core@wmf/1.36.0-wmf.16] vendor: Bump wikimedia/parsoid to 0.13.0-a15

https://gerrit.wikimedia.org/r/639214

ssastry removed a subtask: T267156: Weird list corruption in a DiscussionTools edit (2020-10-29).Nov 4 2020, 8:50 PM
ppelberg edited projects, added Editing QA; removed Editing-team (Tracking).Nov 5 2020, 10:50 PM
ppelberg moved this task from Inbox to High Priority on the Editing QA board.
ssastry removed a subtask: T267158: Parsoid's Cleanup DOM pass incorrectly deletes empty nodes with transclusion info.Nov 9 2020, 7:23 PM
gerritbot added a comment.Nov 10 2020, 3:10 PM

Change 640438 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/services/parsoid@master] Selser: Skip over templates while adding selser-wrapper <span>s

https://gerrit.wikimedia.org/r/640438

gerritbot added a comment.Nov 10 2020, 7:47 PM

Change 640438 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Selser: Skip over templates while adding selser-wrapper <span>s

https://gerrit.wikimedia.org/r/640438

ssastry closed subtask T265094: Line break in a link label was changed to a space as Resolved.Nov 16 2020, 5:41 PM
ssastry reopened subtask T265094: Line break in a link label was changed to a space as Open.
ssastry closed subtask T265094: Line break in a link label was changed to a space as Resolved.Nov 16 2020, 5:54 PM
ssastry added a comment.Nov 16 2020, 9:05 PM

@ppelberg could you all evaluate which of the subtasks are fixed and resolve them? There are some tasks where the dirty diffs would be fixed for list items but not outside list items (ex: unbalanced closing tags might still be removed outside lists).

matmarex closed subtask T262414: {{safesubst:}} else where in doc is substituted when adding a comment as Declined.Nov 16 2020, 10:06 PM
ppelberg added a comment.Nov 16 2020, 10:51 PM
In T262408#6625619, @ssastry wrote:

@ppelberg could you all evaluate which of the subtasks are fixed and resolve them? There are some tasks where the dirty diffs would be fixed for list items but not outside list items (ex: unbalanced closing tags might still be removed outside lists).

Absolutely, @ssastry. Thank you for the ping. We have plans to explicitly test this task's subtasks this week.

ppelberg reassigned this task from ssastry to Ryasmeen.Nov 16 2020, 10:52 PM
ppelberg added a subscriber: Ryasmeen.

Assigning this over to @Ryasmeen for further testing.

  • Before that, I (@ppelberg) need to make sure testing instructions are added to the task description of each of this task's substasks.
ppelberg added a project: Editing-team (FY2021-22 Kanban Board).Nov 16 2020, 10:53 PM
ppelberg moved this task from Incoming to Blocked / Needs More Work on the Editing-team (FY2021-22 Kanban Board) board.
gerritbot added a comment.Nov 17 2020, 1:09 AM

Change 641308 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a17

https://gerrit.wikimedia.org/r/641308

gerritbot added a comment.Nov 17 2020, 1:52 AM

Change 641308 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a17

https://gerrit.wikimedia.org/r/641308

matmarex added subtasks: T268035: Random space inserted in a list item in a DiscussionTools edit, T268037: Template substituted in a DiscussionTools edit, T268038: Parsoid removing unmatched closing tag causes unexpected indent-pre styling.Nov 17 2020, 4:46 PM
ppelberg moved this task from Blocked / Needs More Work to Ready for Sign Off on the Editing-team (FY2021-22 Kanban Board) board.Nov 18 2020, 3:40 PM
Ryasmeen edited projects, added Verified; removed Editing QA.Nov 23 2020, 8:17 PM
ssastry closed subtask T262449: Syntax for empty list item is removed as Resolved.Nov 23 2020, 8:21 PM
JTannerWMF moved this task from Ready for Sign Off to Quarterly Goals on the Editing-team (FY2021-22 Kanban Board) board.Dec 2 2020, 6:30 PM
ssastry closed subtask T262448: Unmatched closing tags are removed as Resolved.Dec 9 2020, 2:38 PM
Esanders reopened subtask T262409: Space at end of another list item removed as Open.Dec 11 2020, 11:32 AM
ssastry closed subtask T262409: Space at end of another list item removed as Resolved.Dec 11 2020, 2:29 PM
ssastry added a subtask: T269901: Dirty diff: Single additional trailing whitespace added to several discussion tools edits in last 3+ weeks.Dec 11 2020, 3:21 PM
ssastry closed subtask T269901: Dirty diff: Single additional trailing whitespace added to several discussion tools edits in last 3+ weeks as Resolved.Dec 12 2020, 5:58 PM
matmarex reopened subtask T265094: Line break in a link label was changed to a space as Open.Jan 2 2021, 8:34 PM
ppelberg closed subtask T266140: HTML entity replaced by the Unicode character in an edit as Resolved.Jan 16 2021, 4:11 AM
ssastry lowered the priority of this task from High to Medium.Jan 29 2021, 5:47 PM
ppelberg edited projects, added Editing-team (Tracking); removed Editing-team (FY2021-22 Kanban Board).Feb 9 2021, 4:12 PM
Arlolra closed subtask T264028: Whitespace in <nowiki>…</nowiki> normalized or something as Resolved.Apr 17 2021, 2:19 PM
ppelberg moved this task from Backlog to Triaged on the DiscussionTools board.Apr 28 2021, 5:47 PM
ppelberg added a subtask: T277592: Parsoid added a newline before a space, resulting in indent-pre markup.
Arlolra moved this task from Current & Upcoming Work to Bugs & Crashers on the Parsoid board.Jul 30 2021, 10:21 PM
Arlolra closed subtask T270180: White space before top of page meta items is removed as Resolved.Sep 20 2021, 6:51 PM
ppelberg closed subtask T264027: <nowiki/> tag removed inside template parameter as Resolved.Sep 25 2021, 1:58 AM
ssastry removed projects: Patch-For-Review, Parsing-Active-Work.Dec 16 2021, 10:39 PM
ssastry closed subtask T277592: Parsoid added a newline before a space, resulting in indent-pre markup as Resolved.Feb 25 2022, 7:18 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL