Page MenuHomePhabricator
Create Task
Maniphest T401454

Uses of TokenUtils::shiftTokenTSR will null offset in TokenStreamPatcher are suspect
Closed, ResolvedPublic
Actions

Description

If tokens contain a template, we'll likely get a crasher in the template handler on null tsr when expanding while reprocessTokens

From https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/1175931 and https://gerrit.wikimedia.org/r/c/mediawiki/services/parsoid/+/1176537

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Bump wikimedia/parsoid to 0.22.0-a17mediawiki/vendormaster+338 -101
Shift tokens after parsing to expandedmediawiki/services/parsoidmaster+31 -38
TSP: Reduce scenarios of passing null offset to reprocessTokensmediawiki/services/parsoidmaster+2 -5
Customize query in gerrit

Related Objects

Event Timeline

ABreault-WMF created this task.Aug 7 2025, 11:24 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 7 2025, 11:24 PM
ABreault-WMF updated the task description. (Show Details)Aug 7 2025, 11:25 PM
gerritbot added a comment.Aug 8 2025, 6:19 PM

Change #1176717 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/services/parsoid@master] TSP: Reduce scenarios of passing null offset to reprocessTokens

https://gerrit.wikimedia.org/r/1176717

gerritbot added a project: Patch-For-Review.Aug 8 2025, 6:19 PM
ssastry triaged this task as Low priority.Aug 8 2025, 6:20 PM
ssastry added a project: Content-Transform-Team (Work In Progress).
ssastry moved this task from Backlog to Code Review on the Content-Transform-Team (Work In Progress) board.
ssastry moved this task from Needs Triage to Tech Debt / Big changes on the Parsoid board.
gerritbot added a comment.Aug 11 2025, 10:55 PM

Change #1176717 merged by jenkins-bot:

[mediawiki/services/parsoid@master] TSP: Reduce scenarios of passing null offset to reprocessTokens

https://gerrit.wikimedia.org/r/1176717

gerritbot added a comment.Aug 11 2025, 11:14 PM

Change #1177517 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/services/parsoid@master] [WIP] Shift tokens after parsing to expanded

https://gerrit.wikimedia.org/r/1177517

gerritbot added a comment.Aug 13 2025, 10:21 PM

Change #1177517 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Shift tokens after parsing to expanded

https://gerrit.wikimedia.org/r/1177517

Maintenance_bot removed a project: Patch-For-Review.Aug 13 2025, 10:30 PM
ABreault-WMF mentioned this in T401413: CTT tasks week of 2025-08-08.Aug 14 2025, 5:19 PM
MSantos moved this task from Code Review to To Deploy on the Content-Transform-Team (Work In Progress) board.Aug 18 2025, 3:13 PM
ABreault-WMF claimed this task.Aug 18 2025, 3:16 PM
gerritbot added a comment.Aug 18 2025, 6:13 PM

Change #1179743 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.22.0-a17

https://gerrit.wikimedia.org/r/1179743

gerritbot added a project: Patch-For-Review.Aug 18 2025, 6:13 PM
gerritbot added a comment.Aug 18 2025, 7:25 PM

Change #1179743 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.22.0-a17

https://gerrit.wikimedia.org/r/1179743

Maintenance_bot removed a project: Patch-For-Review.Aug 18 2025, 7:31 PM
ABreault-WMF moved this task from To Deploy to To Verify on the Content-Transform-Team (Work In Progress) board.Aug 21 2025, 3:19 PM
ABreault-WMF closed this task as Resolved.Aug 21 2025, 3:42 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits