Page MenuHomePhabricator
Create Task
Maniphest T319141

Deprecate prop=sections in action=parse
Open, Needs TriagePublicBUG REPORT
Actions

Description

In action=parse, when getting prop=sections, each section has a byteoffset field. However, that field does not contain a byte offset, but rather a Unicode codepoint offset.

This is extremely misleading! There are two complaints (one by me just now) about this on the action=parse talk page: https://www.mediawiki.org/w/index.php?title=API_talk:Parsing_wikitext&oldid=5499602#%22byteoffset%22_is_a_misnomer

In an ideal world we would rename the field, but I think all we can do now is add a note to the documentation for action=parse.

@cscott: The correct information is available via TOCData, we should export that and deprecate sections.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
ApiParse: deprecate prop=sectionsmediawiki/coremaster+22 -49
ApiParse: remove deprecated 'prop=sections' from default propertiesmediawiki/coremaster+3 -1
Bump parsoid to 0.17.0-a20 and zest-css to 3.0.0mediawiki/vendormaster+357 -205
Report Parsoid section offsets in codepoints, not bytesmediawiki/services/parsoidmaster+50 -3
Parser: Section offsets are in codepoints, not in bytesmediawiki/coremaster+3 -3
Bump wikimedia/parsoid to 0.17.0-a15mediawiki/vendormaster+2 K -528
SectionMetadata: Rename byteOffset to codepointOffset to reflect realitymediawiki/services/parsoidmaster+41 -10
Fix import of non-ASCII sectionsmediawiki/extensions/Flowmaster+4 -2
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenBUG REPORTNoneT319141 Deprecate prop=sections in action=parse
 ResolvedcscottT328605 Export TOCData in the action API

Event Timeline

Enterprisey created this task.Oct 2 2022, 6:36 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 2 2022, 6:36 AM
Func added projects: MediaWiki-Parser, StructuredDiscussions.Oct 3 2022, 2:54 AM
Func subscribed.

A friend of mine told me years ago that they have no idea how to use this "byteoffset" field when developing a talk archiving tool for a Chinese site. Turns out it's measured in codepoints, then everything makes sense.

And a deployed extension also misused this field of section data: https://codesearch.wmcloud.org/deployed/?q=byteoffset&repos=mediawiki/extensions/Flow

Maybe we can introduce a correct field named "codepoint" first.

Restricted Application added a project: Growth-Team. · View Herald TranscriptOct 3 2022, 2:54 AM
gerritbot added a comment.Oct 3 2022, 3:20 AM

Change 837254 had a related patch set uploaded (by Func; author: Func):

[mediawiki/core@master] Parser: Introduce a "codepoint" field for section data

https://gerrit.wikimedia.org/r/837254

gerritbot added a project: Patch-For-Review.Oct 3 2022, 3:20 AM
kostajh moved this task from Inbox to Triaged on the Growth-Team board.Oct 12 2022, 1:13 PM
kostajh subscribed.

Thank you for filing this task and thanks also for the patch. Moving to Triaged, as it's not something Growth can work on in Structured Discussions given our current priorities.

gerritbot added a comment.Feb 1 2023, 9:26 PM

Change 885882 had a related patch set uploaded (by C. Scott Ananian; author: Subramanya Sastry):

[mediawiki/services/parsoid@master] SectionMetadata: Rename byteOffset to cpOffset to reflect reality

https://gerrit.wikimedia.org/r/885882

cscott mentioned this in T328605: Export TOCData in the action API.Feb 1 2023, 9:44 PM
cscott added a subtask: T328605: Export TOCData in the action API.
cscott subscribed.

Should be fixed when T328605 is; the field is being properly (re)named in the new SectionMetadata objects which represent the section data internally.

gerritbot added a comment.Feb 2 2023, 3:44 PM

Change 886077 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/extensions/Flow@master] Fix import of non-ASCII sections

https://gerrit.wikimedia.org/r/886077

gerritbot added a comment.Feb 10 2023, 7:56 PM

Change 886077 merged by jenkins-bot:

[mediawiki/extensions/Flow@master] Fix import of non-ASCII sections

https://gerrit.wikimedia.org/r/886077

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.23; 2023-02-13).Feb 10 2023, 8:00 PM
gerritbot added a comment.Feb 10 2023, 10:39 PM

Change 885882 merged by jenkins-bot:

[mediawiki/services/parsoid@master] SectionMetadata: Rename byteOffset to codepointOffset to reflect reality

https://gerrit.wikimedia.org/r/885882

gerritbot added a comment.Feb 13 2023, 8:21 PM

Change 888790 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.17.0-a15

https://gerrit.wikimedia.org/r/888790

gerritbot added a comment.Feb 13 2023, 9:18 PM

Change 888790 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.17.0-a15

https://gerrit.wikimedia.org/r/888790

cscott mentioned this in T330232: Export TOCData via action API.Feb 21 2023, 8:09 PM
gerritbot added a comment.Feb 23 2023, 4:00 AM

Change 837254 merged by jenkins-bot:

[mediawiki/core@master] Parser: Section offsets are in codepoints, not in bytes

https://gerrit.wikimedia.org/r/837254

Maintenance_bot removed a project: Patch-For-Review.Feb 23 2023, 4:10 AM
ReleaseTaggerBot edited projects, added MW-1.40-notes (1.40.0-wmf.25; 2023-02-27); removed MW-1.40-notes (1.40.0-wmf.23; 2023-02-13).Feb 23 2023, 11:00 AM
gerritbot added a comment.Feb 23 2023, 7:00 PM

Change 891594 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/services/parsoid@master] Report Parsoid section offsets in codepoints, not bytes

https://gerrit.wikimedia.org/r/891594

gerritbot added a project: Patch-For-Review.Feb 23 2023, 7:00 PM
gerritbot added a comment.Mar 7 2023, 8:36 PM

Change 891594 merged by jenkins-bot:

[mediawiki/services/parsoid@master] Report Parsoid section offsets in codepoints, not bytes

https://gerrit.wikimedia.org/r/891594

Maintenance_bot removed a project: Patch-For-Review.Mar 7 2023, 9:10 PM
gerritbot added a comment.Mar 10 2023, 6:23 PM

Change 896413 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@master] Bump parsoid to 0.17.0-a20 and zest-css to 3.0.0

https://gerrit.wikimedia.org/r/896413

gerritbot added a project: Patch-For-Review.Mar 10 2023, 6:23 PM
gerritbot added a comment.Mar 10 2023, 6:56 PM

Change 896413 merged by jenkins-bot:

[mediawiki/vendor@master] Bump parsoid to 0.17.0-a20 and zest-css to 3.0.0

https://gerrit.wikimedia.org/r/896413

Maintenance_bot removed a project: Patch-For-Review.Mar 10 2023, 7:11 PM
cscott mentioned this in T332243: NOTOC causing API action=parse prop=sections to return nothing.Mar 21 2023, 6:08 PM
Urbanecm_WMF removed a project: Growth-Team.Oct 28 2025, 12:27 AM
Maintenance_bot added a project: MW-Interfaces-Team.Oct 28 2025, 12:30 AM
Pppery removed a project: StructuredDiscussions.Oct 28 2025, 12:33 AM
BPirkle edited projects, added Content-Transform-Team; removed MW-Interfaces-Team.Oct 28 2025, 4:58 PM
BPirkle subscribed.

Older issue, not sure if it is still relevant. Looks more like Content Transform than MediaWiki Interfaces, so tagging them and untagging us. Please retag if I'm wrong about that.

cscott renamed this task from byteoffset field in action=parse results is misnamed to Deprecate prop=sections in action=parse.Oct 30 2025, 2:22 PM
cscott updated the task description. (Show Details)
cscott removed a project: MediaWiki-Parser.
cscott moved this task from Backlog to Later on the Content-Transform-Team board.
cscott moved this task from Unsorted to In Dev on the MediaWiki-Action-API board.
gerritbot added a comment.Nov 7 2025, 6:45 PM

Change #1203071 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@master] ApiParse: deprecate prop=sections

https://gerrit.wikimedia.org/r/1203071

gerritbot added a project: Patch-For-Review.Nov 7 2025, 6:45 PM
gerritbot added a comment.Nov 7 2025, 9:52 PM

Change #1203150 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/core@master] ApiParse: remove deprecated 'prop=sections' from default properties

https://gerrit.wikimedia.org/r/1203150

gerritbot added a comment.Nov 7 2025, 10:41 PM

Change #1203071 merged by jenkins-bot:

[mediawiki/core@master] ApiParse: deprecate prop=sections

https://gerrit.wikimedia.org/r/1203071

ReleaseTaggerBot added a project: MW-1.46-notes (1.46.0-wmf.2; 2025-11-12).Nov 7 2025, 11:00 PM
Nirmos subscribed.Nov 18 2025, 11:37 PM

Noting it here because it took some time for me to see it: toclevel is tocLevel in the new API.

Anomie subscribed.Nov 22 2025, 7:58 PM

You forgot to announce this to mediawiki-api-announce.

cscott closed subtask T328605: Export TOCData in the action API as Resolved.Nov 25 2025, 4:03 AM
santhosh mentioned this in T423981: Replace deprecated prop=sections to prop=tocdata.Tue, Apr 21, 4:37 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits