Update Talk API to support in-line replying on Android
Closed, ResolvedPublic
Actions

Description

Deployment Timing

Target deployment date: Wednesday, 23 March 2022 via 1.39.0-wmf.4.

Functionality being requested

In the rest_v1/page/talk/, add comment ids (source: T273292#6851354) to each reply item.

The Beta shows that each reply URL contains the information below:

data-mw-comment='{"type":"comment","level":3,"id":"c-RYasmeen_(WMF)-2022-01-04T16:45:00.000Z-RYasmeen_(WMF)-2022-01-04T16:40:00.000Z","replies":["c-73.48.132.36-2022-02-08T18:31:00.000Z-RYasmeen_(WMF)-2022-01-04T16:45:00.000Z"],"timestamp":"2022-01-04T16:45:00.000Z","author":"RYasmeen (WMF)"}'

which the id may be used for the inline reply API.

The commentid comes from the API below:
https://en.wikipedia.beta.wmflabs.org/w/api.php?action=discussiontoolspageinfo&format=json&formatversion=2&uselang=en&page=Talk:Cats&oldid=536711

and the commentid should be added to the page/talk API.

"id": 4,
"replies": [
	{
	  "sha": "7b4e356696ed4bf3b59afabe0af49d872dfdb5c2350121081f7d38a0ebd3ff45",
	  "depth": 0,
          "commentid": "c-RYasmeen_(WMF)-2022-01-04T16:45:00.000Z-RYasmeen_(WMF)-2022-01-04T16:40:00.000Z"
	  "html": "Comment 1 <a href=\"./User:Cooltey\" title=\"User:Cooltey\">Cooltey</a> (<a href=\"./User_talk:Cooltey#top\" title=\"User talk:Cooltey\">talk</a>) 23:10, 21 June 2021 (UTC)"
	},
	{
	  "sha": "17199edbe8f832e63a860cb0cd1fdb16744d3731e64fc8ee873f6625902bcbc3",
	  "depth": 0,
          "commentid": "c-RYasmeen_(WMF)-2022-01-04T16:45:00.000Z-RYasmeen_(WMF)-2022-01-04T16:40:00.000Z"
	  "html": "Comment 2<a href=\"./User:Cooltey\" title=\"User:Cooltey\">Cooltey</a> (<a href=\"./User_talk:Cooltey#top\" title=\"User talk:Cooltey\">talk</a>) 23:13, 21 June 2021 (UTC)"
	}
]

Question
Not sure but should the /page/talk/ API contains other data like "replies":["c-68.195.105.122-2020-05-13T16:40:00.000Z-213.245.48.203-2020-04-08T20:26:00.000Z","c-173.76.103.59-2021-02-17T17:27:00.000Z-213.245.48.203-2020-04-08T20:26:00.000Z"],"timestamp":"2020-04-08T20:26:00.000Z"?

Reference
When submitting replies, we can use API:
action=discussiontoolsedit and use the id as commentname

Discussion tool
https://github.com/wikimedia/mediawiki-extensions-DiscussionTools/blob/ffcbce4f2dc69c26c77919cc6b581a131e7c52a3/includes/ApiDiscussionToolsEdit.php#L107

Details

	Project	Branch	Lines +/-	Subject
	mediawiki/extensions/DiscussionTools	master	+417 -414	Add item name to the JSON output of HeadingItem
	mediawiki/extensions/DiscussionTools	master	+64 -30	Talk API for apps

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T290186 [EPIC] Empower developers to create new and improve existing experiences using DiscussionTools
		Resolved		• MattCleinman	T285971 Update Talk API to support in-line replying on Android

Event Timeline

ppelberg created this task.Jul 1 2021, 5:49 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 1 2021, 5:49 PM

ppelberg mentioned this in T273292: Create in line replies .Jul 1 2021, 5:50 PM

cooltey updated the task description. (Show Details)Jul 1 2021, 11:25 PM

BPirkle moved this task from Inbox to Tracking/Watching on the Platform Engineering board.Jul 13 2021, 9:14 PM

(I will discuss what's documented in the task with the Editing Engineering Team to see the extent to which they might be uniquely positioned to take on this task. After that conversation, I will follow up with @sdkim to talk about whether Editing or Platform takes on this work).

cooltey removed cooltey as the assignee of this task.Jul 30 2021, 6:06 PM

I feel that updating the /page/talk/ API with those ids would be a bit painful, since it's all defined off outside of the general mediawiki PHP/JS ecosystem. I'm sure we could call in the JS version of the DT comment parser there somehow, but we'd probably need to do all sorts of refactoring-it-as-a-npm-package or similar first.

Also, it's a separate implementation of parsing a comment thread, so I don't think we could guarantee that our interpretations of comment boundaries would line up.

Much-easier would be exposing our own comment model via the DiscussionTools api. We're clearly doing it partially already via the transcludedfrom call, and just returning the full threads doesn't seem unreasonable.

Timing
Based on what @sdkim shared on 6 August 2021, it sounds like @JTannerWMF and the Android Team will need support for in-line replying by the end of this fiscal year (June, 2022). As such, I'm moving this to the backlog for now.

ppelberg moved this task from Backlog to Triaged on the DiscussionTools board.Aug 11 2021, 12:37 AM

ppelberg moved this task from Triaged to New Features on the DiscussionTools board.

ppelberg moved this task from Untriaged to This Fiscal Year on the Editing-team board.

Change 711505 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/DiscussionTools@master] Talk API: a simple approach

https://gerrit.wikimedia.org/r/711505

gerritbot added a project: Patch-For-Review.Aug 11 2021, 2:19 PM

That patch is the most-naïve possible version to us providing our own comment structure. It's just dumping the flat comment list from CommentParser, resulting in something like:

{
    "discussiontools": [
        {
            "type": "heading",
            "level": 0,
            "id": "h-hi!-2020-12-16T21:35:00.000Z",
            "replies": [
                "c-Matma_Rex-2020-12-16T21:35:00.000Z-hi!"
            ],
            "headingLevel": 2,
            "placeholderHeading": false,
            "html": "<span id=\"hi.21\" typeof=\"mw:FallbackId\" data-parsoid='{\"dsr\":[2559,2559,null,null]}'></span>hi!"
        },
        {
            "type": "comment",
            "level": 1,
            "id": "c-Matma_Rex-2020-12-16T21:35:00.000Z-hi!",
            "replies": [],
            "timestamp": "2020-12-16T21:35:00.000Z",
            "author": "Matma Rex",
            "html": "hello <a rel=\"mw:WikiLink\" href=\"./User:Matma_Rex\" title=\"User:Matma Rex\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:Matma_Rex\"},\"sa\":{\"href\":\"User:Matma Rex\"},\"dsr\":[2574,2602,17,2]}'>Matma Rex</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:Matma_Rex\" title=\"User talk:Matma Rex\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:Matma_Rex\"},\"sa\":{\"href\":\"User talk:Matma Rex\"},\"dsr\":[2604,2663,22,2]}'><font color=\"green\" data-parsoid='{\"stx\":\"html\",\"dsr\":[2626,2661,20,7]}'><i data-parsoid='{\"dsr\":[2646,2654,2,2]}'>talk</i></font></a>) 21:35, 16 December 2020 (UTC)"
        },

This does have one major difference compared to the page/talk interpretation of a page, because that version does seem to include non-talk content. I looked at Talk:The Fighting Temeraire which has a notice box above the talk sections, and it's represented as a talk-topic. Our implementation would entirely skip that.

matmarex added a subscriber: matmarex.Aug 11 2021, 3:27 PM

LGoto added projects: Wikipedia-Android-App-Backlog, Wikipedia-iOS-App-Backlog.Aug 13 2021, 3:45 PM

iOS is also going to be implementing this, so it's a shared priority for both android and iOS.

LGoto moved this task from Needs Triage to Tracking on the Wikipedia-Android-App-Backlog board.Aug 16 2021, 4:12 PM

LGoto moved this task from Needs Triage to Tracking on the Wikipedia-iOS-App-Backlog board.Aug 16 2021, 6:42 PM

ppelberg mentioned this in T290186: [EPIC] Empower developers to create new and improve existing experiences using DiscussionTools.Sep 1 2021, 5:46 PM

ppelberg added a parent task: T290186: [EPIC] Empower developers to create new and improve existing experiences using DiscussionTools.

matmarex mentioned this in T295648: Provide API that exposes DiscussionTools data structure.Nov 15 2021, 6:39 PM

Esanders mentioned this in T296018: Android team cannot associate a talk page notification with a specific section anymore.Nov 19 2021, 3:04 PM

ppelberg edited projects, added Editing-team (Kanban Board); removed Editing-team.Feb 7 2022, 9:23 PM

ppelberg moved this task from Incoming to Ready to Be Worked On on the Editing-team (Kanban Board) board.

cooltey updated the task description. (Show Details)Feb 8 2022, 7:02 PM

ppelberg mentioned this in T300316: <Product Initiative> Structured talk APIs.Feb 8 2022, 8:26 PM

While making these changes, two other tweaks (that I think should be fairly easy):

Could the response include a timestamp field for each comment, now that the information is available?
Could the response include a field for the username of who wrote each comment?

@MattCleinman if you'll note my earlier comment with an example it does contain these things. 😁

@DLynch Thank you!!!! That looks absolutely amazing. And apologies for not reading the ticket carefully enough.

In T285971#7275581, @DLynch wrote:

This does have one major difference compared to the page/talk interpretation of a page, because that version does seem to include non-talk content. I looked at Talk:The Fighting Temeraire which has a notice box above the talk sections, and it's represented as a talk-topic. Our implementation would entirely skip that.

It's worth noting that mobile web also hides the notices boxes: https://en.m.wikipedia.org/wiki/Talk:The_Fighting_Temeraire

In T285971#7698333, @MattCleinman wrote:

While making these changes, two other tweaks (that I think should be fairly easy):

Could the response include a timestamp field for each comment, now that the information is available?

Could the response include a field for the username of who wrote each comment?

Will we be able to use this API for some of our other features to see user names and dates then?

In T285971#7699252, @Esanders wrote:

In T285971#7275581, @DLynch wrote:

This does have one major difference compared to the page/talk interpretation of a page, because that version does seem to include non-talk content. I looked at Talk:The Fighting Temeraire which has a notice box above the talk sections, and it's represented as a talk-topic. Our implementation would entirely skip that.

It's worth noting that mobile web also hides the notices boxes: https://en.m.wikipedia.org/wiki/Talk:The_Fighting_Temeraire

Hey @Dbrant I think we display notice boxes as the top most post right ?

@Esanders may be worth checking out our design to see how we hope to display article talk pages

Talk pages redesign2.png (1×720 px, 149 KB)

Also we are super excited yall are working on this for us thank you so much! I will leave you in peace to code now :) @ppelberg @Esanders @DLynch

Will we be able to use this API for some of our other features to see user names and dates then?

If you mean the usernames/dates in the screenshot you just posted, then yes. We are planning something similar on desktop (see past experiment: https://patchdemo.wmflabs.org/wikis/aed6d63dcecc14ee69aa3c7405c32c4c/w/index.php/Talk:Main_Page )

• MattCleinman mentioned this in T301504: Change over to new talk page API.Feb 10 2022, 7:19 PM

Hi y'all, thanks for working on this and for asking for feedback! I'm very excited about inline replies and using a unified implementation. Here are my notes based on the structure in this comment:

The structure overall looks good to me. The bigger pains in my experience with Swift JSON parsing is heterogeneous arrays (missing fields are fine, but I'm talking dramatically different objects) and any keys that are dynamic (I think some MediaWiki endpoints use page ids as some of its keys). I don't see anything like that here.
Are you using that data-parsoid embedded json for anything? We can strip that out on our side, but I just wanted to make sure we wouldn't need to dig into that and parse it.
Replies aren't nested like they are in our current endpoint, but that shouldn't be a problem, we just might need to have an extra step connecting the relationship between the two.
Will the placement order of the replies/comments be based on the order they are in within the outer array, or the order the ids are listed within the replies array? Or will either one work?
One note on stripping infoboxes - just wanted to mention that when building page/talk, we noticed cases of user talk pages that had whole threads going before the first official topic/section was listed. So we added that in the beginning as its own sort of intro topic with empty topic text. I can't find many good examples of this now but here are a couple - this one has archive links and this one has replies. I'm not sure how common this is and if it's worth supporting, but just wanted to point out that this extra handling in page/talk isn't just limited to info boxes.

ppelberg assigned this task to DLynch.Feb 14 2022, 6:30 PM

ppelberg moved this task from Ready to Be Worked On to Blocked / Needs More Work on the Editing-team (Kanban Board) board.

The structure overall looks good to me. The bigger pains in my experience with Swift JSON parsing is heterogeneous arrays (missing fields are fine, but I'm talking dramatically different objects) and any keys that are dynamic (I think some MediaWiki endpoints use page ids as some of its keys). I don't see anything like that here.

I think this only happens if you are using formatversion=1 (which is still the default). formatversion=2 fixes this.

Are you using that data-parsoid embedded json for anything? We can strip that out on our side, but I just wanted to make sure we wouldn't need to dig into that and parse it.

You will not need that unless you are sending the old HTML back to Parsoid from the client. If you are using our APIs to add comments all your HTML will be fresh, and we re-fetch the Parsoid HTML and build the edit on the server side.

Replies aren't nested like they are in our current endpoint, but that shouldn't be a problem, we just might need to have an extra step connecting the relationship between the two.

The point of this patch was to closely mimic the existing API to make the switchover easy for you, but we can just as easily output the nested data structure.

Will the placement order of the replies/comments be based on the order they are in within the outer array, or the order the ids are listed within the replies array? Or will either one work?

A reply will be inserted one level in from the parent comment, and after all other comments in the reply tree beneath the parent comment. This is the convention on almost all talk pages.

One note on stripping infoboxes - just wanted to mention that when building page/talk, we noticed cases of user talk pages that had whole threads going before the first official topic/section was listed. So we added that in the beginning as its own sort of intro topic with empty topic text. I can't find many good examples of this now but here are a couple - this one has archive links and this one has replies. I'm not sure how common this is and if it's worth supporting, but just wanted to point out that this extra handling in page/talk isn't just limited to info boxes.

Those are picked up by the comment parser and placed under a placeholder heading (which has no title). They should be available in the API.

The point of this patch was to closely mimic the existing API to make the switchover easy for you, but we can just as easily output the nested data structure.

@Esanders To be fair, this patch is slightly flatter than the rest API. That outputs an object for each topic which contains a replies key that holds a (flat) array with objects that contain depth key... whereas this patch is completely flat and the entire heading/reply structure is implied by the level keys.

The bigger pains in my experience with Swift JSON parsing is heterogeneous arrays (missing fields are fine, but I'm talking dramatically different objects) and any keys that are dynamic (I think some MediaWiki endpoints use page ids as some of its keys). I don't see anything like that here.

@Tsevener It's minor but present -- there's a type field in the objects which can be heading or comment, which contains slightly different data. They're both representations of PHP-side subclasses of a main ThreadItem class, mind, so they're fundamentally similar.

@Tsevener / @MattCleinman I think if you wanted to give us some example of a hypothetical ideal API response that contains all the data you're going to need in a form that's maximally easy for you to parse, we could make whatever you want happen. (Or tell if you if something in there isn't going to be possible, I guess.)

If your position is just "we can work with whatever" and you don't really care, that's also fine of course. 😁

@Esanders @DLynch thanks for the clarifications! So far I'm still not concerned, but I'll check with @MattCleinman today to see if he has any final thoughts. If it's not too much trouble, can you give us a sample output of what this more complicated talk page structure might look like, as the patch is now?

[[Archive1]]
== Test section 1 title text ==

Test initial text. [[User:User2|User2]] ([[User talk:User2|talk]]) 17:58, 2 June 2013 (UTC)
:Test reply 1 text. [[User:User1|User1]] ([[User talk:User1|talk]]) 18:00, 2 June 2013 (UTC)
::Test reply 2 text. [[User:User2|User2]] ([[User talk: User2|talk]]) 18:02, 2 June 2013 (UTC)
:Test reply 3 text. [[User:User3|User3]] ([[User talk: User3|talk]]) 17:59, 2 June 2013 (UTC)

== Test section 2 title text ==

Test initial text. [[User:User4|User4]] ([[User talk: User4 |talk]]) 20:00, 2 June 2013 (UTC)
:Test reply 1 text. [[User:User1|User1]] ([[User talk:User1|talk]]) 20:01 2 June 2013 (UTC)

@Tsevener Sure! Here's exactly what'd be output from a page containing just that wikitext, in the current state of the patch:

{
    "discussiontoolspageinfo": {
        "threaditemshtml": [
            {
                "type": "heading",
                "level": 0,
                "id": "h-Test_section_1_title_text-2013-06-02T17:58:00.000Z",
                "replies": [
                    "c-User2-2013-06-02T17:58:00.000Z-Test_section_1_title_text"
                ],
                "headingLevel": 2,
                "html": "Test section 1 title text"
            },
            {
                "type": "comment",
                "level": 1,
                "id": "c-User2-2013-06-02T17:58:00.000Z-Test_section_1_title_text",
                "replies": [
                    "c-User1-2013-06-02T18:00:00.000Z-User2-2013-06-02T17:58:00.000Z",
                    "c-User3-2013-06-02T17:59:00.000Z-User2-2013-06-02T17:58:00.000Z"
                ],
                "timestamp": "2013-06-02T17:58:00.000Z",
                "author": "User2",
                "html": "Test initial text. <a rel=\"mw:WikiLink\" href=\"./User:User2\" title=\"User:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User2\"},\"sa\":{\"href\":\"User:User2\"},\"dsr\":[65,85,13,2]}'>User2</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User2\" title=\"User talk:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User2\"},\"sa\":{\"href\":\"User talk:User2\"},\"dsr\":[87,111,18,2]}'>talk</a>) 17:58, 2 June 2013 (UTC)"
            },
            {
                "type": "comment",
                "level": 2,
                "id": "c-User1-2013-06-02T18:00:00.000Z-User2-2013-06-02T17:58:00.000Z",
                "replies": [
                    "c-User2-2013-06-02T18:02:00.000Z-User1-2013-06-02T18:00:00.000Z"
                ],
                "timestamp": "2013-06-02T18:00:00.000Z",
                "author": "User1",
                "html": "Test reply 1 text. <a rel=\"mw:WikiLink\" href=\"./User:User1\" title=\"User:User1\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User1\"},\"sa\":{\"href\":\"User:User1\"},\"dsr\":[158,178,13,2]}'>User1</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User1\" title=\"User talk:User1\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User1\"},\"sa\":{\"href\":\"User talk:User1\"},\"dsr\":[180,204,18,2]}'>talk</a>) 18:00, 2 June 2013 (UTC)"
            },
            {
                "type": "comment",
                "level": 3,
                "id": "c-User2-2013-06-02T18:02:00.000Z-User1-2013-06-02T18:00:00.000Z",
                "replies": [],
                "timestamp": "2013-06-02T18:02:00.000Z",
                "author": "User2",
                "html": "Test reply 2 text. <a rel=\"mw:WikiLink\" href=\"./User:User2\" title=\"User:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User2\"},\"sa\":{\"href\":\"User:User2\"},\"dsr\":[252,272,13,2]}'>User2</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User2\" title=\"User talk:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User2\"},\"sa\":{\"href\":\"User talk: User2\"},\"dsr\":[274,299,19,2]}'>talk</a>) 18:02, 2 June 2013 (UTC)"
            },
            {
                "type": "comment",
                "level": 2,
                "id": "c-User3-2013-06-02T17:59:00.000Z-User2-2013-06-02T17:58:00.000Z",
                "replies": [],
                "timestamp": "2013-06-02T17:59:00.000Z",
                "author": "User3",
                "html": "Test reply 3 text. <a rel=\"mw:WikiLink\" href=\"./User:User3\" title=\"User:User3\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User3\"},\"sa\":{\"href\":\"User:User3\"},\"dsr\":[346,366,13,2]}'>User3</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User3\" title=\"User talk:User3\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User3\"},\"sa\":{\"href\":\"User talk: User3\"},\"dsr\":[368,393,19,2]}'>talk</a>) 17:59, 2 June 2013 (UTC)"
            },
            {
                "type": "heading",
                "level": 0,
                "id": "h-Test_section_2_title_text-2013-06-02T20:00:00.000Z",
                "replies": [
                    "c-User4-2013-06-02T20:00:00.000Z-Test_section_2_title_text"
                ],
                "headingLevel": 2,
                "html": "Test section 2 title text"
            },
            {
                "type": "comment",
                "level": 1,
                "id": "c-User4-2013-06-02T20:00:00.000Z-Test_section_2_title_text",
                "replies": [],
                "timestamp": "2013-06-02T20:00:00.000Z",
                "author": "User4",
                "html": "Test initial text. <a rel=\"mw:WikiLink\" href=\"./User:User4\" title=\"User:User4\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User4\"},\"sa\":{\"href\":\"User:User4\"},\"dsr\":[473,493,13,2]}'>User4</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User4\" title=\"User talk:User4\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User4\"},\"sa\":{\"href\":\"User talk: User4 \"},\"dsr\":[495,521,20,2]}'>talk</a>) 20:00, 2 June 2013 (UTC)"
            }
        ]
    }
}

@DLynch awesome thanks, that's super helpful. I discussed this with @MattCleinman and @Dbrant and we decided that we would actually prefer it to be fully nested, if that's possible, to save us a tree-building step on our side. To be clear, this would be more nested than our current page/talk implementation which is flat replies nested within topics. Below is an example of what we're thinking. If that's doesn't work for y'all, the flat structure as you have it now will be fine too. Thanks!

{
    "discussiontoolspageinfo": {
        "threaditemshtml": [
            {
                "type": "heading",
                "level": 0,
                "id": "h-Test_section_1_title_text-2013-06-02T17:58:00.000Z",
                "headingLevel": 2,
                "html": "Test section 1 title text",
                "replies": [
                    {
                        "type": "comment",
                        "level": 1,
                        "id": "c-User2-2013-06-02T17:58:00.000Z-Test_section_1_title_text",
                        "timestamp": "2013-06-02T17:58:00.000Z",
                        "author": "User2",
                        "html": "Test initial text. <a rel=\"mw:WikiLink\" href=\"./User:User2\" title=\"User:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User2\"},\"sa\":{\"href\":\"User:User2\"},\"dsr\":[65,85,13,2]}'>User2</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User2\" title=\"User talk:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User2\"},\"sa\":{\"href\":\"User talk:User2\"},\"dsr\":[87,111,18,2]}'>talk</a>) 17:58, 2 June 2013 (UTC)",
                        "replies": [
                            {
                                "type": "comment",
                                "level": 2,
                                "id": "c-User1-2013-06-02T18:00:00.000Z-User2-2013-06-02T17:58:00.000Z",
                                "timestamp": "2013-06-02T18:00:00.000Z",
                                "author": "User1",
                                "html": "Test reply 1 text. <a rel=\"mw:WikiLink\" href=\"./User:User1\" title=\"User:User1\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User1\"},\"sa\":{\"href\":\"User:User1\"},\"dsr\":[158,178,13,2]}'>User1</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User1\" title=\"User talk:User1\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User1\"},\"sa\":{\"href\":\"User talk:User1\"},\"dsr\":[180,204,18,2]}'>talk</a>) 18:00, 2 June 2013 (UTC)",
                                "replies": [
                                    {
                                        "type": "comment",
                                        "level": 3,
                                        "id": "c-User2-2013-06-02T18:02:00.000Z-User1-2013-06-02T18:00:00.000Z",
                                        "replies": [],
                                        "timestamp": "2013-06-02T18:02:00.000Z",
                                        "author": "User2",
                                        "html": "Test reply 2 text. <a rel=\"mw:WikiLink\" href=\"./User:User2\" title=\"User:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User2\"},\"sa\":{\"href\":\"User:User2\"},\"dsr\":[252,272,13,2]}'>User2</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User2\" title=\"User talk:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User2\"},\"sa\":{\"href\":\"User talk: User2\"},\"dsr\":[274,299,19,2]}'>talk</a>) 18:02, 2 June 2013 (UTC)"
                                    },
                                ]
                            },
                            {
                                "type": "comment",
                                "level": 2,
                                "id": "c-User3-2013-06-02T17:59:00.000Z-User2-2013-06-02T17:58:00.000Z",
                                "timestamp": "2013-06-02T17:59:00.000Z",
                                "author": "User3",
                                "html": "Test reply 3 text. <a rel=\"mw:WikiLink\" href=\"./User:User3\" title=\"User:User3\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User3\"},\"sa\":{\"href\":\"User:User3\"},\"dsr\":[346,366,13,2]}'>User3</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User3\" title=\"User talk:User3\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User3\"},\"sa\":{\"href\":\"User talk: User3\"},\"dsr\":[368,393,19,2]}'>talk</a>) 17:59, 2 June 2013 (UTC)",
                                    "replies": []
                            },
                        ]
                    },
                ]
            },{
                "type": "heading",
                "level": 0,
                "id": "h-Test_section_2_title_text-2013-06-02T20:00:00.000Z",
                "headingLevel": 2,
                "html": "Test section 2 title text",
                "replies": [
                    {
                        "type": "comment",
                        "level": 1,
                        "id": "c-User4-2013-06-02T20:00:00.000Z-Test_section_2_title_text",
                        "timestamp": "2013-06-02T20:00:00.000Z",
                        "author": "User4",
                        "html": "Test initial text. <a rel=\"mw:WikiLink\" href=\"./User:User4\" title=\"User:User4\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User4\"},\"sa\":{\"href\":\"User:User4\"},\"dsr\":[473,493,13,2]}'>User4</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User4\" title=\"User talk:User4\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User4\"},\"sa\":{\"href\":\"User talk: User4 \"},\"dsr\":[495,521,20,2]}'>talk</a>) 20:00, 2 June 2013 (UTC)",
                            "replies": []
                    }
                ]
            }
        ]
    }
}

There's a new version of the patch for more-nested output: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DiscussionTools/+/762955

Direct from the API sandbox we get this, which I think should be what you want:

    "discussiontoolspageinfo": {
        "threaditemshtml": [
            {
                "type": "heading",
                "level": 0,
                "id": "h-Test_section_1_title_text-2013-06-02T17:58:00.000Z",
                "html": "Test section 1 title text",
                "headingLevel": 2,
                "placeholderHeading": false,
                "replies": [
                    {
                        "type": "comment",
                        "level": 1,
                        "id": "c-User2-2013-06-02T17:58:00.000Z-Test_section_1_title_text",
                        "replies": [
                            {
                                "type": "comment",
                                "level": 2,
                                "id": "c-User1-2013-06-02T18:00:00.000Z-User2-2013-06-02T17:58:00.000Z",
                                "replies": [
                                    {
                                        "type": "comment",
                                        "level": 3,
                                        "id": "c-User2-2013-06-02T18:02:00.000Z-User1-2013-06-02T18:00:00.000Z",
                                        "replies": [],
                                        "html": "Test reply 2 text. <a rel=\"mw:WikiLink\" href=\"./User:User2\" title=\"User:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User2\"},\"sa\":{\"href\":\"User:User2\"},\"dsr\":[252,272,13,2]}'>User2</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User2\" title=\"User talk:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User2\"},\"sa\":{\"href\":\"User talk: User2\"},\"dsr\":[274,299,19,2]}'>talk</a>) 18:02, 2 June 2013 (UTC)",
                                        "timestamp": "2013-06-02T18:02:00.000Z",
                                        "author": "User2"
                                    }
                                ],
                                "html": "Test reply 1 text. <a rel=\"mw:WikiLink\" href=\"./User:User1\" title=\"User:User1\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User1\"},\"sa\":{\"href\":\"User:User1\"},\"dsr\":[158,178,13,2]}'>User1</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User1\" title=\"User talk:User1\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User1\"},\"sa\":{\"href\":\"User talk:User1\"},\"dsr\":[180,204,18,2]}'>talk</a>) 18:00, 2 June 2013 (UTC)",
                                "timestamp": "2013-06-02T18:00:00.000Z",
                                "author": "User1"
                            },
                            {
                                "type": "comment",
                                "level": 2,
                                "id": "c-User3-2013-06-02T17:59:00.000Z-User2-2013-06-02T17:58:00.000Z",
                                "replies": [],
                                "html": "Test reply 3 text. <a rel=\"mw:WikiLink\" href=\"./User:User3\" title=\"User:User3\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User3\"},\"sa\":{\"href\":\"User:User3\"},\"dsr\":[346,366,13,2]}'>User3</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User3\" title=\"User talk:User3\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User3\"},\"sa\":{\"href\":\"User talk: User3\"},\"dsr\":[368,393,19,2]}'>talk</a>) 17:59, 2 June 2013 (UTC)",
                                "timestamp": "2013-06-02T17:59:00.000Z",
                                "author": "User3"
                            }
                        ],
                        "html": "Test initial text. <a rel=\"mw:WikiLink\" href=\"./User:User2\" title=\"User:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User2\"},\"sa\":{\"href\":\"User:User2\"},\"dsr\":[65,85,13,2]}'>User2</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User2\" title=\"User talk:User2\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User2\"},\"sa\":{\"href\":\"User talk:User2\"},\"dsr\":[87,111,18,2]}'>talk</a>) 17:58, 2 June 2013 (UTC)",
                        "timestamp": "2013-06-02T17:58:00.000Z",
                        "author": "User2"
                    }
                ]
            },
            {
                "type": "heading",
                "level": 0,
                "id": "h-Test_section_2_title_text-2013-06-02T20:00:00.000Z",
                "html": "Test section 2 title text",
                "headingLevel": 2,
                "placeholderHeading": false,
                "replies": [
                    {
                        "type": "comment",
                        "level": 1,
                        "id": "c-User4-2013-06-02T20:00:00.000Z-Test_section_2_title_text",
                        "replies": [],
                        "html": "Test initial text. <a rel=\"mw:WikiLink\" href=\"./User:User4\" title=\"User:User4\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:User4\"},\"sa\":{\"href\":\"User:User4\"},\"dsr\":[473,493,13,2]}'>User4</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:User4\" title=\"User talk:User4\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:User4\"},\"sa\":{\"href\":\"User talk: User4 \"},\"dsr\":[495,521,20,2]}'>talk</a>) 20:00, 2 June 2013 (UTC)",
                        "timestamp": "2013-06-02T20:00:00.000Z",
                        "author": "User4"
                    }
                ]
            }
        ]
    }
}

I think our current impression is that we're providing everything you want/need, and so we'll move ahead on getting the patch actually finalized and merged. If this is wrong, please let us know!

In T285971#7748159, @DLynch wrote:

I think our current impression is that we're providing everything you want/need, and so we'll move ahead on getting the patch actually finalized and merged. If this is wrong, please let us know!

@MattCleinman I'm boldly assigning this task over to you to confirm whether the updated API output David shared in T285971#7730299 will meet both the Android and iOS Teams' needs.

If there is someone who you think is better positioned to do the above, please let me know!

Sorry about the delay from my side - this looks good for iOS! A few very minor remaining questions, I wouldn't consider them must-haves:

Based on the answers here, it sounds like that [[Archive 1]] link in the beginning of my sample text will get returned in the json as well. Is that a correct assumption? What would an object look like whose placeholderHeading field is true?
I wonder if we should either change the replies type value to say reply or change the replies array key to comments, just to keep them consistent with each other.
Now that we have a more nested structure, do the level and headingLevel distinctions become redundant on the heading type objects? I can see the argument either way - it seems like one can represent indention and the other font size, but correct me if I'm wrong.

Based on the answers here, it sounds like that [[Archive 1]] link in the beginning of my sample text will get returned in the json as well. Is that a correct assumption?

I think Ed was wrong about that. What would be picked up, in current behavior, is an actual comment that was in the same place as that link. Because there's no comment there, nothing is included.

What would an object look like whose placeholderHeading field is true?

To demonstrate this I will stick a signature after your [[Archive1]] link:

{
    "type": "heading",
    "level": 0,
    "id": "h--2022-03-07T22:30:00.000Z",
    "html": "",
    "replies": [
        {
            "type": "comment",
            "level": 1,
            "id": "c-Admin-2022-03-07T22:30:00.000Z",
            "html": "<a rel=\"mw:WikiLink\" href=\"./Archive1\" title=\"Archive1\" class=\"new\" data-parsoid='{\"stx\":\"simple\",\"a\":{\"href\":\"./Archive1\"},\"sa\":{\"href\":\"Archive1\"},\"dsr\":[0,12,2,2]}'>Archive1</a><a rel=\"mw:WikiLink\" href=\"./User:Admin\" title=\"User:Admin\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User:Admin\"},\"sa\":{\"href\":\"User:Admin\"},\"dsr\":[12,32,13,2]}'>Admin</a> (<a rel=\"mw:WikiLink\" href=\"./User_talk:Admin\" title=\"User talk:Admin\" class=\"new\" data-parsoid='{\"stx\":\"piped\",\"a\":{\"href\":\"./User_talk:Admin\"},\"sa\":{\"href\":\"User talk:Admin\"},\"dsr\":[34,58,18,2]}'>talk</a>) 22:30, 7 March 2022 (UTC)",
            "replies": [],
            "timestamp": "2022-03-07T22:30:00.000Z",
            "author": "Admin"
        }
    ],
    "headingLevel": 99,
    "placeholderHeading": true
}

I wonder if we should either change the replies type value to say reply or change the replies array key to comments, just to keep them consistent with each other.

Technically the replies can be comments or headings, so keeping it generic seems wiser.

Now that we have a more nested structure, do the level and headingLevel distinctions become redundant on the heading type objects?

level is mostly in there because what you're getting currently is just our regular JSON serialization of a ThreadItem, and it was already included. I'm inclined to leave it in just because avoiding customizing the output is simpler.

headingLevel, though, should definitely stay, because it corresponds to the <h1/2/3/4/5/6> level from the source, and you'll probably need to reproduce that to avoid misleading page-rendering in the app.

To demonstrate this I will stick a signature after your [[Archive1]] link:

Got it - actually looking closer I had specifically asked about comments appearing before the first section getting picked up, which is what Ed answered. Good to know we'll still pick up signed replies here!

Technically the replies can be comments or headings, so keeping it generic seems wiser.

Ah, good to know, does this mean we'll need to be prepared for any replies array to contain both comment and header types? I'm not really sure what the UI would look like in that situation.

level is mostly in there because what you're getting currently is just our regular JSON serialization of a ThreadItem, and it was already included. I'm inclined to leave it in just because avoiding customizing the output is simpler.
headingLevel, though, should definitely stay, because it corresponds to the <h1/2/3/4/5/6> level from the source, and you'll probably need to reproduce that to avoid misleading page-rendering in the app.

Sounds good!

I'm not really sure what the UI would look like in that situation.

Sorry I probably should have dug more before blurting out this question - I found an example of how we handle it in this topic - looks like on iOS we just treat them like a top level talk page sections. But it makes sense to me that that interpretation lives in the client.

With that, that's the last of my questions on the iOS-side, this looks good to me. Thanks for the quick answers.

Just wanted to check in on what needs to happen next for this to hit the train?

JTannerWMF triaged this task as High priority.Mar 11 2022, 10:03 AM

@JTannerWMF We need to clean up the patch and merge it. There's also an issue with the data that we noticed (subheadings and their threads are being duplicated) that's a side-effect of a parser choice we made that we'd like to fix up, though that's not strictly essential to getting this API on the train because subheadings are rare.