You can do this now with external tools such as hypothes.is.

@Pginer-WMF: Is more work planned / needed here or was this task about an isolated Hackathon project that could be closed?
If more work is planned / needed, which project should be associated to this task? Thanks!

In T173011#3958246, @Aklapper wrote:

@Pginer-WMF: Is more work planned / needed here or was this task about an isolated Hackathon project that could be closed?
If more work is planned / needed, which project should be associated to this task? Thanks!

@SBisson did some work in the past hackathon. I'm not sure if there are future plans to keep exploring the idea. If this were to become part of a project, I guess it would be StructuredDiscussions, so I'll add the tag if it helps for the classification.

There has been a lot of discussion on implementation recently in T185607: Provide an inline discussion feature, "DiscussThis".

The mock-up illustration creates a very over-optimistic impression of how well this would work in reality. The sample comments do not resemble real world content.

I used the random article button and sampled 70 actual comments. (I excluded an additional 15 bot messages, almost all were over 1000 characters.) Of the human comments, only 10% were as short as the comments in the sample image. About half were at least 300 characters, and 10% were between 700 and 2800 characters. The sample also included instances of images and list-formatting.

Trying to view real world discussion-content in such a narrow sidebar would be painful. Any formatting or reply-threading is essentially impossible. A single URL can wrap across four or five lines. A single comment may well require a vertical scroll bar - and adding a scroll bar would squeeze the available width even more severely.

My sample didn't come across any reflists, tables, galleries, multi-level lists, templates etc... but those things do sometimes pop up in discussions. Anything like that would fail badly such a narrow sidebar.

In T173011#3958246, @Aklapper wrote:

@Pginer-WMF: Is more work planned / needed here or was this task about an isolated Hackathon project that could be closed?
If more work is planned / needed, which project should be associated to this task? Thanks!

This is more of a recurring dream than a specific project at this point. No one is actively working on it but many people, including C Scott and JD Forrester (whom I've just subscribed) have been interested over the years. I don't know of any specific tag for it, but I'm glad to see Growth team's tag here, because this type of discussion is often cited as something that would make the editing process more transparent for new users.

In T173011#4425826, @Alsee wrote:

The mock-up illustration creates a very over-optimistic impression of how well this would work in reality. The sample comments do not resemble real world content.

I used the random article button and sampled 70 actual comments. (I excluded an additional 15 bot messages, almost all were over 1000 characters.) Of the human comments, only 10% were as short as the comments in the sample image.

Alsee, I think the issues you raise can be handled through a combination of means/features. E.g.:

1. My belief from discussions with Pau—though I don't see it reflected in these sketches—is that these in-line comments would also be reflected on the talk page. I.e., they are an alternate view, not a replacement. Which leads to my next comment...
2. The use-case for this type of comment is that a user wants to remark about a specific line, phrase or idea in the article. But, as you know better than I, discussions on talk pages cover a range of subjects, some directly related to the actual text and others more generally about an approach or process, etc.. So my thinking would be that linking the comment to a particular word or phrase would be, again, an option not a replacement.
3. The notes shown in the sketches are missing a key element familiar to any Google docs Comments user: the "More" link you click to see the full extent of longer comments. Google shows the first X characters and then truncates behind that More link. In our case, one might imagine the More link either showing the full text in place or, when the text is quite long, simply popping up a new tab open to the right spot in the talk page.

I think this type of commenting could do a few things things. One is that it would enable more direct discussion of the actual writing and phrasing. I.e., if inline commenting does not support long arguments well, as you say, perhaps the converse is also true. Maybe it will encourage a more fine-grained commenting that is now not well supported by talk pages. If users can make short, in-context questions and suggestions, pinging each other for their opinions on the changes, it might lead to a more collaborative working style and overall better prose.

I think it also could provide what I believe some will find a more lively entry point into discussions. If someone reading an article could click a button to see what others think about the specifics in it, that might be very engaging—a very different experience from reading a discussion on a separate page, disconnected from the content.

This last point does invite the question of whether that increased participation might have down sides, possibly in the form of encouraging more editorializing from readers about the article's subject ("that guy is such a loser!" etc.). But that is the sort of thing that we'd expect to flush out during design-research and testing.