Wherever the saveSuccess event is missing, there should instead be a saveFailure event, which should contain the error details. We should analyze those. I don't remember right now where to access the data.

OK, I figured out where I can access it. For future reference: https://superset.wikimedia.org/superset/sqllab → presto_analytics_hive → event.

Then I ran this query:

SELECT event.save_failure_type, event.save_failure_message, count(*) as count
from editattemptstep
where event.integration='discussiontools' and event.action='saveFailure'
group by event.save_failure_type, event.save_failure_message
order by count desc

Here's the result, with notes from me:

save_failure_type	save_failure_message	count			Notes
responseUnknown	blocked	20			User is blocked. Looks like we show the reply tool interface to blocked users, and only fail when they try posting (and the message lacks details). We should improve this.
responseUnknown	abusefilter-warning	15			Warning from an abuse filter. As this is only a warning, the user should be able to click "Reply" again to post their comment.
extensionCaptcha	captcha	15			CAPTCHA (heh, I expected this to be the top one). Usually triggered by trying to post an external link in the comment. The user should be able to post their comment after filling out the CAPTCHA.
responseUnknown	discussiontools-commentid-notfound	14			"Could not find the comment you're replying to on the page." The comment could have been deleted while the user was writing their reply (or it could be a bug on our side).
extensionAbuseFilter	abusefilter-disallowed	11			Disallowed by abuse filter.
responseUnknown	apierror-visualeditor-docserver-http	8			Internal error where we couldn't contact Parsoid.
responseUnknown	http	7			Usually means that the user's network connection got disconnected (and, if this entry arrived in our database, that means it re-connected later and they were hopefully able to retry and post their comment).
responseUnknown	wikimedia-globalblocking-ipblocked-range	2			User's IP address is globally blocked.
responseUnknown	assertanonfailed	2			The user started replying while logged out, then logged in in another browser tab, and we show them a message to confirm they know which username they're posting under. The user should be able to click "Reply" again to post their comment.
extensionSpamBlacklist	spamblacklist	2			Disallowed by spam blacklist.
responseUnknown	latestnotfound	1			Internal error where the latest revision of the page they tried posting on doesn't exist? Pretty weird, maybe that page got deleted. I'd expect that to give a different error message that says so directly, though.

(That's the total number of times each type of error happened, not the number of sessions or users who experienced it. It might be just one person who was blocked and clicked the button 20 times before they gave up. I don't want to spend too much time on this, but it should be possible to do this better or merge it with the report somehow.)

(These are all the details we have. We don't log the actual error message the user saw, or further details like which abuse filter was triggered or which comment they were replying to. The schema EditAttemptStep doesn't have fields for anything else, they could be added but it would take a bit of work.)

In T270346#6702699, @matmarex wrote:

OK, I figured out where I can access it. For future reference: https://superset.wikimedia.org/superset/sqllab → presto_analytics_hive → event.

This is great, @matmarex.

Then I ran this query:

SELECT event.save_failure_type, event.save_failure_message, count(*) as count
from editattemptstep
where event.integration='discussiontools' and event.action='saveFailure'
group by event.save_failure_type, event.save_failure_message
order by count desc

• @matmarex: To confirm: the query above searches all edits that have ever been made with discussiontools, correct?

Note: here is a permalink to this query for us to reference in the future: https://superset.wikimedia.org/superset/sqllab?savedQueryId=178.

Here's the result, with notes from me:

save_failure_type	save_failure_message	count			Notes
responseUnknown	blocked	20			User is blocked. Looks like we show the reply tool interface to blocked users, and only fail when they try posting (and the message lacks details). We should improve this.

See T270803.

responseUnknown	abusefilter-warning	15			Warning from an abuse filter. As this is only a warning, the user should be able to click "Reply" again to post their comment.

This doesn't seem too concerning to me considering I expect people who are new to trigger abuse filters more often than people who have more editing experience.

extensionCaptcha	captcha	15			CAPTCHA (heh, I expected this to be the top one). Usually triggered by trying to post an external link in the comment. The user should be able to post their comment after filling out the CAPTCHA.

Same as above RE newer people.

responseUnknown	discussiontools-commentid-notfound	14			"Could not find the comment you're replying to on the page." The comment could have been deleted while the user was writing their reply (or it could be a bug on our side).

• @matmarex: does this need a ticket? If so, I can file. Note: I'm not too concerned about this considering how infrequently this seems to be happening.

extensionAbuseFilter	abusefilter-disallowed	11			Disallowed by abuse filter.

Same as above RE newer people.

responseUnknown	latestnotfound	1			Internal error where the latest revision of the page they tried posting on doesn't exist? Pretty weird, maybe that page got deleted. I'd expect that to give a different error message that says so directly, though.

• @matmarex: What is the error message that is currently shown?

(That's the total number of times each type of error happened, not the number of sessions or users who experienced it. It might be just one person who was blocked and clicked the button 20 times before they gave up. I don't want to spend too much time on this, but it should be possible to do this better or merge it with the report somehow.)

Noted. This is a great starting place. I've created a "saveFailure incidences" sheet within the existing DiscussionTools usage report workbook; I'll re-run the query very two weeks for the next month or so and see if we see much movement in these numbers.

(These are all the details we have. We don't log the actual error message the user saw, or further details like which abuse filter was triggered or which comment they were replying to. The schema EditAttemptStep doesn't have fields for anything else, they could be added but it would take a bit of work.)

Understood. I don't think this is necessary just yet.

In T270346#6711298, @ppelberg wrote:

• @matmarex: To confirm: the query above searches all edits that have ever been made with discussiontools, correct?

No, it looks like it searches the last 90 days (running select min(dt) from editattemptstep returns '2020-10-13' right now). Older data is probably deleted permanently (per https://meta.wikimedia.org/wiki/Data_retention_guidelines#How_long_do_we_retain_non-public_data?).

responseUnknown	discussiontools-commentid-notfound	14			"Could not find the comment you're replying to on the page." The comment could have been deleted while the user was writing their reply (or it could be a bug on our side).

• @matmarex: does this need a ticket? If so, I can file. Note: I'm not too concerned about this considering how infrequently this seems to be happening.

No, comments being deleted is normal, although uncommon.

responseUnknown	latestnotfound	1			Internal error where the latest revision of the page they tried posting on doesn't exist? Pretty weird, maybe that page got deleted. I'd expect that to give a different error message that says so directly, though.

• @matmarex: What is the error message that is currently shown?

"Could not find latest revision for title"

I tried now and I can consistently reproduce this by opening the reply widget, deleting the talk page (or moving it without creating a redirect), then trying to save a reply.

In T270346#6737970, @matmarex wrote:

In T270346#6711298, @ppelberg wrote:

• @matmarex: To confirm: the query above searches all edits that have ever been made with discussiontools, correct?

No, it looks like it searches the last 90 days (running select min(dt) from editattemptstep returns '2020-10-13' right now). Older data is probably deleted permanently (per https://meta.wikimedia.org/wiki/Data_retention_guidelines#How_long_do_we_retain_non-public_data?).

Got it.

responseUnknown	discussiontools-commentid-notfound	14			"Could not find the comment you're replying to on the page." The comment could have been deleted while the user was writing their reply (or it could be a bug on our side).

• @matmarex: does this need a ticket? If so, I can file. Note: I'm not too concerned about this considering how infrequently this seems to be happening.

No, comments being deleted is normal, although uncommon.

Understood.

responseUnknown	latestnotfound	1			Internal error where the latest revision of the page they tried posting on doesn't exist? Pretty weird, maybe that page got deleted. I'd expect that to give a different error message that says so directly, though.

• @matmarex: What is the error message that is currently shown?

"Could not find latest revision for title"

I tried now and I can consistently reproduce this by opening the reply widget, deleting the talk page (or moving it without creating a redirect), then trying to save a reply.

I see.

• And to be sure: title in the message above would be the title of the page the person is attempting to save a comment to, not the word "title," right?

• And to be sure: title in the message above would be the title of the page the person is attempting to save a comment to, not the word "title," right?

No, that is literally the error message.

In T270346#6756209, @matmarex wrote:

• And to be sure: title in the message above would be the title of the page the person is attempting to save a comment to, not the word "title," right?

No, that is literally the error message.

Understood.

Next steps

• @ppelberg create new ticket for copy change and associate this to-be-created ticket with T241403.

In T270346#6763273, @ppelberg wrote:

create new ticket for copy change and associate this to-be-created ticket with T241403.

I filed T273068.