Page MenuHomePhabricator

Tons of garbage logged in EditAttemptStep save_failure_type and save_failure_message
Closed, ResolvedPublic

Description

The EditAttemptStep schema's save_failure_type and save_failure_message fields contain lots of useless data.

Some editing interfaces actually log the error message shown to the user in save_failure_message. This text is localised, so it's difficult to compare the data between different wikis, or estimate how often errors occur globally. Also, it can contain personally identifiable information (e.g. your IP address could be shown in the error message for when you're blocked), forcing us to be very careful when sharing the result of any queries on that data.

Some other editing interfaces instead log the name of the error message in save_failure_message, which is only slightly better, because many of the messages are defined by the abuse filter settings, and also differ between different wikis.

save_failure_type is supposed to group some error types (e.g. 'extensionAbuseFilter' for all abuse filter messages), but it doesn't work correctly in many cases, partially because of the above and partially because the format of the errors has changed since this code was written and they're not detected right.


I think the original idea was for save_failure_type to cover all possible errors, in which case it would be okay for save_failure_message to contain this excessively-detailed data for debugging, but it doesn't even come close, and new and exciting error types are continually being invented.

All editing interfaces should follow what we did with DiscussionTools (and which is actually useful for analysis like T270346#6702699) and log API error codes in the save_failure_message field.

Event Timeline

Just for reference, here's the kind of stuff we have, in some of the most common errors:

select event.save_failure_type, substr(event.save_failure_message, 1, 50) as save_failure_message, event.integration, event.platform, event.editor_interface, count(*) as count
from editattemptstep
where event.action='saveFailure'
group by event.save_failure_type, substr(event.save_failure_message, 1, 50), event.integration, event.platform, event.editor_interface
order by count desc
save_failure_typesave_failure_messageintegrationplatformeditor_interfacecount(Notes)
(1)responseUnknownpagephonevisualeditor66678Why is this empty?
(2)extensionCaptchahookabortedpagedesktopwikitext30752
(3)responseUnknownunicode-support-failpagedesktopwikitext30588
(4)extensionCaptchaNULLpagedesktopvisualeditor14642Same error as (2), but different message
(5)extensionCaptchapagephonewikitext11480Same error as (2), but different message
(6)extensionAbuseFilterNULLpagedesktopvisualeditor9319
(7)responseUnknownmissingsummarypagedesktopwikitext6749
(8)editConflictNULLpagedesktopwikitext6703
(9)extensionAbuseFilterabusefilter-disallowedpagedesktopwikitext6503Same error as (6), but different message
(10)responseUnknownspamprotectionmatchpagedesktopwikitext4570
(11)editConflictError, another user has edited this page.pagephonevisualeditor3426
(12)responseUnknownedit-conflictpagedesktopwikitext2426Same error as (11), but different message – not marked as editConflict
(13)responseUnknownpagephonewikitext2381Why is this empty?
(14)editConflictError, another user has edited this page.pagephonewikitext2240Same error as (11), but different message
(15)editConflictErro, esta página foi editada por outro utilizadorpagephonewikitext2132Same error as (11), but different message
(16)responseUnknownblankarticlepagedesktopwikitext1949
(17)editConflictErro, esta página foi editada por outro utilizadorpagephonevisualeditor1593Same error as (11), but different message
(18)responseUnknownYou have been blocked from editing.pagephonevisualeditor1531
(19)editConflictNULLpagedesktopvisualeditor1471Same error as (11), but different message
(20)responseUnknownYou have been blocked from editing.pagephonewikitext1357
(21)extensionAbuseFilterEsta edición no se ha grabado porque ha sido identpagephonevisualeditor1356Same error as (6), but different message
(22)extensionCaptchaspam-blacklisted-linkpagedesktopwikitext1210
(23)responseUnknownactionthrottledtextpagedesktopwikitext1187
(24)responseUnknownabusefilter-disallowed-blankingpagedesktopwikitext1179Same error as (6), but different message – not marked as extensionAbuseFilter

Change 656498 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/extensions/DiscussionTools@master] Update 'save_failure_type' map for EditAttemptStep schema

https://gerrit.wikimedia.org/r/656498

Change 656499 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/extensions/VisualEditor@master] Update save failure logging for EditAttemptStep schema, simplify code

https://gerrit.wikimedia.org/r/656499

Change 656500 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/extensions/MobileFrontend@master] Update save failure logging for EditAttemptStep schema

https://gerrit.wikimedia.org/r/656500

Change 656501 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/extensions/WikiEditor@master] Update save failure logging for EditAttemptStep schema

https://gerrit.wikimedia.org/r/656501

(I'm not planning to do anything with this data right now, but I wanted to get this done while the DiscussionTools implementation is still fairly fresh in my mind, before I inevitably forget how it all works again.)

Change 656501 merged by jenkins-bot:
[mediawiki/extensions/WikiEditor@master] Update save failure logging for EditAttemptStep schema

https://gerrit.wikimedia.org/r/656501

Change 656499 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Update save failure logging for EditAttemptStep schema, simplify code

https://gerrit.wikimedia.org/r/656499

Change 656498 merged by jenkins-bot:
[mediawiki/extensions/DiscussionTools@master] Update 'save_failure_type' map for EditAttemptStep schema

https://gerrit.wikimedia.org/r/656498

Change 656500 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Update save failure logging for EditAttemptStep schema

https://gerrit.wikimedia.org/r/656500

Nice spot, @matmarex. A resulting question:

  • Is it best practice to reflect the changes you implemented to log API error codes in EditAttemptStep's save_failure_message field [i] in the documentation on Meta as well [i]?

cc @MNeisler


i. https://meta.wikimedia.org/wiki/Schema:EditAttemptStep

Nice spot, @matmarex. A resulting question:

  • Is it best practice to reflect the changes you implemented to log API error codes in EditAttemptStep's save_failure_message field [i] in the documentation on Meta as well [i]?

@matmarex confirmed the changes in this task affect the potential values delivered through the save_failure_messagefield. These "values" are not currently documented within EditAttemptStep. As such, we are assuming no changes are needed to the documentation page considering what was changed in this patch is not currently represented on the schema's meta page.

cc @MNeisler