Page MenuHomePhabricator

[SPIKE] Determine how we can identify blocked save attempts
Closed, ResolvedPublic

Description

As part of T277825, we will seek to learn whether the New Discussion Tool causes a greater percentage of Junior Contributors to publish the new discussions they start without a significant increase in disruption.

As part of learning the above, we will want to exclude edit attempts in which people have been blocked from publishing an edit. In order to make this exclusion, we will need a way to detect when this happens...thus this task.

Open question(s)

  • 1. Where and how – if at all – do we log when someone is not able to publish an edit because they have been blocked?

Done

  • All ===Open questions are answered and documented within this ticket

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

This does show up in the EditAttemptStep schema. There will be an action=saveFailure event, with a save_failure_type and a save_failure_message that can be investigated to see why the failure happened.

Type information about action=saveFailure
userBadToken: Save attempt failed because the user's token was bad
userNewUser: Save attempt failed because the user's session has changed user
extensionAbuseFilter: Save attempt failed because of the AbuseFilter extension
extensionCaptcha: Save attempt failed because of the CAPTCHA extensions
extensionSpamBlacklist: Save attempt failed because of the SpamBlacklist extension
responseEmpty: Save attempt failed but the response was empty
responseUnknown: Save attempt failed but the response was not recognised
editPageDeleted: Save attempt failed because the page was deleted
editConflict: Save attempt failed because of an edit conflict

Is that sufficient to work out the type of blocked-edits you want to not-count?

ppelberg added a subscriber: MNeisler.

Is that sufficient to work out the type of blocked-edits you want to not-count?

(Thank you, David – I'm assigning this task over to @MNeisler to answer the question you posed above.)

@DLynch - Would edits blocked due to IP blocks (such as, for belonging to one of the IP ranges that are being used for the new Apple Relay Service) be logged in EditAttemptStep as action=saveFailure event? If so, what save_failure_type enum would be logged?

I think that'd be a responseUnknown -- quick code reading suggests that the error would come back as one of apierror-blocked/apierror-autoblocked/apierror-blocked-partial, and the schema doesn't specifically know about those so it'd turn it into responseUnknown.

Thanks @DLynch. This should be sufficient to identify and exclude blocked edit attempts for the purposes of the New Discussion Tool AB test.

Quick note: A review of the data looks like responseUnknown is usually the most frequently logged save_failure_type followed by extensionAbuseFilter and extensionCaptcha. This seems expected if reponseUnknown includes IP blocks as well as any other save failures the schema doesn't know about.

@ppelberg - Reassinging to you for sign-off.