Page MenuHomePhabricator

Ensure the A/B test bucket is logged when a logged out user edits a full page in wikitext
Closed, ResolvedPublic

Description

In an attempt to rectify the the bucketing imbalances identified in T291308, we will use this ticket to improve tracking for logged-out users who're doing full-page wikitext editing.

Minimum test case

This can be tested in the normal ways of watching for EditAttemptStep events -- just verify when doing full-page wikitext editing on a talk page (WikiEditor and NWE, logged in and out) on an abtest wiki that the bucket and (if logged out) anonymous_user_token fields are present on the event.

Event Timeline

Change 761997 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Improve DiscussionTools a/b test bucket logging

https://gerrit.wikimedia.org/r/761997

Change 761998 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/WikiEditor@master] Improve DiscussionTools a/b test bucket logging

https://gerrit.wikimedia.org/r/761998

Change 761997 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Improve DiscussionTools a/b test bucket logging

https://gerrit.wikimedia.org/r/761997

Change 761998 merged by jenkins-bot:

[mediawiki/extensions/WikiEditor@master] Improve DiscussionTools a/b test bucket logging

https://gerrit.wikimedia.org/r/761998

Change 762055 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/WikiEditor@master] Misnamed field on a/b test logging

https://gerrit.wikimedia.org/r/762055

Change 762056 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Misnamed field on a/b test logging

https://gerrit.wikimedia.org/r/762056

Oh hey, I think I worked out how we could have gotten those events with a bucket but without an anonymous_user_token. Also, purely coincidentally, I feel somewhat sheepish about flubbing a field-name.

For future reference: this can be tested in the normal ways of watching for EditAttemptStep events -- just verify when doing full page wikitext editing on a talk page (WikiEditor and NWE, logged in and out) on an abtest wiki that the bucket and (if logged out) anonymous_user_token fields are present on the event.

Change 762055 merged by jenkins-bot:

[mediawiki/extensions/WikiEditor@master] Misnamed field on a/b test logging

https://gerrit.wikimedia.org/r/762055

Change 762056 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Misnamed field on a/b test logging

https://gerrit.wikimedia.org/r/762056

@DLynch: can you please add the steps QA will need to verify whether this patch is working as expected to the task description?

Ryasmeen subscribed.

EditAttemptStep events are getting logged for both logged in and logged out users when doing full page editing on a talk page using WikiEditor and NWE.

ppelberg added a subscriber: DLynch.

Next steps

  • @MNeisler to verify that logged out users are being bucketed as expected
MNeisler edited projects, added Product-Analytics (Kanban); removed Product-Analytics.

I reviewed the aggregate data logged in editattemptstep and verfied we are now recording an anonymous_user_token for all logged-out users included in the AB test, including full page wikitext edit events.

Logged out users also appear to be bucketed as expected, with the split between the test and control groups much more balanced.

I believe we can now resolve this task as AB test data appears as expected.

@ppelberg - Reassigning to you for sign-off.

Notebook