In an attempt to identify the underlying causes for the bucketing imbalances identified in T291308, we will us this ticket to investigate what init type is associated with the sessions where a ready event is being emitted, but an anonymous user token is not being logged.
Description
Description
| Status | Subtype | Assigned | Task | |
|---|---|---|---|---|
| · · · | ||||
| Resolved | mpopov | T277825 Run A/B test to evaluate impact of New Topic Tool | ||
| Resolved | Spike | MNeisler | T301495 [SPIKE] Investigate what init type is associated with the sessions where a ready event is being emitted, but an anonymous user token is not being logged | |
| · · · |
Event Timeline
Comment Actions
Here's are the session init_types logged for ready events in the AB test where the anonymous user token was not assigned. These instances occur for both init_types but the majority in both test groups occur where the session init_type = page. Also, as I mentioned in my comment on T291308#7693635, the anonymous_user_token is only missing for bucketed ready events where event.integration = 'page' and editor_interface = 'wikitext'.
| bucket | integration | anonymous_user_token | editor_interface | init_type | num_ready_events |
|---|---|---|---|---|---|
| control | page | NULL | wikitext | page | 1255 |
| control | page | NULL | wikitext | section | 108 |
| test | page | NULL | wikitext | page | 1228 |
| test | page | NULL | wikitext | section | 81 |
Let me know if any other data would be helpful. Here's the query I used to collect the above data for reference:
WITH init_events AS ( SELECT event.editing_session_id as edit_attempt_id, event.init_type as init_type, wiki As init_wiki FROM event.editattemptstep WHERE -- following deployment of patch YEAR = 2022 AND month = 02 AND day >= 03 -- inits events on desktop AND event.platform = 'desktop' AND event.action = 'init' -- only talk pages AND event.page_ns % 2 = 1 --by anon not bot users AND useragent.is_bot = false AND event.user_id = 0 AND event.user_class = 'IP' --- test wikis AND wiki IN ('amwiki', 'bnwiki', 'zhwiki', 'nlwiki', 'arzwiki', 'frwiki', 'hewiki', 'hiwiki', 'idwiki', 'itwiki', 'jawiki', 'kowiki', 'omwiki', 'fawiki', 'plwiki', 'ptwiki', 'eswiki', 'thwiki', 'ukwiki', 'viwiki') ) SELECT event.editing_session_id as edit_attempt_id, event.bucket AS bucket, wiki As wiki, event.integration AS integration, init_events.init_type AS init_type, event.anonymous_user_token, event.editor_interface AS editor_interface FROM event.editattemptstep eas INNER JOIN init_events ON eas.event.editing_session_id = init_events.edit_attempt_id AND eas.wiki = init_events.init_wiki WHERE YEAR = 2022 AND month = 02 AND day >= 03 -- look at only desktop ready events AND event.platform = 'desktop' AND event.action = 'ready' --by anon not bot users AND useragent.is_bot = false AND event.user_id = 0 AND event.user_class = 'IP' -- only talk page events AND event.page_ns % 2 = 1 -- bucketed events only AND event.bucket in ('test', 'control') -- test wikis AND wiki IN ('amwiki', 'bnwiki', 'zhwiki', 'nlwiki', 'arzwiki', 'frwiki', 'hewiki', 'hiwiki', 'idwiki', 'itwiki', 'jawiki', 'kowiki', 'omwiki', 'fawiki', 'plwiki', 'ptwiki', 'eswiki', 'thwiki', 'ukwiki', 'viwiki')
Comment Actions
Resolving this task as the init_type results were shared and the patch to fix the underlying issue has been created in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/762056/