Large parts of EditAttemptStep logging code are duplicated in VisualEditor, DiscussionTools, MobileFrontend, and WikiEditor. As part of the IP masking work we would need to update this code in each of the extensions. This might be a good opportunity to centralize it.
I like how the Readers-Web team's logging for DesktopWebUIActionsTracking works, we should do ours similarly.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| In Progress | Niharika | T324492 Temporary accounts - MVP | |||
| Open | None | T326816 Update features for IP Masking | |||
| Open | None | T326876 Update Editing Team-owned products that may be affected by temporary users | |||
| Resolved | matmarex | T332432 Update DiscussionTools for IP masking | |||
| Resolved | matmarex | T332433 Update page editing features in MobileFrontend for IP masking | |||
| Resolved | matmarex | T332435 Update VisualEditor for IP masking | |||
| Resolved | matmarex | T332436 Update WikiEditor for IP masking | |||
| Resolved | matmarex | T332437 Change EditAttemptStep and other schemas and logging to support temp users | |||
| Resolved | matmarex | T332438 Centralize EditAttemptStep logging code in WikimediaEvents |
Change 900754 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):
[mediawiki/extensions/WikimediaEvents@master] Centralize EditAttemptStep logging code in WikimediaEvents
Change 900755 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):
[mediawiki/extensions/DiscussionTools@master] Centralize EditAttemptStep logging code in WikimediaEvents
Change 900757 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):
[mediawiki/extensions/MobileFrontend@master] Centralize EditAttemptStep logging code in WikimediaEvents
Change 900759 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):
[mediawiki/extensions/VisualEditor@master] Centralize EditAttemptStep logging code in WikimediaEvents
Change 900760 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):
[mediawiki/extensions/WikiEditor@master] Centralize EditAttemptStep logging code in WikimediaEvents
You can set an oversample flag via URL or a dedicated hook (see WikimediaEventsHooks::shouldSchemaEditAttemptStepOversample()). When it is set, the event is always recorded and an is_oversampled field is set in the event data. GrowthExperiments always sets that flag for suggested edits because there aren't that many of them so sampling would make the statistical power of the data very weak. We only analyze these events (the session ID can also be overridden by a similar mechanism and we use that to connect these events to Special:Homepage events that initiate the edits, and only care about events where the ID matches). When others do generic analysis of editors, they can discard the events with an oversample flag (so they don't skew the stats).
Does that help?
(cc @nettrom_WMF for awareness)
One thing to maybe be mindful of that with all the JS-config-var-based overriding that happens, module load order might be significant and the refactoring might change that. For GrowthExperiments that's not an issue, all overriding is initiated on the server side.
Thanks for the heads up! I'll definitely take a look at the patches.
Regarding the sampling rate, I'm fairly confident that it can be is 100% (see T312016: Increase EditAttemptStep sampling rate(s) to 100%). I understand not wanting to change that while making other changes though.
Edit
I misremembered the state of that task. The sampling rate for EAS is 100% on all wikis.
Change 900760 merged by jenkins-bot:
[mediawiki/extensions/WikiEditor@master] Centralize EditAttemptStep logging code in WikimediaEvents
Change 900755 merged by jenkins-bot:
[mediawiki/extensions/DiscussionTools@master] Centralize EditAttemptStep logging code in WikimediaEvents
Change 900759 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Centralize EditAttemptStep logging code in WikimediaEvents
Change 900757 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Centralize EditAttemptStep logging code in WikimediaEvents
Change 900754 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Centralize EditAttemptStep logging code in WikimediaEvents
It's probably worth QAing this to the level of making an edit in WikiEditor/VisualEditor/MobileFrontend/DiscussionTools and verifying that there's not any obvious JS errors. If you want to be more thorough and verify the types of events that're being logged then you can, but that'd be a fair bit more work. (If you do, then speculating on what's most likely to go wrong I would focus on the new way we're adding editingStatsId to API requests, and changes to how we're regenerating the session ID for editing sessions after the first one on a given pageload.)
Oh, also, @MNeisler: hopefully this entire patchset should make no visible different whatsoever to the EditAttemptStep/VisualEditorFeatureUse data you're analyzing. However, if the levels of anything suddenly change during this train deployment, then this is almost certainly the cause...
I think we could close this now. We discovered some minor issues in production and we're working on them (T237063#8763888, T334157). Anyone, feel free to resolve if you agree.