Background
As part of https://phabricator.wikimedia.org/T387150, @jwang found that there was far too many sessions with tick events compared to sessions with click search events. Tick events are supposed to only start after a user first clicks on the searchbar.
User story
- As a data analyst, I need session events to only show for users that receive the treatment or control search experience
Requirements
- Sessions in the Search AB test with session tick events should always be associated with click events too
Acceptance criteria
- We switch from storing sessionTick and lastSessionTime in localStorage to sessionStorage
- We remove any existing items from localStorage using a call to mw.storage.remove that is documented. This should be called outside the mediawiki.web_AB_test_enrollment hook to make sure it's removed wherever found.
- The change to fix this is backported.
- We update the experiment name to include "v2.0" so we can distinguish cached events from new events in our data validation. This should be done in the config repo and MUST include the string RelatedArticles test in the name. The config change should happen after the change has been backported.
QA
We will perform data QA to confirm (or not confirm) whether the volume is now meeting expectations.
We confirm on super set that our data volume is expected.
-- Count unique sessions that have tick events SELECT mediawiki."database", count(distinct performer.session_id) AS sessions, count(1) AS events FROM event.product_metrics_web_base_search_ab_test_session_ticks WHERE year=2025 and month=3 and day=3 AND NOT performer.is_logged_in GROUP BY mediawiki."database", element_at(experiments.assigned, 'RelatedArticles test experiment v2') , day -- Count unique sessions that have clicked on the search bar SELECT mediawiki."database", element_at(experiments.assigned, 'RelatedArticles test experiment v2') AS assigned_group , day, count(distinct performer.session_id) AS sessions, count(1) AS events FROM event.product_metrics_web_base_search_ab_test_clicks WHERE year=2025 and month=3 AND day=3 AND action='click' AND action_source='search_box' GROUP BY mediawiki."database", element_at(experiments.assigned, 'RelatedArticles test experiment v2') , day
Communication criteria - does this need an announcement or discussion?
- Add communication criteria
Sign off
- Make sure there is a task for removing the localStorage cleanup e.g. "Removing the instrumentation altogether" or removing https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/1123448/8/modules/ext.wikimediaEvents/sessionLength/mixin.js#L175
