Background
Context why we need a search activity id:
"The count of clicks is not reliable. For the click-through rate, a user must have a maximum of 1 click per 1 impression. However, in past projects, I sometimes observed more clicks than impressions. In such cases, I need to cap the clicks at 1 per search_session_id.
It’s quite common for event logging to miss some events. For example, we might receive click events but not the corresponding init events. Or we might expect to receive the same number of events for two equivalent triggers, but that’s not the case. I have put two examples in our 1:1 doc. Same thing could happen for any events in a sequence like user’s search journey. With search_session_id , we can exclude each search journey which missed impression events, or cap the clicks if they seems to be overcounted.
Without search session id, we cannot further clean the data. It might lead to inconclusive analysis result."
According to the spec, we need the search activity id associated with every click tracking event. This task is for adding code to WikimediaEvents that will manage search activity id, and ensure the search activity id is sent out with the click tracking
We can pull some of this code from searchSatisfaction.js in WikimediaEvents (i.e. how to generate the unique id), but their definition of a "search session" is more complicated, and describes a more general journey of a user trying to find information and potentially including multiple searches. Our definition is much simpler.
This patch will likely require code in MobileFrontend and WikimediaEvents
Requirements
A search activity is started when the searchbox receives focus.
A search activity is ended when the user exits the searchOverlay OR when the user proceeds with a search query/search recommendation link.
- WikimediaEvents should generate a unique search activity id when the searchOverlay is opened. Note, this is different than ext.MobileFrontend.searchOverlay.empty.
- Closing the searchOverlay should reset the search activity id
- The search activity id should be send along with all the click tracking events
- The search activity id should be logged as funnel_entry_token in all events
Acceptance criteria
- Click tracking event for each search overlay should have a unique search activity id
- The search activity id should be logged as funnel_entry_token in all events
BDD
Feature: Implement search activity ID for Empty Search A/B test click tracking
Scenario: Generate unique search activity ID when search overlay opens
Given a user focuses on the search box
When the search overlay opens
Then a unique search activity ID should be generated
Scenario: Reset search activity ID when search overlay closes
Given a user has an active search activity ID
When the user closes the search overlay
Then the search activity ID should be reset
Scenario: Include search activity ID in click tracking events
Given a user interacts with search recommendations or submits a query
When a click tracking event is fired
Then the event should include the search activity ID as funnel_entry_token
Scenario: Maintain the same search activity ID during a single search session
Given a user performs multiple actions within an open search overlay
When click tracking events are fired
Then the same search activity ID should be included in all events until the overlay is closedTest Steps
Test Case 1: Verify unique search activity ID is generated when search overlay opens
- Open an incognito window and visit https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=T352930.
- Open the browser console and execute:
mw.loader.using('mediawiki.api').then(function () {
new mw.Api().saveOption('eventlogging-display-console', '1');
});- Click on the search box to open the search overlay.
- Observe the console logs for click tracking events.
- Verify that a unique search activity ID is generated and included as funnel_entry_token.
- AC1: A unique search activity ID is generated when the search overlay opens.
Test Case 2: Verify search activity ID is reset when search overlay closes
- Follow Test Case 1 to open the search overlay and generate an ID.
- Close the search overlay.
- Reopen the search overlay.
- Observe the console logs for new events.
- Verify that a new search activity ID is generated, different from the previous one.
- AC2: The search activity ID is reset when the search overlay is closed.
Test Case 3: Verify search activity ID is included in all click tracking events
- Open the search overlay as in Test Case 1.
- Interact with search recommendations (e.g., click on a suggestion).
- Submit a search query.
- Observe the console logs for click tracking events.
- Verify that each event includes the search activity ID as funnel_entry_token.
- AC3: The search activity ID is included in all click tracking events.
Test Case 4: Verify the same search activity ID is maintained within a single session
- Open the search overlay.
- Perform multiple actions without closing the overlay (e.g., type queries, click recommendations).
- Observe the console logs for click tracking events.
- Verify that all events share the same search activity ID.
- AC4: The same search activity ID is maintained during the session until the overlay is closed.
Communication criteria - does this need an announcement or discussion?
- no
QA Results - Beta
| AC | Status | Details |
|---|---|---|
| 1 | ✅ | T383936#10533386 |
| 2 | ✅ | T383936#10533386 |
| 3 | ✅ | T383936#10533386 |
| 4 | ✅ | T383936#10533386 |
