How is this going? Need any help?

@ovasileva any progress here...?

Chatted today with @ovasileva I advise a few things:

• don't assume a user will be bucketed correctly. If a user exhausts localstorage on their browser (very easy in mediawiki) they will be re-bucketed on subsequent page views

Look at unique session ids in each bucket and compare to total amount to get a sense of click through rate in each bucket.
e.g.

• events in feature-enabled bucket with unique user token / all events in feature-enabled bucket
• events in feature-disabled bucket with unique user token / all events in feature-disabled bucket

@Jdlrobson Could you also check and if necessary correct the description at https://meta.wikimedia.org/wiki/Schema_talk:RelatedArticles#Sampling_method_and_rate ?

Done.

@Jdlrobson Thanks! Just to be extra sure, is this sentence true too: "On each new page load, we decide randomly whether to record events during that pageview (1% probability) or not (99%)" - or is that instead decided once per session (when the new value for userSessionToken is generated)?

The first statement is not true. All sampling is per session.

I.e. If a user is bucketed for logging events and also bucketed with related pages disabled this should stay true until they clear their browser cache.

OK, I have corrected this too at https://meta.wikimedia.org/wiki/Schema_talk:RelatedArticles#Sampling_method_and_rate .

now here: https://www.mediawiki.org/wiki/Reading/Web/Projects/Related_pages

Note for the record: If I understand @phuedx correctly (T175918#3663909 ), the underlying data (from Schema:RelatedArticles) is affected by {T175918]}, i.e. all session-based metrics will be invalid.