Page MenuHomePhabricator

Remove sampling rate artifacts for language switcher after enough data collected
Closed, DuplicatePublic0.5 Story Points

Description

As a product owner, I want to avoid logging events at a high sampling threshold for a long period, so that my queries run faster and we use less disk space.

This should be deployed once both the new language switcher button affordance and language switcher overlay have been in production for a couple weeks.

Event Timeline

dr0ptp4kt raised the priority of this task from to Needs Triage.
dr0ptp4kt updated the task description. (Show Details)
dr0ptp4kt added a subscriber: dr0ptp4kt.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 17 2016, 5:44 PM
dr0ptp4kt renamed this task from Reduce sampling rate for language switcher after enough days at higher sampling rate to Reduce sampling rate for language switcher after enough days of the A/B test at the higher sampling rate.Feb 17 2016, 5:44 PM
dr0ptp4kt triaged this task as Normal priority.
dr0ptp4kt set Security to None.
Jdlrobson moved this task from Backlog to Tasks on the MobileFrontend board.Feb 18 2016, 6:24 PM

Change 272724 had a related patch set uploaded (by Bmansurov):
Reduce sampling rate for language switcher

https://gerrit.wikimedia.org/r/272724

Jdlrobson added a subscriber: Jdlrobson.

Given there are issues with bucketing I suspect we'll need to bump this to the sprint after this one?

+1ed
Someone needs to SWAT this for it to be done.

As a formality does this need to go back to To Do so it can be SWAT'd? I'm okay with it being SWAT'd. At this point there's plenty of data.

Let's hold off on this piece right now. The data are presently inconclusive, and part of this has to do with the relatively lower number of events with version B, structured-overlay. I'll open a fresh task to make the A/B test head-to-head!

Overlay start of funnel


select left(t2.timestamp, 8) ts, t2.event_languageOverlayVersion, t1.event_languageButtonTappedBucket, count(*)
from
MobileWebLanguageSwitcher_15302503 t2
inner join
MobileWebLanguageSwitcher_15302503 t1 on t2.event_FunnelToken = t1.event_FunnelToken
inner join
MobileWebLanguageSwitcher_15302503 t0 on t1.event_FunnelToken = t0.event_FunnelToken
where
t2.timestamp > '20160227' and t2.timestamp < '20160304' and t2.event_event = 'languageListLoaded' and t2.event_MobileMode = 'stable'
and t1.timestamp > '20160227' and t1.timestamp < '20160304' and t1.event_event = 'languageButtonTap'
and t0.timestamp > '20160227' and t0.timestamp < '20160304' and t0.event_event = 'pageLoaded' and t0.event_beaconCapable = 1
group by ts, t2.event_languageOverlayVersion, t1.event_languageButtonTappedBucket;



ts	event_languageOverlayVersion	event_languageButtonTappedBucket	count(*)
20160227	simpler-overlay	1-4 taps	69
20160227	simpler-overlay	20+ taps	12
20160227	simpler-overlay	5-20 taps	41
20160227	simpler-overlay	unknown	145
20160227	structured-overlay	1-4 taps	9
20160227	structured-overlay	20+ taps	1
20160227	structured-overlay	5-20 taps	2
20160227	structured-overlay	unknown	20
20160228	simpler-overlay	1-4 taps	94
20160228	simpler-overlay	20+ taps	10
20160228	simpler-overlay	5-20 taps	26
20160228	simpler-overlay	unknown	162
20160228	structured-overlay	1-4 taps	7
20160228	structured-overlay	20+ taps	1
20160228	structured-overlay	5-20 taps	1
20160228	structured-overlay	unknown	22
20160229	simpler-overlay	1-4 taps	80
20160229	simpler-overlay	20+ taps	13
20160229	simpler-overlay	5-20 taps	32
20160229	simpler-overlay	unknown	166
20160229	structured-overlay	1-4 taps	5
20160229	structured-overlay	20+ taps	2
20160229	structured-overlay	5-20 taps	2
20160229	structured-overlay	unknown	9
20160301	simpler-overlay	0 taps	1
20160301	simpler-overlay	1-4 taps	84
20160301	simpler-overlay	20+ taps	8
20160301	simpler-overlay	5-20 taps	26
20160301	simpler-overlay	unknown	142
20160301	structured-overlay	1-4 taps	8
20160301	structured-overlay	20+ taps	5
20160301	structured-overlay	5-20 taps	5
20160301	structured-overlay	unknown	13
20160302	simpler-overlay	0 taps	3
20160302	simpler-overlay	1-4 taps	69
20160302	simpler-overlay	20+ taps	12
20160302	simpler-overlay	5-20 taps	27
20160302	simpler-overlay	unknown	137
20160302	structured-overlay	1-4 taps	7
20160302	structured-overlay	20+ taps	1
20160302	structured-overlay	5-20 taps	2
20160302	structured-overlay	unknown	11
20160303	simpler-overlay	0 taps	25
20160303	simpler-overlay	1-4 taps	73
20160303	simpler-overlay	20+ taps	7
20160303	simpler-overlay	5-20 taps	24
20160303	simpler-overlay	unknown	117
20160303	structured-overlay	0 taps	4
20160303	structured-overlay	1-4 taps	4
20160303	structured-overlay	20+ taps	1
20160303	structured-overlay	5-20 taps	7
20160303	structured-overlay	unknown	9




Completed funnels

select left(t3.timestamp, 8) ts, t2.event_languageOverlayVersion, t1.event_languageButtonTappedBucket, t3.event_exitModal, t3.event_searchInputHasQuery, count(*)
from MobileWebLanguageSwitcher_15302503 t3
inner join
MobileWebLanguageSwitcher_15302503 t2 on t3.event_FunnelToken = t2.event_FunnelToken
inner join
MobileWebLanguageSwitcher_15302503 t1 on t2.event_FunnelToken = t1.event_FunnelToken
inner join
MobileWebLanguageSwitcher_15302503 t0 on t1.event_FunnelToken = t0.event_FunnelToken
where
t3.timestamp > '20160227' and t3.timestamp < '20160304' and t3.event_event = 'exitModal' and t3.event_mobileMode = 'stable'
and t2.timestamp > '20160227' and t2.timestamp < '20160304' and t2.event_event = 'languageListLoaded'
and t1.timestamp > '20160227' and t1.timestamp < '20160304' and t1.event_event = 'languageButtonTap'
and t0.timestamp > '20160227' and t0.timestamp < '20160304' and t0.event_event = 'pageLoaded' and t0.event_beaconCapable = 1
group by ts, t2.event_languageOverlayVersion, t1.event_languageButtonTappedBucket, t3.event_exitModal, t3.event_searchInputHasQuery;





ts	event_languageOverlayVersion	event_languageButtonTappedBucket	event_exitModal	event_searchInputHasQuery	count(*)
20160227	simpler-overlay	1-4 taps	dismissed	0	1
20160227	simpler-overlay	1-4 taps	tapped-on-result	0	60
20160227	simpler-overlay	20+ taps	dismissed	0	1
20160227	simpler-overlay	20+ taps	tapped-on-result	0	6
20160227	simpler-overlay	5-20 taps	dismissed	0	1
20160227	simpler-overlay	5-20 taps	tapped-on-result	0	34
20160227	simpler-overlay	unknown	dismissed	0	1
20160227	simpler-overlay	unknown	dismissed	1	2
20160227	simpler-overlay	unknown	tapped-on-result	0	108
20160227	simpler-overlay	unknown	tapped-on-result	1	6
20160227	structured-overlay	1-4 taps	tapped-on-result	0	9
20160227	structured-overlay	20+ taps	tapped-on-result	0	1
20160227	structured-overlay	5-20 taps	tapped-on-result	0	1
20160227	structured-overlay	unknown	dismissed	0	1
20160227	structured-overlay	unknown	tapped-on-result	0	12
20160228	simpler-overlay	1-4 taps	dismissed	0	4
20160228	simpler-overlay	1-4 taps	tapped-on-result	0	82
20160228	simpler-overlay	20+ taps	tapped-on-result	0	12
20160228	simpler-overlay	5-20 taps	tapped-on-result	0	24
20160228	simpler-overlay	unknown	dismissed	0	11
20160228	simpler-overlay	unknown	dismissed	1	4
20160228	simpler-overlay	unknown	tapped-on-result	0	87
20160228	structured-overlay	1-4 taps	tapped-on-result	0	8
20160228	structured-overlay	20+ taps	tapped-on-result	0	1
20160228	structured-overlay	unknown	dismissed	0	1
20160228	structured-overlay	unknown	tapped-on-result	0	13
20160229	simpler-overlay	1-4 taps	dismissed	0	5
20160229	simpler-overlay	1-4 taps	dismissed	1	1
20160229	simpler-overlay	1-4 taps	tapped-on-result	0	64
20160229	simpler-overlay	20+ taps	tapped-on-result	0	8
20160229	simpler-overlay	5-20 taps	dismissed	0	1
20160229	simpler-overlay	5-20 taps	tapped-on-result	0	29
20160229	simpler-overlay	unknown	dismissed	0	9
20160229	simpler-overlay	unknown	dismissed	1	3
20160229	simpler-overlay	unknown	tapped-on-result	0	108
20160229	structured-overlay	1-4 taps	tapped-on-result	0	3
20160229	structured-overlay	20+ taps	tapped-on-result	0	2
20160229	structured-overlay	5-20 taps	tapped-on-result	0	2
20160229	structured-overlay	unknown	dismissed	1	1
20160229	structured-overlay	unknown	tapped-on-result	0	1
20160301	simpler-overlay	0 taps	tapped-on-result	0	6
20160301	simpler-overlay	1-4 taps	dismissed	0	3
20160301	simpler-overlay	1-4 taps	tapped-on-result	0	59
20160301	simpler-overlay	20+ taps	tapped-on-result	0	8
20160301	simpler-overlay	5-20 taps	dismissed	0	1
20160301	simpler-overlay	5-20 taps	tapped-on-result	0	18
20160301	simpler-overlay	unknown	dismissed	0	12
20160301	simpler-overlay	unknown	dismissed	1	3
20160301	simpler-overlay	unknown	tapped-on-result	0	84
20160301	simpler-overlay	unknown	tapped-on-result	1	3
20160301	structured-overlay	1-4 taps	dismissed	0	1
20160301	structured-overlay	1-4 taps	tapped-on-result	0	3
20160301	structured-overlay	20+ taps	tapped-on-result	0	3
20160301	structured-overlay	5-20 taps	tapped-on-result	0	4
20160301	structured-overlay	unknown	tapped-on-result	0	10
20160302	simpler-overlay	0 taps	tapped-on-result	0	3
20160302	simpler-overlay	1-4 taps	dismissed	0	1
20160302	simpler-overlay	1-4 taps	tapped-on-result	0	79
20160302	simpler-overlay	20+ taps	dismissed	0	1
20160302	simpler-overlay	20+ taps	tapped-on-result	0	8
20160302	simpler-overlay	5-20 taps	dismissed	0	3
20160302	simpler-overlay	5-20 taps	tapped-on-result	0	19
20160302	simpler-overlay	unknown	dismissed	0	8
20160302	simpler-overlay	unknown	dismissed	1	5
20160302	simpler-overlay	unknown	tapped-on-result	0	77
20160302	structured-overlay	1-4 taps	tapped-on-result	0	8
20160302	structured-overlay	5-20 taps	tapped-on-result	0	1
20160302	structured-overlay	unknown	tapped-on-result	0	3
20160303	simpler-overlay	0 taps	dismissed	0	2
20160303	simpler-overlay	0 taps	tapped-on-result	0	24
20160303	simpler-overlay	0 taps	tapped-on-result	1	1
20160303	simpler-overlay	1-4 taps	dismissed	0	3
20160303	simpler-overlay	1-4 taps	tapped-on-result	0	66
20160303	simpler-overlay	20+ taps	tapped-on-result	0	5
20160303	simpler-overlay	5-20 taps	dismissed	0	2
20160303	simpler-overlay	5-20 taps	tapped-on-result	0	23
20160303	simpler-overlay	unknown	dismissed	0	12
20160303	simpler-overlay	unknown	dismissed	1	3
20160303	simpler-overlay	unknown	tapped-on-result	0	75
20160303	structured-overlay	0 taps	dismissed	0	1
20160303	structured-overlay	0 taps	tapped-on-result	0	2
20160303	structured-overlay	1-4 taps	tapped-on-result	0	21
20160303	structured-overlay	20+ taps	tapped-on-result	0	1
20160303	structured-overlay	5-20 taps	tapped-on-result	0	3
20160303	structured-overlay	unknown	tapped-on-result	0	6
20160303	structured-overlay	unknown	tapped-on-result	1	1
dr0ptp4kt updated the task description. (Show Details)Mar 15 2016, 3:06 PM

@bmansurov I updated the description to indicate we should deploy this reduction in sampling after both the button and the modal have been running live in production for a couple weeks. No rush on this. It's in sprint 69, probably toward the end of the sprint, but could conceivably move to sprint 70. We'll see.

Jdlrobson changed the task status from Open to Stalled.Mar 17 2016, 6:18 PM
dr0ptp4kt renamed this task from Reduce sampling rate for language switcher after enough days of the A/B test at the higher sampling rate to Remove sampling rate artifacts for language switcher after enough data collected.Apr 4 2016, 4:55 PM
dr0ptp4kt set the point value for this task to 0.5.

Based on release timing, I think this looks more like sprint 71 now. Moving.

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptApr 19 2016, 9:48 PM

Change 272724 abandoned by Bmansurov:
Reduce sampling rate for language switcher

Reason:
Resurrect if needed.

https://gerrit.wikimedia.org/r/272724