Page MenuHomePhabricator

en6C_dsk_FR campaign: Bucket D behind in impressions
Closed, ResolvedPublic

Description

Hey @AndyRussG, tagging you bc it seems relevant! We've got a test up in our en6C_dsk_FR campaign - I'm looking at buckets C & D. Two banners in each bucket and the impressions count for the banners seem suspiciously out of sync:

Bucket C:

Banner 1 impressions = 1,211,480
Banner 2 impressions = 1,214,810

Bucket D:

Banner 3 impressions = 1,178,290
Banner 4 impressions = 1,176,450

Event Timeline

Hmmm, that does look dicey! What time frame are those impressions from? I'll check the data on Hive... Hope it's just a sampling-related fluke!! Thanks...! ;p

Thanks for the fast response, Andy. This test went up at 22:50 UTC today and is still live.

One of the banners in bucket D appears in a different position: at the top of the page instead of at the end of the first section. It could be that more people are closing that, and hence not seeing more banners for 7 days.

Thanks much!! :)

Checking Hive data for today, for that campaign, I these numbers for banner-cancelled-due-to-previously-clicking-on-close-button status (statusCode 2.1):

bucketcount
A127391
B123249
C4026093
D4238714

So, @Pcoombe's theory is confirmed! About 200000 more users in bucket D closed a banner in the last 7 days, and as a result aren't seeing any!

I see that the campaign is switching banners quite a bit... Maybe you'd wish to use a new campaign to re-shuffle buckets? (Just a thought... ;) )

Thanks again for noticing this? Mmm what are your thoughts on closing the task? Or do you think there may be more that we can/should dig into here? :)

Seems like this mystery is solved. Thanks for the Hive digging!