Write a query to compare daily page views between control and trending groups in KaiOS app experiment

Closed, DeclinedPublic

Actions

@AMuigai @SBisson what do you hope to learn from this query? Is it just to confirm that the experiment isn't broken?

I understand it's early but given the early numbers you shared, ~3.5k devices per group per day, we could start seeing a difference between the groups.

Also, it's good to have confirmation that the data is usable for its intended purposes. There's still time to tweak it if needed.

In T280178#7000128, @SBisson wrote:

I understand it's early but given the early numbers you shared, ~3.5k devices per group per day, we could start seeing a difference between the groups.

Ah, I see. I definitely understand the desire to learn the outcome as soon as possible, but I don't think it's a good idea. We planned the experiment length ahead of time so that we wouldn't form a conclusion without sufficient data. Looking at the data every day would defeat that purpose.

Also, it's good to have confirmation that the data is usable for its intended purposes. There's still time to tweak it if needed.

Hmm...I don't think this would give us any assurance that we didn't already get from my first query. If I was doing a deeper data check now, there would be several other things I would check first.