This task represents the work with conducting an analysis to learn how QuickSurvey sampling works.
User Story
As a data analyst I want to be able to send the survey to a sample of users so that I can get statistically viable results.
What we know
Weβre looking to show the survey to a random sample of logged-in users who have made x amount of edits in the last x months at the time of the sampling
- QuickSurveys samples global users based on a percentage, and then filters logged in users and other criteria to see if they should see the survey This means that for a given survey, if we choose a sampling rate of 0.4, 40% of all users will be bucketed for the survey. Of that 40%, only the ones that match other criteria, like being logged in, will have the survey shown to them.
- Sampling is done based on session tokens, which are semi-permanent and let us get a percentage of device sessions, rather than a percentage of pageviews
Open Questions
- Can we sample based on logged-in status?
- β Yes, configuring the audience like so:
'audience' => [ 'anons' => false, ]
- Can we sample based on edit count?
- β Yes, configuring the audience like so:
'audience' => [ 'minEdits' => 0, ]
- Can we sample based on a time frame for a number of edits?
- Not easily
- Investigation in progress... β³
- Alternative criteria that could proxy this requirement:
- getLatestEditTimestamp() *
- We could expose the last time a user edited and filter users based on that and minEdits
- getLatestEditTimestamp() *