It would be great if we could build on the current structure of our campaign configurations to be able to support A/B testing (of different verbiage, etc).
Here is our current way of encoding campaign assets:
assets: {
en: {
text: "...",
actions: { ... }
}
}And here is the new proposed style:
When not running an A/B test:
assets: {
en: [
{
text: "...",
actions: { ... }
}
]
}...And when running an A/B test:
assets: {
en: [
{
weight: 0.5,
id: "verbiageA",
text: "This is verbiage A!",
actions: { ... }
},
{
weight: 0.5,
id: "verbiageB",
text: "This is verbiage B!",
actions: { ... }
},
]
}- Notice how, when not running an A/B test, the structure is almost the same, except it's an array. (and the new weight and id parameters are optional)
- And when running an A/B test, the weight and id parameters are used, and you could have an arbitrary number of verbiage objects - not just two, which will be chosen on the client side, based on the weight.
- On the client side, the app will randomly put the current user (based on app_install_id) into one of the asset buckets, taking the weights into account.
- This will necessitate bumping the version of the top-level campaign object to 2 (currently 1). Note that different versions of campaign objects can coexist in the same top-level array of objects, so we'll be able to make lists of campaigns that target both new versions of apps, as well as older versions that don't support a/b testing.
- For tracking impressions, we could use the top-level campaign id (as we do currently), and append the newly-added id of the a/b verbiage that gets chosen. So it will become something like BR_2025_03_verbiageA / BR_2025_03_verbiageB.
Effort estimate
Large task for a single developer (per team).