As stated in the documentation of Central Notice, only two of the four buckets are assigned automatically. The other buckets have to be assigned programmatically. That means, that users will first see banner A or B and can then be moved to other buckets to see banner C or D.

Technically, it is possible to do what we want, the current setup on meta just won't allow us.

Thank you for clarifying. Does that mean, that we can only have 2 different banners? Or would it still work if we for example delete banner (3)?
Then it Would be 25%, 25%, 12,5% 12,5%.

We can only have two different banners. Nobody would initally see banner 3 or 4, unless code in banner 1 or 2 would assign users to buckets C or D. That is not what we want, though.

Obviously, that does not correspond to the Campaign design as provided in the KickOff new editor autumn edition slides.

I would highly appreciate if anyone could provide me with a better understanding of banner bucketing, and especially in respect to the following questions:

• Will the introduction of banner bucketing affect the campaign user tracking as described in https://phabricator.wikimedia.org/T174952#3595367 and https://phabricator.wikimedia.org/T174952#3599738 in any way, or not? The bucket field is present in our databases and if I need to track that info too, then I should learn about it.

• Will the introduction of banner bucketing change the campaign design as described in the the KickOff new editor autumn edition slides in any way, and if yes, then what is the new campaign design? I have to ask this since from you previous discussion it seems like you're close to the decision to actually change the campaign design (seeing Banners 1, 2, then 3 or 4, etc.)

ASAP, please. Thank you.

@GoranSMilovanovic

Will the introduction of banner bucketing affect the campaign user tracking

No, even if we change the campaign design the tracking would remain the same. The buckets are just to direct users to the appropriate banner. It is not relevant for tracking. With T174952#3599738 and T174952#3595367 we have all relevant information.

Will the introduction of banner bucketing change the campaign design

We need to discuss this scenario on Monday with our team. I hope we can stick to the plan! We will communicate possible changes asap and in every affacted Ticket!

In General:
I just had a chat with @kai.nissen. It would be possible to work without buckets and thus still show all banners. But then we cannot make sure that one user always sees the same banner. It could be, that the same user sees two different banners in two different sessions in the campaign period.
@GoranSMilovanovic: If we do it this way, would it sophisticate our findings about the conversions?

@Stefan_Schneider_WMDE No no no please use banner bucketing, now I understand what it does. Certainly there is a way to handle the situation where the user sees two or more banners during the campaign analytically, but I can already sense complication there.

@GoranSMilovanovic If we can only use 2 banners, this would massively affect our design and I would love to make 4 banners possible. Please get specific on possible complications, so we can discuss scenarios in our team on Monday. Technically we can test the scenarios beforehand.

@Stefan_Schneider_WMDE Ok, I see: if we use banner bucketing, we can have only two different banner (please confirm).

Ok. Well, don't do it then. Let the user be able to see different banners during the campaign, we can incorporate that to our Multi-Channel Attribution Model of the campaign.

A minor refinement of the research design for the Campaign Analytics in order to incorporate this fact is now introduced to Slide 11 of the KickOff Presentation.

Now, what you can't really see from the Slide is that the dataset will change: it will encompass all observed user journeys, include those where a user sees a banner and does nothing. That will enable us to estimate the capacity of the banner to induce a click upon itself in the scope of the model encompassing all campaign elements.

I will change the Campaign Simulation in order to accommodate for this; the same code will be used for real campaign analytics, of course.

@GoranSMilovanovic Right, using bucketing only 2 banners can be used. Thank you for integrating this scenario in the presentation. After the meeting on Monday I'll confirm whether we use bucketing or not.

@Stefan_Schneider_WMDE Simulation updated.

Thank you @GoranSMilovanovic!

We decided to continue without bucketing. It would be not possible to show the same banner to the same person, but it's more crucial to show the different banners and test them. Let's see how it'll work in the testing.

You're welcome @Stefan_Schneider_WMDE

Proceeding with the campaign design as initially planned + recently introduced corrections to account for the fact that a single user may see different banners during the campaign.

Stay in touch.