I tested it using a copy of a past campaign. Error log entry is on Kibana

Change 593502 had a related patch set uploaded (by Abban Dunne; owner: Abban Dunne):
[mediawiki/extensions/WikimediaEvents@master] Update Banner Interaction Schema

https://gerrit.wikimedia.org/r/593502

Change 594693 had a related patch set uploaded (by Gabriel Birke; owner: Abban Dunne):
[mediawiki/extensions/WikimediaEvents@wmf/1.35.0-wmf.30] Update Banner Interaction Schema

https://gerrit.wikimedia.org/r/594693

Change 594694 had a related patch set uploaded (by Gabriel Birke; owner: Abban Dunne):
[mediawiki/extensions/WikimediaEvents@wmf/1.35.0-wmf.31] Update Banner Interaction Schema

https://gerrit.wikimedia.org/r/594694

Change 593502 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Update Banner Interaction Schema

https://gerrit.wikimedia.org/r/593502

Change 594694 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@wmf/1.35.0-wmf.31] Update Banner Interaction Schema

https://gerrit.wikimedia.org/r/594694

Change 594693 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@wmf/1.35.0-wmf.30] Update Banner Interaction Schema

https://gerrit.wikimedia.org/r/594693

Mentioned in SAL (#wikimedia-operations) [2020-05-11T11:23:25Z] <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.35.0-wmf.31/extensions/WikimediaEvents/: SWAT: [[gerrit:594694|Update Banner Interaction Schema (T250791, wmf.31)]] (duration: 01m 07s)

Mentioned in SAL (#wikimedia-operations) [2020-05-11T11:30:12Z] <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.35.0-wmf.30/extensions/WikimediaEvents/: SWAT: [[gerrit:594693|Update Banner Interaction Schema (T250791, wmf.30)]] (duration: 01m 08s)

I tested this with a slightly modified version of the test banner. In order to test for errors, please have a look at my example query.

@kai.nissen @Verena @WMDE-leszek I have tested the revised WMDEBannerInteractions schema:

select dt, event.bannerName, event.bannerAction, event.bannerImpressions, event.userID from event.wmdebannerinteractions where year=2020 and month=05 and (day=10 or day=11) limit 100;

results in (userid field values stripped, while both records refer to the one and same user):

dt                      bannername                              banneraction    bannerimpressions       userid
2020-05-11T16:18:09Z    WMDE_oceditors_spring_2020_mobile       banner-closed   30      
2020-05-11T16:18:09Z    WMDE_oceditors_spring_2020_mobile       banner-seen     30

Q1. First row: does this mean that a particular user has seen the banner 30 times before closing the banner on 2020-05-11T16:18:09Z?
Q2. Second: does this mean that a particular user has received the 30th banner impression on 2020-05-11T16:18:09Z?

Also,

Q3. The campaign Central Notice setup defines different weights (see section: Assigned banners) for banner A, B, and C: I guess these numbers refer to banner impression rates?

Q4. From the ticket description: "The banner name also contains a suffix that can be used to distinguish between banner 1 and 2" - I do not see a suffix in the banner names in these test results, and also the campaign doc and Central Notice setup mention three banners (A, B, C), while only two (A, B) are mentioned in the ticket description here.

And finally,

Q5. In T249617 we find:

The a/b test scope is comparing target group specific banners (1 and 2) against banner 3:
banner 1: target group A,
banner 2: target group B,
banner 3: target group A+B

So, banner 1 will be presented to group A, banner 2 will be presented to group B, but banner 3 will be seen by both groups: is it ensured that an editor from group A who sees banner 1 does not also see banner 3, and that an editor in group B who sees banner 2 does not also see banner 3?

Thank you.

In T250791#6128795, @GoranSMilovanovic wrote:

Q1. First row: does this mean that a particular user has seen the banner 30 times before closing the banner on 2020-05-11T16:18:09Z?
Q2. Second: does this mean that a particular user has received the 30th banner impression on 2020-05-11T16:18:09Z?

Both is correct.

Q3. The campaign Central Notice setup defines different weights (see section: Assigned banners) for banner A, B, and C: I guess these numbers refer to banner impression rates?

The setup will be changed before starting. It will consist of two equally weighted banners.

Q4. From the ticket description: "The banner name also contains a suffix that can be used to distinguish between banner 1 and 2" - I do not see a suffix in the banner names in these test results, and also the campaign doc and Central Notice setup mention three banners (A, B, C), while only two (A, B) are mentioned in the ticket description here.

I just tested the event logging functionality yesterday. The test banner did not include suffixing the banner-name.

So, banner 1 will be presented to group A, banner 2 will be presented to group B, but banner 3 will be seen by both groups: is it ensured that an editor from group A who sees banner 1 does not also see banner 3, and that an editor in group B who sees banner 2 does not also see banner 3?

Yes. banner C will be in bucket B. Unless users clear their cookies, they will see only banner A/B or banner C.

@kai.nissen We're superfine to track and analyze the campaign then. Thank you very much.

@kai.nissen @GoranSMilovanovic This is great, thank you!

Thank you all. @GoranSMilovanovic That means we are set and ready? Or is further testing before the campaign set recommended?

@Verena

Or is further testing before the campaign set recommended?

I would create a test account or two, just in case.

@Verena @Christine_Domgoergen_WMDE Given that the user registration procedure for our concurrently running campaign (see: T251524#6133796) fails to send the campaign tag to the event.serversideaccountcreation table, I would now strongly suggest to test user registrations for this campaign too.

@GoranSMilovanovic Since we are targeting only logged in users this is not necessary, we are lucky here :-)

@Verena @Christine_Domgoergen_WMDE In relation to this campaign, please see: T251524#6133866 from @Addshore. Keyword: patch.

@Christine_Domgoergen_WMDE Oh, right... Thanks. I am beginning to get lost in concurrently testing two campaigns. @Verena please disregard my comment in T250791#6133879 (just above).

@GoranSMilovanovic No worries :-) we can focus on https://phabricator.wikimedia.org/T251535#6132281 for tonight