Hi @kai.nissen
I think we can try it with the following

• banner diet: 100%
• impression rate per user: 5

Just for testing, or would 100% be too much?

I would aim the 20.09 for testing.

Since this is setup on beta-meta, I think we can test with no traffic limit (i. e. 100%).

Giving the links for testing landingpages on T175911#3614574 and the deployed hack on Thursday, all information should be in place now. Would it be possible to have a setup until Thursday, when the hack is deployed or do you need something else?

The banners will be dummies, right? So I just make up "content" and set the target links to the respective landing pages?

Right

Dummy banners are set up:

https://de.wikipedia.org/?banner=WMDE_editor_campaign_fall17_a
https://de.wikipedia.org/?banner=WMDE_editor_campaign_fall17_b
https://de.wikipedia.org/?banner=WMDE_editor_campaign_fall17_c
https://de.wikipedia.org/?banner=WMDE_editor_campaign_fall17_d
https://de.wikipedia.org/?banner=WMDE_editor_campaign_fall17_e

Banner link targets are set as defined in T175911#3614574, campaign parameters are appended as defined in T174952#3595367.

Awesome - thank you @kai.nissen!
Then we just need to test it from Thursday on and @GoranSMilovanovic can create the report next week.
I will make sure that the banners will be clicked and that the data will be created.

@Stefan_Schneider_WMDE What elements of the Campaign will the test encompass:

• only banner impressions and clicks, or
• landing pages, registrations, guided tours, edits etc. too?

I guess only impressions and clicks, right?

I'm working on the R code right now.

@Stefan_Schneider_WMDE @kai.nissen Kai, if a user sees a General Invitation Banner and goes to the landing page

from https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/Mach_mit?campaign=wmde_abc2017_gib_lp

(we're talking about your dummy Banner D), and the clicks "ACCOUNT ANLEGEN" in the image at the bottom of the page, he or she will be redirected to:

https://de.wikipedia.org/wiki/Spezial:Benutzerkonto_anlegen

which does not contain any campaign url parameter.

Q: How will we track how many users came to the registration page from this landing page?

EDIT. The same seems to hold for all other banners, indeed. Probably I can track the referral page for every page view of the Registration page, I guess?

@Stefan_Schneider_WMDE Ok, please let me know when you have some test data in; the R test code is ready. Thanks.

@Stefan_Schneider_WMDE @kai.nissen

Ok, edited:

• I can see the test banner clicks in the referer field of the wmf.webrequest table on the landing pages;

• I can also track the hits to the registration page from the respetive landing pages and/or the banner that point directly to it.

@Stefan_Schneider_WMDE Let's go clicking on these test banners tomorrow for a while and I will deliver a test report in line with https://phabricator.wikimedia.org/T175914 during the day.

@kai.nissen Thanks a lot!

@GoranSMilovanovic

Great - than everything seems to work. I'll provide you a summary of the data needed for the test tracking in the respective task T175914.

Thanks Groan for the hint.

@kai.nissen: In T175911#3614657 you mentioned the following:

We can, of course, create test banners that will not be part of an active campaign, but can be forced to display by using the banner parameter. Calling banners with that parameters will not trigger an impression count, though.

Does that mean, that there will be no record in webrequest-table and thus no page-views tracked?

That impressions are not trackable was clear before starting this testing, though. If the pageviews are not trackable with the webrequest-table we can still track the banner clicks (@GoranSMilovanovic: please confirm) and thus the page views.

@Stefan_Schneider_WMDE No worries about the banner clicks and landing page/registration page views; I've already tracked the test data and I'm working on the report right now. Ok, so no banner impressions in the test report.

Q: Did you engage in any user registration, guided tour or editing from these pages during the test? Thank you.

Some people of our team clicked on every banner and I created some test accounts that you should be able to track. Unfortunately there was no Guided Tour shown after the registration.

@Addshore: Am I right, that the hack should start the guided tour after the registration? Is there something not working properly? I just tried again and the guided tour does not show up.

@kai.nissen @Stefan_Schneider_WMDE

Could you please check whether you are tracking the referer for the Registration Page (Spezial:Benutzerkonto_anlegen) when the source is /wiki/Wikipedia:Wikimedia_Deutschland/Mach_mit?

I can't see any cases where users starts from /wiki/Wikipedia:Wikimedia_Deutschland/Mach_mit and lands on Spezial:Benutzerkonto_anlegen - could it be because the the user needs to click on an image on /wiki/Wikipedia:Wikimedia_Deutschland/Mach_mit in order to get there? Thanks.

Looks like there is a small issue with the code T174948#3626972 https://gerrit.wikimedia.org/r/#/c/379748/1

I can deploy this fix on Monday.

@Stefan_Schneider_WMDE @Addshore

"Some people of our team clicked on every banner and I created some test accounts that you should be able to track. Unfortunately there was no Guided Tour shown after the registration."

There are no event_campaign field values remotely similar to our campaign tags in the log.ServerSideAccountCreation_5487345 table. Please check whether the campaign user registrations are tracked properly.

Please let's avoid digging this out from server logs like we did last time. Also, please check whether Guided Tour tracking is enabled for this campaign.

@Addshore

Does the fix, that was deployed today, only correct the guided-tour-issue or also that the campaign tags are not recorded in the log.ServerSideAccountCreation_5487345 table. I got, that last time it worked like that. Could you have a look on it, please?

If all the bugs are fixed I would test it again and make sure everything works properly.

@Stefan_Schneider_WMDE Are you sure that you have some test data there, entered following the deployment of the fix? If yes, please confirm, and I will test everything today.

@Stefan_Schneider_WMDE it will fix both issues. It will be deployed in the next 2 hours.

@Stefan_Schneider_WMDE @Addshore Awesome. Please let me know when some test data are in (I will be testing too, with a bit of assistance from the Google Translate service :)

@Addshore Thank You. Just leave a comment, when it's done and we will create some test data.

@Stefan_Schneider_WMDE Done! You should be able to create the test data now!

@Addshore Thanks for the message. I created some test data now. The guided tour is still not shown to me as a new user. I don't know the reason. Did I do something wrong? Could you test it from your side, please?

So I can confirm the code seems to be working.
Here are the server logs for the test I just carried out:

2017-09-25 15:46:47 [Wckk5wpAIDUAACychPkAAAAG] mw1183 dewiki 1.30.0-wmf.19 WMDE INFO: wmde_abc2017_bt2 - 1 - Banner click by anon user without cookie
2017-09-25 15:46:50 [Wckk6gpAEKoAAHE41goAAAAT] mw1215 dewiki 1.30.0-wmf.19 WMDE INFO: wmde_abc2017_bt2 - 2 - Inject campaign value on CreateAccount
2017-09-25 15:47:33 [WcklFQpAMFAAAIpZqdQAAABA] mw1245 dewiki 1.30.0-wmf.19 WMDE INFO: wmde_abc2017_bt2 - 3.e - User older than the campaign.

The last log line right now is generated by https://github.com/wikimedia/mediawiki-extensions-WikimediaEvents/blob/59d2eea7aa13a05a7de1fb16599ef310e4cd5b15/WikimediaEventsHooks.php#L762
The start time is set @ https://github.com/wikimedia/mediawiki-extensions-WikimediaEvents/blob/59d2eea7aa13a05a7de1fb16599ef310e4cd5b15/WikimediaEventsHooks.php#L697
Which is set to 20171004000000 (as I understand it this is essentially the start of the campaign)

No users will be shown guided tours until after that timestamp, but the guided tours will be shown then :)

Here's what I've found in log.ServerSideAccountCreation_5487345 last night (sometime between 00:30 and 1:40 CEST):

• wmde_abc2017_bt1 (2 registrations)
• wmde_abc2017_bt2 (1 registration)

Feedback, please. @Stefan_Schneider_WMDE I am now modifying the Daily Report in line with https://phabricator.wikimedia.org/T175914#3631667

@GoranSMilovanovic

Thanks for testing. That should be the data I created. It would have been more, but Wikipedia denied to create more accounts from my IP.
Ok, Great - then it's working now. Can't wait to see your updated report.

@Addshore

Now I get, why it wasn't working these days. We planned to test the guided tour beforehand. Is there a particular reason to start the guided tour only from the beginning of our campaign and not for testing issues right now? Then we could be sure that the tracking is working.

In T175912#3635116, @Stefan_Schneider_WMDE wrote:

@Addshore

Now I get, why it wasn't working these days. We planned to test the guided tour beforehand. Is there a particular reason to start the guided tour only from the beginning of our campaign and not for testing issues right now? Then we could be sure that the tracking is working.

So firstly the reason the 'hack' broke for the tracking this time was due to changes that I made to make it easier to reuse for each campaign. The code that triggers the guided tour has not changed.
For the next campaign I may be able to simply make the campaign options configurable and avoid re deploying this hack each time, and just leave it in place and turn it on and off with different options.

The reason for the date limit is otherwise the following can happen (which happened in the first campaign)

1. User registered since 2015 is logged out but views and clicks on a banner adding the session cookie to their browser.
2. User logs in and views a mainspace page, as they have started the banner process and now have an account they would be shown the guided tour.

It might be possible to think of another way around this issue, perhaps by using multiple different cookies to identify each step / make sure a user lands on the registration page, or maybe even the successful registration page before having the chance of being shown the guided tour.
I'm not sure if it is worth working on this though as at one of the last meetings I understand we were thinking about stopping the guided tour all together?

@Addshore

It's no problem that it initially didn't work - that's why we are doing the test. So now it's cured :)

About the guided tour. Yeah, I remember this problem and totally get the date now. For now we will not change the user journey or any campaign tags, as it is too close to the start of the campaign. We will work on other ways of getting started in the wikipedia but if we reuse the guided tour again we will revisit that problem.