@GoranSMilovanovic Hi Goran, I will be on vacation for the next weeks but @Merle_von_Wittich_WMDE will take over from here and looks forward to the updated report :-)

@kai.nissen Great thank you!

Great, thank you! I have some questions:

@GoranSMilovanovic Perfect, thank you! When will you be able to finish the preliminary report?

@GoranSMilovanovic can you continue the daily reporting anyways? I guess it will take a bit to figure out the confusion and we might need @kai.nissen for that...

@GoranSMilovanovic Okay, thank you!

@GoranSMilovanovic @gabriel-wmde @AbbanWMDE
Hello,
I just checked the tracking spreadsheet and I am confused by the banner names. Kai mentioned above

Yes, we can. Would you like to see the distinction in the daily reports, or in the summary report? Because if it needs to be done for daily reporting I will also need to change the existing analytics code. It is easier done once we have all the data for the final report.

@GoranSMilovanovic Okay, thanks! Can we distinguish in the number of impressions and clicks if they come from logged-in or non-logged-in users?

@GoranSMilovanovic Oh perfect, thank you! Banner impressions look a lot more plausible right now

@GoranSMilovanovic @kai.nissen
Hi Goran, perfect, thank you! I had a look at the interactions table and am a bit confused: according to the table there were only 7709 impressions but 1014981 closing clicks and 519389 banner clicks. Could the impression number be wrong here?

@GoranSMilovanovic Yes of course, self-care first, especially in these times!! No worries, I am just always a bit curious about the numbers but it can easily wait until tomorrow or even the day after tomorrow. Take care!

@GoranSMilovanovic Hi Goran, would you share the spreadsheet with the daily reporting here as soon as you have it? Thank you!

@GoranSMilovanovic Ah perfect, thanks for checking and updating! Thanks also for double checking the figures. The missing edit indeed sounds weird... but I guess we'll have to work what the databases are giving us ;-)

@GoranSMilovanovic Great, thank you! The intro says the last update of user edit data is from November 24th, is that true? Because for the tracking of six week retention we would need the editing behavior until December 23rd as stated in the task description.

@GoranSMilovanovic Ah perfect, thank you! I guess they might have started the Challenge a bit later maybe or decided later to register while the cookie was still in place. Although we stated the 11.11. as the end of the campaign I think it would be good to include them also. This would be the last change of the report I hope! :-)

@GoranSMilovanovic Great, thank you.

@GoranSMilovanovic Great, thank you! It will be really interesting how the editing numbers develop until the end of the year....

Hi @GoranSMilovanovic, I had a look at the report and it would be great if you could include the following:

@GoranSMilovanovic Thank you! I will have a look next week and get back to you. For your planning: As far as I know there are no other campaigns planned from our side for the rest of the year. At the beginning of next year, we probably will need your expertise again though ;-)

@GoranSMilovanovic I just had a quick glance at the report and it looks good, so we can resolve the ticket yes! I checked on the other tickets, thank you.

@Aklapper done :-) thanks for helping to tidy this up!

@Aklapper This should remain active for non campaign related reportings, but this one could be archived indeed: https://phabricator.wikimedia.org/project/view/4409/

@GoranSMilovanovic great, thank you. No we track the pageviews from LP1 via Matomo separately and I don't think its necessary to include them in your report.

great, this looks really good :-) thank you!

@GoranSMilovanovic Thank you. The reporting of user registrations needs to be continued till November 11 because users receive emails until then. Could you update the spreadsheet? Thank you!

@GoranSMilovanovic Hi Goran, thank youu! I just checked the report and came across some differences in the impression numbers. In the second last version of the report the numbers of impressions and closed-by in the tables 1.1.A and 1.2.1 A are different to the numbers in this last version (13500 vs. 67500). Why is that? Could you check this? Thank you!

@GoranSMilovanovic Yes, no worries, we do not need the pageviews of all sub-landing pages because there are so many. We focus on some of them and decided not to track others. So it is possible that you get user registrations from other sub-landing pages.

@GoranSMilovanovic Yes please. In the other campaign we have an offwiki landingpage so the data from the schema is our only source. In this campaign we have better data from the wmf.webrequest table, thats why we stick with this.

@GoranSMilovanovic Hi Goran, do you have an update on this? Thank you :-)

@GoranSMilovanovic Hi, Goran, thank you for the update!

Great, thank you

@GoranSMilovanovic Great, thank you! So everything works and we are good to go :-) @kai.nissen

@GoranSMilovanovic
Just to be safe and to test the second part of the tracking as well I just created a user account from email 1 (campaign tag: ?campaign=WMDE_2020_challenge_1) with the user name "Testuser Challengebnr". Can you check, if you find it in the database? Thank you!

@GoranSMilovanovic
So I clicked twice on each banner from the links above, and produced one banner click and one closing click each. Can you find data now?

In the previous campaigns we were able to test the banners before actually running the campaign, or at least that is what I remember.

@kai.nissen Can we test with the dummy banners if the tracking of banner impressions and banner clicks via the schema works before the campaign starts?

@kai.nissen Great, thank you. Can we change the starting and eding time to 10.00 UTC (12.00 our time)?

@GoranSMilovanovic Thank you for the summary! Please see my comments below:

In T262534#6478963, @GoranSMilovanovic wrote:

@kai.nissen @Christine_Domgoergen_WMDE Are these the test banners for this campaign: T262670#6474604?

@GoranSMilovanovic Yes we will track the page views via Matomo also. I would like to verify how unreliable it really is when we do not have a complicated tracking of the target groups and it really is no extra effort to use all the data from the schema, right?

Perfect, thank you!

@GoranSMilovanovic Yes, but it would need to be replaced by the pageviews you counted from the wmf.webrequest table. And then all the ratios, where the banner clicks from the schema were used need to be recalculated also: all the click rates etc. Could you do that?

As far as I understand from https://phabricator.wikimedia.org/T249617 the banner clicks are not recorded correctly in event.WMDEBannerActions schema and we will use pageviews instead: a banner click == a pageview.

@kai.nissen Thank you! I will let the contractor do the changes we have talked about and get back to you

@AbbanWMDE Hey Abban, just fyi: we are currently asking the contractor to do some minor changes because the design is not implemented correctly in all banner sizes (see task https://phabricator.wikimedia.org/T262670. I will let you and Kai know, when the changes are implemented.

@GoranSMilovanovic Yes exactly! Thank you.

Thank you Kai!

@kai.nissen Thank you.

@kai.nissen Hey Kai, the contractor finished the banner development, you can find all code in the github fork. Could you integrate the code to see if everything works? Thank you! Please let me know if you need anything else!

@kai.nissen Hey Kai, we will be tracking the banner data from two sources now, the banner impressions from the wmf.webrequest table and the banner clicks and closing clicks fom the event.WMDEBannerInteractions schema. We were wondering how reliable the data from the schema will be this time (targeting only non-logged in users, no targeting in the browser). Can we have your opinion on this? Thank you!

@kai.nissen Kai, can we get your opinion on this question:

Hence the question: how much do we trust the new event.WMDEBannerActions schema? Do we want (A) to rely on it completely, or do we want to (B) rely on the assumption that its seen_by field reports the banner impressions data correctly while its clicked_by field reports incorrect data, or (C) do we want to rely on this schema at all?

As I understood it, it is better to trust the seen_by data from the schema because the banner impressions from the wmf.webrequest table does just count all banner impressions before the targeting filter is applied. Correct? This would mean option (B) would be what we go for. Or might there be an error in the "seen_by" data as well which would maybe explain the very very high conversion rate of about 25 percent, which we get when we calculate pageviews from wmf.webrequest / seen_by from the schema?

@GoranSMilovanovic Hi Goran, so I talked to Kai and he also cannot really explain, why the number of seen by is not equal the number of page views from the database. However he says in doubt the numbers you got from the database is more likely to be correct. Could you adapt the final report accordingly? Thank you! And afterwards we can finally close the tickent :-)

@GoranSMilovanovic Since we still have no idea why the ratios from the schema and the page view/seen by numbers are different I want to get some feedback from @kai.nissen about this before we close the ticket. Kai will be back from vacation in two weeks and I hope we can close the ticket when he has responded.

@GoranSMilovanovic Hi Goran, this ticket is is done, no further questions :-) I already published the report and will close the ticket. Thank you for your work on this!

@GoranSMilovanovic Thank you for the explanations.

@GoranSMilovanovic Okay, thank you. I still don't understand why there is such a big difference. I understand the rate is calculated on the basis of the numbers in "clicked_by", but why are they in sum so different from the number of page views from the database? And when I divide the clicked_by/seen_by I get a click_rate of 9,94% and not 9 percent straight. Maybe @kai.nissen can help here?

See section 3.4.4. in the following update: 2020_WMDE_OccasionalEditorsBannerCampaign .nb.html3 MBDownload

@GoranSMilovanovic Great, thank you for the quick update! In our meeting yesterday two more questions came up:

@GoranSMilovanovic Did you have time to look at this yet? We are having a meeting on this tomorrow and it would be great to have the numbers. Thank you!

@GoranSMilovanovic Just to be sure:

@GoranSMilovanovic Thank you, I will have a look at the report now! For your questions:

@GoranSMilovanovic Thank you! I will get back to you about this one next week.

@GoranSMilovanovic Sorry for the late reply and thank you! I believe this is one of the best reports we ever had :-) And some really interesting results in the edit behavior! Still, while writing the summary some questions have come up: