@GoranSMilovanovic Ah perfect, thanks for checking and updating! Thanks also for double checking the figures. The missing edit indeed sounds weird... but I guess we'll have to work what the databases are giving us ;-)

@GoranSMilovanovic Great, thank you! The intro says the last update of user edit data is from November 24th, is that true? Because for the tracking of six week retention we would need the editing behavior until December 23rd as stated in the task description.

@GoranSMilovanovic Ah perfect, thank you! I guess they might have started the Challenge a bit later maybe or decided later to register while the cookie was still in place. Although we stated the 11.11. as the end of the campaign I think it would be good to include them also. This would be the last change of the report I hope! :-)

@GoranSMilovanovic Great, thank you.

@GoranSMilovanovic Great, thank you! It will be really interesting how the editing numbers develop until the end of the year....

Hi @GoranSMilovanovic, I had a look at the report and it would be great if you could include the following:

@GoranSMilovanovic Thank you! I will have a look next week and get back to you. For your planning: As far as I know there are no other campaigns planned from our side for the rest of the year. At the beginning of next year, we probably will need your expertise again though ;-)

@GoranSMilovanovic I just had a quick glance at the report and it looks good, so we can resolve the ticket yes! I checked on the other tickets, thank you.

@Aklapper done :-) thanks for helping to tidy this up!

@Aklapper This should remain active for non campaign related reportings, but this one could be archived indeed: https://phabricator.wikimedia.org/project/view/4409/

@GoranSMilovanovic great, thank you. No we track the pageviews from LP1 via Matomo separately and I don't think its necessary to include them in your report.

great, this looks really good :-) thank you!

@GoranSMilovanovic Thank you. The reporting of user registrations needs to be continued till November 11 because users receive emails until then. Could you update the spreadsheet? Thank you!

@GoranSMilovanovic Hi Goran, thank youu! I just checked the report and came across some differences in the impression numbers. In the second last version of the report the numbers of impressions and closed-by in the tables 1.1.A and 1.2.1 A are different to the numbers in this last version (13500 vs. 67500). Why is that? Could you check this? Thank you!

@GoranSMilovanovic Yes, no worries, we do not need the pageviews of all sub-landing pages because there are so many. We focus on some of them and decided not to track others. So it is possible that you get user registrations from other sub-landing pages.

@GoranSMilovanovic Yes please. In the other campaign we have an offwiki landingpage so the data from the schema is our only source. In this campaign we have better data from the wmf.webrequest table, thats why we stick with this.

@GoranSMilovanovic Hi Goran, do you have an update on this? Thank you :-)

@GoranSMilovanovic Hi, Goran, thank you for the update!

Great, thank you

@GoranSMilovanovic Great, thank you! So everything works and we are good to go :-) @kai.nissen

@GoranSMilovanovic
Just to be safe and to test the second part of the tracking as well I just created a user account from email 1 (campaign tag: ?campaign=WMDE_2020_challenge_1) with the user name "Testuser Challengebnr". Can you check, if you find it in the database? Thank you!

@GoranSMilovanovic
So I clicked twice on each banner from the links above, and produced one banner click and one closing click each. Can you find data now?

In the previous campaigns we were able to test the banners before actually running the campaign, or at least that is what I remember.

@kai.nissen Can we test with the dummy banners if the tracking of banner impressions and banner clicks via the schema works before the campaign starts?

@kai.nissen Great, thank you. Can we change the starting and eding time to 10.00 UTC (12.00 our time)?

@GoranSMilovanovic Thank you for the summary! Please see my comments below:

In T262534#6478963, @GoranSMilovanovic wrote:

@kai.nissen @Christine_Domgoergen_WMDE Are these the test banners for this campaign: T262670#6474604?

@GoranSMilovanovic Yes we will track the page views via Matomo also. I would like to verify how unreliable it really is when we do not have a complicated tracking of the target groups and it really is no extra effort to use all the data from the schema, right?

Perfect, thank you!

@GoranSMilovanovic Yes, but it would need to be replaced by the pageviews you counted from the wmf.webrequest table. And then all the ratios, where the banner clicks from the schema were used need to be recalculated also: all the click rates etc. Could you do that?

As far as I understand from https://phabricator.wikimedia.org/T249617 the banner clicks are not recorded correctly in event.WMDEBannerActions schema and we will use pageviews instead: a banner click == a pageview.

@kai.nissen Thank you! I will let the contractor do the changes we have talked about and get back to you

@AbbanWMDE Hey Abban, just fyi: we are currently asking the contractor to do some minor changes because the design is not implemented correctly in all banner sizes (see task https://phabricator.wikimedia.org/T262670. I will let you and Kai know, when the changes are implemented.

@GoranSMilovanovic Yes exactly! Thank you.

Thank you Kai!

@kai.nissen Thank you.

@kai.nissen Hey Kai, the contractor finished the banner development, you can find all code in the github fork. Could you integrate the code to see if everything works? Thank you! Please let me know if you need anything else!

@kai.nissen Hey Kai, we will be tracking the banner data from two sources now, the banner impressions from the wmf.webrequest table and the banner clicks and closing clicks fom the event.WMDEBannerInteractions schema. We were wondering how reliable the data from the schema will be this time (targeting only non-logged in users, no targeting in the browser). Can we have your opinion on this? Thank you!

@kai.nissen Kai, can we get your opinion on this question:

Hence the question: how much do we trust the new event.WMDEBannerActions schema? Do we want (A) to rely on it completely, or do we want to (B) rely on the assumption that its seen_by field reports the banner impressions data correctly while its clicked_by field reports incorrect data, or (C) do we want to rely on this schema at all?

As I understood it, it is better to trust the seen_by data from the schema because the banner impressions from the wmf.webrequest table does just count all banner impressions before the targeting filter is applied. Correct? This would mean option (B) would be what we go for. Or might there be an error in the "seen_by" data as well which would maybe explain the very very high conversion rate of about 25 percent, which we get when we calculate pageviews from wmf.webrequest / seen_by from the schema?

@GoranSMilovanovic Hi Goran, so I talked to Kai and he also cannot really explain, why the number of seen by is not equal the number of page views from the database. However he says in doubt the numbers you got from the database is more likely to be correct. Could you adapt the final report accordingly? Thank you! And afterwards we can finally close the tickent :-)

@GoranSMilovanovic Since we still have no idea why the ratios from the schema and the page view/seen by numbers are different I want to get some feedback from @kai.nissen about this before we close the ticket. Kai will be back from vacation in two weeks and I hope we can close the ticket when he has responded.

@GoranSMilovanovic Hi Goran, this ticket is is done, no further questions :-) I already published the report and will close the ticket. Thank you for your work on this!

@GoranSMilovanovic Thank you for the explanations.

@GoranSMilovanovic Okay, thank you. I still don't understand why there is such a big difference. I understand the rate is calculated on the basis of the numbers in "clicked_by", but why are they in sum so different from the number of page views from the database? And when I divide the clicked_by/seen_by I get a click_rate of 9,94% and not 9 percent straight. Maybe @kai.nissen can help here?

See section 3.4.4. in the following update: 2020_WMDE_OccasionalEditorsBannerCampaign .nb.html3 MBDownload

@GoranSMilovanovic Great, thank you for the quick update! In our meeting yesterday two more questions came up:

@GoranSMilovanovic Did you have time to look at this yet? We are having a meeting on this tomorrow and it would be great to have the numbers. Thank you!

@GoranSMilovanovic Just to be sure:

@GoranSMilovanovic Thank you, I will have a look at the report now! For your questions:

@GoranSMilovanovic Thank you! I will get back to you about this one next week.

@GoranSMilovanovic Sorry for the late reply and thank you! I believe this is one of the best reports we ever had :-) And some really interesting results in the edit behavior! Still, while writing the summary some questions have come up:

@Tobi_WMDE_SW @Addshore Okay, thank you! I changed the link.

@Addshore Hi Adam, thank you for bringing this to our attention. I checked the links in challenge 10 and for me they all work. If I replace the ? with a & I am directed to a non existing page in Wikidata. Could you specify which links you are referring to?

@GoranSMilovanovic Oh I forgot one question: the campaign spreadsheet includes the numbers until 27.05.2020, the campaign ended on 28.5.2020 12.00 UTC though. Could you include the last day as well? Thank you!

@GoranSMilovanovic Hi Goran, I had a look at the report and your questions and for a better overview I added our research questions here in the tracking document. Please indicate if something is not clear! A lot of them are already answered in the report.

@GoranSMilovanovic
Perfect, thank you! I will have a look at it and get back to you if there are questions.

@GoranSMilovanovic Okay, thank you for the clarification!

@Addshore Hi Adam, thank you for your help!

@GoranSMilovanovic Yes, thank you for setting up everything! We expect the first data on dewiki on day 4 of the campaign. The first data in one of the other projects is not expected until day 7, so if the patch is deployed unitl then we are good. I will be out of office next week but @Verena can help you if any questions come up.

@GoranSMilovanovic The campaign started yesterday evening as planned. Apart from the problem with tracking page views and user registrations in projects other than dewiki and the dynamic pages the tracking is place and running, right?

@Addshore Could you help us here with deploying the patch? And if you have an idea about Goran's two questions about the dynamically generated content of special pages it would be very appreciated as well :-) Thank you!

@GoranSMilovanovic No, due to the testing and some issues with the code we decided to start today

@GoranSMilovanovic Oh this is really great, thank you! :-)

@GoranSMilovanovic
Yes, thank you!

@GoranSMilovanovic just shortly before I made the comment, arount 2.30 pm

but I can't see any campaign tags in the respective uri_query fields; I was specifically looking for anything similar to WMDE_oceditors_spring_2020.