Page MenuHomePhabricator

Create Tracking Report: Email Campaign Donors for New Editors 2019
Closed, ResolvedPublic

Description

We will setup an email campaign and reuse the landingpage we used in our last campaigns:

**Landing page link
https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia

Campaign tag
There will be 3 different email templates with similar messages. They all link to the Landing Page and should contain the following campaign tag:
?campaign=WMDE_2019_email

Landing page including the campaign tag
https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia?campaign=WMDE_2019_email

Please find further campaign information here.

Campaign start: 29.09.2019

We will need to track..
(a) new registrations via the campaign tags
(b) Landingpage impressions via the campaign tags

Event Timeline

Hi @Boris_Brunst_ext_WMDE. Can you please associate at least one active project with this task (via the Add Action...Change Project Tags dropdown). Maybe this is about WMDE-Fundraising-Tech ? This will allow others to get notified and see this task when looking at the corresponding project workboard. Thanks!

No reply from @Boris_Brunst_ext_WMDE, hence adding a (slightly random) project tag to this task. Feel free to correct.

@Stefan_Schneider_WMDE I think we should meet:

  • I do not know who will be setting up the tracking for this campaign, while
  • I would need to consult with that person to (1) make sure all the details are clear and (2) organize the tests as we do for each campaign.

@GoranSMilovanovic I talked to @Stefan_Schneider_WMDE : the tracking should be set up by now.

I just made a test account for several campaign tags and passed the trainings for WMDE_2019_emaila and WMDE_2019_emailb. Could you please check these campaign tags for new registries:
WMDE_2019_emaila
WMDE_2019_emailb
WMDE_2019_emailb_ort
WMDE_2019_emailb_gender
WMDE_2019_emailc

..and give me a feedback if the tracking works :) Thank you!!

@Boris_Brunst_ext_WMDE @Stefan_Schneider_WMDE

Test done for September 20, 2019:

event_userId	event_userName	event_isSelfMade	event_campaign	 timestamp	date	campaign
3299618	        Borrrrrris	1	                WMDE_2019_emaila 20190920145456	2019-09-20	2019_EmailCampaign
3299623	        Borrrrrrisb	1	                WMDE_2019_emailb 20190920150055	2019-09-20	2019_EmailCampaign

So, on September 20, 2019, we had two test registrations: one from WMDE_2019_emaila and another from WMDE_2019_emailb.

Please check your tracking for the remaining tags. Also, please check if some test registrations (maybe those for the missing tags) were made before or after Sep 20, 2019 (I chose this date because you indicated that the test took place on that date - I am correct in assuming that all tests took place on Sep 20, 2019?).

Edit. I have just checked September 19, 21, and 22 data: no user registrations for any of the campaign tags. I think you need to check your tracking for _emailb_ort, _emailb_gender, and _emailc - if you've used them at all in your tests.

Ok, thank you @GoranSMilovanovic for the feedback. All the tests took place on Sep 20!
I wonder why the other tags are missing.. Nevertheless, I will re-test today for the missing tags and post it here.

@Boris_Brunst_ext_WMDE @Stefan_Schneider_WMDE

No data, test just finished.

@Boris_Brunst_ext_WMDE You are asking me to test the user registrations - why are you providing links to the landing page?

@Boris_Brunst_ext_WMDE @Stefan_Schneider_WMDE

Tested again; no data on user registrations for the three remaining tags.

@GoranSMilovanovic I tested It from my side to just have another approach. Could you look it up? I created them just a few minutes ago.
THX!

Campaign Tag: WMDE_2019_emailb_ort
https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia?campaign=WMDE_2019_emailb_ort

Account created: Testtttteet

Campaign Tag: WMDE_2019_emailb_gender
https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia?campaign=WMDE_2019_emailb_gender

Account created: Testetetett

Campaign Tag: WMDE_2019_emailc
https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia?campaign=WMDE_2019_emailc

Account created: Testeteteeeettt

@Stefan_Schneider_WMDE @Boris_Brunst_ext_WMDE

Sorry, but I still cannot find any user registrations from the three remaining tags. I checked it manually (e.g. selected all user registrations from September 20th and until today and searched for your test user names).

However: @Boris are you sure that you did not make any test user registrations while carrying the following tag: WMDE_neweditors_summer2018_LP2, on September 20th? There are some user names similar to your test user names created for WMDE_2019_emaila and WMDE_2019_emailb found in the data set... I don't even know if this insight helps or not, just asking. On other hand, @Stefan_Schneider_WMDE : nothing similar to the test user names that you've reported in T232332#5516193, sorry.

Hi @GoranSMilovanovic , thank you for the feedback. It is still mysterious.. Yes, the other usernames I used are very similar and like these:

  1. September:

Borrrrrrisb_ort
Borrrrrrisb_gender
Borrrrrrisc

  1. September

Borrrrisb_ort
Borrrrisb_gender
Borrrrisc

Are there any entries or belonging campaign tags?

@Boris_Brunst_ext_WMDE @Stefan_Schneider_WMDE

September 20.

Borrrrrrisb ort    WMDE_neweditors_summer2018_LP2    20190920153846
Borrrrrrisb g      WMDE_neweditors_summer2018_LP2    20190920154045
Borrrrrrisc        WMDE_neweditors_summer2018_LP2    20190920154438

So these are your test user names, the timestamps are right, but the tags are not.

September 23.

Nothing similar to your test usernames is found.

@Boris_Brunst_ext_WMDE User names are private data in WMF/WMDE and thus have to be handled very carefully. Here, we know what test user names you were using and under which campaign tags you tried to register. However, even in tests we need to be careful: in T232332#5518443 you state the following user names

Borrrrrrisb_ort
Borrrrrrisb_gender
Borrrrrrisc

but you actually did not use and underscore (_) - see the test data.

Please, in your next test, use the following user names:

BorisB_Test_WMDE_2019_emailA_1
BorisB_Test_WMDE_2019_emailB_1
BorisB_Test_WMDE_2019_ort_1
BorisB_Test_WMDE_2019_gender_1
BorisB_Test_WMDE_2019_emailC_1

So, we will start with 1 at the end of the test user name for the next test, and then 2 if we need another test, and so on. That is how we will know exactly what test users are we looking for and minimize the probability that we accidentally pick up some real user data. Thank you.

@GoranSMilovanovic Just made a new test. Since Wikipedia seems to not support underscore I had to setup the names with hyphen:

BorisB-Test-WMDE-2019-emailA-1
BorisB-Test-WMDE-2019-emailB-1
BorisB-Test-WMDE-2019-ort-1
BorisB-Test-WMDE-2019-gender-1
BorisB-Test-WMDE-2019-emailC-1

  1. Question: would it be possible to analyze the landing page impressions according to the campaign tags? (for example to find out that people come to the landing page via a specific campaign tag but do not sign up for any reason..)

THX

@Boris_Brunst_ext_WMDE

Just made a new test.

I am on it now.

Since Wikipedia seems to not support underscore I had to setup the names with hyphen

Fine, thanks for noticing the _ thing in the user names, I didn't know that they were not supported.

Question: would it be possible to analyze the landing page impressions according to the campaign tags? (for example to find out that people come to the landing page via a specific campaign tag but do not sign up for any reason..)

Yes. In order to assess the pageviews from a specific tag we don't need the user to be logged in.

@Boris_Brunst_ext_WMDE Still nothing. Again, I have searched through all registrations since 2019/09/20, not specifically for our campaign tags. Only the first two test registrations are there (what we've already found in T232332#5515435).

@Addshore Hi Adam, we currently try to test a tracking setup for a mailing campaign. We use the extensions in the task description. And right now some of the test are just not working. Now I wonder if something in the tags or the hack you made maybe got wrong? Could you assist here? The campaign starts on Monday and we need feedback asap. Sry for the quick Task here... I hope you can halp!

Hey @GoranSMilovanovic
can you please support us here again. We are lost :(
It could be, that we have an error-prone test setup: Maybe the browser tracks a different tag, than we are klicking. Honestly, we don't know. I'll check up with a techy in the software team upstairs.

But another question: It would really help to approve from your side, that the tags are not working. Would it be possible that you create a test account with one of the (seemingly not working) campaign tags? If we ourselves are the source of the error, that would really help!

Still - I would double check with someone else and test it from my side again.

@Stefan_Schneider_WMDE @Boris_Brunst_ext_WMDE

For September 23, 2019, found from Hadoop (event.serversideaccountcreation - thanks @Ladsgroup for an insight):

Borrrrisb ort : WMDE_2019_emailb_ort
Borrrrisb gender : WMDE_2019_emailb_ort
Borrrrisc : WMDE_2019_emailb_ort
Testetetett : WMDE_2019_emailb_gender
Testtttteet : WMDE_2019_emailb_ort
Testeteteeeettt : WMDE_2019_emailc

Please confirm if these data accurately reflect your test user registrations.

The question remains: why didn't we get to see this data in our MySQL storage? This is strange, all our previous campaigns worked perfectly with the ServerSideAccountCreation schema there.

@GoranSMilovanovic Awesome, that we found a solution. Your question to the MySQL storage ist one, that I cannot answer. Honestly I don't even know where to adress it. But is the hadoop storage a workaround, that would work for at least this campaign?

The three tests, that I did are confirmed with your data.

For the whole testing I would love to have a confirmation from @Boris_Brunst_ext_WMDE.

@Stefan_Schneider_WMDE

But is the hadoop storage a workaround, that would work for at least this campaign?

Not that it presents a workaround only - we can easily switch our campaign analytics to work with Hadoop entirely.
In conclusion, we can safely proceed with tests & the campaign itself, while I will be using the event.serversideaccountcreation table in Hadoop.

Hey @Stefan_Schneider_WMDE @GoranSMilovanovic

thx again for testing and passion for the project. The Test-user names are all correct, but it seems in my case that I took the same cookie for different campaign tags. So I just made a (hopefully) last test for the following user names:

BorisB-Test-WMDE-2019-gender-2
BorisB-Test-WMDE-2019-ort-3
BorisB-Test-WMDE-2019-emailA-2

I would be pleased if you can check these user names and I hope we are ready to go :)

@Boris_Brunst_ext_WMDE The data might not be in Hadoop yet, I will take a look at them a bit later (tested just now: nothing).

@Boris_Brunst_ext_WMDE at this point, nothing for September 26 yet, this is everything that we have:

HOUR    DAY     CAMPAIGN                TEST_USER
20      14      WMDE_2019_emaila        Borrrrrris
20      15      WMDE_2019_emailb        Borrrrrrisb
23      10      WMDE_2019_emailb_ort    Borrrrisb ort
23      10      WMDE_2019_emailb_ort    Borrrrisb gender
23      10      WMDE_2019_emailb_ort    Borrrrisc
23      13      WMDE_2019_emailb_gender Testetetett
23      13      WMDE_2019_emailb_ort    Testtttteet
23      13      WMDE_2019_emailc        Testeteteeeettt
24      14      WMDE_2019_emaila        BorisB-Test-WMDE-2019-emailC-1
24      14      WMDE_2019_emaila        BorisB-Test-WMDE-2019-ort-1
24      14      WMDE_2019_emaila        BorisB-Test-WMDE-2019-gender-1
24      14      WMDE_2019_emaila        BorisB-Test-WMDE-2019-emailA-1
24      14      WMDE_2019_emaila        BorisB-Test-WMDE-2019-emailB-1

but let's give it some time, maybe the processing is not yet done.

@Boris_Brunst_ext_WMDE It seems like we are even making some profit from your tests, the data says we have a bit more than what would be expected from T232332#5526905:

26      14      WMDE_2019_emailb_ort    BorisB-Test-WMDE-2019-ort-3
26      14      WMDE_2019_emailb_ort    BorisB-Test-WMDE-2019-ort-2
26      14      WMDE_2019_emaila        BorisB-Test-WMDE-2019-emailA-2
26      14      WMDE_2019_emailb_gender BorisB-Test-WMDE-2019-gender-2

The BorisB-Test-WMDE-2019-ort-2 you did not mention :)

Hey thx @GoranSMilovanovic HAHA, the BorisB-Test-WMDE-2019-ort-2 seemed not to work so I had to switch the name..

Great, I think we are now finally on track. I will get back after we send out the Email.

@GoranSMilovanovic: For your information: due to several other issues we had to reschedule the campaign by short notice to Sunday, 13th October.

@Boris_Brunst_ext_WMDE Thanks for letting me know. What about the social media campaign (T232335)?

@GoranSMilovanovic The campaign was sent out yesterday morning. It would be great if you check the leads tomorrow late afternoon/ evening to finalize the reporting on Wednesday. THX

@Boris_Brunst_ext_WMDE

2019-10-13 pageviews:

WMDE_2019_emaila                           /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia 39 pageviews
WMDE_2019_emaila_kalendar                  /wiki/Wikipedia:Treffen_der_Wikipedianer             13 pageviews
WMDE_2019_emailb                           /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia 32 pageviews
WMDE_2019_emailb_gender                    /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia 6 pageviews
WMDE_2019_emailb_ort                       /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia 5 pageviews

2019-10-13 user registrations: none.


2019-10-14 pageviews:

WMDE_2019_emaila                           /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia 11 pageviews
WMDE_2019_emaila_kalendar                  /wiki/Wikipedia:Treffen_der_Wikipedianer             2 pageviews
WMDE_2019_emailb                           /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia 12 pageviews

2019-10-14 user registrations: 1 registration from WMDE_2019_emaila.

@GoranSMilovanovic THX !! Could you please check again for yesterday? And.. from which campaign did the registration come from?

@Boris_Brunst_ext_WMDE See change in T232332#5574455 in respect to

And.. from which campaign did the registration come from?

Could you please check again for yesterday?

The ETL procedure is now running.

@Boris_Brunst_ext_WMDE

2019-10-15 pageviews:

WMDE_2019_emaila                       /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 8 pageviews
WMDE_2019_emaila_kalendar              /wiki/Wikipedia:Treffen_der_Wikipedianer:             1 pageview
WMDE_2019_emailb                       /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 3 pageviews

2019-10-15 user registrations: 1 registration from WMDE_2019_emaila.

@Boris_Brunst_ext_WMDE

2019-10-16 pageviews:

WMDE_2019_emaila        /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia:              5 pageviews

2019-10-16 user registrations: none.


2019-10-17 pageviews:

WMDE_2019_emaila        /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia:              3 pageviews
WMDE_2019_emailb        /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia:              2 pageviews

2019-10-17 user registrations: none.

@Boris_Brunst_ext_WMDE

2019-10-18 pageviews

WMDE_2019_emaila              /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 1 pageview                                                                                    
WMDE_2019_emailb              /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 2 pageviews

2019-10-18 user registrations: None.

@GoranSMilovanovic
Hi Goran, for your information: we skipped the send out for template C and made another send out yesterday with the existing templates A and B again (identically to the prior send out with the same campaign tags, just to expand the target group..).

So we should please check the pageviews and user registrations ongoing for the period 20.10. 2019 - 25.10.2019.

Thank you in advance!!

@Boris_Brunst_ext_WMDE Got it. Here we go (Oct 19 and Oct 20):

2019-10-19 pageviews:

WMDE_2019_emailb              /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 1 pageview

2019-10-19 user registrations: none.


2019-10-20 pageviews:

WMDE_2019_emaila               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 37 pageviews
WMDE_2019_emaila_kalendar      /wiki/Wikipedia:Treffen_der_Wikipedianer:              7 pageviews
WMDE_2019_emailb               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 53 pageviews
WMDE_2019_emailb_gender        /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia:  5 pageviews
WMDE_2019_emailb_ort           /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia:  3 pageviews

2019-10-20 user registrations: 3 user registrations, all 3 from WMDE_2019_emailb.

2019-10-21 pageviews:

WMDE_2019_emaila               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 13 pageviews
WMDE_2019_emaila_kalendar      /wiki/Wikipedia:Treffen_der_Wikipedianer:              3 pageviews
WMDE_2019_emailb               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 7 pageviews
WMDE_2019_emailb_ort           /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia:  1 pageview

2019-10-21 user registrations: 2 user registrations, both from WMDE_2019_emaila.

@Boris_Brunst_ext_WMDE The latest numbers are here.

2019-10-22 pageviews:

WMDE_2019_emaila               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 8 pageviews
WMDE_2019_emaila_kalendar      /wiki/Wikipedia:Treffen_der_Wikipedianer:              2 pageviews
WMDE_2019_emailb               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 5 pageviews
WMDE_2019_emailb_gender           /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia:  1 pageview

2019-10-22 user registrations: None.

2019-10-23 pageviews:

WMDE_2019_emaila               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 1 pageviews
WMDE_2019_emailb               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 2 pageviews

2019-10-23 user registrations: None.

2019-10-24 pageviews:

WMDE_2019_emaila               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 2 pageviews
WMDE_2019_emailb               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 3 pageviews
WMDE_2019_emailb_ort           /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia:  1 pageview

2019-10-24 user registrations: 1 (one) from WMDE_2019_emailb.

2019-10-25 pageviews:

WMDE_2019_emaila               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 2 pageviews
WMDE_2019_emailb               /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 3 pageviews
WMDE_2019_emailc           /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia:  10 pageview

2019-10-25 user registrations: 1 (one) from WMDE_2019_emailb.

In addition to the data requested until Oct 25, I have noticed the following:

  • seven (7) user registrations from WMDE_2019_emailc and two (2) WMDE_2019_emailb on Oct 27;
  • one (1) user registration from WMDE_2019_emailc on Oct 28.

@GoranSMilovanovic Perfect, thx. There was another send out yesterday to get some feedbacks about the prior mailing. Seems to me some people used the reminder to signup by now. Great!

Please check again 29/10/19 and 30/10/19 before we are done and will close the ticket.

@Boris_Brunst_ext_WMDE Oct 28 user registrations:

WMDE_2019_emaila - 2 registrations
WMDE_2019_emailb - 1 registration
WMDE_2019_emailc - 2 registrations

@Boris_Brunst_ext_WMDE Oct 30 user registrations: one (1) from WMDE_2019_emailc. Do we need anything else here?

We are done :) I will close the ticket! Thank you for the good collaboration! Best...Boris

@GoranSMilovanovic
Hi Goran, could you maybe check again once in two weeks, if we had any further registrations? We don't need daily reports till then!

@Boris_Brunst_ext_WMDE Did we also track the edits of the newly registered users?

@Christine_Domgoergen_WMDE No problem:

  • the ticket is re-opened,
  • next update is on November 13, 2019 (two weeks after October 30 when the last report was delivered).

As of the user edits @Boris_Brunst_ext_WMDE we did not track them, but we can since we know the user IDs from their registrations.

@Christine_Domgoergen_WMDE Do we want this follow-up done for T232335 too?

@GoranSMilovanovic Great, thank you! And yes, please track the edits for both of the campaigns, also the follow-up would be good for the social media campaign as well.

@Christine_Domgoergen_WMDE @Boris_Brunst_ext_WMDE

  • there were six (6) new user registrations in November;
  • none of the campaign registered users made any edits in October or November 2019.

Closing the ticket.

@GoranSMilovanovic Great, thank you. One question: would it be a lot of effort to put all the data from this campaign in one spreadsheet? This is so much easier to work with for us...

@Christine_Domgoergen_WMDE The data sets are shared in a Google Spreadsheet + invite sent to you over email.