Page MenuHomePhabricator

Create Tracking Report: Social Media Campaign Type-Test 2019
Closed, ResolvedPublic

Description

We will setup a social media campaign to promote a type-test:

Landing page link 1
https://wikimedia.de/mitmachen/

People who passed the type-test will be asked to sign up at wikipedia and follow a link to..

Landing page link 2
https://de.wikipedia.org/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia

Campaign tag
There will be up to 10 different social media banners. They all link to the landing page 1.

New Editors Landing Page and should contain the following campaign tag:
?pk_campaign=WMDE_2019_smc

Campaign NoCampaign NameCampaign Tag
1DokumentatorInWMDE_2019_smc_fb1
2Star-AutorInWMDE_2019_smc_fb2
3Technik-OptimiererInWMDE_2019_smc_fb3
4KulturagentInWMDE_2019_smc_fb4
5SpenderInWMDE_2019_smc_fb51
WMDE_2019_smc_fb52
6StrippenzieherInWMDE_2019_smc_fb6
7Facebook AllgemeinWMDE_2019_smc_fb71
WMDE_2019_smc_fb72
8Facebook Allgemein LookalikeWMDE_2019_smc_fb8
9Instagram FeedWMDE_2019_smc_ig9
10Instagram StoryWMDE_2019_smc_ig10

Landing page including the campaign tag
https://wikimedia.de/mitmachen/?pk_campaign=WMDE_2019_scm

Please find further campaign information here.

  1. Campaign Flight: 01.10.2019 - 06.10.2019
  2. Campaign Flight: 16.10.2019 - 27.10.2019

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 9 2019, 1:28 PM

Hi @Boris_Brunst_ext_WMDE. Can you please associate at least one active project with this task (via the Add Action...Change Project Tags dropdown). Maybe this is about WMDE-Fundraising-Tech ? This will allow others to get notified and see this task when looking at the corresponding project workboard. Thanks!

@Aklapper Thanks for noticing this.

@Boris_Brunst_ext_WMDE I have associated two projects (User-GoranSMilovanovic + WMDE-Analytics-Engineering) to the ticket.
Please let me know as soon as the tracking for the campaigns is in place so that we can begin testing. Thanks!

@Stefan_Schneider_WMDE I think we should meet:

  • I do not know who will be setting up the tracking for this campaign, while
  • I would need to consult with that person to
    • (1) make sure all the details are clear,
    • (2) figure out where will the Matomo data be found (if Matomo is to be used with this campaign),
    • and (3) organize the tests as we do for each campaign.
Boris_Brunst_ext_WMDE updated the task description. (Show Details)

@Boris_Brunst_ext_WMDE @Stefan_Schneider_WMDE

Campaign start: 29.09.2019

(from the ticket description).

So: did this campaign start, or not yet? Nothing was tested in relation to it.

Hi @GoranSMilovanovic, the Social Media campaign will start tomorrow, October 1.

We have Matomo installed on the landing page and will use Matomo for the tracking of the different social media banners.
For the transfer from Landing Page 1 to Landing Page 2 we will use the existing campaign tags "WMDE_2019_staa" and "WMDE_2019_strz" which have been in use before. So we should be safe regarding the tracking.

GoranSMilovanovic added a comment.EditedSep 30 2019, 3:10 PM

@Boris_Brunst_ext_WMDE @Stefan_Schneider_WMDE @WMDE-leszek

Please note:

  • nothing in this campaign has been tested, while it is about to start tomorrow and thus it will be somewhat risky;
  • As of the required data section in the campaign specification doc:
    • Performance of each creative (Facebook ads and Instagram ads) in terms of views, impressions, conversion rate.
    • Visits of the page on a daily basis
    • Visits of the page in total
    • Link-Clicks (first landing page Button “Jetzt testen”) on a daily basis
    • Link-Clicks (first landing page Button “Jetzt testen”) in total
    • How many people start the test?:

I will not be able to address any these questions without access to the Matomo data, and at this point I do not know how will these data be delivered.

Also:

For the transfer from Landing Page 1 to Landing Page 2 we will use the existing campaign tags "WMDE_2019_staa" and "WMDE_2019_strz"

What is the difference between these two tags in the context of this campaign?

And finally:

What tags will be encountered at the user registration?

@Boris_Brunst_ext_WMDE @Stefan_Schneider_WMDE
Please do not start online campaigns without any previous tests. In the previous days we have spent some testing the WMDE 2019 E-mail campaign (see: T232332) only to learn that it is going to be postponed until mid-October, while this campaign starts tomorrow and untested. It is not safe. We might not be getting the analytics timely from this campaign because of the absence of tests. Thank you.

Nota bene. I will not be available tomorrow 15:30 - 21:00 Berlin/Belgrade time. Please try to answer to my questions (Matomo data sets, user registration tags, the role of WMDE_2019_staa and WMDE_2019_strz tags in this campaign) until 21:00. I will then start testing this campaign with whatever data are collected until then. Thank you.

@GoranSMilovanovic

Thank you for your message. We just moved the landing page today so there was no chance to test earlier. Since we are working with Matomo we are able to track most of the addressed tracking goals easily:

Performance of each creative (Facebook ads and Instagram ads) in terms of views, impressions, conversion rate.

--> these data are delivered by Facebook Admanager

Visits of the page on a daily basis

--> these data are provided by Matomo

Visits of the page in total

--> these data are provided by Matomo

Link-Clicks (first landing page Button “Jetzt testen”) on a daily basis
Link-Clicks (first landing page Button “Jetzt testen”) in total
How many people start the test?:

--> these data are provided by Matomo as we can see how many people started and passed the test.

I checked every campaign tag with Matomo today.

For the transfer from Landing Page 1 to Landing Page 2 we will use the existing campaign tags "WMDE_2019_staa" and "WMDE_2019_strz"
What is the difference between these two tags in the context of this campaign?

The two campaign tags link from different subsites to Landing Page 2. So we will be able to see which registration belongs to which subsite and therefore to which banner.

@GoranSMilovanovic Do you have a Matomo Wikimedia LogIn?

Best, Boris

GoranSMilovanovic added a comment.EditedSep 30 2019, 4:14 PM

@Boris_Brunst_ext_WMDE

Do you have a Matomo Wikimedia LogIn?

No. In fact, I've never used Matomo in my life before.

For the transfer from Landing Page 1 to Landing Page 2 we will use the existing campaign tags "WMDE_2019_staa" and "WMDE_2019_strz"
What is the difference between these two tags in the context of this campaign?

The two campaign tags link from different subsites to Landing Page 2. So we will be able to see which registration belongs to which subsite and therefore to which banner.

So, we have 10 campaign tags in the task description, but then only two subsites and their corresponding tags: how will we reverse engineer the 10 campaign tags from the 2 observed tags? I am not sure if I understand the approach.

Also:

What tags will be encountered at the user registration?

So, we have 10 campaign tags in the task description, but then only two subsites and their corresponding tags: how will we reverse engineer the 10 campaign tags from the 2 observed tags? I am not sure if I understand the approach.

@GoranSMilovanovic Yes, the campaign works like that:
There are 10 different banners that link to the website and the user test. There are 9 different possible results for the test. But only two test results link to Landing Page 2 (..and are relevant for user registration). We use the two mentioned campaign tags for the registrations.

The first campaign flight is now running till this Sunday, 6.10.2019. Afterwards we will take a look at the numbers and decide how to optimize the campaign setting for a second flight.

@Boris_Brunst_ext_WMDE @Stefan_Schneider_WMDE

Pageviews test, from incomplete 2019/10/01 data:

"","Tag","Page","Pageviews","date","campaign"
"1","WMDE_2019_staa","/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia",2,"2019-10-01","2019_SocialMedia"
"2","WMDE_2019_strz","/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia",3,"2019-10-01","2019_SocialMedia"

which means:

  • 2 pageviews of the LerneWikipedia page from WMDE_2019_staa, and
  • 3 pageviews of the LerneWikipedia page from WMDE_2019_strz.

User registration test, from incomplete 2019/10/01 data:
(NOTE. Using event.serversideaccountcreation in hdfs as @Ladsgroup suggested to @Stefan_Schneider_WMDE in the Email campaign test, see: T232332#5523890)

  • No user registrations from WMDE_2019_staa or WMDE_2019_strz yet.

@Boris_Brunst_ext_WMDE As far as I understand all the remaining data should come through Matomo, right?
Correct me if I am wrong, this is what we expect from Matomo (in reference to the campaign doc):

  • pageviews of the https://wikimedia.de/mitmachen/ page (with tags from different social media banners, I guess),
  • impressions of the https://wikimedia.de/mitmachen/ page,
  • link-clicks (first landing page Button “Jetzt testen”),
  • how many people start the test, and
  • how many people pass the full test?

Please confirm or correct me - it is very important for me to understand the organization of the data at this point. Thanks a lot!

Hi @GoranSMilovanovic

thank you for the analysis of the first campaign day!

Yes, we find all these information on Matomo and the Wordpress CMS! Also, there is an overview for the link-clicks to the landing page. Here we have:

2 pageviews of the LerneWikipedia page from WMDE_2019_staa, and

Yes, we also track 2 link-clicks

3 pageviews of the LerneWikipedia page from WMDE_2019_strz.

We count 4 link-clicks :)

The campaign runs till Sunday night. I think we are fine if we then take a look afterwards at the analysis. We will start the second campaign flight not before the 15./16.10.
(I will be away the next week from 7.10. - 12.10. and will take a deeper look on the analysis latest on Monday 14.10.)

@Boris_Brunst_ext_WMDE

Complete 10/01/2019 data:

  • 4 pageviews for WMDE_2019_staa and also 4 pageviews for WMDE_2019_strz;
  • no user registrations yet found in the event.serversideaccountcreation hdfs table.
GoranSMilovanovic added a comment.EditedOct 3 2019, 11:47 AM

Complete 10/02/2019 data:

  • 7 pageview for WMDE_2019_staa and 2 pageviews for WMDE_2019_strz;
  • no user registrations yet found in the event.serversideaccountcreation hdfs table.
GoranSMilovanovic added a comment.EditedOct 6 2019, 6:50 PM

Complete 10/03/2019 data:

  • 4 pageviews for WMDE_2019_staa and 8 pageviews for WMDE_2019_strz;
  • one (1) user registration from WMDE_2019_strz found in the event.serversideaccountcreation hdfs table.

Complete 10/04/2019 data:

  • 6 pageviews for WMDE_2019_staa and 7 pageviews for WMDE_2019_strz;
  • no user registrations found in the event.serversideaccountcreation hdfs table.

Complete 10/05/2019 data:

  • 2 pageviews for WMDE_2019_staa and 11 pageviews for WMDE_2019_strz;
  • one (1) user registration from WMDE_2019_strz found in the event.serversideaccountcreation hdfs table.

Complete 10/06/2019 data:

  • 3 pageviews for WMDE_2019_staa and 6 pageviews for WMDE_2019_strz;
  • no new user registration found in the event.serversideaccountcreation hdfs table.

Complete 10/07/2019 data:

  • no pageviews for WMDE_2019_staa and 2 pageviews for WMDE_2019_strz;
  • no new user registration found in the event.serversideaccountcreation hdfs table.

Complete 10/08/2019 data:

  • 1 pageview for WMDE_2019_staa and no pageviews for WMDE_2019_strz;
  • no new user registration found in the event.serversideaccountcreation hdfs table.
GoranSMilovanovic added a comment.EditedOct 9 2019, 11:58 AM

Complete 10/09/2019 data:

  • no pageviews for WMDE_2019_staa and no pageviews for WMDE_2019_strz;
  • no new user registration found in the event.serversideaccountcreation hdfs table;
  • to be re-assessed until the end of the day.

Hi @GoranSMilovanovic,

thank you for the overview. As stated before the first campaign flight ended last Sunday. There have been just two registrations, correct?

Best, BB

@Boris_Brunst_ext_WMDE

Yes, only two registrations, and that would be the following:

year    month   day     hour    campaign        
2019    10      3       13      WMDE_2019_strz
2019    10      5       13      WMDE_2019_strz

@GoranSMilovanovic
Hi Goran, for your information: the second campaign flight started last Wednesday, 16.10.2019 and runs till 27.10.2019 with the same campaign tags and banners as before.
Best...Boris

@Boris_Brunst_ext_WMDE

2019-10-16 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 2 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 2 pageviews

2019-10-16 user registrations: 1 registration from WMDE_2019_staa.


2019-10-17 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 5 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 8 pageviews

2019-10-17 user registrations: 1 registration from WMDE_2019_staa.


2019-10-18 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 2 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 11 pageviews

2019-10-18 user registrations: 1 registration from WMDE_2019_strz.


2019-10-19 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 6 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 11 pageviews

2019-10-19 user registrations: none.


2019-10-20 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 6 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 12 pageviews

2019-10-20 user registrations: none.


2019-10-21 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 9 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 9 pageviews

2019-10-21 user registrations: 1 registration from WMDE_2019_staa.

@Boris_Brunst_ext_WMDE Here are the latest numbers. Please let me know what are the next steps in analytics for this campaign. Thanks.

2019-10-22 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 3 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 15 pageviews

2019-10-22 user registrations: None.


2019-10-23 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 2 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 5 pageviews

2019-10-23 user registrations: None.


2019-10-24 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 3 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 15 pageviews

2019-10-24 user registrations: None.


2019-10-25 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 2 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 15 pageviews

2019-10-25 user registrations: None.


2019-10-26 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 3 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 5 pageviews

2019-10-26 user registrations: None.


2019-10-27 pageviews:

WMDE_2019_staa     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 1 pageviews
WMDE_2019_strz     /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia: 20 pageviews

2019-10-27 user registrations: None.


@GoranSMilovanovic Thank you for the update. The campaign ended yesterday and we are here done for now.

@Boris_Brunst_ext_WMDE Shall we close this ticket then?

Boris_Brunst_ext_WMDE closed this task as Resolved.Oct 28 2019, 10:23 AM
GoranSMilovanovic reopened this task as Open.Nov 4 2019, 12:23 PM

@Christine_Domgoergen_WMDE

  • the ticket is re-opened,
  • next update is on November 13, 2019 (two weeks after October 30 when the last report was delivered);
  • user edits will be tracked too.
GoranSMilovanovic closed this task as Resolved.Nov 14 2019, 12:59 PM

@Boris_Brunst_ext_WMDE @Christine_Domgoergen_WMDE

No changes here:

  • no new user registrations;
  • the registered users did not make any edits.

Closing the ticket.