⚓ T203954 Inspect the Autumn Banner Campaign 2018 proposed User Journey

GoranSMilovanovic created this task.Sep 10 2018, 12:01 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 10 2018, 12:01 PM

@Stefan_Schneider_WMDE
The analytical schema of the campaign and my comments are placed in the User Journey doc.

@GoranSMilovanovic Sry Goran, I'm quite full right now. You'll get feedback tomorrow!

@GoranSMilovanovic I just went through the document. It really made the process a lot clearer. I commented everything and hope to clarify last questions there.

@Stefan_Schneider_WMDE

Pageviews and campaign tags tested:

testResults.csv25 KBDownload

Test query:

USE wmf;
  SELECT uri_path, uri_query, referer FROM webrequest
  WHERE uri_host = 'de.wikipedia.org'
  AND (uri_path = '/wiki/Wikipedia:Wikipedia_vor_Ort_2018' OR uri_path = '/wiki/Wikipedia:Wikipedia_vor_Ort_2018/Wien' OR uri_path = '/wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia' OR uri_path = '/wiki/Spezial:Benutzerkonto_anlegen') 
  AND ((year = 2018 AND month = 09 AND day = 30 AND (hour = 22 OR hour = 23)) OR (year = 2018 AND month = 10 AND day = 01 AND (hour >= 0 OR hour <= 23)) 
  OR (year = 2018 AND month = 10 AND day = 02 AND (hour >= 0))) ;

In other words: collect all pageviews of the desired pages beginning from October 1st 00:00 CEST.

Scroll down the table and look into the uri_query column to check whether your campaign tags are in place.

Checking for test user registration now.

@Stefan_Schneider_WMDE User registration test completed:

one test user with event_userName = 'Testtttet' found;
the event_campaign field for that user has the value of: 'WMDE_neweditors_autumn_2018_lp1'.

@GoranSMilovanovic Thanks for checking. I'm not quite sure, I can understand the table properly and ask for your help again: I would need an explicit answer to the orange questions in the Doc to each User Journey. For now I cannot answer that questions with your table.

Did you find any other registration with the campaign tag ?campaign=WMDE_neweditors_autumn_2018_lpn and would you find the registration also withour the username given - only with the tags?

@GoranSMilovanovic What I see from my perspective (but I'm not so much technically competent)

I do not see any referrer, that shows from which banner the people on the registration page came from. Just the LerneWikipedia-page. Do you see something different in the registration data?
I do see, that the people on the different local pages came from banner or newsletter, because the campaign tag is given in the referrer.
Same for LerneWikipedia page: the referrer shows from where they came from.

So from my perspective the only question is, if we can guarantee that the registration contains the campaign tag from the banner of newsletter. Then the journey would be trackable.

What do you say?

GoranSMilovanovic moved this task from New Editors/Campaigns to Prioritized on the User-GoranSMilovanovic board.Oct 4 2018, 9:52 AM

Re-checking and getting back to you. I think we would need to get @kai.nissen involved in the discussion then.

@Stefan_Schneider_WMDE @kai.nissen

I have re-checked the campaign tags for the Autumn Banner Campaign 2018, beginning from Oct 1.

The following table should help you answer your questions @Stefan_Schneider_WMDE. There goes the table and then the explanation:

Crosstabs_testABC2018.xlsx5 KBDownload

The uri_path column: the page at de.wikipedia.org that we want to take a look at, for example: /wiki/Spezial:Benutzerkonto_anlegen, /wiki/Wikipedia:Wikimedia_Deutschland/LerneWikipedia, /wiki/Wikipedia:Wikipedia_vor_Ort_2018, etc.
The uri_quert column (NOTE: here is where you are looking for the campaign tags): the query passed onto the respective page (in the same row of the table), for example: ?campaign=WMDE_2018_sprbt2 (not interesting for us in the context of the Autumn Banner Campaign 2018), or ?campaign=WMDE_neweditors_autumn_2018_lpn (interesting for us in this context). All queries from the current campaign are in put bold typeface in the table.
The Freq column: how many times in the dataset do we obtain a particular combination of the uri_path and uri_query values?

Now, as of your questions, this is what you need to do in order to obtain the answers:

for every page that you list in the document, you need to know what query (or queries) you expect to obtain; only you and people who are working on the campaign setup can know that because that is the part of the campaign's user journey design;
if the respective query (or queries) where properly matched to the respective pages, having in mind the design of the user journey that you want to implement, and then tested, in the attached table you must find that the exact combination of the page (uri_path) and query (uri_query) must have the value of Freq larger than zero, which means: someone has visited that particular page carrying that particular campaign tag that you expect to see there.

I hope this helps. I guess @kai.nissen can provide additional technical details. Also, I would need an additional meeting with both of you because I am myself not sure if I understand exactly what user journey are you designing here after following the developments on T205720 :)

@Stefan_Schneider_WMDE @kai.nissen

I can confirm that the registrations for both tested campaign tags can be obtained from the relevant schema now.

Query:

select * from log.ServerSideAccountCreation_17719237 
  where (
  (webHost = 'de.wikipedia.org') and 
  (timestamp >= 20181001000000)
  and (event_campaign = 'WMDE_neweditors_autumn_2018_lpn' or event_campaign = 'WMDE_neweditors_autumn_2018_lp1')
  );

Result:

+---------+----------------------------------+------+----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------+---------------------------------+---------------------+-------------+------------------+----------------+----------------+---------------------+-------------+-------------------+--------------+----------------+
| id      | uuid                             | dt   | timestamp      | userAgent                                                                                                                                                                                                                        | webHost          | wiki   | event_campaign                  | event_displayMobile | event_isApi | event_isSelfMade | event_isStable | event_returnTo | event_returnToQuery | event_token | event_userBuckets | event_userId | event_userName |
+---------+----------------------------------+------+----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------+---------------------------------+---------------------+-------------+------------------+----------------+----------------+---------------------+-------------+-------------------+--------------+----------------+
| 2476492 | 5736ecd0b3305fd1a3ce6a955376cc70 | NULL | 20181001130705 | {"wmf_app_version": "-", "os_minor": null, "os_major": null, "is_bot": false, "device_family": "Other", "os_family": "Ubuntu", "browser_minor": "0", "is_mediawiki": false, "browser_major": "69", "browser_family": "Chromium"} | de.wikipedia.org | dewiki | WMDE_neweditors_autumn_2018_lp1 |                   0 |           0 |                1 | NULL           | NULL           | NULL                |             |                   |      3027206 | Testtttet      |
| 2585662 | ed00feaa2cfb5526a86dcda164f2c059 | NULL | 20181010125822 | {"wmf_app_version": "-", "os_minor": null, "os_major": null, "is_bot": false, "device_family": "Other", "os_family": "Ubuntu", "browser_minor": "0", "is_mediawiki": false, "browser_major": "69", "browser_family": "Chromium"} | de.wikipedia.org | dewiki | WMDE_neweditors_autumn_2018_lpn |                   0 |           0 |                1 | NULL           | NULL           | NULL                |             |                   |      3034687 | Testttstststst |
+---------+----------------------------------+------+----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------+---------------------------------+---------------------+-------------+------------------+----------------+----------------+---------------------+-------------+-------------------+--------------+----------------+

Both test users Testtttet (campaign tag: WMDE_neweditors_autumn_2018_lp1) and Testttstststst (campaign tag: WMDE_neweditors_autumn_2018_lpn) are in.

As of the test registrations that you've made during our call: I guess it will take some time until they can be found in the relevant log database schema, however, after this test there is no reason to believe that they will not get there.

@GoranSMilovanovic Thx for testing! Good, that it's working. Kai also created one account with lpnTag and one account with lp1Tag during our meeting. Can find them too?

@Stefan_Schneider_WMDE Not yet; I guess we will have to wait some time for these two to show up in the table.

@GoranSMilovanovic Thx! Awesome, that it works :)

@Stefan_Schneider_WMDE @kai.nissen

Warning: two accounts that were registered by Kai during our discussion earlier today are not found in log.ServerSideAccountCreation_17719237 (the place where user registrations live).

@kai.nissen Kai, could you please let me know what campaign tags do we expect to find in relation to the two test registrations that you have made during our discussion with @Stefan_Schneider_WMDE yesterday? Thank you. If they are 'lp1' or 'lpn' - your test user registrations did not hit the log database.

@GoranSMilovanovic I used the actual campaign tags, so you should have found them. Can you look up user names? If so, please check users Tester2525, Tester2526 and Tester2527 for campaign tags.

@kai.nissen

Very nice, all three found:

| id      | uuid                             | dt   | timestamp      | userAgent                                                                                                                                                                                                                        | webHost          | wiki   | event_campaign                     | event_displayMobile | event_isApi | event_isSelfMade | event_isStable | event_returnTo | event_returnToQuery | event_token | event_userBuckets | event_userId | event_userName |
| 2585668 | 7e2f64b7b2ee50389336aa38a20c37d5 | NULL | 20181010130109 | {"wmf_app_version": "-", "os_minor": null, "os_major": null, "is_bot": false, "device_family": "Other", "os_family": "Ubuntu", "browser_minor": "0", "is_mediawiki": false, "browser_major": "69", "browser_family": "Chromium"} | de.wikipedia.org | dewiki |                                    |                   0 |           0 |                0 | NULL           | NULL           | NULL                |             |                   |      3034691 | Tester2525     |
| 2585672 | 1c044706c47051d2ab4378c589eafa9b | NULL | 20181010130235 | {"wmf_app_version": "-", "os_minor": null, "os_major": null, "is_bot": false, "device_family": "Other", "os_family": "Ubuntu", "browser_minor": "0", "is_mediawiki": false, "browser_major": "69", "browser_family": "Chromium"} | de.wikipedia.org | dewiki | B18WMDE_neweditors_autumn_2018_lpn |                   0 |           0 |                1 | NULL           | NULL           | NULL                |             |                   |      3034693 | Tester2526     |
| 2585708 | 1d15016a9cda5e95b82c68a84ef1eed4 | NULL | 20181010130412 | {"wmf_app_version": "-", "os_minor": null, "os_major": null, "is_bot": false, "device_family": "Other", "os_family": "Ubuntu", "browser_minor": "0", "is_mediawiki": false, "browser_major": "69", "browser_family": "Chromium"} | de.wikipedia.org | dewiki | B18WMDE_neweditors_autumn_2018_lp1 |                   0 |           0 |                1 | NULL           | NULL           | NULL                |             |                   |      3034695 | Tester2527     |

Note. Campaign tags:

none recorded for Tester2525
B18WMDE_neweditors_autumn_2018_lpn for Tester2526
B18WMDE_neweditors_autumn_2018_lp1 for Tester2527.

What happened during the Tester2525 test user registration?

@GoranSMilovanovic Thank you Goran - that looks good!

@GoranSMilovanovic
Thanks a lot! Apparently I was using a wrong prefix (B18WMDE_ instead of WMDE_). Sorry for that.

By the way, Tester2525 was the user I registered when I was already logged in.

GoranSMilovanovic moved this task from Prioritized to New Editors/Campaigns on the User-GoranSMilovanovic board.Oct 11 2018, 1:04 PM

@Stefan_Schneider_WMDE Should we close this one as resolved?

@GoranSMilovanovic Yes :) Thx for reminding me.

Inspect the Autumn Banner Campaign 2018 proposed User Journey
Closed, ResolvedPublic
Actions

Description

Related Objects

Event Timeline

	F26274507: testResults.csv
	Oct 2 2018, 11:03 AM

Inspect the Autumn Banner Campaign 2018 proposed User JourneyClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Inspect the Autumn Banner Campaign 2018 proposed User Journey
Closed, ResolvedPublic
Actions