This ticket will be for the Discovery-Analysis team to verify that the test is progressing as we want it to be, once it has been pushed into production.
Please refer to the epic for full details: https://phabricator.wikimedia.org/T131238
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • debt | T133732 Release to production (Egg): adding descriptive text to sister project links | |||
| Resolved | • debt | T131238 [EPIC] Wikipedia.org A/B Test (Egg): add descriptive text to sister project links | |||
| Resolved | mpopov | T132519 A/B Test (Egg): Check that all is well with the addition of descriptive text to sister project links |
Event Timeline
Comment Actions
USE log;
SELECT
cohort, accept_lang_has_english,
COUNT(1) AS sessions,
SUM(events) AS total_events
FROM (
SELECT
event_session_id AS session_id, event_cohort AS cohort,
CASE WHEN INSTR(event_accept_language, 'en') THEN 'yes'
ELSE 'no' END AS accept_lang_has_english,
COUNT(1) AS events
FROM WikipediaPortal_14377354
WHERE LEFT(timestamp, 8) = '20160504'
AND event_cohort IN('descriptive-text-a', 'descriptive-text-b')
GROUP BY session_id, cohort, accept_lang_has_english
) AS sessions
GROUP BY cohort, accept_lang_has_english;| cohort | accept_lang_has_english | sessions | total_events |
| descriptive-text-a | yes | 1342 | 2670 |
| descriptive-text-b | yes | 1326 | 2658 |
But if we look at number of sessions with recorded 'other projects' clicks…
SELECT
cohort, accept_lang_has_english,
COUNT(1) AS sessions,
SUM(events) AS total_events
FROM (
SELECT
event_session_id AS session_id, event_cohort AS cohort,
CASE WHEN INSTR(event_accept_language, 'en') THEN 'yes'
ELSE 'no' END AS accept_lang_has_english,
COUNT(1) AS events
FROM WikipediaPortal_14377354
WHERE LEFT(timestamp, 8) = '20160504'
AND event_cohort IN('descriptive-text-a', 'descriptive-text-b')
AND event_section_used = 'other projects'
GROUP BY session_id, cohort, accept_lang_has_english
) AS sessions
GROUP BY cohort, accept_lang_has_english;| cohort | accept_lang_has_english | sessions | total_events |
| descriptive-text-a | yes | 4 | 4 |
| descriptive-text-b | -- | 0 | 0 |
I'm guessing that since the user has to scroll down quite a bit to even see the change (if they're even enrolled in the A/B test and have been put into the test bucket), we're probably not going to see a lot of clicks from that section (if any!). Hopefully there will be enough to see some impact, and, like, it's still too early to say, but maybe this is something we should have done after collapsing languages.
Comment Actions
Thanks, @mpopov - we knew that this region isn't high in clickthroughs. Let's see what the rest of the test week brings us. :)