Page MenuHomePhabricator

A/B Test (Caterpiller): Analyze the results of the languages by article count test
Closed, ResolvedPublic4 Estimated Story Points

Description

This ticket is to capture the work needed to be done to analyze the results of the A/B test for the languages by article count test.

Please see the epic ticket for further information: https://phabricator.wikimedia.org/T131526

Event Timeline

debt triaged this task as Medium priority.May 31 2016, 8:29 PM
debt moved this task from Needs triage to Up Next on the Discovery-Analysis board.

This test has now been turned off - @mpopov it's all yours now! :)

mpopov moved this task from Backlog to In progress on the Discovery-Analysis (Current work) board.
mpopov set the point value for this task to 4.

As part of this analysis, I've released some additions to the BCDA package (https://github.com/bearloga/BCDA), namely making it better for reports: https://github.com/bearloga/BCDA#tabular-presentation-of-the-results

First draft of the report is up on GitHub now: https://github.com/wikimedia-research/Discovery-Portal-Test-SecondaryLinks/blob/master/report.pdf

If the draft jives with @debt & @TJones, I'll upload it to Commons. It's a pretty short one so reviewing shouldn't take too long :)

Hey @mpopov - this looks pretty good! I'll let @TJones and @Jdrewniak take a look at it as well before calling it completely done.

I've also uploaded a screen shot of the exec summary for those that don't want to click through and read the pdf.

The data gathered and analyzed shows that the cleaner page results in more actions taken, yay!

Screen Shot 2016-06-20 at 8.52.24 AM.png (626×744 px, 164 KB)

This looks really good to me. As always, all the included links are appreciated! The results seem pretty straightforward. We should do it!

looks good to me, the results are certainly compelling :)