Page MenuHomePhabricator
Create Task
Maniphest T143853

Wikipedia.org dashboard: Determine if new layout change caused any decreases
Closed, ResolvedPublic4 Estimated Story Points
Actions

Description

Per some comments in a thread on mediawiki.org, there is a suspicion that the deployment of the collapsed language links has decreased the page views to the Ukrainian Wikipedia as a result of some users browsers being configured to Russian, thereby burying the link to the Ukrainian Wikipedia. Our previous analysis suggests that this suspicion is incorrect, and the page views dashboard also does not seem to indicate there has been a drop, but we should confirm whether this is really the case using statistical analysis.

Hypothesis: deployment of collapsed links to wikipedia.org has significantly decrease page views to https://uk.wikipedia.org. Let's perform whatever analysis we need to to decide whether to reject or accept this hypothesis.

Note: We recently released this new dashboard page to view any changes in the language clickthroughs from the Wikipedia.org portal page (ticket T140816)

Related Objects
Search...

Event Timeline

• Deskana created this task.Aug 25 2016, 4:43 AM
Restricted Application added subscribers: Base, Aklapper. · View Herald TranscriptAug 25 2016, 4:43 AM
Ata subscribed.Aug 25 2016, 6:07 AM
Ijon awarded a token.Aug 25 2016, 6:23 AM
Antanana subscribed.Aug 25 2016, 8:07 AM
debt added a subtask: T140816: Wikipedia.org Portal Dashboard: add clicks by language.Aug 25 2016, 4:23 PM
• Deskana triaged this task as High priority.Aug 25 2016, 4:40 PM
• Deskana updated the task description. (Show Details)
• Deskana added a subscriber: debt.

Setting to high priority; this relates to a recent deployment, so should be done as soon as is possible. @debt, please triage accordingly. :-)

debt assigned this task to mpopov.Aug 25 2016, 8:29 PM
debt lowered the priority of this task from High to Medium.

We'll take a look but please be aware, that we only have about a week's worth of data to look at right not, as we only pushed the new portal page layout into production last week (Aug 16, 2016).

debt moved this task from Needs triage to Current work on the Discovery-Analysis board.Aug 25 2016, 8:30 PM
debt edited projects, added Discovery-Analysis (Current work); removed Discovery-Analysis.
Ijon subscribed.Aug 25 2016, 8:45 PM
mpopov set the point value for this task to 4.Aug 25 2016, 8:45 PM
mpopov moved this task from Backlog to In progress on the Discovery-Analysis (Current work) board.Aug 25 2016, 11:29 PM
debt closed subtask T140816: Wikipedia.org Portal Dashboard: add clicks by language as Resolved.Aug 26 2016, 4:14 PM
debt renamed this task from Determine if recent deployment to wikipedia.org caused decrease to Wikipedia.org dashboard: Determine if new layout change caused any decreases.Aug 26 2016, 4:25 PM
debt updated the task description. (Show Details)
mpopov added a subscriber: • chelsyx.Sep 7 2016, 6:23 PM

First draft up at: http://wikimedia-research.github.io/Discovery-Research-Portal/ukrainian/

@chelsyx Please review it when you have a chance. Thanks!

mpopov moved this task from In progress to Needs review on the Discovery-Analysis (Current work) board.Sep 7 2016, 6:23 PM
• Deskana added a comment.Sep 7 2016, 6:25 PM

Thanks, @mpopov!

In the executive summary, I assume that the "we do have sufficient evidence" is meant to be "we do not have sufficient evidence"? :-)

mpopov added a comment.Sep 7 2016, 6:39 PM
In T143853#2616163, @Deskana wrote:

Thanks, @mpopov!

In the executive summary, I assume that the "we do have sufficient evidence" is meant to be "we do not have sufficient evidence"? :-)

Whoops! Will fix. Thanks!

debt added a comment.Sep 8 2016, 5:59 PM

There's a table (Table 2) that has missing data (at least it looks like it) in the description of the table:

  1. The following control time series:
  2. The following control time series:

...and slightly disappointed that you used the dynamic time warping-matched pageview time series but didn't reference The B-52's Time Warp song. :-P

mpopov added a subscriber: TJones.Sep 9 2016, 8:32 PM

@chelsyx & @TJones: please review the new 2nd draft: http://wikimedia-research.github.io/Discovery-Research-Portal/ukrainian/

@Deskana & @debt: Thanks for pointing those mistakes out! I've fixed them in 2nd draft.

TJones added a comment.Sep 9 2016, 9:38 PM

@mpopov: comments sent in email.

• chelsyx added a comment.Sep 10 2016, 1:03 AM

@mpopov Looks great! Sent you my comments via email :)

mpopov moved this task from Needs review to In progress on the Discovery-Analysis (Current work) board.Sep 13 2016, 4:39 PM
mpopov added a comment.Sep 13 2016, 9:17 PM

3rd draft: http://wikimedia-research.github.io/Discovery-Research-Portal/ukrainian/

PDF version: https://github.com/wikimedia-research/Discovery-Research-Portal/blob/master/Analyses/Ukrainian%20Wikipedia/report.pdf

@TJones: getting the reference linking right like you suggested took some tinkering but I did it :D you were right, it is better this way.

@chelsyx, @debt, @Deskana, @TJones What do you think? Is this the final draft or does there need to be a 4th?

mpopov added a comment.Sep 13 2016, 9:19 PM

(I will remove the "3rd draft" subtitle if we agree that it's done, and I will make "Download PDF" into an actual link once the final draft PDF is uploaded on Commons.)

mpopov moved this task from In progress to Needs review on the Discovery-Analysis (Current work) board.Sep 13 2016, 9:19 PM
TJones added a comment.Sep 15 2016, 8:02 PM

Overall, it looks good. I like the breakout by Ukrainian/non-Ukrainian IPs in fig 2. The ref links were worth it! :)

Did Table 1 disappear? It seems to only be a header.

debt added a comment.Sep 15 2016, 11:34 PM

I've been struggling with this sentence:

"On 16 August 2016, the team deployed the final major patch (for the foreseeable future) wherein the links to Wikipedia..."

Can it be changed to (something like) this:

On 16 August 2016, the team deployed a major patch wherein the links to Wikipedia

I also can't find Table 1 and am also curious as to why the images that are referenced aren't near where we referenced them at (embedded rather than all at the bottom of the report)?

mpopov moved this task from Needs review to In progress on the Discovery-Analysis (Current work) board.Sep 16 2016, 6:21 PM
mpopov added a comment.Sep 19 2016, 5:27 PM

Final draft: http://wikimedia-research.github.io/Discovery-Research-Portal/ukrainian/ & https://commons.wikimedia.org/wiki/File:The_Wikipedia.org_Portal_and_Ukrainian_Wikipedia.pdf

mpopov moved this task from In progress to Done on the Discovery-Analysis (Current work) board.Sep 19 2016, 5:28 PM
debt added a comment.Sep 19 2016, 11:05 PM

Thanks! I've added the links to the Portal page for Analysis work.

debt closed this task as Resolved.Sep 23 2016, 9:12 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL