There was a significant drop in KaiOS web pageviews during November 2019, with the daily average dropping from ~700 k to ~350 k over the course of several weeks.
Let's figure out why this happened.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | nshahquinn-wmf | T262810 Understand decline in KaiOS web traffic | |||
| Resolved | nshahquinn-wmf | T242853 Investigate significant drop in KaiOS pageviews |
Event Timeline
Comment Actions
@AMuigai, this seems important to look into and I will get to it as soon as I can; however, I won't be working Tuesday–Friday next week and will be at All Hands the following week. Is it okay if this waits until the week of 3 Feb? If not, you can email my team (product-analytics@wikimedia.org) and @kzimmerman can try to find someone to work on it next week.
Comment Actions
Adding some initial thoughts from an email I wrote to Angie:
That is a very strange drop indeed. It doesn't really look like it's related to the big reclassification from FirefoxOS to KaiOS; that happened on 22 November, but this decline happened gradually between 17 November and 26 November. Also, although the countries other than India only play tiny roles, it looks like they didn't see similar declines, which means it could be an India-specific issue.
I thought it might be related to the internet shutdowns the government has been using to suppress the protests against the Citizenship Amendment Act, but overall India pageviews show little impact from those and in any case they started in early December so the timing doesn't line up either. It looks like Jio (along with India's other big mobile ISPs) raised prices a fair bit in December, but apparently those went into effect in early December so again the timing doesn't fit.
Comment Actions
I think the aggreggation you are looking at might be missleading, The drop seems more gradual that your graph indicates. Please take a look to pageview_hourly (same dataset with higher granularity for the last 3 months)
If you split by referrer you can see is the traffic via search engine who is affected the most (slope of decay is more pronounced). So this might be just an artifact of KaiOS serach results
There is also a drop in FF mobile externally referred traffic overall:
Comment Actions
I shared my results with @AMuigai in our sync meeting today; my analysis is available at https://github.com/wikimedia-research/KaiOS-readership/blob/master/2019-11-pageviews-drop.ipynb.
From that notebook:
This drop seems to be very specific to KaiOS, rather than a broader trend within India (the overwhelming majority of KaiOS users are in India). Within KaiOS, the drop is consistent across Indian states. However, within KaiOS pageviews search-engine referred pageviews dropped significantly more than pageviews from other referrer types (in Indian mobile pageviews generally, pageviews of each referrer type were stable). Although internally-referred pageviews dropped too, the ratio of internal to external pageviews actually rose, suggesting that the decline in internally-referred pageviews is just an effect of the drop in search-engine referred pageviews.
All this suggests that something specific to KaiOS changed in mid-November that decreased how often KaiOS users visited Wikipedia through search engine results. Mostly likely, this was a software change that decreased how often those users searched the web. Less likely explanations are a decrease in the number of KaiOS users or the amount of phone usage by each of those users (such behavior changes probably would not have happened so sharply) and a decrease in Wikipedia's positioning in Google results or a decrease in clickthrough on those results (overall search engine referrals in India probably would have decreased too).