The December 2015–March 2016 level of mobile app edits was fairly stable around 100 K to 110 K. However, it dropped to 87 K in April and 67 K in May. The number so far in June (50 K on 21 June) extrapolates to 71 K.
What's going on?
The December 2015–March 2016 level of mobile app edits was fairly stable around 100 K to 110 K. However, it dropped to 87 K in April and 67 K in May. The number so far in June (50 K on 21 June) extrapolates to 71 K.
What's going on?
Hmm, the decline seems to be evenly split between the Android and iOS apps, which isn't consistent with some sort of regression preventing users from editing. Perhaps they switched to sampling the editing events?
SELECT LEFT(timestamp, 6) as "month", COUNT(*) as "mobile app edits", SUM(IF(userAgent LIKE "%(iPhone%", 1, 0)) as "iOS edits", SUM(IF(userAgent LIKE "%(Android%", 1, 0)) as "Android edits" FROM log.MobileWikiAppEdit_9003125 WHERE event_action = "saved" AND wiki LIKE "%wiki" AND wiki NOT IN ( "commonswiki", "wikidatawiki", "metawiki", "loginwiki", "mediawikiwiki", "sourceswiki", "specieswiki", "testwiki", "test2wiki", "testwikidatawiki", "incubatorwiki", "advisorywiki", "outreachwiki", "qualitywiki", "strategywiki", "usabilitywiki", "collabwiki", "donatewiki", "execwiki", "officewiki", "legalteamwiki", "movementroleswiki", "internalwiki", "auditcomwiki", "boardgovcomwiki", "boardwiki", "chairwiki", "chapcomwiki", "checkuser_wiki", "foundationwiki", "grantswiki", "iegcomwiki", "nostalgiawiki", "ombudsmenwiki", "otrs_wikiwiki", "searchcomwiki", "spcomwiki", "stewardwiki", "tenwiki", "transitionteamwiki", "votewiki", "wg_enwiki", "zerowiki", "checkuserwiki", "fdcwiki" ) AND wiki NOT LIKE "wikimania%" AND wiki NOT LIKE "arbcom%" AND event_pageNS IS NULL AND timestamp BETWEEN "201601" AND "201607" GROUP BY month; month mobile app edits iOS edits Android edits 201601 111921 19632 92287 201602 102102 19779 82323 201603 101795 11995 89799 201604 87315 5538 81776 201605 66800 5015 61785 201606 50020 2919 47101
@ori: They haven't deployed a new version of the same schema, but yeah, it's possible that they've done something more drastic like removing the logging of edit events altogether.
@JMinor , @Dbrant, do you know of any changes to the logging of edit events from the mobile apps in the past couple of months?
I just talked to @JMinor. There hasn't been any intentional change to the logging; the only other thing he could think of was an Nginx bug two months ago which made it appear that edits hadn't saved properly and affected both apps. However, it was promptly fixed, so it doesn't seem a likely candidate.
@Dbrant, I couldn't find the task for that bug. Could you point me to it?
It also looks like the logging code hasn't been touched either in the iOS app or in the Android app.
Here is the task that was related to the nginx issue that prevented users from saving edits: T134758 However, this issue only surfaced in May, and wouldn't explain the drop starting in April or even earlier.
My best theory so far would be as follows:
It's possible (but unlikely) that the issue started before April, but we weren't made *aware* of it until May. And even though we patched the issue quickly, the rate of users updating to the latest version of the app can be a bit sluggish. This would explain the edit rate continuing to fall in May, but picking back up (by extrapolation) in June.
On the iOS side, T134758 had the effect of showing a false error message "Could not connect to network." but edits did go through. So the impact was not as direct, but obviously users seeing an error message during save may have reduced editing or future motivation to edit.
I've pulled daily edit numbers, broken down by platform, and graphed them in this spreadsheet.
@Dbrant, the only noticeable event in the Android numbers is a significant drop on 4 May, which we've mostly recovered from since. That's clearly the Nginx bug.
@JMinor, the iOS numbers look considerably stranger. There's no noticeable effect from the Nginx bug, but on 11 March, edits abruptly drop from 600-700 a day to 100-200 a day, and have held at that lower level ever since. Any ideas?
@JMinor, I made a graph in that spreadsheet of iOS edits broken down by version. It seems like the drop happened when people largely stopped editing with 4.1.7.171, but didn't largely start editing with 5.0.0.775. Perhaps there was a design change that deemphasized the edit function?
Didn't v.5 of the iOS app bring in opt-in stats (instead of opt-out)? If that covers this, the numbers will be totally useless for tracking.
I just talked with @JMinor and he confirmed that version 5 of the iOS app included opt-in stats and that these event logs were included in that.
So it looks like the drop was due to the Nginx bug on the Android side and the switch to opt-in stats in the iOS app.