Page MenuHomePhabricator

Investigate non-bot mobile edit growth reduction
Closed, ResolvedPublic

Description

Did something change in tracking or engineering that would explain a reduction in the growth of mobile editing? Mobile non bot edits have continuously increased for the last years. In October 2021 the number of edits started to decrease and the reduction continued in November and December. We see a reduction in the YoY growth in mobile editing when compared to last year.superset chart

  • T249038 is not likely to be impacting in this case. As I understand, that is a change tag specific to user_interface whereas here we're looking at data from edits_hourly that has a mobile related platform revision tag.
  • Mobile talk pages to anons on enwiki (T293946) is interesting to note here but not likely a contributing factor
  • The greatest YoY mobile nonbot edit decreases were on wikinews and wikidata but together these made up less than 4% of mobile nonbot edits in Dec. Wikipedia made up 89.4% of all mobile nonbot edits in Dec and saw a 0.4% increase YoY this month...which while a positive YoY growth is the smallest YoY growth that we've seen in the last 6 years.
  • By language there doesn't appear to be a market based difference. the 25 languages with the most mobile nonbot edits are all seeing growth trajectory decreases in mobile nonbot edits. And many wikis are seeing the same pattern. See the related slack comment

References:

Event Timeline

@SNowick_WMF may have related mobile instrumentation tickets to share

Iflorez triaged this task as Medium priority.

@Iflorez I am in the process of pulling together all the iOS/Android editing counts for the past 6 months for comparison, will post here when counts are ready.

In order to get more insight into where and to what level we are seeing the edit growth reduction on mobile I modified the edits_hourly queries from @Iflorez's superset chart to split the data between mobile web edits and mobile app edits. Comparison data is collated here.

Key takeaways:
Comparing monthly edit counts from edits_hourly to an earlier monthly average (2021-06, 2021-07 & 2021-08) for the time period in question (2021-09 - 2021-12) the segment with the largest reduction in edit counts was mobile web. However as of 2022-01 those reductions are no longer apparent but this something we should be keeping an eye on.

MonthMobile Web Edits % Difference from BaselineMobile App Edits % Difference from Baseline
2021-092.6% ↓1.4% ↓
2021-103.8%. ↓1.6%. ↓
2021-1110.3%. ↓1.4% ↑
2021-126.8% ↓8.1% ↑
2022-014.2% ↑11.5% ↑

I also pulled together iOS and Android edit count data from mediawiki_history and will add any relevant findings when I compare with edits_hourly data, but since mobile app edits look to be less affected in this reduction, the mobile web data is likely more important for resolving this issue.

mpopov edited projects, added: Product-Analytics; removed: Product-Analytics (Kanban).
mpopov subscribed.

This work needs to be properly triaged, and Irene is currently busy with higher priority items.

Update from the Status & Planning mtg:
We'll keep an eye on this in the monthly metrics and will revisit this task after February KHM data is available.
It's helpful to see @SNowick_WMF's dig and we'll keep this in consideration.
A few open questions: Is this connected in any way to data loss? blocks? Apple relay?

The trend continues
The reduction in non-bot mobile monthly edits continues to see a decrease: https://superset.wikimedia.org/r/1305
In February there was .3% YoY growth, a continued reduction from the previous months and years: https://superset.wikimedia.org/r/1306

The issue documented here may be caused/impacted by blocks.
We’ll continue to monitor this at the high level through Monthly Key Metrics reporting.

Iflorez changed the task status from Declined to Resolved.Mar 15 2022, 4:50 PM