EditAttemptStep Event Counts:
Date range 2022-02-01 - 2022-02-22

For all page namespaces: Data

For Talk page namespaces:

Important caveats to this comparison matrix:
Android data:
*We do not track page namespace (aka page_ns and/or pagens) for Android in EditAttemptStep data because of impediments, partly due to the way the EditAttemptStep table was recreated from Legacy to MEP. This means the edit completion/step data we have for Android from EditAttemptStep is for all pages. In order to ascertain the Android edit success rate for Talk pages I used the Talk Page specific conversion rate from Mobilewikiappedit and mediawiki_history. The data from Mediawikiappedit is not exact because, similarly to the pagens value issue in MobilewikiappTalk, the pagens value in Mobilewikiappedit is a text string, rather than a numeric value. Additionally, there are null values for pagens in Mediawikiappedit, so the mediawiki_history ratio of edits by page type should be considered the source of truth.

According to data from mediawiki_history, for all Android app edits made in 2022-01 the unreverted edits split was 2.4% Talk page edits and 97.6% non Talk page edits.

EditAttemptStep platform data:
** It appears that page_ns data is also missing or not tracked for phone platform data in EditAttemptStep as there are also a significant amount of NULL values for `page_ns', so the comparison data presented above should be considered incomplete. See data in Turnilo for further details.
Note: Query failed to filter out integration != app-android values, the rows missing page_ns values were coming from the Android app. See Turnilo for platform = phone page_ns value distribution. Thanks to @MNeisler for pointing this out.

Steps we can take to resolve missing page type data issue:

1. Add page_ns tracking to data we are sending to EditAttemptStep (if it is possible to resolve outstanding complications). And/Or
2. Change pagens value sent to Mobilewikiappedit (and Mobilewikiapptalk) from string/page name to numeric value , this will ensure accuracy for query by page type. (This is a pending fix request for Mobilewikiapptalk already - https://phabricator.wikimedia.org/T296310#7552053). We will also need to resolve the NULL pagens values in that table, without complete data it is not possible to determine ratios of Talk to Non-Talk page edits.
3. Follow up on proposed Android EditAttemptStep schema that emulates data as recorded in EditAttemptStep as closely as possible. This could also be accomplished by repairing Mobilewikiappedit and/or adding any missing steps commensurate to events in EditAttemptStep.

Work with Growth team to resolve or eliminate missing pagens values for phone platform data if it is invalid. This may be due to the 'mislabeled as mobile' data issue https://phabricator.wikimedia.org/T249944#6481858 or another unrelated issue. Without these values we will not have a complete picture of edit completion rates coming from mobile edits.

@JTannerWMF, @Sharvaniharan & @Dbrant - let's discuss solutions in next meeting. I will follow up with @MNeisler re. missing page_ns attribution for platform=phone in EditAttemptStep data.

@JTannerWMF Comparison data from EditAttemptStep (above) has been updated for corrected platform=phone edit event counts.

Per our most recent meeting, @Sharvaniharan will add pagens values to data sent to EditAttemptStep, to be included in next app beta release. Following that, the page_ns values sent to MobileWikiAppEdit and MobileWikiAppTalk will be changed to page ID code rather than the current string value.

@Dbrant will explore adding Android-specific values for editor_interface to data sent to EditAttemptStep to allow us more granular insight into what interface is being used when edits ate made on the Android app.

Updated platform comparison edit data with corrected Android EditAttemptStep event counts:
Date range 2022-03-03 - 2022-03-15 Data

Article Talk Pages

User Talk Pages

@JTannerWMF As we discussed, I will look into Phone (Mobile Web) edit event start accuracy in EditAttemptStep, note that this data shows that the low completion rate is only apparent on Article Talk pages. It may be that the init/start event is premature and not indicative of an intentional started edit. Will post findings here.

Data and charts visualizing edit funnels were more than can fit into one ticket, see slide deck here. @JTannerWMF Please let me know if any of the tables in the deck would be useful to insert into this ticket for reference and I can add them.

Note there is not a straight path from init (start) to edit complete or edit fail, some events, like saveIntent only happen when users are editing via Wikitext. Bar charts showing event counts were the most effective way to visualize this data, see deck for more details.

See also: Treemap charts for event counts in Superset.

To summarize the Edit Funnel data findings (also in deck):

• Desktop platform Edit Success Rates for Article Talk pages is 15.6% and 26.5% for User Talk pages. The abort rate is 17.9% and 14.3% and the measured Failure rate is 2.6% and 3.2%.
• Phone platform Edit Success Rates for Article Talk pages is 2.7% and 21.3% for User Talk pages. The abort rate is 39.6% and 50.0% , actual Failure events are only counted in certain scenarios, for Article pages Failure is at .42% but commensurate with Desktop on User pages at 3.4%.
• Android Edit Success Rates for Article Talk pages is 30.1% and 37.9% for User Talk pages. The abort rate is 13.5% and 14.4% and the measured Failure rate is 5.7% and 6.8%.