@scblr Results have been sorted (with some cleanup due to translation anomalies needed) for EN, AR, ID and JA, answers are tabbed in response spreadsheets. Will wrap up FR and HI next. Lmk if you have questions.

@scblr English survey answers have been counted (1498 total) and sorted, spreadsheet access is limited for privacy.
Sorting the rest now using a combination of Sheets and Python, will list when completed here.

These event strings look good, please go ahead and implement.

@scblr and I reviewed survey results, I am currently translating and collating results. Will link completed spreadsheets by Wiki here as they are sorted. Will also be joining all answers into one master set for Design usage and link here when ready.

Updating team on survey response quantity thus far:

Added editing data for 2022-06 to analysis, the average monthly edits did decrease somewhat but we still see an overall increase in edits across 44.4% of wikis, with a median .74% positive difference (the average difference is 3.4%).

Avg. percent of DAU using Edit History Interaction: 0.92%

Verifying we are seeing small counts of Rollback events in MobileWikiAppEdit

Link to iOS Market Share Charts

Thanks @Ottomata, we do not need all data so I'm fine with starting with 50% and following @mpopov's recommended settings. Will keep an eye on eventgate and post updates here when we're up and running.

Note: Verified w @Dbrant that we can use click tracking data from android_breadcrumbs_event for successful namespace prompts in Search. Adding this insight to the pageview data we get from MobileWikiAppLinkPreview will give us a more thorough picture of user engagement with this new feature.

Based on data volume estimates using Beta breadcrumb events per user and DAU and event counts from MobileWikiAppSessions (link) we anticipate events in android_breadcrumbs_event to be ~183% higher than what we see daily in MobileWikiAppSessions. I'm cc'ing @Ottomata as we discussed to give DataEng a heads up on our anticipated daily event data volume. We currently have this feature-flagged for Beta only, pending signoff (or a decision that we should indeed sample this data).

See https://phabricator.wikimedia.org/T311532 for Breadcrumb Onboarding Analysis

Android Quarterly DAU, MAU and DAU/MAU Ratio 2020-10 - 2022-06

Baseline pageviews by Source

Verifying after this fix we are seeing test Rollback events now in MobileWikiAppEdit, will report on uptake when app goes to production.

We have not been seeing test events in schema MobileWikiAppEdit as of 2022-06-23. Added Rollback events ("rollbackAttempt", "rollbackSuccess", "rollbackFailure") to schema doc as action is an enum field. Code needs to be updated to Revision 23442324 for new schema to work.

Post-notification implementation editing analysis:
Looking at editing activity post-notification shows a median +10.4% difference in edits made month over month, across 114 different language wikis. This smooths out with more data, using an average of edits by wiki for all of 2022 prior to Notification release, the median increase is +2.3% difference. We will need to look at next month's mediawiki_history editing data to see if this is an ongoing increase or a temporary lift.
Charts and Data

Streaming data streams=android.breadcrumbs_event

Hi @MNeisler - we have been discussing the way we currently use phone and tablet values, it looks like tablet data is only from Android but both Android and MobileWeb use the phone data value (Data). We do want to keep the distinction between phone and tablet in our data, so we are considering using android-app-phone and android-app-tablet values to avoid further confusion. (We should also reserve the values ios-app-phone and ios-app-tablet for future usage down the road). Having the platform value will let us compare MobileWeb and Desktop to Android (and iOS eventually) from within Turnilo.

Caveat that usage will likely be very rare, don't expect large counts for these events.

1. Most users will not be accessing edit history/diff screens where button resides
2. Even fewer editors have Rollback privileges - button is only visible to those editors

Noting here: Depending on volume of data we see with beta release we may want to consider implementing data sampling on stream config (should be by device not by session, so we get whole sets of user data). Will follow up when we have a better grasp on volume in a few days.

Hi @cooltey (adding @Dbrant for oversight) - we need to add the value app-android to our data (in the platform column) that's sent to EditAttemptStep - Sharvani was working on this but we were waiting for approval which we now have. Let me know if any of this is unclear, it should be a fairly straightforward code change that Sharvani may have already prepped.

Chart has been updated to consolidate LATAM and The Caribbean.

Chart in GDocs.
Will be consolidating LATAM and The Caribbean in next version.

@cooltey sounds good, yes please use for those events.

Hi @ppelberg checking in on this, @Sharvaniharan will be away on sabbatical next week, we can work with another Android engineer but we want to get this done for next quarter's editing OKR baselines if possible. Please let me know what next steps are to add a value in platform column for Android.

Verifying we are seeing all expected action values in data filtering by version 2.7.50401.

How many users have opted to receive native push notifications?
We track device_level_enabled for users who interact with notifications (66.6% enabled) and all iOS users overall (3335, 5.1%) (from schema mobilewikiappiosuserhistory). There is a high count of notDetermined which I will be looking into.

In preparation for this I pulled 2022 editing data for iOS app users from mediawiki_history using revison_tag = ios app edit. I expected to see edits on commonswiki, particularly because data on ios_notification_interaction shows the most frequent source of notifications is commonswiki, with the majority of those notification types are edit-user-talk (see chart).

We will add data tracking for this function to the MobileWikiAppEdit schema, since a rollback is technically an edit. We can use the value action = rollback and add the requisite pageNS and revID values with each event.

For prior data see: https://analytics.wikimedia.org/published/dashboards/Wikipedia_C-19_Comms_Stats/#pageviews

Validating that all possible values for theme are working in schema MobileWikiAppiOSUserHistory:

Data has been validated for version 6.9.0.1922, values for inbox_count and device_level_enabled now populated in MobileWikiAppiOSUserHistory.

Data is validated - see Superset Dashboard for notification event frequency charts/data.

@DLynch this request will enable us to use Turnilo to compare Android app data with desktop and mobile using Split. As it is currently implemented (with integration=app-android) I need to download each platform dataset (or query separately in Superset) and then compare it manually in GDoc pivot tables. We can't take advantage of all the ad hoc filtering tools that Turnilo offers, including self-serve segmentation.

Thank you @ppelberg, we are flexible on time frame, if we can get it implemented by end of May that would be great, if that is possible.If this is something our engineers (specifically @Sharvaniharan) can help with, please let us know.

Deck has been reviewed, this is complete for now. The funnel for Talk page edits will be updated when planned Android upgrades are implemented.

We're seeing a small amount of data in the new ios_notification_interaction schema. Looks like a healthy variety of events are being counted. Users who appear in ios_notification_interaction are also showing values for inbox_count and device_level_enabled in MobileWikiAppiOSUserHistory.

Yes, the PageNamespace values are useful, converting to the string that you've already defined from the integer is probably the best value to work with. So a user clicks a link to namespace integer = 9 should be action = link_mediawikiTalk, a user clicks a link to namespace integer = 11, the value will be action = link_templateTalk. That way we aren't converting values after data is pulled from schema.

Hi thanks for checking on these -

1. Since the schema is for interactions, yes action should be a required field
2. Added values to Spec spreadsheet with definitions, as follows:

We will add tracking for both additional in-depth metrics; sort order selection (and what options user selects) and thread expand/collapse (both global and by item). @Sharvaniharan will follow up when implementation is ready to be tested.

Draft deck currently under review, will finalize when approved.

Discoverability
We have baseline data on returning vs new users as percent of population based on a snapshot taken in late 2022-03, this snapshot will be updated just prior to the release of new Talk page features so this metric should be considered a temporary stand-in for values we will update with each new release.

Charts and Analysis for beta App users is in deck. Also more in depth data available here. Data from 35077 unique users.

@Mazevedo, @Tsevener
Schema slide deck: here
Schema Spec Spreadsheet based on latest version of notification events.

Note: we are seeing event data for beta app users in schema android_article_toolbar_interaction. Data here is from 2022-04-04 onward. Will be reporting on these initial findings at next week's app offsite.

Note: For analytics we will be using string values for event tracking, rather than the numeric code described above as action_rank. Hopefully this will let us sidestep the order/API code issues. I'm working on the schema doc now , will use Toni's spreadsheet and screenshots to illustrate the string values for each action. Will share doc when it's laid out but let me know if there are any questions before then. Thanks!

To summarize the Edit Funnel data findings (also in deck):

Discoverability
Baseline KR is average daily percent of user population that is new to Talk pages (not seen using Talk since we started measuring visits). Records of active Talk page users can be recorded prior to each new release, so we will always be able to detect discovery and new users. Will post Baseline average here when we have accumulated an acceptable sample size in the next week.

Data and charts visualizing edit funnels were more than can fit into one ticket, see slide deck here. @JTannerWMF Please let me know if any of the tables in the deck would be useful to insert into this ticket for reference and I can add them.