Wed, Sep 28
V1 is ready Android Monthly Metrics Dashboard. We can use this ticket to track data additions, any issues, questions, etc.
Tue, Sep 27
Thu, Sep 22
Verifying we are seeing various InsertMedia actions made by 2 users so far in beta event data for app version: Beta: 2.7.50420-beta-2022-09-12. data
Wed, Sep 21
Currently seeing 4 results in event data for Beta version generated by 2 unique users. Data It looks like we are not expecting large counts of these events so this may be ok but will wait for engineering to verify that this validates data (ie. if these counts are lower than expected there may be issues).
Wed, Sep 14
Tue, Sep 13
Adding Android Chinese Language data here for reference. The same data collection/extrapolation process applies to this data. Most notably, 90.5% of the Taiwan Android users are using Chinese language wikis. The rest of the countries hover at around the same percentages as iOS.
Fri, Sep 9
The iOS app store does not give us stats on user language, so instead we used the distribution of users with Chinese language selected as primary language in app (from MobileWikiAppiOSSessions) to extrapolate against total users by the country data we get from App Store Connect.
Thu, Sep 8
Wed, Sep 7
Deck is complete, will add summary of results/findings for review.
Tue, Sep 6
Aug 29 2022
Aug 27 2022
Results are as follows:
Aug 26 2022
Aug 18 2022
Primary analysis is completed, deck is ready for initial review. Will complete secondary metrics after I've caught up on other pending tasks.
Aug 17 2022
Aug 16 2022
Aug 15 2022
Hopefully this more clearly answers what your team was looking for w/r/t edit notice impact, I pulled edit counts and revert rates from before and after we implemented edit notices. The following is monthly edit data (edit notices were released in 2021-11). It appears that edit notices did not discourage edit completion, edit revert rates also decreased.
Aug 11 2022
Last Android analysis: results of our ABC test for a new notification Icon, in this deck are slides that show notification interactions w/edit counts before and after notifications release. Results are segmented by our test groups. Deck
Aug 10 2022
Also relevant - Android editor analysis - notifications by type: Data
@ppelberg and @MNeisler
I've done analysis for both iOS and Android post-notification editing data, can share data but most useful right away would be the iOS editing impact analysis I did in June 2022: https://phabricator.wikimedia.org/T308793#8018661, links to all data are in the ticket, please let me know if you have trouble accessing. Note that data is not edit notice specific, just notifications in general so may not be useful.
Aug 5 2022
Aug 3 2022
@cooltey Thanks for clarifying, I will include all event/source counts in my data analysis for archive talk page activity.
@cooltey Verifying data in MobileWikiAppTalk for event: open_talk source: archivedTalkActivity(count: 15), however for event: open_archive I am seeing both source: archivedTalkActivity (count: 1) and talkActivity (count: 4).
Aug 2 2022
Jul 29 2022
Currently we have implemented a 50% sample rate for beta app data which appears to have been effective. Comparing current data to previous daily unique and event averages shows a 50% count of unique users and 59% of daily events. This is 2 days of data, will keep an eye on data volume but patch looks successful.
Jul 27 2022
@cooltey Thanks, yes I think we should track these events. If the breadcrumb event names clearly indicate each step we can use that, otherwise we will need a glossary of event name meaning for the data we track.
We will mainly want a count of attempted image or link adds and successful image or link adds to verify engagement with this new feature.
Thanks @DLynch this looks good. When using this field to pivot can we clarify here which integration values represent mobile web vs. desktop? Obviously mobilefrontend is mobile web, does that mean page and visualeditor would be desktop editing data? Other than that I don't have further questions, thanks for your work on this.
Jul 26 2022
Jul 25 2022
Jul 23 2022
@scblr Results have been sorted (with some cleanup due to translation anomalies needed) for EN, AR, ID and JA, answers are tabbed in response spreadsheets. Will wrap up FR and HI next. Lmk if you have questions.
Jul 21 2022
Jul 20 2022
@scblr English survey answers have been counted (1498 total) and sorted, spreadsheet access is limited for privacy.
Sorting the rest now using a combination of Sheets and Python, will list when completed here.
These event strings look good, please go ahead and implement.
Jul 19 2022
@scblr and I reviewed survey results, I am currently translating and collating results. Will link completed spreadsheets by Wiki here as they are sorted. Will also be joining all answers into one master set for Design usage and link here when ready.
Jul 11 2022
Updating team on survey response quantity thus far:
Jul 7 2022
Added editing data for 2022-06 to analysis, the average monthly edits did decrease somewhat but we still see an overall increase in edits across 44.4% of wikis, with a median .74% positive difference (the average difference is 3.4%).
Jul 6 2022
Avg. percent of DAU using Edit History Interaction: 0.92%
Verifying we are seeing small counts of Rollback events in MobileWikiAppEdit
Jul 2 2022
Jul 1 2022
Thanks @Ottomata, we do not need all data so I'm fine with starting with 50% and following @mpopov's recommended settings. Will keep an eye on eventgate and post updates here when we're up and running.
Jun 30 2022
Note: Verified w @Dbrant that we can use click tracking data from android_breadcrumbs_event for successful namespace prompts in Search. Adding this insight to the pageview data we get from MobileWikiAppLinkPreview will give us a more thorough picture of user engagement with this new feature.
Based on data volume estimates using Beta breadcrumb events per user and DAU and event counts from MobileWikiAppSessions (link) we anticipate events in android_breadcrumbs_event to be ~183% higher than what we see daily in MobileWikiAppSessions. I'm cc'ing @Ottomata as we discussed to give DataEng a heads up on our anticipated daily event data volume. We currently have this feature-flagged for Beta only, pending signoff (or a decision that we should indeed sample this data).
See https://phabricator.wikimedia.org/T311532 for Breadcrumb Onboarding Analysis
Jun 29 2022
Jun 28 2022
Jun 24 2022
Verifying after this fix we are seeing test Rollback events now in MobileWikiAppEdit, will report on uptake when app goes to production.
Jun 23 2022
We have not been seeing test events in schema MobileWikiAppEdit as of 2022-06-23. Added Rollback events ("rollbackAttempt", "rollbackSuccess", "rollbackFailure") to schema doc as action is an enum field. Code needs to be updated to Revision 23442324 for new schema to work.
Jun 22 2022
Post-notification implementation editing analysis:
Looking at editing activity post-notification shows a median +10.4% difference in edits made month over month, across 114 different language wikis. This smooths out with more data, using an average of edits by wiki for all of 2022 prior to Notification release, the median increase is +2.3% difference. We will need to look at next month's mediawiki_history editing data to see if this is an ongoing increase or a temporary lift.
Charts and Data
Jun 21 2022
Streaming data streams=android.breadcrumbs_event
Jun 17 2022
Hi @MNeisler - we have been discussing the way we currently use phone and tablet values, it looks like tablet data is only from Android but both Android and MobileWeb use the phone data value (Data). We do want to keep the distinction between phone and tablet in our data, so we are considering using android-app-phone and android-app-tablet values to avoid further confusion. (We should also reserve the values ios-app-phone and ios-app-tablet for future usage down the road). Having the platform value will let us compare MobileWeb and Desktop to Android (and iOS eventually) from within Turnilo.
Jun 16 2022
Caveat that usage will likely be very rare, don't expect large counts for these events.
- Most users will not be accessing edit history/diff screens where button resides
- Even fewer editors have Rollback privileges - button is only visible to those editors
Jun 15 2022
Noting here: Depending on volume of data we see with beta release we may want to consider implementing data sampling on stream config (should be by device not by session, so we get whole sets of user data). Will follow up when we have a better grasp on volume in a few days.
Jun 14 2022
Hi @cooltey (adding @Dbrant for oversight) - we need to add the value app-android to our data (in the platform column) that's sent to EditAttemptStep - Sharvani was working on this but we were waiting for approval which we now have. Let me know if any of this is unclear, it should be a fairly straightforward code change that Sharvani may have already prepped.
Chart has been updated to consolidate LATAM and The Caribbean.
Chart in GDocs.
Will be consolidating LATAM and The Caribbean in next version.