Page MenuHomePhabricator

Utilize `integration` value to distinguish Android app, mobile web and desktop editing data in EditAttemptStep schema
Open, LowPublic

Description

Following the successful Android data integration with the EditAttemptStep schema we needed a way to differentiate Android app editing data from mobile web and desktop. The shared field integration will be used for this purpose.

Deprecated: Following the successful Android data integration with the EditAttemptStep schema we are requesting to add a distinguishing value for all Android edit data using the platform attribute. This will enable us to use Turnilo to separate our data using a singular attribute from platform such as app-android. Currently we are using the Filter setting: Event integration = app-android, but we also need to specify platform = phone and Useragent OS Family = Android. See: https://phabricator.wikimedia.org/T294503#7749333.

Another benefit for this change is that other data users may not know about the Event integration filter for app-android and inadvertently end up using data that includes Android data. If app-android is explicitly available in integration this will be avoided.

We want to be sure this won't cause any unanticipated issues with editing data so if anyone has feedback please let us know.

More context:
https://phabricator.wikimedia.org/T294503#7568135

Intended outcome(s)

Simplify queries so that:

  1. Non-Android Teams have a lower risk of including Android event data in the queries they are running
  2. Android Team can write simpler queries to visualize data in Turnilo

Schema change(s)

Proposed new event nameField new event will "belong" toSchema where event will be logged
app-androidintegrationEditAttemptStep

Implementation Process

StepPeople/Team ResponsibleDescriptionStatus
1.AndroidFile ticket requesting instrumentation✅ Done in T307698
2.AndroidPopulate the ===Schema change(s) section with proposed changes, once defined✅ Done
3.@MNeisler + @DLynchReview the ===Schema change(s) Community Tech is proposing to introduceNext
4.AndroidImplement the instrumentation changes defined in ======Schema change(s)
5.Editing TeamCode review the instrumentation changes Community Tech will have implemented
6.AndroidVerify newly instrumented events are being emitted as expected by clients
7.Product AnalyticsVerify newly instrumented events are being logged in database/schema(s) as expected, once the new instrumentation has landed in production.
8.AndroidUpdate schema documentation with the events spec'd in this task once they have been verified to be implemented as expected. Read: clients are emitting events as expected and said events are being logged in the database as expected as well.

Event Timeline

Hi @SNowick_WMF. Could you please associate a project tag with this task (via the Add Action...Change Project Tags dropdown)? This will allow others to get notified, or see this task when searching via projects / look at workboards. Thanks in advance! :)

hi @SNowick_WMF – next week, you can expect me to have updated the task description with the steps/process that will guide the changes to EditAttemptStep y'all are requesting. More context in T304556.

In the meantime, can you share when the Android Team needs the changes this ticket is asking for to be implemented?

Isn't integration=app-android already a perfectly-distinguishing method of filtering for Android app edits? I don't understand why you'd need to do anything else.

LGoto triaged this task as Low priority.May 9 2022, 4:17 PM

Thank you @ppelberg, we are flexible on time frame, if we can get it implemented by end of May that would be great, if that is possible.If this is something our engineers (specifically @Sharvaniharan) can help with, please let us know.

@DLynch this request will enable us to use Turnilo to compare Android app data with desktop and mobile using Split. As it is currently implemented (with integration=app-android) I need to download each platform dataset (or query separately in Superset) and then compare it manually in GDoc pivot tables. We can't take advantage of all the ad hoc filtering tools that Turnilo offers, including self-serve segmentation.

Hi @ppelberg checking in on this, @Sharvaniharan will be away on sabbatical next week, we can work with another Android engineer but we want to get this done for next quarter's editing OKR baselines if possible. Please let me know what next steps are to add a value in platform column for Android.

ppelberg updated the task description. (Show Details)
ppelberg added a subscriber: MNeisler.

Hi @ppelberg checking in on this, @Sharvaniharan will be away on sabbatical next week, we can work with another Android engineer but we want to get this done for next quarter's editing OKR baselines if possible. Please let me know what next steps are to add a value in platform column for Android.

hi @SNowick_WMF – thank you for the reminder about this and I'm sorry for the delay in getting back to you.

I've updated the task description to include the following new sections:

  • ===Intended outcome(s)
  • ===Schema change(s)
  • ===Implementation process

    I hope will these new sections will address the question you posed around next steps. [i]

Of course, if what I've added prompts any new thoughts or questions, please let me know.


i. For context: we've documented the need for an explicit process for adding events to EditAttemptStep in T304556.

SNowick_WMF added subscribers: cooltey, Dbrant.

Hi @cooltey (adding @Dbrant for oversight) - we need to add the value app-android to our data (in the platform column) that's sent to EditAttemptStep - Sharvani was working on this but we were waiting for approval which we now have. Let me know if any of this is unclear, it should be a fairly straightforward code change that Sharvani may have already prepped.

ppelberg added a project: Product-Analytics.

Hi @cooltey (adding @Dbrant for oversight) - we need to add the value app-android to our data (in the platform column) that's sent to EditAttemptStep - Sharvani was working on this but we were waiting for approval which we now have.

@SNowick_WMF: thank you for populating the Schema change(s) with the instrumentation y'all are proposing to add.

Tho, the approval to move forward with what y'all are proposing will need to come from @DLynch and @MNeisler as two of the maintainers of EditAttemptStep. I've updated the task description to reflect where in the process I perceive us to be.

@SNowick_WMF
The proposed change (adding app-android to the platform field in EditAttemptStep) seems fine to me but I want to confirm a couple details:

  • With this change, will platform = phone no longer include android app edits? Wondering if it would be worth adding some brief platform definitions to the field descriptions since the addition of this new value might make the definition of phone less clear.
  • Will the existing app-android value in the integration field be removed?

Hi @MNeisler - we have been discussing the way we currently use phone and tablet values, it looks like tablet data is only from Android but both Android and MobileWeb use the phone data value (Data). We do want to keep the distinction between phone and tablet in our data, so we are considering using android-app-phone and android-app-tablet values to avoid further confusion. (We should also reserve the values ios-app-phone and ios-app-tablet for future usage down the road). Having the platform value will let us compare MobileWeb and Desktop to Android (and iOS eventually) from within Turnilo.

We do not need to use app-android in integration if we have the platform value. However, we have discussed using integration to store the actual values for the edit mechanism we use, which is what the column was intended for, but we have quite a few like suggested edit, etc. - we can explore that down the road.

Adding @cooltey since he is the engineer who will be working on this (and suggested using the phone/tablet distinction in our data).

There's a ticket for problems with platform already (T249944) which might be worth reviewing. (Basically: on web it currently just means "is MobileFrontend being used?" which is not necessarily what people would expect, and there's discussion of whether/how that should get changed.)

Proposal coming from a meeting we just had: we're going to change how some current editors use integration. Android won't need to make any changes, because they're already in-line with what we want to do.

We're going to change integration so that it refers to the overall tool being used to make the edit. Mostly this just means that we're going to split out a bunch of current editors that're all overloading the single "page" integration.

editorintegration
2010 Wikitextpage
Desktop VisualEditor including 2017 Wikitext Editorvisualeditor
MobileFrontend visual and sourcemobilefrontend
Androidapp-android (no change)
iOSapp-ios (no change)

Several of these have the visual/source split, and that'll still be logged to the editor_interface field. (Also, I've left out some other integrations that won't change.)

Future specialized edit experiences inside the apps would possibly get a new integration (app-android-talk?), depending on how it affects the analysis; we are inconclusive about this currently.

Proposal coming from a meeting we just had...

Thank you, @DLynch. I'm assigning this task over to @SNowick_WMF to review T307698#8109699 and ask questions about any uncertainties it may bring to mind.

Shay, once you and David resolve any questions that emerge through the review you will be doing, can you please assign this task over to @MNeisler so that she can review the proposal?

Once all of the above happens, I'll decide when the Editing Team will prioritize work on implementing what y'all will have converged on doing.

SNowick_WMF renamed this task from Add `platform` value to Android app editing data in EditAttemptStep to Utilize `integration` value to distinguish Android app, mobile web and desktop editing data in EditAttemptStep schema.Wed, Jul 27, 7:27 PM
SNowick_WMF reassigned this task from SNowick_WMF to MNeisler.
SNowick_WMF updated the task description. (Show Details)
SNowick_WMF updated the task description. (Show Details)

Thanks @DLynch this looks good. When using this field to pivot can we clarify here which integration values represent mobile web vs. desktop? Obviously mobilefrontend is mobile web, does that mean page and visualeditor would be desktop editing data? Other than that I don't have further questions, thanks for your work on this.

@SNowick_WMF Yes, page and visualeditor would be on the desktop site. (But, you know, could be someone accessing said site from a mobile device. Making that distinction will await T249944 being resolved somehow.)

@DLynch

The proposed changes to the integration field make look good to me. Page integration events currently account for about 91% of all edit attempts so further breaking these out will be helpful in analytics.
I just have one clarifying question below and then I think this can be implemented as proposed.

Several of these have the visual/source split, and that'll still be logged to the editor_interface field.

Just to confirm my understanding, the editor_interface field can be used to help determine the visual/source split for the new mobilefrontend and app integrations. For the 'page' and 'visualeditor' integrations, the following should always be true. Is this correct?
If integration=visualeditor and platform = desktop, then editor_interface = visualeditor
If integration=page and platform = desktop, then editor_interface = wikitext or editor_interface =wikitext-2017

Other way around for those, actually.

If integration = visualeditor and platform = desktop, then editor_interface = visualeditor or editor_interface = wikitext-2017. This is because the wikitext-2017 mode is part of the VisualEditor extension.

We should wind up with integration = page only ever accompanying platform = desktop (unless/until we change platform's meaning, etc etc etc) *and* editor_interface = wikitext.