Page MenuHomePhabricator

Log app errors to app/base schema
Open, MediumPublic

Description

We want to start logging errors of interest or concern to our backend so that we can have more visibility into issues users may be having, such as hanging launch times, etc. Per @Dbrant, we should use the existing app/base schema.

https://schema.wikimedia.org/repositories/secondary/jsonschema/analytics/product_metrics/app/base/2.0.0.yaml

  1. Create new ErrorFunnel that logs to app/base schema. Stream name TBD.

Proposed structure:

action: "error"
action_context: stuffed json object with "domain" and "code" fields.

  • instrument_name**: Area of error. Can be a feature name identifier or an insightful area of the codebase like "launch_sequence". TBD on if we have approval to use this.

domain: String

Error type / namespace. This would include the name of our custom errors (like WMFData.WMFCoreDataStoreError) or a generic system error type (like NSURLErrorDomain)

code: String

Specific error case. In our case this could be another custom error name (like setupMissingDataModelFileURL) or a generic system error code (like -1000)

  1. Log launch sequence errors for now.

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone

Event Timeline

Hi @Dbrant, I would like to get your opinion on this approach. Would a new schema like this for Android be helpful? I'm open to any changes.

@SNowick_WMF I would like to get your opinion as well. With this structure, would we be able to easily see if events of a particular domain + code combination are becoming more prevalent? Would we be able to investigate details for particular errors? Would we be able to look up a particular app_install_id and see all error events? Assuming yes on all of these but wanted to double-check.

Explorations were made in the error-funnel branch.

Tsevener renamed this task from Create error schema for instrumentation + log HCaptcha errors to Log app errors to app_base schema.Tue, Feb 17, 3:08 PM
Tsevener renamed this task from Log app errors to app_base schema to Log app errors to app/base schema.
Tsevener updated the task description. (Show Details)
Seddon triaged this task as Medium priority.Tue, Feb 17, 6:14 PM
Seddon moved this task from Needs Triage to Up next on the Wikipedia-iOS-App-Backlog board.

@Tsevener not sure if you've been updated but we are cleared to use app_base (for queries it's event.product_metrics_app_base ) for our data needs so that hurdle has been cleared.

Instead of using instrument_name to indicate where error is coming from it makes more sense to use the action_source column - that would be where you send 'feature name identifier or an insightful area of the codebase like "launch_sequence"'

Ideally I can make Instrumentation docs for this because they're very helpful for me later when looking for data - you don't need to wait for me to start instrumenting but we will want to clearly map how we're sending data. (Save any screens you think might be tricky, I know a lot of errors don't come with special screens but if you have any, that would be great to add).

We should talk about whether you need to use a Funnel for this - is it often the case that a user will see multiple error events in a sequence? If that's true then we can use an error funnel by including a value for funnel_name like funnel_name = error_log with all events, as long as it is consistent throughout that's all that matters. This could be useful since with each funnel a unique funnel_entry_token: is generated for us and the
funnel_event_sequence_position: automatically tracks the sequence of events if that becomes important. There's also a column`action_subtype` if there's more data you want to send like 'what button user clicked' etc. (See Test Kitchen Guidelines but the naming of data is up to us, we just want to be sure we are consistent for both apps, etc.)

For all your questions - yes we can answer all those by looking at the data, we could also set up monitoring in Superset to keep an eye on patterns, trends, etc.