Page MenuHomePhabricator
Create Task
Maniphest T259714

Test schema and stream for analytics events
Closed, ResolvedPublic
Actions

Description

There's currently a /test/event schema in primary repository for simple testing of "production" events, but not for testing "analytics" events.

Specifically, it's missing client_dt field (from /fragments/analytics/common schema fragment in secondary repo) that Event Platform Clients (e.g. EventLogging on MediaWiki) include in events. In case of Event Platform Client for iOS in the Wikipedia app, that also attaches a device_id and a session_id to all events, which are fields that are being added (along with a pageview_id field) in the /fragments/analytics/identifiers schema fragment, currently in code review)

This came up in a conversation between @mpopov and @Tsevener about adding calls to test event logging using the new client library in the Wikipedia app for iOS. Those events would be sent, sure, but they wouldn't be valid against the existing test schema.

So, we need a /analytics/test schema in the secondary repo that will ref /fragments/analytics/common and /fragments/analytics/identifiers and – just like the /test/event schema in primary repo – it would just have these fields:

test:
  type: string
  default: default value
test_map:
  description: >
    We want to support 'map' types using additionalProperties to specify the value types.  (Keys are always strings.)
  type: object
  additionalProperties:
    type: string

We would also need to deploy a stream that uses that schema – "analytics.test" or "test.analytics"?

Details

SubjectRepoBranchLines +/-
wgEventStreams: Streams for testing MEP-based analytics instrumentsoperations/mediawiki-configmaster+24 -2
Create analytics/test schemaschemas/event/secondarymaster+407 -0
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 Open• DAbadT282003 Metrics Platform: Streamline the process to create new instrumentations
 DeclinedNoneT259734 BUOD-KR1-Q4+: Certify that analytics schema and instruments have been upgraded to use the MEP system (clearing the legacy system for sunsetting)
 DeclinedOttomataT259157 BUOD-KR1-Q3: Require that all new schema/instruments are created with the MEP system
 DeclinedNoneT259704 BUOD-KR1-Q2: Upgrade MEP clients to full release status
 ResolvedkzimmermanT259169 BUOD-KR1-Q1: Pilot MEP clients with 2 product teams
 ResolvedmpopovT259709 MEP instrument/schema pilot #1 (EditHistoryCompare)
 OpenNoneT276378 EPIC: Release Metrics Platform v1
 Resolved• DAbadT281997 Metrics Platform: Centralize code location and organization
 Resolved• jlinehanT280182 Consolidate Metrics Platform client libraries into mediawiki/libs/metrics-platform
 OpenNoneT228175 Event Platform Client Libraries
 Resolved• MhollowayT239996 Send batches of events from EPC app libraries (Java, Swift)
 OpenNoneT281768 Incorporate librarized Metrics Platform Swift client into the iOS app
 OpenNoneT281770 Ensure that the Swift client library can be included via SwiftPM
 Resolved• MhollowayT281758 Consolidate Metrics Platform Swift client into mediawiki/libs/metrics-platform
 OpenNoneT261987 MEP Client iOS (Revision)
 ResolvedmpopovT228180 MEP Client iOS (MVP)
 ResolvedmpopovT259714 Test schema and stream for analytics events
 ResolvedphuedxT281772 [EPIC] Incorporate librarized Metrics Platform Java client into the Android app
 ResolvedcjmingT281773 Publish librarized Metrics Platform Java client to Maven
 Resolved• MhollowayT281757 Conslidate Metrics Platform Java client into mediawiki/libs/metrics-platform
 ResolvedcjmingT318934 Extend the Java MPC to detect session inactivity
 ResolvedcjmingT318935 Refactor Java MPC's event sending code
 ResolvedcjmingT318947 Add dispatch method to Java MPC
 ResolvedcjmingT318948 Refactor Java MPC's stream config fetching code

Event Timeline

mpopov triaged this task as High priority.Aug 5 2020, 2:05 PM
mpopov created this task.
mpopov moved this task from Inbox to Backlog on the Better Use Of Data board.

Marking this as high priority since we need to have simple ability to test event logging as soon as possible.

mpopov moved this task from Triage to Current Quarter on the Product-Analytics board.Aug 5 2020, 2:07 PM
mpopov edited projects, added Product-Analytics (Kanban); removed Product-Analytics.
mpopov moved this task from Inbox to Goals on the Product-Data-Infrastructure board.
mpopov added a parent task: T228180: MEP Client iOS (MVP).Aug 5 2020, 2:44 PM
• jlinehan moved this task from Goals to Doing on the Product-Data-Infrastructure board.Aug 5 2020, 5:14 PM
mpopov mentioned this in T259747: Review identifiers schema fragment (Shay).Aug 5 2020, 8:13 PM
mpopov mentioned this in T259748: Review identifiers schema fragment (Morten).
mpopov mentioned this in T259749: Review identifiers schema fragment (Neil).
• jlinehan removed a parent task: T259169: BUOD-KR1-Q1: Pilot MEP clients with 2 product teams.Aug 5 2020, 8:51 PM
mpopov mentioned this in T259917: Streams with empty configs should be rendered as {} in the JSON returned by StreamConfig API.Aug 12 2020, 5:25 PM
Ottomata subscribed.Aug 12 2020, 5:48 PM
gerritbot added a comment.Aug 24 2020, 4:08 PM

Change 622169 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[schemas/event/secondary@master] Create analytics/test/event schema

https://gerrit.wikimedia.org/r/622169

gerritbot added a project: Patch-For-Review.Aug 24 2020, 4:08 PM
gerritbot added a comment.Aug 26 2020, 2:57 PM

Change 622169 merged by Bearloga:
[schemas/event/secondary@master] Create analytics/test schema

https://gerrit.wikimedia.org/r/622169

Maintenance_bot removed a project: Patch-For-Review.Aug 26 2020, 3:10 PM
gerritbot added a comment.Aug 26 2020, 3:48 PM

Change 622604 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[operations/mediawiki-config@master] wgEventStreams: Streams for testing MEP-based analytics instruments

https://gerrit.wikimedia.org/r/622604

gerritbot added a project: Patch-For-Review.Aug 26 2020, 3:48 PM
gerritbot added a comment.Aug 27 2020, 8:41 PM

Change 622604 merged by Ottomata:
[operations/mediawiki-config@master] wgEventStreams: Streams for testing MEP-based analytics instruments

https://gerrit.wikimedia.org/r/622604

Stashbot added a comment.Aug 27 2020, 8:43 PM

Mentioned in SAL (#wikimedia-operations) [2020-08-27T20:43:15Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: wgEventStreams: Streams for testing MEP-based analytics instruments - T259714 (duration: 00m 55s)

Maintenance_bot removed a project: Patch-For-Review.Aug 27 2020, 9:10 PM
mpopov added a comment.Aug 27 2020, 10:59 PM
2020-08-27 18:55:05.180043-0400 Wikipedia[8891:260268] 2020-08-27 18:55:05:180 Wikipedia[main] log(stream:schema:data:domain:)#L612 D: EPC: Event logged to stream 'test.instrumentation' (schema: '/analytics/test/1.0.0'):
{
  "test_map" : {
    "file" : "Application\/App View Controller\/WMFAppViewController.m",
    "method" : "showSettingsWithSubViewController()"
  },
  "test_string" : "Opened Settings screen",
  "meta" : {

  },
  "client_dt" : "2020-08-27T22:55:05Z",
  "app_session_id" : "8e69dacadb9ebd5148c7"
}
2020-08-27 18:55:05.180436-0400 Wikipedia[8891:260267] 2020-08-27 18:55:05:180 Wikipedia[main] log(stream:schema:data:domain:)#L657 D: EPC: Sending HTTP request to https://intake-analytics.wikimedia.org/v1/events with POST body: {"test_map":{"file":"Application\/App View Controller\/WMFAppViewController.m","method":"showSettingsWithSubViewController()"},"$schema":"\/analytics\/test\/1.0.0","meta":{"stream":"test.instrumentation","id":"4C625BFF-0BF9-4F9D-9C1D-2B516E8C7940"},"test_string":"Opened Settings screen","client_dt":"2020-08-27T22:55:05Z","app_install_id":"BC84A66D-6603-4BE0-9812-4967374C2812","app_session_id":"8e69dacadb9ebd5148c7"}

020-08-27 18:56:35.148531-0400 Wikipedia[8891:261117] 2020-08-27 18:56:35:148 Wikipedia[main] log(stream:schema:data:domain:)#L612 D: EPC: Event logged to stream 'test.instrumentation' (schema: '/analytics/test/1.0.0'):
{
  "test_map" : {
    "method" : "refreshControlActivated",
    "file" : "Features\/Feed\/ExploreViewController.swift"
  },
  "client_dt" : "2020-08-27T22:56:35Z",
  "test_string" : "Explore Feed refreshed",
  "app_session_id" : "8e69dacadb9ebd5148c7",
  "meta" : {

  }
}
2020-08-27 18:56:35.148672-0400 Wikipedia[8891:261117] 2020-08-27 18:56:35:149 Wikipedia[main] log(stream:schema:data:domain:)#L657 D: EPC: Sending HTTP request to https://intake-analytics.wikimedia.org/v1/events with POST body: {"meta":{"stream":"test.instrumentation","id":"DD9A6CBD-C94C-4332-8880-131A0ED72EEF"},"client_dt":"2020-08-27T22:56:35Z","app_install_id":"BC84A66D-6603-4BE0-9812-4967374C2812","test_map":{"method":"refreshControlActivated","file":"Features\/Feed\/ExploreViewController.swift"},"app_session_id":"8e69dacadb9ebd5148c7","$schema":"\/analytics\/test\/1.0.0","test_string":"Explore Feed refreshed"}

These two events in Grafana:

Screen Shot 2020-08-27 at 6.58.07 PM.png (1×3 px, 287 KB)

Hoping to see them in events.test_instrumentation soon!

mpopov closed this task as Resolved.Aug 28 2020, 7:32 PM

Verified that the events made it into the database:

Screen Shot 2020-08-28 at 9.59.13 AM.png (1×2 px, 316 KB)

mpopov moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.Aug 28 2020, 7:33 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL