Maniphest T196783

Rename appInstallID, ts, sessionToken, timeSpent/duration property names in EL schemas
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	mpopov
	Jun 8 2018, 9:35 PM

Description

Per recommendations from Analytics Engineering (and my own personal preferences, tbh), we should use snake_case instead of camelCase for property names in EventLogging schemas.

Also in T191859#4255283 Andrew wrote:

We are trying to use the convention that fields named after dt are ISO-8601, and ‘timestamps’ or ts fields are unix epoch timestamps.

Since Sharvani is instrumenting parts of multilingual analytics in T190931 from scratch, it makes sense for that to use the new naming standards from the start, but there are dependencies on existing instrumentation that uses the existing naming conventions. We discussed this issue and decided that rather than having half of the new schema follow one convention and the other half follow the other convention, it's better to just rename these three common fields across all active schemas/funnels now.

Details

	Subject	Repo	Branch	Lines +/-
	Renaming properties in Analytics schemas	apps/android/wikipedia	master	+33 -33

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	None	T160567 [EPIC] Improve the multi-lingual experience on the Android app
Declined	mpopov	T196714 Update mobile app search metrics after multilingual update
Resolved	Sharvaniharan	T190931 Implement Analytics/Event logging instrumentation to measure new multilingual features
Resolved	Sharvaniharan	T196783 Rename appInstallID, ts, sessionToken, timeSpent/duration property names in EL schemas

Event Timeline

mpopov triaged this task as Medium priority.Jun 8 2018, 9:35 PM

mpopov created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 8 2018, 9:35 PM

@mpopov also the timeSpent property please. Shoudl it be 'duration' from now on?

Sharvaniharan moved this task from Backlog to Doing on the Wikipedia-Android-App-Backlog (Android-app-release-v2.7.24x-I-Ice-lolly) board.Jun 8 2018, 11:14 PM

In T196783#4268407, @Sharvaniharan wrote:

@mpopov also the timeSpent property please. Shoudl it be 'duration' from now on?

Great catch, thanks! I'll change duration to time_spent in the new schemas and change timeSpent to time_spent in existing ones.

mpopov renamed this task from Rename appInstallID, ts, and sessionToken property names in EL schemas to Rename appInstallID, ts, sessionToken, timeSpent/duration property names in EL schemas.Jun 8 2018, 11:53 PM

Change 439384 had a related patch set uploaded (by Sharvaniharan; owner: Sharvaniharan):
[apps/android/wikipedia@master] [WIP] Renaming properties in Analytics schemas

https://gerrit.wikimedia.org/r/439384

gerritbot added a project: Patch-For-Review.Jun 9 2018, 5:38 AM

mpopov mentioned this in T190931: Implement Analytics/Event logging instrumentation to measure new multilingual features .Jun 10 2018, 2:32 AM

For @Sharvaniharan:

Funnels part of multilingual analytics

Schema	Old Revision ID	New Revision ID	Changed Fields
MobileWikiAppDailyStats	18111418	18115101	`AppInstallID`→`app_install_id`, `ts`→`client_dt`
MobileWikiAppFeedConfigure	18111413	18115104	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `timeSpent`→`time_spent`
MobileWikiAppLanguageSettings	18110934	18113720	`duration`→`time_spent`
MobileWikiAppLanguageSearching	18110978	18113721	`duration`→`time_spent`
MobileWikiAppSearch	18111443	18115103	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `sessionToken`→`session_token`
MobileWikiAppSessions	18111417	18115099	`AppInstallID`→`app_install_id`, `ts`→`client_dt`

Note: MobileWikiAppFeed schema has been modified too. Refer to T190931#4269717

Other funnels

Here are just the new revision IDs. In one case a funnel (Appearance Settings) will need special attention (marked) but otherwise it's just a list of schemas affected by the field name standardization and which fields specifically were renamed.

Schema	Old Revision ID	New Revision ID	Changed Fields
MobileWikiAppAppearanceSettings	17836887	18113727	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `currentValue`→`current_value`, `newValue`→`new_value` ★
MobileWikiAppArticleSuggestions	17837021	18115091	`AppInstallID`→`app_install_id`, `ts`→`client_dt`
MobileWikiAppCreateAccount	17836914	18115095	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `sessionToken`→`session_token`
MobileWikiAppEdit	17837072	18115551	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `sessionToken`→`session_token`
MobileWikiAppFindInPage	17836920	18115552	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `timeSpent`→`time_spent`
MobileWikiAppInstallReferrer	17837162	18115554	`AppInstallID`→`app_install_id`, `ts`→`client_dt`
MobileWikiAppIntents	17836922	18115555	`AppInstallID`→`app_install_id`, `ts`→`client_dt`
MobileWikiAppLinkPreview	17836926	18115548	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `timeSpent`→`time_spent`, `sessionToken`→`session_token`
MobileWikiAppLogin	17836928	18115556	`AppInstallID`→`app_install_id`, `ts`→`client_dt`
MobileWikiAppMediaGallery	17836947	18115560	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `timeSpent`→`time_spent`, `sessionToken`→`session_token`
MobileWikiAppNavMenu	17836951	18118517	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `sessionToken`→`session_token`
MobileWikiAppOfflineLibrary	17836955	18118712	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `timeSpent`→`time_spent`
MobileWikiAppOnThisDay	17836957	18118721	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `timeSpent`→`time_spent`
MobileWikiAppPageScroll	17836977	18118723	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `timeSpent`→`time_spent`
MobileWikiAppProtectedEditAttempt	17836991	18118725	`AppInstallID`→`app_install_id`, `ts`→`client_dt`
MobileWikiAppRandomizer	17836997	18118733	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `timeSpent`→`time_spent`
MobileWikiAppReadingLists	17837080	18118739	`AppInstallID`→`app_install_id`, `ts`→`client_dt`
MobileWikiAppShareAFact	18071215	18118760	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `sessionToken`→`session_token`
MobileWikiAppToCInteraction	18071228	18118763	`AppInstallID`→`app_install_id`, `ts`→`client_dt`
MobileWikiAppTabs	17837025	18118767	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `sessionToken`→`session_token`
MobileWikiAppWiktionaryPopup	17837038	18118768	`AppInstallID`→`app_install_id`, `ts`→`client_dt`, `timeSpent`→`time_spent`

@mpopov We will not be sending anymore events to MobileWikiAppOfflineLibrary. All Offline Library related code has been removed.

Change 439384 merged by jenkins-bot:
[apps/android/wikipedia@master] Renaming properties in Analytics schemas

https://gerrit.wikimedia.org/r/439384

Sharvaniharan moved this task from Doing to Done on the Wikipedia-Android-App-Backlog (Android-app-release-v2.7.24x-I-Ice-lolly) board.Jun 12 2018, 10:09 PM

mpopov closed this task as Resolved.Jun 27 2018, 7:58 PM

• Vvjjkkii renamed this task from Rename appInstallID, ts, sessionToken, timeSpent/duration property names in EL schemas to 4cbaaaaaaa.Jul 1 2018, 1:04 AM

• Vvjjkkii reopened this task as Open.

• Vvjjkkii removed Sharvaniharan as the assignee of this task.

• Vvjjkkii raised the priority of this task from Medium to High.

• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).

• Vvjjkkii updated the task description. (Show Details)

• Vvjjkkii edited subscribers, added: Sharvaniharan; removed: gerritbot, Aklapper.

CommunityTechBot renamed this task from 4cbaaaaaaa to Rename appInstallID, ts, sessionToken, timeSpent/duration property names in EL schemas.Jul 2 2018, 12:57 PM

CommunityTechBot closed this task as Resolved.

CommunityTechBot assigned this task to Sharvaniharan.

CommunityTechBot lowered the priority of this task from High to Medium.

CommunityTechBot updated the task description. (Show Details)

CommunityTechBot removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.

CommunityTechBot edited subscribers, added: gerritbot, Aklapper; removed: Sharvaniharan.

Rename appInstallID, ts, sessionToken, timeSpent/duration property names in EL schemasClosed, ResolvedPublicActions