Page MenuHomePhabricator
Create Task
Maniphest T205641

Add ability to bucketize integers as part of event ingestion
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

Add ability to bucketize integers as part of druid event ingestion

Details

SubjectRepoBranchLines +/-
Correct and improve EventLoggingToDruid time measure bucketinganalytics/refinery/sourcemaster+32 -21
Allow EventLoggingToDruid to bucketize integers into rangesanalytics/refinery/sourcemaster+75 -61
Customize query in gerrit

Related Objects
Search...

Event Timeline

• Nuria created this task.Sep 27 2018, 4:29 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 27 2018, 4:29 PM
• Nuria added a parent task: T205562: Ingest data aggregate ReadingDepth data into Druid .Sep 27 2018, 4:33 PM
• fdans mentioned this in T205562: Ingest data aggregate ReadingDepth data into Druid .Sep 27 2018, 5:00 PM
• Tbayer subscribed.Sep 28 2018, 8:43 PM

As indicated over at T205562#4626349 , in many or most cases we will want to treat such integer fields as measures, rather than as dimensions. It seems bucketing only makes sense for the latter.

• fdans triaged this task as Medium priority.Oct 1 2018, 4:12 PM
• fdans moved this task from Incoming to Smart Tools for Better Data on the Analytics board.
mforns claimed this task.Oct 2 2018, 1:35 PM
gerritbot subscribed.Oct 2 2018, 2:51 PM

Change 463968 had a related patch set uploaded (by Mforns; owner: Mforns):
[analytics/refinery/source@master] Allow EventLoggingToDruid to bucketize integers into ranges

https://gerrit.wikimedia.org/r/463968

gerritbot added a project: Patch-For-Review.Oct 2 2018, 2:51 PM
mforns moved this task from Next Up to In Code Review on the Analytics-Kanban board.Oct 2 2018, 3:01 PM
mforns moved this task from In Code Review to In Progress on the Analytics-Kanban board.Oct 2 2018, 6:03 PM
mforns moved this task from In Progress to In Code Review on the Analytics-Kanban board.Oct 2 2018, 6:43 PM
gerritbot added a comment.Oct 2 2018, 8:12 PM

Change 463968 merged by Mforns:
[analytics/refinery/source@master] Allow EventLoggingToDruid to bucketize integers into ranges

https://gerrit.wikimedia.org/r/463968

mforns moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.Oct 2 2018, 8:12 PM
mforns added a comment.Oct 2 2018, 8:22 PM

I also updated https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/Schema_Guidelines

gerritbot added a comment.Oct 3 2018, 3:40 PM

Change 464171 had a related patch set uploaded (by Mforns; owner: Mforns):
[analytics/refinery/source@master] Rearrange bucket ranges in EventLoggingToDruid time measure bucketing

https://gerrit.wikimedia.org/r/464171

mforns moved this task from Ready to Deploy to In Code Review on the Analytics-Kanban board.Oct 4 2018, 4:01 PM
gerritbot added a comment.Oct 4 2018, 5:58 PM

Change 464171 merged by Nuria:
[analytics/refinery/source@master] Correct and improve EventLoggingToDruid time measure bucketing

https://gerrit.wikimedia.org/r/464171

mforns moved this task from In Code Review to Done on the Analytics-Kanban board.Oct 5 2018, 4:01 PM
mforns set the point value for this task to 5.Oct 8 2018, 4:01 PM
• Nuria closed this task as Resolved.Oct 8 2018, 7:28 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL