Druid doesn't allow for array fields.
But if the array is of simple type (Array[String], Array[Int], Array[Boolean]), then we can stringify it and ingest it as a string.
Description
Description
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | mforns | T202751 Ingest data from PageIssues EventLogging schema into Druid | |||
Invalid | None | T201873 [EventLoggingToDruid] Allow ingestion of simple-type arrays by converting them to strings |
Event Timeline
Comment Actions
Should we sort the array before stringifying?
Like: [1,2] and [2,1] generate the same string "[1,2]"?
This would reduce the number of possible string values, and make differently ordered arrays match.
However, if the order in the array has a specific semantics, that would be lost in the stringification.
Comment Actions
After discussing with the team, it turns out Druid does allow ingestion of array types!
I tested that it works for this data set and indeed it does.
So, sorry for the confusion, my bad.
Will invalidate this task.