I think using git submodules would be the least confusing. In all the other options, there is some 'duplication' of either materialized version files, or even worse of current.yaml schemas.
Hue needs to have accounts manually created. Just did. Try now with 'daisy'
Nuria I think we can decline this yes? Doing so, feel free to reopen if I am wrong.
Not sure about the is_cloud_vps name...can the dashboard just examine the ip network and differentiate directly, rather than us adding a new field? Not sure.
Or even more DRY:
The capsule is coming back!
Haha, not quite. The fields would still be defined explicitly. But yes, I also don't like this idea.
Fri, Sep 20
We won't be fixing this as part of Modern Event Platform. Can test2wiki's config just be updated to use meta.wm.org?
Closing this; Modern Event Platform uses draft-07. Won't change for EventLogging.
Thanks for comments!
Hm, your example isn't quite right. geocoded_data is defined to be a map of string -> string, and blah: ... looks like an object (or another map?).
An issue with keeping the ingestion annotation in the event schema is that anyone who has merge rights to the analytics event schema repository will have the ability to change the ingestion information.
In T214093: Modern Event Platform: Schema Guidelines and Conventions we are discussing how to annotate event schemas with druid ingestion information, to ease automated ingestion into druid.
@JAllemandou any thoughts?
FYI Schema and Design Guideline documentation WIP here:
Working on 'guidelines' here:
@Pchelolo any thoughts?
Actually blocked on T208589: [EventLoggingToDruid] Add support for ingesting subfields of map columns.
It sounds like the Druid use case needs a little more thought. I still think this is a good idea, but let's hold for now.
@Pchelolo should we close?
Thu, Sep 19
Hm that sounds harder than I thought. Why can't sqoop map the MySQL schemas?
@Nuria, perhaps sqooping this into Hive wouldn't be too hard?
So far this mediawiki:Event Platform/Schemas draft is mostly documentation about how to create and modify schemas in a git repository. We'll also need some documentation about fields and naming conventions, TBD.
Wed, Sep 18
Phew ok after all those patches, I think we are good with puppet code cleanup!
AWESOME thank youuuu
apt-get purge python-zmq python-tornado python-ua-parser python-urllib3 python-jsonschema python-mysqldb python-graypy python-pygments python-sprockets-mixins-statsd python-snappy python-six python-service-checker python-sqlalchemy python-statsd