As a user of DataHub I expect the columns of the tables in the event database to automatically have descriptions based on the corresponding schemas. @Milimetric suggested that since the hive metastore in ingested into DataHub what we actually need is to populate the metastore table metadata with field descriptions from the corresponding schemas
- For Event Platform tables created out of configured streams such as mediawiki.skin_diff I would expect event.mediawiki_skin_diff to use analytics/pref_diff/latest.json
- Likewise for migrated EventLogging schemas (e.g. eventlogging_SearchSatisfaction stream, event.searchsatisfaction table, analytics/legacy/searchsatisfaction schema latest.json)
- Since the intention is to eventually shut down legacy EventLogging, I don't think we need to update metastore with details from not-yet-migrated legacy EL schemas on Meta wiki