Page MenuHomePhabricator

Decide how to store calculated movement metrics
Open, MediumPublic

Description

Currently, we store our computed metric values in TSV files in the movement-metrics repo. It's not certain that we can continue to do that (can an Airflow job push to a Git repo?) and in any case, it seems sensible to at least _add_ storage of the metrics in the Data Lake, so the values can be more easily retrieved by analysts. On the other hand, we want to maintain access to the data in Google spreadsheets, which is most easily accomplished by having a public TSV somewhere and using the IMPORT_DATA function.

So, given all that, how should we store our computed metrics going forward?