In T366627: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration and linked documents, it was decided that Metrics Platform will use static stream configuration. This means there will be only 1 or a few tables per schema.

To ensure query efficiency, the tables should be partitioned by something other than just hourly time fields (experiment? instrument?).

This will be easier to do for Iceberg tables, as partitioning can be changed after the table is created.
As long as the Metrics Platform tables will be Iceberg, we probably don't need to support this for basic Parquet tables.

Done is

• Custom Iceberg Hive table partitioning can be configured (probably in EventStreamConfig)
• Partitioning is applied during by Refine during table schema evolution

we discuss how it would be interesting to partition some tables generated by refine using an extra field not already included in standard refine partitioning (datacenter, year, month, day, hour).
This task is about implementing the parametrization of a refine job, to accept information about which extra fields refine should use as partitions.