As per the parent ticket, there are two DAGs running on the product-analaytics team's airflow instance which use WMF Data Workflow Utils to build a runtime conda environment, based on miniconda.
- product-analytics/moderation-mariadb-jobs # Several versions
- product-analytics/automoderator-metrics-jobs # Several versions
- Update the artifacts.yaml of analytics_product and the affected airflow DAGs to the point to new package versions.
Due to upcoming licence changes in the Anaconda project, we wish to ensure that all of these environments are using the latest version of the workflow utils conda pipeline, in which we switch from miniconda to miniforge.
Please would you take steps to upgrade these DAGs and deploy them, when convenient?
It should just be a case of ensuring the gitlab-ci.yml file references v0.19.0 of repos/data-engineering/workflow_utils as stated here:
https://wikitech.wikimedia.org/wiki/Data_Platform/Systems/Airflow/Developer_guide/Python_Job_Repos#GitLab_CI_setup
There is a reference GitLab MR here, which you may find useful:
https://gitlab.wikimedia.org/repos/data-engineering/example-job-project/-/merge_requests/36/