Data-Engineering upgraded the analytics cluster to Spark 3, see T295072: Install spark3 in analytics clusters.
- in stat machines, switch to the latest conda environment setup
- run the pipeline and check whether changes to the code base are needed
- update the production conda-environment.yaml
- add the pipeline to the list of jobs to migrate