Implement the functionality of synchronizing HDFS locations of refinery HQL files automatically on merging into the master branch of GitLab repository.
Follow the project proposal as described in [[ https://docs.google.com/document/d/1gytt1rzO5wO1IWmmrGKdfuwSlVvFyQ9pFK87wePUYFE | this document ]].
Design work done here: {T360968}
**Done is**:
[x] Existing workflow_utils project modified to support its original use cases as well as this new one
[x] Add support for interacting with GitLab repositories through a designated script
[x] If needed, modify artifact.py script to support any additional functionality
[x] Implemented stand-alone Flask Web application that:
[x] Supports configuration of different repositories through a config file
[x] Provides REST API endpoints to initiate the synchronization process for each of the configured repositories
[x] Supports pulling the latest Git repository updates through workflow_utils package
[x] Recognizes files/folders to be synchronized to HDFS locations, through config file(s)
[x] Performs transactional synchronization from Git repository to HDFS locations
[] Implemented synchronization task log web page
[] Provides a web endpoint that lists all synchronization tasks received from Gitlab, with their status, in chronological order
[x] Application that is fully operable within the context of our Kubernetes cluster
[x] App/service is accessible to REST API calls from WMF GitLab
[x] Has functional access to WMF GitLab repositories
[x] Has functional access to WMF HDFS
[x] Will not be public facing
[x] Will not be accessible to any hosts other than GitLab
[] Will require authentication.
[x] repository, k8s, DNS and other names are named 'blunderbuss' and not 'hdfs-synchronizer'
[] T382348 Develop a GitLab CI/CD Component to be used by projects/repositories to integrate their CI/CD workflows with Blunderbuss
---
this is blunderbuss :)