Component source code repository: https://gitlab.wikimedia.org/repos/data-engineering/blunderbuss-bugler

Following up:

In Slack we discussed if if would be possible to reduce the # of repositories to manage and include this component CI template code in either the main blunderbuss repo, or in workflow_utils.

Hopefully it is, and if so we can move the code and delete repos/data-engineering/blunderbuss-bugler

@Ottomata I've addressed your concerns re: number of repositories in this ticket: https://phabricator.wikimedia.org/T382430

Thank you!

@amastilovic thanks for all of your work on this. - Referring back to the original design doc, we did list one of the functional requirements as being:

Use some sort of authentication for this API - possibly GitLab’s secret tokens

We discussed on Slack that this hasn't been implemented yet, although an IP address based authentication system is in place.

Only GitLab trusted runners have access to the trigger URL at https://blunderbuss.discovery.wmnet:30443 by means of IPtables rules. (ref: Add Blunderbuss firewall rule to GitLab runner set)

While this is a good basis, I would prefer to see some kind of additional authentication layer on top of this, if possible. The current situation would mean that any of these projects could trigger a Blunderbuss sync if they have a protected branch, or indeed the GitLab runner machines themselves could also trigger a sync.

I'd be more comfortable if we could have a means by which the API calls themselves could require some level of authentication from Blunderbuss that they were sent from a specific and pre-approved project.
Is there a plan for adding this support to Blunderbuss, or should we collaborate on such a plan? Tyler mentioned GitLab's ID token support, but I'm not really familiar with it. I haven't got a particular implementation in-mind, but I was thinking of some kind of pre-shared key authentication.

In T382348#10416529, @BTullis wrote:

While this is a good basis, I would prefer to see some kind of additional authentication layer on top of this, if possible.

I have no qualms about implementing an additional auth layer on top, but I think we need to discuss the details of what that would look like. We haven't really drilled down into that topic, and there are many different paths the auth feature could take.

The current situation would mean that any of these projects could trigger a Blunderbuss sync if they have a protected branch, or indeed the GitLab runner machines themselves could also trigger a sync.

This is true, although it should be noted that they can only trigger a Blunderbuss sync of themselves and not of some other repositories.

Is there a plan for adding this support to Blunderbuss, or should we collaborate on such a plan? Tyler mentioned GitLab's ID token support, but I'm not really familiar with it. I haven't got a particular implementation in-mind, but I was thinking of some kind of pre-shared key authentication.

There's no plan as of yet, we should collaborate and come up with something viable and useful. I'll take a look with ID token auth you linked.