Document long-term requirements for GitLab job runners
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	brennen
	Jul 19 2021, 9:16 PM

Description

With a view to our long-term solution for hosting GitLab job runners (beyond the initial pass using WMCS VMs in the project requested at T285913), we should document what we want and need, covering at least:

Compute
Privacy
Elastic demand
Data we want for measuring use and performance
Whether these can safely run on a third-party platform

(See T287279 for setting up runners on WMCS.)

Related Objects

Mentioned In: T295481: Setup GitLab Runner in trusted environment
T295045: Allow a shared, protected runner for the data-engineering group in GitLab
T282842: Early adoption signup for WMF GitLab
T292094: Limit GitLab shared runners to trusted contributors
T291978: Limit GitLab shared runners to images from Wikimedia Docker registry
T290335: mwcli: Automate upload of new version to a releases server
Mentioned Here: T295045: Allow a shared, protected runner for the data-engineering group in GitLab
T287279: Experiment with gitlab-runners project and provision a pool of shared runners for general use
T285913: Request creation of gitlab-runners VPS project

Event Timeline

brennen created this task.Jul 19 2021, 9:16 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 19 2021, 9:16 PM

brennen updated the task description. (Show Details)Jul 27 2021, 5:17 PM

brennen added subscribers: Jelto, • wkandek.

brennen moved this task from INBOX to Seen on the Release-Engineering-Team board.Jul 27 2021, 5:28 PM

brennen edited projects, added Release-Engineering-Team (Seen); removed Release-Engineering-Team.

I started to collect some thoughts for the long-term GitLab Runner setups here: https://wikitech.wikimedia.org/wiki/GitLab/Gitlab_Runner#Future_Gitlab_Runner_setup_(T286958)

Feel free to add, comment or edit. There are some open questions. So we should chat/talk soon about your goals and requirements for the GitLab Runner setup. Maybe you have some practical experience from https://phabricator.wikimedia.org/T287279 already.

Also ccing @Joe here, because he offered to give some technical input as well.

kostajh mentioned this in T290335: mwcli: Automate upload of new version to a releases server.Sep 13 2021, 12:36 PM

Jdforrester-WMF subscribed.Sep 13 2021, 7:53 PM

brennen moved this task from Inbox to CI & Job Runners on the GitLab board.Sep 21 2021, 7:46 PM

brennen edited projects, added GitLab (CI & Job Runners); removed GitLab.

PDF diagram that shows Separation into untrusted and trusted environments that has been talked about in SvcOps before

Tagging Security-Team for awareness, per discussion.

dduvall mentioned this in T291978: Limit GitLab shared runners to images from Wikimedia Docker registry.Oct 8 2021, 10:09 PM

brennen mentioned this in T292094: Limit GitLab shared runners to trusted contributors.Oct 19 2021, 7:56 PM

BTullis subscribed.Oct 22 2021, 12:10 PM

Hello,

The Data-Engineering (aka Analytics) team have been discussing a potential GitLab CI based solution to a current requirement and I thought that this ticket might be a good way to start the conversation about it with ServiceOps.

The current requirement focuses around the development and deployment of Airflow DAGs.

We would like to host a new repository (probably) called airflow-dags on GitLab, but we do not yet know what group/namespace we should use, nor precisely what permissions will be required. Multiple teams will maintain code within the repository, but we have yet to determine exactly what the levels of authorization and automation will be for deployment.

We would like the deployment part of the system to be capable of:

accessing our current Airflow instances, which are in the Analytics VLAN.
accessing HDFS, which requires the use of Kerberos
uploading artefacts (Jars and python wheels) to Archiva

It struck me that a private runner, sited within the analytics VLAN, might be a good way of achieving this result.
I don't think it particularly matters whether it is a shell based runner, or whether it uses the docker executor. Happy to look at either option.

Copying in @Ottomata , @odimitrijevic, @mforns for reference.

I'm happy to make a separate ticket for this, or to discuss elsewhere.
Thanks.

(edit) - I have made a follow-up ticket here: T295045: Allow a shared, protected runner for the data-engineering group in GitLab

Ottomata added a subscriber: gmodena.Oct 25 2021, 1:14 PM

BTullis mentioned this in T282842: Early adoption signup for WMF GitLab.Oct 25 2021, 3:25 PM

sbassett edited projects, added SecTeam-Processed; removed Security-Team.Oct 26 2021, 7:44 PM

The security team would be more than happy to review the documentation when it gets to the draft phase

BTullis mentioned this in T295045: Allow a shared, protected runner for the data-engineering group in GitLab.Nov 4 2021, 2:56 PM

Jelto mentioned this in T295481: Setup GitLab Runner in trusted environment.Nov 10 2021, 3:35 PM