Page MenuHomePhabricator
Create Task
Maniphest T296893

Replace Diffusion integration with Gitlab integration in Striker (toolsadmin)
Closed, ResolvedPublic
Actions

Description

Supporting Toolforge tools with an automated git repo creation mechanism is currently the main blocker for removing Differential from our Phabricator service. Gerrit is also currently being treated as a legacy system, with GitLab as the actively developed replacement solution.

Gitlab will allow some amount of self-service git repo creation, but taht will not remove a need to make the process easily discoverable by tool maintainers or to assist them in organizing their repos in a discoverable location for others. There is also a need to migrate existing Diffusion repos to the new preferred system. These are all opportunities for adding automation to reduce human toil.

See also:

Details

SubjectRepoBranchLines +/-
striker: bump container version to 2022-09-07-191738-productionoperations/puppetproduction+2 -2
feat: update git goal for gitlab provider changelabs/strikermaster+11 -4
striker: bump container version to 2022-09-06-213820-productionoperations/puppetproduction+2 -2
fix: Use kwargs to lookup reverse url for GitlabRepo modellabs/strikermaster+6 -1
striker: Bump deployed version to 2022-09-04-055313-productionoperations/puppetproduction+2 -2
fix: unwrap user info from gitlab lookup responselabs/strikermaster+2 -2
dev: Add dummy GITLAB_ACCESS_TOKEN value for prep buildslabs/strikermaster+1 -0
feat: create and view gitlab reposlabs/strikermaster+308 -86
dev: Add gitlab container to docker-compose configlabs/strikermaster+156 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

bd808 created this task.Dec 1 2021, 11:19 PM
bd808 mentioned this in T224676: Add Gerrit support to Striker (toolsadmin).
bd808 added projects: GitLab, Tools.
Xover subscribed.Dec 2 2021, 5:13 AM
Xover added a comment.Dec 2 2021, 5:49 AM

@bd808 I still can't find any mention of Toolforge on mw:GitLab or its subpages, so do we need a separate task to set policy for access/groups/directory layout in GitLab? What's currently documented only deals with WMF and WMDE components, and I am not immediately convinced that the assumptions for those hold in the context of Toolforge.

I also wonder whether this would be an opportunity to migrate from separate developer accounts to general SUL accounts, and whether that needs a separate task.

Reedy subscribed.Dec 2 2021, 5:52 AM
In T296893#7542832, @Xover wrote:

I also wonder whether this would be an opportunity to migrate from separate developer accounts to general SUL accounts, and whether that needs a separate task.

T161859: Make Wikitech an SUL wiki

• mmodell subscribed.Dec 2 2021, 9:18 AM
bd808 added a comment.Dec 2 2021, 3:42 PM
In T296893#7542832, @Xover wrote:

@bd808 I still can't find any mention of Toolforge on mw:GitLab or its subpages

Seems reasonable since I just created this task and haven't started talking with the fine folks who have been working on Gitlab about it yet. We will figure it out, that's what we do here.

Legoktm subscribed.Dec 2 2021, 4:52 PM

It would be nice if we could automatically synchronize Tool maintainer lists and Git repository owners, whether that's automated via LDAP or manually by Striker.

• mmodell added a comment.EditedDec 7 2021, 1:04 AM
In T296893#7544085, @Legoktm wrote:

It would be nice if we could automatically synchronize Tool maintainer lists and Git repository owners, whether that's automated via LDAP or manually by Striker.

I'd like to automate management of git repos as well, for the sake of easy management and tracking what exists in gitlab, diffusion, gerrit and github. My ideal would be a tool that can easily set up a repo's authoritative source and then set up mirroring automatically everywhere that it should be.

diegodlh subscribed.Jan 12 2022, 3:30 PM
brennen moved this task from Inbox to Project Migration on the GitLab board.Jan 25 2022, 9:24 PM
brennen edited projects, added GitLab (Project Migration); removed GitLab.
Aklapper mentioned this in T191182: Migrate active repositories in Phabricator Differential to GitLab.Mar 30 2022, 9:28 AM
bd808 mentioned this in T310493: Renamed tool diffusion repository failed to be updated in toolsadmin.wikimedia.org.Jun 21 2022, 3:46 PM
Stang subscribed.Jun 21 2022, 3:52 PM
bd808 moved this task from Backlog to Ready on the Striker board.Jul 7 2022, 7:39 PM
gerritbot added a comment.Jul 7 2022, 7:43 PM

Change 812066 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[labs/striker@master] dev: Add gitlab container to docker-compose config

https://gerrit.wikimedia.org/r/812066

gerritbot added a project: Patch-For-Review.Jul 7 2022, 7:43 PM
gerritbot added a comment.Jul 7 2022, 8:32 PM

Change 812066 merged by jenkins-bot:

[labs/striker@master] dev: Add gitlab container to docker-compose config

https://gerrit.wikimedia.org/r/812066

Maintenance_bot removed a project: Patch-For-Review.Jul 7 2022, 9:30 PM
bd808 triaged this task as High priority.Jul 20 2022, 8:02 PM
bd808 added a project: Goal.
bd808 moved this task from Ready to Doing on the Striker board.Aug 5 2022, 9:49 PM
• demon closed subtask T313366: Figure out workflow for programatically adding GitLab users as Resolved.Aug 9 2022, 5:10 PM
• demon mentioned this in T313366: Figure out workflow for programatically adding GitLab users.
bd808 removed a subtask: T316295: Connect Striker created GitLab projects with Striker created Phabricator projects.Aug 25 2022, 9:39 PM
bd808 added a parent task: T316295: Connect Striker created GitLab projects with Striker created Phabricator projects.
bd808 changed the task status from Open to In Progress.Aug 26 2022, 9:37 PM
bd808 claimed this task.
Restricted Application added a project: User-bd808. · View Herald TranscriptAug 26 2022, 9:37 PM
bd808 changed the status of subtask T315706: Migrate existing Striker created Diffusion repos to GitLab from Open to In Progress.Sep 1 2022, 12:02 AM
bd808 added a project: Developer-Advocacy (Jul-Sep 2022).
gerritbot added a comment.Sep 1 2022, 12:08 AM

Change 828659 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[labs/striker@master] feat: create and view gitlab repos

https://gerrit.wikimedia.org/r/828659

gerritbot added a project: Patch-For-Review.Sep 1 2022, 12:08 AM
gerritbot added a comment.Sep 1 2022, 7:33 PM

Change 828659 merged by jenkins-bot:

[labs/striker@master] feat: create and view gitlab repos

https://gerrit.wikimedia.org/r/828659

gerritbot added a comment.Sep 1 2022, 8:05 PM

Change 829058 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[labs/striker@master] dev: Add dummy GITLAB_ACCESS_TOKEN value for prep builds

https://gerrit.wikimedia.org/r/829058

gerritbot added a comment.Sep 1 2022, 8:30 PM

Change 829058 merged by jenkins-bot:

[labs/striker@master] dev: Add dummy GITLAB_ACCESS_TOKEN value for prep builds

https://gerrit.wikimedia.org/r/829058

Maintenance_bot removed a project: Patch-For-Review.Sep 1 2022, 8:31 PM
gerritbot added a comment.Sep 2 2022, 6:38 PM

Change 829235 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[labs/striker@master] bug: unwrap user info from gitlab lookup response

https://gerrit.wikimedia.org/r/829235

gerritbot added a project: Patch-For-Review.Sep 2 2022, 6:38 PM
gerritbot added a comment.Sep 2 2022, 8:58 PM

Change 829235 merged by jenkins-bot:

[labs/striker@master] fix: unwrap user info from gitlab lookup response

https://gerrit.wikimedia.org/r/829235

Maintenance_bot removed a project: Patch-For-Review.Sep 2 2022, 9:30 PM
gerritbot added a comment.Sep 6 2022, 7:57 PM

Change 830252 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/puppet@production] striker: Bump deployed version to 2022-09-04-055313-production

https://gerrit.wikimedia.org/r/830252

gerritbot added a project: Patch-For-Review.Sep 6 2022, 7:57 PM
gerritbot added a comment.Sep 6 2022, 8:08 PM

Change 830252 merged by Andrew Bogott:

[operations/puppet@production] striker: Bump deployed version to 2022-09-04-055313-production

https://gerrit.wikimedia.org/r/830252

Stashbot added a comment.Sep 6 2022, 8:13 PM

Mentioned in SAL (#wikimedia-operations) [2022-09-06T20:13:23Z] <bd808> Running database migrations for Striker (T296893)

Stashbot added a comment.Sep 6 2022, 8:16 PM

Mentioned in SAL (#wikimedia-operations) [2022-09-06T20:16:27Z] <bd808> Forcing puppet runs on cloudweb100[34] to deploy new version of Striker (T296893)

Maintenance_bot removed a project: Patch-For-Review.Sep 6 2022, 8:31 PM
gerritbot added a comment.Sep 6 2022, 9:26 PM

Change 830270 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[labs/striker@master] fix: Use kwargs to lookup reverse url for GitlabRepo model

https://gerrit.wikimedia.org/r/830270

gerritbot added a project: Patch-For-Review.Sep 6 2022, 9:26 PM
gerritbot added a comment.Sep 6 2022, 9:37 PM

Change 830270 merged by jenkins-bot:

[labs/striker@master] fix: Use kwargs to lookup reverse url for GitlabRepo model

https://gerrit.wikimedia.org/r/830270

gerritbot added a comment.Sep 6 2022, 9:53 PM

Change 830275 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/puppet@production] striker: bump container version to 2022-09-06-213820-production

https://gerrit.wikimedia.org/r/830275

gerritbot added a comment.Sep 6 2022, 10:00 PM

Change 830275 merged by Andrew Bogott:

[operations/puppet@production] striker: bump container version to 2022-09-06-213820-production

https://gerrit.wikimedia.org/r/830275

gerritbot added a comment.Sep 7 2022, 7:04 PM

Change 830679 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[labs/striker@master] feat: update git goal for gitlab provider change

https://gerrit.wikimedia.org/r/830679

gerritbot added a comment.Sep 7 2022, 7:17 PM

Change 830679 merged by jenkins-bot:

[labs/striker@master] feat: update git goal for gitlab provider change

https://gerrit.wikimedia.org/r/830679

gerritbot added a comment.Sep 7 2022, 7:46 PM

Change 830688 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/puppet@production] striker: bump container version to 2022-09-07-191738-production

https://gerrit.wikimedia.org/r/830688

gerritbot added a comment.Sep 7 2022, 7:53 PM

Change 830688 merged by Andrew Bogott:

[operations/puppet@production] striker: bump container version to 2022-09-07-191738-production

https://gerrit.wikimedia.org/r/830688

bd808 closed subtask T315706: Migrate existing Striker created Diffusion repos to GitLab as Resolved.Sep 8 2022, 2:04 AM
bd808 closed subtask T317272: Remove legacy Diffusion related code from Striker as Declined.Sep 8 2022, 4:38 PM
bd808 updated the task description. (Show Details)Oct 13 2022, 8:17 PM
bd808 mentioned this in T320438: Fix misleading "Diffusion" creation in Striker (actually creates on GitLab).Oct 13 2022, 8:19 PM
bd808 closed subtask T320438: Fix misleading "Diffusion" creation in Striker (actually creates on GitLab) as Resolved.Oct 13 2022, 8:44 PM
bd808 changed the status of subtask T317345: Mirror Striker managed GitLab repos in Diffusion from Open to In Progress.Oct 17 2022, 10:41 PM
bd808 closed this task as Resolved.Oct 24 2022, 4:33 PM
bd808 closed subtask T317345: Mirror Striker managed GitLab repos in Diffusion as Resolved.
Stang unsubscribed.Oct 27 2022, 10:33 PM
Aklapper mentioned this in T329428: Update all "Source Repo" fields of Phabricator projects which have a Diffusion URL.Feb 11 2023, 6:54 PM
Maintenance_bot removed a project: Patch-For-Review.Feb 11 2023, 7:30 PM
bd808 mentioned this in T324151: GitLab repos are not correctly mirrored to Diffusion.Jun 8 2023, 4:33 PM
bd808 mentioned this in T359549: Drop our mirroring of code to Diffusion and empty the repos.Apr 5 2024, 4:48 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL