Page MenuHomePhabricator

Investigate replacing Archiva with Gitlab repositories
Open, MediumPublic

Description

Since Gitlab provides some level of repository hosting, it make sense to investigate if we can replace Archiva with it. This would allow to decommission one service and simplify operations.

We have an open discussion about transferring ownership of Archiva to collaboration-services & Release-Engineering-Team, which contains some context.

Use cases of Archiva

  • hosting of Java binaries released by WMF for public consumption
  • hosting of Java binaries released by WMF for deployment on WMF infrastructure
  • hosting of Java binaries released by WMF for reuse in other WMF projects (libraries)
  • mirror of Maven Central (and a few other repositories) for use in build and CI, providing a level of isolation to protect against supply chain attacks
  • hosting of Python wheels released by WMF for deployment on WMF infrastructure
  • integration with git-fat for scap deployment of binaries

General document deescribing the migration: https://docs.google.com/document/d/113xqLy4hGU-3izKGNrmATI0RKLAFjdvXnRJ0rvVszBw/edit

Related Objects

StatusSubtypeAssignedTask
OpenNone

Event Timeline

Gehel triaged this task as Medium priority.Feb 28 2024, 9:28 AM
Gehel moved this task from Incoming to Toil / Automation on the Data-Platform-SRE board.

+1! to this idea!

integration with git-fat for scap deployment of binaries

We probably want to and can stop using git-fat (T279509) if we do this. We made an 'artifact syncing' tool for airflow artifacts that works with gitlab package repos, and will work with maven artifacts. We then use scap hooks (e.g. here) to run the artifact sync script to download the artifacts to the target host.

The system probably needs improvements before we can replace the git-fat system though. There is no artifact cleanup, there isn't any checksum verification after the copy (a change in the source file will not be reflected).

One notable change which makes this task more important is that Archiva development has officially stopped, quoting from https://attic.apache.org/projects/archiva.html

Apache Archiva moved into the Attic in 2024-02. Apache Archiva mission was the creation and maintenance of software related to Build Artifact Repository Manager.

I dug a little in the mailing list archives and the relevant thread appears to be https://lists.apache.org/thread/nt9prhzm57fzbwvk63lxcfps8drlh6g6

Gehel added a subtask: Restricted Task.Mar 6 2024, 2:45 PM
BTullis added a parent task: Restricted Task.Mar 25 2024, 4:55 PM

The dependence on Archiva for git-fat has been almost completely removed now, as far as I am aware. There is one patch relating to blazegraph/wdqs that is still to be merged, then we will have completed that part of it.