Investigate again a central cache for package managers
Open, Stalled, NormalPublic

Description

Package managers need to hit some cache to speed them up. We have a homegrown system based on rsync nicknamed Castor. I would like to replace it with something more robust.

Some past investigation is on T112560 and sub tasks.

Sonatype Nexus 2.x did not support npm/rubygems/pip. Version 3 does. On https://www.sonatype.com/download-oss-sonatype

Nexus Repository Manager OSS 3.xx

Manage these formats:

Docker NuGet npm Bower PyPI Ruby Gems

Related docs:

Wikimedia has Apache Archiva for Maven artifacts but it is not in proxy mode and I dont think it supports npm/pypi/rubygems.

There is a feature matrix that is apparently kept up-to-date by maintainers of Archiva, Artifactory, Sonatype Nexus, Eclipse Package Drone, and ProGet: https://binary-repositories-comparison.github.io/

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 7 2016, 12:34 PM

Change 314751 had a related patch set uploaded (by Hashar):
(WIP) contint: Sonatype Nexus (WIP)

https://gerrit.wikimedia.org/r/314751

Did a basic puppet sprint and applied it on repository.integration.eqiad.wmflabs

Nexus got installed from the tar.gz

I have done a basic configuration which has a proxy for each of maven/npm/rubygems and pypi.

I crafted a very lame puppet manifest. Sonatype provides their software as a tar.gz that has 266 .jar probably not worth trying to make a Debian package out of it.

Marko pointed out that it would be a good use case for scap3 deployment which would ship the code AND contain the setting files. So in theory we could have the instance to be a deploy server then just git pull && scap deploy to upgrade Nexus.

hashar triaged this task as Normal priority.Oct 11 2016, 8:48 AM

Change 314751 abandoned by Hashar:
(WIP) contint: Sonatype Nexus (WIP)

Reason:
The instance works on labs, I used with a few jenkins jobs and it works properly in the few use cases I tried.

However I don't have time to figure out how to deploy the Nexus java application itself. So abandoning until one has time to allocate to that.

https://gerrit.wikimedia.org/r/314751

Mentioned in SAL (#wikimedia-releng) [2017-02-17T14:07:58Z] <hashar> integration: deleting "repository" instance. No time to figure out how to ship Sonatype Nexus to it. T147635

The instance worked on labs, I used with a few jenkins jobs and it works properly in the few use cases I tried.

However I don't have time to figure out how to deploy the Nexus java application itself. So abandoning until one has time to allocate to that.

I have deleted the instance.

hashar changed the task status from Open to Stalled.Feb 17 2017, 2:09 PM

Stalled until we get bandwidth to figure out how to ship Sonatype Nexus in a better way than wget && tar -xvf.