Maybe this means git-lfs, but I'm thinking we could possibly leverage docker here since we already do that in train-dev.
Gerrit is deployed from a ~ 80MB war files + several jar files for plugins. The repository is https://gerrit.wikimedia.org/g/operations/software/gerrit.git and the artifacts are held in |deploy/wmf/stable-3.4` branch.
When updating Gerrit or its plugin, we commit the artifacts with git-fat and upload them using a Python script deploy_artifacts.py which wraps mvn deploy:deploy-file (see Gerrit/Upgrade#Upload_artifacts.
The process has a few issues:
- one need an account on archiva.wikimedia.org
- maven requires a user XML configuration with a locally encrypted credential
- artifacts are uploaded with an arbitrary version and a reupload in case of mistake requires a different version to be passed
- the python wrapper adds complexity
Since LFS storage is provided by Gerrit itself, deploying Gerrit with git-lfs would depend on Gerrit running. The upgrade process is managed by scap with a few manual tasks (Gerrit/Upgrade#Deploying) and it boils down to:
- deployment server: git pull
- deployment server: scap deploy
- on the deployment target hosts scap invokes git fat to pull the artifact from Archiva
- We manually stop Gerrit and refresh artifacts, refresh plugins then start Gerrit.
With git-lfs when scap fetch the artifacts Gerrit would be up and running and there will be no issue.
If the upgrade ever goes wrong and Gerrit is not running anymore or not serving LFS (for example if the LFS service itself is broken due to the upgrade) we would not be able to ship new artifacts. However given the artifacts are kept on the deployment targets scap cache, we should be able to rollback. If Gerrit is down, we would need to ensure we can indeed rollback to a previously deployed state and that git lfs being run by scap does not attempt to reach Gerrit but works entirely from the local disk.