Page MenuHomePhabricator

Developer productivity: arm64 versions of CI docker images
Closed, DeclinedPublic

Description

The current operations-puppet CI image and the images it relies on (buster, ci-buster and ci-common) are currently built only for amd64, which makes running local CI test jobs extremely long on M1 Mac.

For comparison, running utils/run_ci_locally.sh from operations/puppet on a small patchset takes :

run systemtime MM:SS
amd64 SRE X1 Carbon00:21
jenkins runner2:27
M1 Max docker-desktop with experimental virtualization.framework12:22
M1 Max docker-desktop with standard hypervisor.framework (qemu)13:46

Are there any plans to cross-compile ci images for arm64?

Event Timeline

hashar subscribed.

This got previously requested via T274140: Create and publish arm64 images of wikimedia-stretch and wikimedia-buster and declined:

In T274140#6832335, @MoritzMuehlenhoff wrote:

This can't be easily done, our repository is currently only built for amd64 (64 bit x86, which is the only architecture we use to run the site) and binary packages such as PHP would need to be built for arm64. Given that we currently only use a single architecture we also don't have a buildd infrastructure and CI. All the tooling exists in general for Debian, but building and maintaining the infrastructure would take quite a bit of time and resourcing.

Your task is very similar to T272500 for Developer Productivity images.

The utils/run_ci_locally.sh scripts runs docker-registry.wikimedia.org/releng/operations-puppet which is defined at https://gerrit.wikimedia.org/g/integration/config/+/refs/heads/master/dockerfiles/operations-puppet/Dockerfile.template . It depends on various Debian packages which are the same we use on production and are only build for amd64 architecture. To support running natively on Apple M1 Mac we would need to rebuild those packages which circles back to Moritz declining message above.

Then if you want to validate puppet changes locally you could get bundler/ruby/python installed on your local machine and invoke bundle exec rake test locally. Potentially wrap that in a local image.

Understood. I actually did get the testing suite installed locally, and the runtime is perfectly fine, so it's not that much of an issue for the particular use case of testing puppet changes.

I'll follow the related tasks if need be, and mark this one as declined since a workaround exists.

Thanks