While we are largely dependent on upstream there are probably some things we could do to improve performance on macOS.
Description
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Open | None | T273305 [epic] Improve MediaWiki-Docker performance on macOS | |||
Open | None | T272500 Create arm64 image variants of releng/dev-images used by MediaWiki-Docker | |||
Declined | None | T274140 Create and publish arm64 images of wikimedia-stretch and wikimedia-buster | |||
Declined | None | T273307 Investigate using Mutagen to boost performance | |||
Open | None | T273309 Consider moving SQLite database into a volume, don't use cache directory in mediawiki-docker | |||
Open | None | T273533 Investigate VSCode Remote Containers |
Event Timeline
There's a new experimental build of Docker for Mac using a macOS 12 virtualization feature to boost performance that interested people might want to test out: https://github.com/docker/roadmap/issues/7#issuecomment-975447079. For the moment, I believe it's for M1 systems only.
@thcipriani and I were talking about this general problem today and started wondering if we could unblock things by pivoting to some other base images and container registry that split from the constraints of WMF production container constraints. Using WMF managed apt repos including custom package builds is, if I'm understanding things correctly, the main blocker we have for publishing multi-arch images supporting both AMD and ARM. WMF local apt at least was the death of T274140: Create and publish arm64 images of wikimedia-stretch and wikimedia-buster.
What software and config are actually in docker-registry.wikimedia.org/dev/buster-php81-fpm:1.0.1-s2 that are needed for the dev environment? Is there a supported upstream base image from Debian that we could build on instead to get multi-arch parity? Would it actually make some sense to move to using a Dockerfile in mediawiki/core.git that builds upon an upstream base image instead of buster-php81-fpm?
What software and config are actually in docker-registry.wikimedia.org/dev/buster-php81-fpm:1.0.1-s2 that are needed for the dev environment?
At that layer, we're just installing php-fpm and some config:
FROM {{ "buster-php81" | image_tag }} AS buster-php80 # We don't want the default index.html for running PHP applications, and # xdebug is a major performance drag, so disable it here (it's already # been disabled for CLI PHP in buster-php72). RUN {{ "php8.1-fpm" | apt_install }} # php-fpm configuration: COPY php-fpm.conf /etc/php/8.1/fpm/php-fpm.conf RUN cp /docker/www.conf /etc/php/8.1/fpm/pool.d/www.conf # MediaWiki-specific php.ini settings # upload_max_filesize and post_max_size here match production, per T246930 RUN echo 'opcache.file_update_protection=0\n\ opcache.memory_consumption=256\n\ opcache.max_accelerated_files=24000\n\ opcache.max_wasted_percentage=10\n\ zlib.output_compression=On\n\ upload_max_filesize=100M\n\ post_max_size=100M\n' | tee -a /etc/php/8.1/fpm/php.ini /etc/php/8.1/cli/php.ini RUN cp --force /docker/xdebug.ini "/etc/php/$PHP_VERSION/fpm/conf.d/20-xdebug.ini" EXPOSE 9000 WORKDIR "/var/www/html/w" ENTRYPOINT ["/bin/bash", "/php_entrypoint.sh"]
That's on top of buster-php81, which installs the PHP base, the CLI, and a list of extensions (plus the laundry list that is build-essential, for reasons I can't now remember) from the Sury repos:
FROM {{ "buster-php-sury" | image_tag }} AS buster-php-sury ENV PHP_VERSION=8.1 # Install PHP dependencies: # Not installing php-tidy here - see https://phabricator.wikimedia.org/T191771 {% set mediawiki_deps|replace('\n', ' ') -%} php8.1 php8.1-apcu php8.1-ast php8.1-bcmath php8.1-cli php8.1-curl php8.1-gd php8.1-gmp php8.1-intl php8.1-mbstring php8.1-sqlite3 php8.1-xml php8.1-zip php8.1-redis php8.1-dev php8.1-ldap php8.1-mysql php8.1-pgsql php8.1-xdebug build-essential {%- endset -%} RUN {{ mediawiki_deps | apt_install }} RUN cp -f /docker/xdebug.ini "/etc/php/$PHP_VERSION/cli/conf.d/20-xdebug.ini"
I think we were only using buster-php-sury (just adds the apt sources) where there aren't yet WMF packages for a PHP version. (That may be a bit out of date by now.)
Beyond that, the environment's using:
- an image with Apache, a handful of modules, and some config
- a jobrunner image that adds ffmpeg to buster-php81
In principle: I don't think there's anything here that couldn't run on some upstream image. This is mostly a pretty standard httpd + php-fpm setup, with some fiddly bits.