Page MenuHomePhabricator

Since the update to bookworm, unable to start mediawiki-web on my M2 Mac
Closed, ResolvedPublic

Description

Since T381728, I am unable to start mediawiki-web from docker-compose.yml using my M2 Mac:

urbanecm@wmf3345 core % docker compose up mediawiki-web
[+] Building 0.0s (0/0)                                                                                                                                                                                                                              
[+] Running 1/0
 ✔ Container core-mediawiki-web-1  Created                                                                                                                                                                                                      0.0s 
Attaching to core-mediawiki-web-1
core-mediawiki-web-1  | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.31.0.8. Set the 'ServerName' directive globally to suppress this message
core-mediawiki-web-1  | [Mon Dec 23 13:53:54.773460 2024] [core:emerg] [pid 14:tid 14] (95)Operation not supported: AH00023: Couldn't create the proxy mutex 
core-mediawiki-web-1  | [Mon Dec 23 13:53:54.776302 2024] [proxy:crit] [pid 14:tid 14] (95)Operation not supported: AH02478: failed to create proxy mutex
core-mediawiki-web-1  | AH00016: Configuration Failed
core-mediawiki-web-1  | Action '-D FOREGROUND' failed.
core-mediawiki-web-1  | The Apache error log may have more information.
core-mediawiki-web-1 exited with code 1
urbanecm@wmf3345 core %

Unfortunately, this functionally breaks my local development setup. Changing image for mediawiki-web to docker-registry.wikimedia.org/dev/buster-apache2:2.0.1 (the previous value) "fixes" the issue, but it means I'm continuing to use buster (rather than bookworm).

Event Timeline

Adding Mutex posixsem to apache2 in the container appears to fix the issue. That being said, I'm not sure how exactly (and whether that might break things on other systems).

Urbanecm_WMF added a project: Growth-Team.
Urbanecm_WMF moved this task from Inbox to Tracking on the Growth-Team board.

Adding Mutex posixsem to apache2 in the container appears to fix the issue. That being said, I'm not sure how exactly (and whether that might break things on other systems).

Indeed, that's what I'm seeing recommended in a few place, but I also have no idea what that fix would encompass. I also came across an interesting comment:

For anyone who stumbles onto this, I'm able to reproduce the issue by running an apache2 docker container on a non-native platform via qemu emulation.
If you're seeing the issue from a Docker image, try changing the platform to match the host system.

Is this something you could try?

T272500 should be somewhat related too, I guess.

For anyone who stumbles onto this, I'm able to reproduce the issue by running an apache2 docker container on a non-native platform via qemu emulation.
If you're seeing the issue from a Docker image, try changing the platform to match the host system.

Is this something you could try?

Possibly, but I am unsure what I should do to try that. Do you want me to try setting the development environment on a different computer than my work laptop? Or do you want me to change some of my Docker settings (on the Mac)? Or something else?

In case that matters: In my Docker settings, I have Use Virtualization framework enabled, but Use Rosetta for x86/amd64 emulation on Apple Silicon disabled, see screenshots:

image.png (623×1 px, 123 KB)
image.png (507×1 px, 111 KB)

FWIW, our own documentation specifically recommends to turn off Rosetta-based emulation, which is the primary reason for this configuration.

For anyone who stumbles onto this, I'm able to reproduce the issue by running an apache2 docker container on a non-native platform via qemu emulation.
If you're seeing the issue from a Docker image, try changing the platform to match the host system.

Is this something you could try?

Possibly, but I am unsure what I should do to try that. Do you want me to try setting the development environment on a different computer than my work laptop? Or do you want me to change some of my Docker settings (on the Mac)? Or something else?

The idea was to add a platform attribute to the compose.yml spec to see if it makes any difference. I don't see how that would work, though, since we don't have arm versions of dev images (T272500, T274140, etc.). However, judging from a conversation elsewhere, it seems like the issue is not necessarily related to M2?

In case that matters: In my Docker settings, I have Use Virtualization framework enabled, but Use Rosetta for x86/amd64 emulation on Apple Silicon disabled [...] FWIW, our own documentation specifically recommends to turn off Rosetta-based emulation, which is the primary reason for this configuration.

Makes sense to me, although I'm not familiar with (development on) mac. More generally, whatever configuration you had before should still be working now... In theory, that is.

Adding Mutex posixsem to apache2 in the container appears to fix the issue. That being said, I'm not sure how exactly (and whether that might break things on other systems).

I won't break other systems. It just fixes a bug in autodetection under emulation. I made the same fix to dev environments for Striker and Toolhub in the past.

What version of Docker/Docker Desktop are you using? Your screenshot shows an update is available. iirc there were two relatively recent (last year or two) Docker Desktop on Mac versions which had bugs which prevented Mediawiki containers from working

Does this persist after bringing containers down, removing all images (docker system prune -af), then bringing everything back up?

If it helps, my config screen looks like this:

Screenshot 2025-01-02 at 4.01.25 PM.png (1×3 px, 1 MB)

FWIW, our own documentation specifically recommends to turn off Rosetta-based emulation, which is the primary reason for this configuration.

Which reads:

Mac users: If you're using Docker Desktop and have the Use Rosetta for x86/amd64 emulation on Apple Silicon setting enabled, you may encounter an issue where loading the wiki results in a blank page or a 503 error. To resolve this, disable the setting and restart Docker Desktop. See this page ( https://phabricator.wikimedia.org/P49617 ) for more information.

^ This is almost certainly out of date / wrong. I added a comment to the paste

Change #1108495 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[mediawiki/core@master] dev(docker): Bump mediawiki-web container to dev/bookworm-apache2:1.0.1

https://gerrit.wikimedia.org/r/1108495

What version of Docker/Docker Desktop are you using? Your screenshot shows an update is available. iirc there were two relatively recent (last year or two) Docker Desktop on Mac versions which had bugs which prevented Mediawiki containers from working

urbanecm@wmf3345 ~ % docker version
Client:
 Cloud integration: v1.0.33
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:16 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.20.1 (110738)
 Engine:
  Version:          24.0.2-38-g8e70a1b23e
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       8e70a1b23e965d86ec8c2feb77605196ae124630
  Built:            Fri Jun  2 15:58:50 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
urbanecm@wmf3345 ~ %

Does this persist after bringing containers down, removing all images (docker system prune -af), then bringing everything back up?

I can reproduce even after recreating the whole environment. @bd808's patch fixes the issue though.

Change #1108495 merged by jenkins-bot:

[mediawiki/core@master] dev(docker): Bump mediawiki-web container to dev/bookworm-apache2:1.0.1

https://gerrit.wikimedia.org/r/1108495

Change #1108789 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[mediawiki/core@REL1_43] dev(docker): Bump mediawiki-web container to dev/bookworm-apache2:1.0.1

https://gerrit.wikimedia.org/r/1108789

Change #1108792 had a related patch set uploaded (by Jforrester; author: Bryan Davis):

[mediawiki/core@REL1_42] dev(docker): Bump mediawiki-web container to dev/bookworm-apache2:1.0.1

https://gerrit.wikimedia.org/r/1108792

bd808 claimed this task.

I can reproduce even after recreating the whole environment. @bd808's patch fixes the issue though.

Closing per this comment. Please do reopen if there are still known issues that we should address.

@Urbanecm_WMF

It would be interesting to see if the issue is present on Docker Desktop 4.37.1

You're on Docker Desktop 4.20.1 from Jun 2023

Change #1108792 merged by jenkins-bot:

[mediawiki/core@REL1_42] dev(docker): Bump mediawiki-web container to dev/bookworm-apache2:1.0.1

https://gerrit.wikimedia.org/r/1108792

Change #1108789 merged by jenkins-bot:

[mediawiki/core@REL1_43] dev(docker): Bump mediawiki-web container to dev/bookworm-apache2:1.0.1

https://gerrit.wikimedia.org/r/1108789

@Urbanecm_WMF

It would be interesting to see if the issue is present on Docker Desktop 4.37.1

You're on Docker Desktop 4.20.1 from Jun 2023

The bug that was exposed is in the Apache2 + QEMU emulating ARM64 combination. The main way that newer Docker Desktop for macOS avoids it is by replacing QEMU with another virtualization container. Apple Virtualization framework in macOS 12.5+ is how most of us missed this particular problem.