Page MenuHomePhabricator

wbs-deploy-wikibase-1 is unhealthy
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

Followed directions here: https://github.com/wmde/wikibase-release-pipeline/blob/main/deploy/README.md.

  • Install Docker.
  • Open Git Bash.
  • Run git clone https://github.com/wmde/wikibase-release-pipeline.
  • Then cd wikibase-release-pipeline/deploy.
  • Then git checkout deploy-3.
  • Next cp template.env .env.
  • Open .env and change WIKIBASE_PUBLIC_HOST to localhost.
  • Uncomment line 194 in docker-compose.yml.
  • Run docker-compose up --wait.

What happens?:

The Docker containers do not instantiate correctly:

$ docker compose up --wait
 Network wbs-deploy_default  Creating
 Network wbs-deploy_default  Created
 Container wbs-deploy-mysql-1  Creating
 Container wbs-deploy-traefik-1  Creating
 Container wbs-deploy-elasticsearch-1  Creating
 Container wbs-deploy-traefik-1  Created
 Container wbs-deploy-mysql-1  Created
 Container wbs-deploy-wikibase-1  Creating
 Container wbs-deploy-elasticsearch-1  Created
 Container wbs-deploy-wikibase-1  Created
 Container wbs-deploy-wikibase-jobrunner-1  Creating
 Container wbs-deploy-wdqs-1  Creating
 Container wbs-deploy-quickstatements-1  Creating
 Container wbs-deploy-wdqs-1  Created
 Container wbs-deploy-wdqs-proxy-1  Creating
 Container wbs-deploy-wdqs-updater-1  Creating
 Container wbs-deploy-wikibase-jobrunner-1  Created
 Container wbs-deploy-quickstatements-1  Created
 Container wbs-deploy-wdqs-updater-1  Created
 Container wbs-deploy-wdqs-proxy-1  Created
 Container wbs-deploy-wdqs-frontend-1  Creating
 Container wbs-deploy-wdqs-frontend-1  Created
 Container wbs-deploy-elasticsearch-1  Starting
 Container wbs-deploy-mysql-1  Starting
 Container wbs-deploy-traefik-1  Starting
 Container wbs-deploy-mysql-1  Started
 Container wbs-deploy-mysql-1  Waiting
 Container wbs-deploy-traefik-1  Started
 Container wbs-deploy-elasticsearch-1  Started
 Container wbs-deploy-mysql-1  Healthy
 Container wbs-deploy-wikibase-1  Starting
 Container wbs-deploy-wikibase-1  Started
 Container wbs-deploy-wikibase-1  Waiting
 Container wbs-deploy-wikibase-1  Waiting
 Container wbs-deploy-wikibase-1  Waiting
 Container wbs-deploy-wikibase-1  Error
 Container wbs-deploy-wikibase-1  Error
 Container wbs-deploy-wikibase-1  Error
dependency failed to start: container wbs-deploy-wikibase-1 is unhealthy

What should have happened instead?:

The Docker containers should have been spawned healthily.

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):

  • Microsoft Windows 11 Pro
  • Git version 2.45.windows.1
  • Visual Studio Code version 1.92.2 (for editing .env and docker-compose.yml
  • Docker version 4.33.1
  • Docker Compose version v2.29.1-desktop.1

I am able to successfully ping Google, so it isn't a network connection issue. Plenty of free space on the computer as well.

Was spawning normally until two days ago with same command set up. Reinstalled everything just in case something else was causing an issue, but no dice.

Also tried without the uncommented line 194 and without the change to WIKIBASE_PUBLIC_HOST. Same results.

Other information (browser name/version, screenshots, etc.):

N/A

Event Timeline

I thought maybe the issue was something to do with line 194 in docker-compose.yml, so I waited a couple days to re-spawn the system (until just now), but it is still not working:

[+] Running 17/17
 ✔ Network wbs-deploy_default                    Created          0.0s
 ✔ Volume "wbs-deploy_wdqs-data"                 Created          0.0s
 ✔ Volume "wbs-deploy_traefik-letsencrypt-data"  Created          0.0s
 ✔ Volume "wbs-deploy_elasticsearch-data"        Created          0.0s
 ✔ Volume "wbs-deploy_quickstatements-data"      Created          0.0s
 ✔ Volume "wbs-deploy_wikibase-image-data"       Created          0.0s
 ✔ Volume "wbs-deploy_mysql-data"                Created          0.0s
 ✔ Container wbs-deploy-elasticsearch-1          Started          0.8s
 ✔ Container wbs-deploy-mysql-1                  Healthy         11.6s
 ✔ Container wbs-deploy-traefik-1                Started          1.1s
 ✘ Container wbs-deploy-wikibase-1               Error           13.6s
 ✔ Container wbs-deploy-wikibase-jobrunner-1     Created          0.1s
 ✔ Container wbs-deploy-wdqs-1                   Created          0.1s
 ✔ Container wbs-deploy-quickstatements-1        Created          0.1s
 ✔ Container wbs-deploy-wdqs-updater-1           Created          0.1s
 ✔ Container wbs-deploy-wdqs-proxy-1             Created          0.1s
 ✔ Container wbs-deploy-wdqs-frontend-1          Created          0.1s
dependency failed to start: container wbs-deploy-wikibase-1 is unhealthy

This is almost certainly related to Traefik rate limits (see here: https://github.com/wmde/wikibase-release-pipeline/blob/main/deploy/README.md, under "Removing Wikibase Suite Completely with all its Data"). Because of this I'm closing for now, but I do think this should be mentioned earlier in documentation, as it seems to be an error multiple users are having.

When this happens again, please start the stack with docker compose up and omit the --wait. This will print status messages and could probably help debugging the issue of wikibase not switching to healthy state.