Page MenuHomePhabricator

wikibase-docker_wdqs-frontend_1 stopped working
Closed, ResolvedPublic

Description

VM: mixnmatch in mix-n-match project
It worked fine since setup about a week ago. This morning it was 502. Did

sudo docker-compose up --no-build -d

All containers come up, but:

magnus@mixnmatch:~/wikibase-docker$ sudo docker-compose ps
              Name                             Command               State           Ports
--------------------------------------------------------------------------------------------------
wikibase-docker_elasticsearch_1     /docker-entrypoint.sh elas ...   Up       9200/tcp, 9300/tcp
wikibase-docker_mysql_1             docker-entrypoint.sh mysqld      Up       3306/tcp
wikibase-docker_quickstatements_1   /bin/bash /entrypoint.sh         Up       0.0.0.0:9191->80/tcp
wikibase-docker_wdqs-frontend_1     /entrypoint.sh nginx -g da ...   Exit 1
wikibase-docker_wdqs-proxy_1        /bin/sh -c "/entrypoint.sh"      Up       0.0.0.0:8989->80/tcp
wikibase-docker_wdqs-updater_1      /entrypoint.sh /runUpdate.sh     Up
wikibase-docker_wdqs_1              /entrypoint.sh /runBlazegr ...   Up       9999/tcp
wikibase-docker_wikibase_1          /bin/bash /entrypoint.sh         Up       0.0.0.0:8181->80/tcp

I have no idea how to debug blazegraph (which is why I went for the "all-in-one-package" in the first place), let alone how to do this in docker.
This breaks essentially all functionality on the tool. Someone please help!

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

can you run "docker-compose logs wdqs-frontend" and send the output?

You might just be able to restart it and it come back up again. (it depends on why it has died)

The frontend not coming up shouldn't stop you from talking to the wdqs-proxy which is what you should be using as the sparql endpoint.

End of the logfile:

wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:17:15 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?query=prefix%20schema:%20%3Chttp://schema.org/%3E%20SELECT%20*%20WHERE%20%7B%3Chttp://www.wikidata.org%3E%20schema:dateModified%20?y%7D&nocache=25661236 HTTP/1.1" 502 173 "http://mixnmatch-query.wmflabs.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:63.0) Gecko/20100101 Firefox/63.0" "193.62.205.8"
wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:24:55 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?format=json&query=SELECT+%3Fq+%3FqLabel+%7B+%3Fq+wdt%3AP2+wd%3AQ2+.+SERVICE+wikibase%3Alabel+%7B+bd%3AserviceParam+wikibase%3Alanguage+%27%5BAUTO_LANGUAGE%5D%2Cen%27+%7D+%7D HTTP/1.1" 502 173 "-" "-" "172.16.1.251"
wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:24:58 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?format=json&query=SELECT+%3Fq+%3FqLabel+%7B+%3Fq+wdt%3AP2+wd%3AQ2+.+SERVICE+wikibase%3Alabel+%7B+bd%3AserviceParam+wikibase%3Alanguage+%27%5BAUTO_LANGUAGE%5D%2Cen%27+%7D+%7D HTTP/1.1" 502 173 "-" "-" "172.16.1.251"
wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:25:33 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?format=json&query=SELECT+%3Fq+%3FqLabel+%7B+%3Fq+wdt%3AP2+wd%3AQ2+.+SERVICE+wikibase%3Alabel+%7B+bd%3AserviceParam+wikibase%3Alanguage+%27%5BAUTO_LANGUAGE%5D%2Cen%27+%7D+%7D HTTP/1.1" 502 173 "-" "-" "172.16.1.251"
wdqs-frontend_1    | 10.68.21.68 - - [16/Oct/2018:07:25:43 +0000] "GET /proxy/wdqs/bigdata/namespace/wdq/sparql?format=json&query=SELECT+%3Fq+%3FqLabel+%7B+%3Fq+wdt%3AP2+wd%3AQ2+.+SERVICE+wikibase%3Alabel+%7B+bd%3AserviceParam+wikibase%3Alanguage+%27%5BAUTO_LANGUAGE%5D%2Cen%27+%7D+%7D HTTP/1.1" 502 173 "-" "-" "172.16.1.251"
wdqs-frontend_1    | 2018/10/16 07:26:38 [emerg] 1#1: host not found in upstream "wikibase.svc" in /etc/nginx/conf.d/default.conf:8
wdqs-frontend_1    | nginx: [emerg] host not found in upstream "wikibase.svc" in /etc/nginx/conf.d/default.conf:8

I'll try the proxy.

So, it looks like the only reason the wdqs-frontend container stopped is because it couldn't connect to the wikibase service any more..

wdqs-frontend_1    | 2018/10/16 07:26:38 [emerg] 1#1: host not found in upstream "wikibase.svc" in /etc/nginx/conf.d/default.conf:8
wdqs-frontend_1    | nginx: [emerg] host not found in upstream "wikibase.svc" in /etc/nginx/conf.d/default.conf:8

So as long as the wikibase service is running you should just be able to start the front end service again.

OK, another sudo docker-compose up --no-build -d did it. Thanks.

Is this normal? Is there a startup order defined somewhere, or can I define one, so that the services come up as they should?

Magnus claimed this task.

If you are using the example then yes there is sort of a predefined start up order.

https://docs.docker.com/compose/startup-order/ are the docs for how it works

The important bit is in the second paragraph though:

You can control the order of service startup with the depends_on option. Compose always starts containers in dependency order, where dependencies are determined by depends_on, links, volumes_from, and network_mode: "service:...".

However, Compose does not wait until a container is “ready” (whatever that means for your particular application) - only until it’s running. There’s a good reason for this.