Change Details

Recently a user and bd808 reported that the job builder is failing. The cause seems to be harbor, which has some flapping services: ``` harbor-core /harbor/entrypoint.sh Restarting harbor-exporter /harbor/entrypoint.sh Up harbor-jobservice /harbor/entrypoint.sh Restarting harbor-log /bin/sh -c /usr/local/bin/ ... Up (healthy) 127.0.0.1:1514->10514/tcp harbor-portal nginx -g daemon off; Up (healthy) nginx nginx -g daemon off; Up (healthy) 0.0.0.0:80->8080/tcp, 0.0.0.0:9090->9090/tcp redis redis-server /etc/redis.conf Up (healthy) registry /home/harbor/entrypoint.sh Up (healthy) registryctl /home/harbor/start.sh Up (healthy) ``` Occasionally those 'restarting' services switch to "Up (health: starting)" and then drop back into 'restarting. Initially we assumed that this was an issue with the recent builds-builder upgrade (resulting in a revert, and then a reverted revert.)

Recently @Hawkeye7 and @bd808 reported that the job builder is failing. ```lang=irc [22:44] < Guest85> BuildClientError: The build service seems to be down – please retry in a few minutes. [22:44] < Guest85> Please report this issue to the Toolforge admins if it persists: https://w.wiki/6Zuu [22:46] < bd808> Guest85: did the error persist? [22:47] < Guest85> Been that way for half an hour [22:48] < Guest85> Ah. bd808. I am Hawkeye7 logged on as a guest ``` The cause seems to be harbor, which has some flapping services: ``` harbor-core /harbor/entrypoint.sh Restarting harbor-exporter /harbor/entrypoint.sh Up harbor-jobservice /harbor/entrypoint.sh Restarting harbor-log /bin/sh -c /usr/local/bin/ ... Up (healthy) 127.0.0.1:1514->10514/tcp harbor-portal nginx -g daemon off; Up (healthy) nginx nginx -g daemon off; Up (healthy) 0.0.0.0:80->8080/tcp, 0.0.0.0:9090->9090/tcp redis redis-server /etc/redis.conf Up (healthy) registry /home/harbor/entrypoint.sh Up (healthy) registryctl /home/harbor/start.sh Up (healthy) ``` Occasionally those 'restarting' services switch to "Up (health: starting)" and then drop back into 'restarting. Initially we assumed that this was an issue with the recent builds-builder upgrade (resulting in a revert, and then a reverted revert.)

Recently a user@Hawkeye7 and @bd808 reported that the job builder is failing. ```lang=irc [22:44] < Guest85> BuildClientError: The build service seems to be down – please retry in a few minutes. [22:44] < Guest85> Please report this issue to the Toolforge admins if it persists: https://w.wiki/6Zuu [22:46] < bd808> Guest85: did the error persist? [22:47] < Guest85> Been that way for half an hour [22:48] < Guest85> Ah. bd808. I am Hawkeye7 logged on as a guest ``` The cause seems to be harbor, which has some flapping services: ``` harbor-core /harbor/entrypoint.sh Restarting harbor-exporter /harbor/entrypoint.sh Up harbor-jobservice /harbor/entrypoint.sh Restarting harbor-log /bin/sh -c /usr/local/bin/ ... Up (healthy) 127.0.0.1:1514->10514/tcp harbor-portal nginx -g daemon off; Up (healthy) nginx nginx -g daemon off; Up (healthy) 0.0.0.0:80->8080/tcp, 0.0.0.0:9090->9090/tcp redis redis-server /etc/redis.conf Up (healthy) registry /home/harbor/entrypoint.sh Up (healthy) registryctl /home/harbor/start.sh Up (healthy) ``` Occasionally those 'restarting' services switch to "Up (health: starting)" and then drop back into 'restarting. Initially we assumed that this was an issue with the recent builds-builder upgrade (resulting in a revert, and then a reverted revert.)