Page MenuHomePhabricator

Ensure zuul-executor can prepare workspaces and dispatch builds
Open, MediumPublic

Description

Now that we have a tenant configuration (see T406384: Add a zuul tenant config on the zuul scheduler host (zuul1001)) and communication between the scheduler and Gerrit, we need to make sure zuul-executor can prepare workspaces for and successfully dispatch builds.

The zuul-executor services is currently having trouble connecting from zuul1002 to zookeeper on zuul1001.

Apr 21 16:30:42 zuul1002 docker[3945930]: 2026-04-21 16:30:42,157 WARNING zuul.zk.base.ZooKeeperClient: Retrying zookeeper connection

It appears to be a network level issue, and we'll likely need a new firewall rule.

dduvall@zuul1002:~$ nc -vz -w 3 zuul1001 2281
nc: connect to zuul1001 (2620:0:861:103:10:64:32:104) port 2281 (tcp) timed out: Operation now in progress
nc: connect to zuul1001 (10.64.32.104) port 2281 (tcp) timed out: Operation now in progress

Event Timeline

Change #1275972 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/puppet@production] zuul: Allow connections to zookeeper from executors

https://gerrit.wikimedia.org/r/1275972

Change #1275972 merged by Dzahn:

[operations/puppet@production] zuul: Allow connections to zookeeper from executors

https://gerrit.wikimedia.org/r/1275972

Since https://gerrit.wikimedia.org/r/c/operations/puppet/+/1275972 added a firewall rule the executors can now talk to zookeeper on the main machines.

Change #1300849 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/puppet@production] zuul: Remove `tls-server-name` from kubeconfig

https://gerrit.wikimedia.org/r/1300849

Change #1300849 merged by Dzahn:

[operations/puppet@production] zuul: Remove `tls-server-name` from kubeconfig

https://gerrit.wikimedia.org/r/1300849

Change #1300922 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/puppet@production] zuul: Update certificate_authority_data for new cluster

https://gerrit.wikimedia.org/r/1300922

Change #1300922 merged by Dzahn:

[operations/puppet@production] zuul: Update certificate_authority_data for new cluster

https://gerrit.wikimedia.org/r/1300922