Refill tool stuck "waiting for an available worker"
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	Keith_D
	Jun 15 2022, 10:18 PM

Description

Usual problem of tool just sitting waiting for a worker.

Related Objects

Mentioned In: T310754: Recurrent API worker failures

Event Timeline

Keith_D created this task.Jun 15 2022, 10:18 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 15 2022, 10:18 PM

Keith_D added a subscriber: Curb_Safe_Charmer.Jun 15 2022, 10:24 PM

TheresNoTime claimed this task.Jun 15 2022, 10:26 PM

TheresNoTime triaged this task as High priority.

@Keith_D thank you for the report — can you confirm the tool is working as expected now?

For reference @Curb_Safe_Charmer, to resolve this I deleted and re-created deployment by:

Step 1

Delete deployment

tools.refill-api@tools-sgebastion-10:~$ kubectl delete -f worker-deployment.yml

deployment.apps "refill-api-worker" deleted
deployment.apps "refill-api-scheduler" deleted

Step 2

Ensure only refill-api-* is running

tools.refill-api@tools-sgebastion-10:~$ kubectl get pods

NAME                          READY   STATUS    RESTARTS   AGE
refill-api-85457fc487-mlqrk   1/1     Running   0          23d

Step 3

Recreate deployment

tools.refill-api@tools-sgebastion-10:~$ kubectl apply -f worker-deployment.yml

deployment.apps/refill-api-worker created
deployment.apps/refill-api-scheduler created

Step 4

Confirm redeployment

tools.refill-api@tools-sgebastion-10:~$ kubectl get pods

NAME                                    READY   STATUS    RESTARTS   AGE
refill-api-85457fc487-mlqrk             1/1     Running   0          23d
refill-api-scheduler-6d748f867d-lfcwl   1/1     Running   0          4m39s
refill-api-worker-586c968f6c-pw9ps      1/1     Running   0          4m39s
refill-api-worker-586c968f6c-r425m      1/1     Running   0          4m39s

TheresNoTime mentioned this in T310754: Recurrent API worker failures.Jun 15 2022, 10:49 PM

TheresNoTime changed the subtype of this task from "Task" to "Bug Report".Jun 15 2022, 10:52 PM

TheresNoTime moved this task from Backlog to Bugs on the Tool-refill board.

Hello, thanks for quick response, the tool is working as expected now.

Resolved and confirmed.

Refill tool stuck "waiting for an available worker"Closed, ResolvedPublicBUG REPORTActions