Page MenuHomePhabricator

[jobs-api] when running a command with wrong quoting, no logs nor useful feedback is given to the user
Open, LowPublicBUG REPORT

Description

I was running this ill-formed command today:

toolsbeta.test@toolsbeta-sgebastion-05:~$ toolforge jobs run --command "sh -c 'env; echo somethingsomethingsomething; echo; env''" --image tool-test/tool-test:latest test

and the cli only shows:

ERROR: An internal error occured while executing this command.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 117, in _make_request
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url: https://api.svc.toolsbeta.eqiad1.wikimedia.cloud:30003/jobs/api/v1/jobs/

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 797, in main
    run_subcommand(args=args, api=api)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 718, in run_subcommand
    mount=args.mount,
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 474, in op_run
    api.post("/jobs/", json=payload)
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 150, in post
    return self._make_request("POST", url, **kwargs).json()
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 130, in _make_request
    raise self.exception_handler(e) from e
tjf_cli.api.TjfCliHttpError: Unable to start job (No closing quotation)
ERROR: Please report this issue to the Toolforge admins: https://w.wiki/6Zuu

and on the server side you get just one line:

root@toolsbeta-test-k8s-control-6:~# kubectl logs -n jobs-api pod/jobs-api-84687bbcf-4vb44 -c webservice | grep 500
[pid: 8|app: 0|req: 2/16] 192.168.104.171 () {44 vars in 664 bytes} [Wed Jan 31 13:51:13 2024] POST /api/v1/jobs/ => generated 67 bytes in 224 msecs (HTTP/1.0 500) 2 headers in 90 bytes (1 switches on core 0)
[pid: 10|app: 0|req: 21/43] 192.168.17.249 () {44 vars in 663 bytes} [Wed Jan 31 13:52:28 2024] POST /api/v1/jobs/ => generated 67 bytes in 262 msecs (HTTP/1.0 500) 2 headers in 90 bytes (1 switches on core 0)
[pid: 8|app: 0|req: 9/47] 192.168.17.249 () {44 vars in 663 bytes} [Wed Jan 31 13:52:39 2024] POST /api/v1/jobs/ => generated 67 bytes in 190 msecs (HTTP/1.0 500) 2 headers in 90 bytes (1 switches on core 0)

The issue is the extra ' quote there, we should give a message that's a bit more useful (took me 15min to figure out the typo xd)

Event Timeline

bd808 changed the subtype of this task from "Task" to "Bug Report".Feb 12 2024, 11:08 PM
dcaro triaged this task as Low priority.Feb 21 2024, 4:45 PM
dcaro edited projects, added: Toolforge; removed: Toolforge Jobs framework.
dcaro moved this task from Backlog to Ready to be worked on on the Toolforge board.

this might work here

>>> import subprocess
>>> subprocess.run(
... ["sh", "-n", "-c", "sh -c 'env; echo somethingsomethingsomething; echo; env''"],
... check=True,
... stdout=subprocess.PIPE,
... stderr=subprocess.PIPE,
... )
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    subprocess.run(
    ~~~~~~~~~~~~~~^
    ["sh", "-n", "-c", "sh -c 'env; echo somethingsomethingsomething; echo; env''"],
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
    stderr=subprocess.PIPE,
    ^^^^^^^^^^^^^^^^^^^^^^^
    )

we can handle the error and display something more explanatory like "There seems to be something wrong with the command you are trying to run" or something. This can be made part of the command validation (maybe just before converting the command to k8s command? )

Raymond_Ndibe changed the task status from Open to In Progress.Apr 29 2025, 5:54 AM
fnegri changed the task status from In Progress to Open.Jan 13 2026, 5:00 PM

Based on

tjf_cli.api.TjfCliHttpError: Unable to start job (No closing quotation)

presumably to fix this one just needs to figure out where that "No closing quotation" error comes from and throw that as a a 4xx instead of a 5xx.