Page MenuHomePhabricator

[jobs-cli] AttributeError: module 'requests.exceptions' has no attribute 'InvalidJSONError' when getting 5xx from the server
Closed, DeclinedPublic

Description

For example, when running a one-off job and checking the logs right after:

tools.jupytest@tools-sgebastion-10:~$ toolforge jobs run --image tool-milhistbot/liftwing:latest --mount=all --command "heroku_output/Liftwing 'Hanford Engineer Works'" liftwing

tools.jupytest@tools-sgebastion-10:~$ toolforge jobs logs liftwing
ERROR: An internal error occured while executing this command.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 117, in _make_request
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url: https://api.svc.tools.eqiad1.wikimedia.cloud:30003/jobs/api/v1/jobs/liftwing/logs?follow=false

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tjf_cli/api.py", line 47, in handle_http_exception
    json = original.response.json()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 518, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 787, in main
    run_subcommand(args=args, api=api)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 722, in run_subcommand
    op_logs(api, args.name, args.follow, args.last)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 527, in op_logs
    params=params,
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 167, in get_raw_lines
    **kwargs,
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 130, in _make_request
    raise self.exception_handler(e) from e
  File "/usr/lib/python3/dist-packages/tjf_cli/api.py", line 59, in handle_http_exception
    except requests.exceptions.InvalidJSONError:
AttributeError: module 'requests.exceptions' has no attribute 'InvalidJSONError'

Event Timeline

This looks like a case of the requests version from Buster that's installed on the bastions being older than what I wrote the code with. It'll magically go away once the grid is away and we upgrade the bastions, or we can fix it now.

I ran into the same error yesterday with builds-cli when trying to start a build. Upon retry a few moments later, it worked fine.

dcaro removed dcaro as the assignee of this task.Jan 10 2024, 11:40 AM
dcaro removed a project: Toolforge Jobs framework.

Let's wait until the grid goes away

dcaro changed the task status from Declined to Resolved.Jan 23 2024, 2:25 PM
dcaro moved this task from Next Up to Done on the Toolforge (Toolforge iteration 03) board.
taavi changed the task status from Resolved to Declined.Jan 23 2024, 2:27 PM