Page MenuHomePhabricator

unidata.py: Exiting due to uncaught exception JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Closed, ResolvedPublicBUG REPORT

Description

Command line:

python pwb.py unidata

Output:

Traceback (most recent call last):
  File "C:\Users\Mohammed\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\requests\models.py", line 976, in json
    return complexjson.loads(self.text, **kwargs)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.3312.0_x64__qbz5n2kfra8p0\Lib\json\__init__.py", line 352, in loads
    return _default_decoder.decode(s)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.3312.0_x64__qbz5n2kfra8p0\Lib\json\decoder.py", line 345, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.3312.0_x64__qbz5n2kfra8p0\Lib\json\decoder.py", line 363, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Mohammed\Downloads\core\pwb.py", line 40, in <module>
    sys.exit(main())
             ~~~~^^
  File "C:\Users\Mohammed\Downloads\core\pwb.py", line 36, in main
    runpy.run_path(str(path), run_name='__main__')
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen runpy>", line 287, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Mohammed\Downloads\core\pywikibot\scripts\wrapper.py", line 567, in <module>
    main()
    ~~~~^^
  File "C:\Users\Mohammed\Downloads\core\pywikibot\scripts\wrapper.py", line 551, in main
    if not execute():
           ~~~~~~~^^
  File "C:\Users\Mohammed\Downloads\core\pywikibot\scripts\wrapper.py", line 538, in execute
    run_python_file(filename, script_args, module)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Mohammed\Downloads\core\pywikibot\scripts\wrapper.py", line 155, in run_python_file
    exec(compile(source, filename, 'exec', dont_inherit=True),
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         main_mod.__dict__)
         ^^^^^^^^^^^^^^^^^^
  File "C:\Users\Mohammed\Downloads\core\scripts\maintenance\unidata.py", line 166, in <module>
    print(process_site('wiktionary', 'fr'))  # noqa: T201
          ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Mohammed\Downloads\core\scripts\maintenance\unidata.py", line 73, in process_site
    ).json()
      ~~~~^^
  File "C:\Users\Mohammed\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\LocalCache\local-packages\Python313\site-packages\requests\models.py", line 980, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
CRITICAL: Exiting due to uncaught exception JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Notes:

  • I tried to print the error message returned by the server and this what I got:
Status code: 403
Response text: Please set a user-agent and respect our robot policy https://w.wiki/4wJS. See also https://phabricator.wikimedia.org/T400119.

Software version:

Pywikibot: [https] r-pywikibot-core (2816408, g20230, 2026/02/11, 19:53:42, master)
Release version: 11.1.0.dev0
packaging version: 26.0
mwparserfromhell version: 0.7.2
wikitextparser version: 0.56.4
requests version: 2.32.5
    certificate test: ok
Python: 3.13.12 (tags/v3.13.12:1cbe481, Feb  3 2026, 18:22:25) [MSC v.1944 64 bit (AMD64)]
User-Agent: MenoBot/version (wikipedia:ar; User:MenoBot) Pywikibot/11.1.0.dev0 (g20230) Python/3.13.12.final.0 requests/2.32.5

Event Timeline

I’d like to work on this task. I’ll try to reproduce the error locally and investigate the JSONDecodeError related to the 403 response and revert with my findings.

I was able to reproduce the issue locally. Found that the failure occurs in process_site() where the response from session.post() is immediately parsed with .json() without checking the HTTP status. When the API returns a 403 response, it leads to an uncaught JSONDecodeError. I’ll work on improving the error handling so HTTP failures are handled properly.

Change #1239515 had a related patch set uploaded (by Tanvi_reddy; author: Tanvi_reddy):

[pywikibot/core@master] maintenance: Handle HTTP errors before parsing JSON in unidata

https://gerrit.wikimedia.org/r/1239515

Submitted a patch to handle HTTP errors before parsing JSON:
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1239515

The change calls raise_for_status() before parsing the response to avoid the uncaught JSONDecodeError when a non-200 response is returned.
If preferred, I can instead raise a RuntimeError including the response status and body for clearer debugging output. Thanks!

Thank you @TanviReddy30 for your patch but it does not solve the underlying problem: There is not UA send with that request. session should not be used directly. Because a site is used here, the corresponding function is http.request() which must be used here. Or much better an action API if available.

Change #1239517 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] Fix: use http.request for parse section instead of section.post

https://gerrit.wikimedia.org/r/1239517

Thanks for the clarification. I’ll update the patch to use http.request() (or the appropriate Action API call) instead of calling session.post() directly.

Xqt triaged this task as Low priority.Sun, Feb 15, 11:13 AM

Thanks for the clarification. I’ll update the patch to use http.request() (or the appropriate Action API call) instead of calling session.post() directly.

I already submitted a patch in https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1239517

Change #1239517 merged by jenkins-bot:

[pywikibot/core@master] Fix: use http.request for parse section instead of section.post

https://gerrit.wikimedia.org/r/1239517

Change #1239515 abandoned by Xqt:

[pywikibot/core@master] maintenance: Handle HTTP errors before parsing JSON in unidata

Reason:

task already solved; merge conflict

https://gerrit.wikimedia.org/r/1239515