Page MenuHomePhabricator
Create Task
Maniphest T266137

uncaught exception <class 'LookupError'>: unknown encoding / threadedhttp.py and reflinks.py
Closed, ResolvedPublicBUG REPORT
Actions

Description

@PAWS:~$ pwb.py ~/reflinks2.py -start:Solariola_zoiai -v -debug

=== Pywikibot framework v5.0.0 -- Logging header ===
COMMAND: ['/home/paws/reflinks2.py', '-start:Solariola_zoiai', '-v', '-debug']
DATE: 2020-10-21 14:19:12.311050 UTC
VERSION: [https] r-pywikibot-core.git (5ebba28, g13454, 2020/10/19, 16:57:22, n/a)
SYSTEM: posix.uname_result(sysname='Linux', nodename='jupyter--52ubinbot', release='4.19.0-11-amd64', version='#1 SMP Debian 4.19.146-1 (2020-09-17)', machine='x86_64')
CONFIG FILE DIR: /srv/paws
PACKAGES:
  __main__ (/home/paws/reflinks2.py) = ??
  _brotli (/srv/paws/lib/python3.6/site-packages/brotli/_brotli.abi3.so) = ??
  _cffi_backend (/srv/paws/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so) = 1.14.1
  _ctypes (/usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so) = 1.1.0
  _cython_0_29_21 ([path unknown]) = ??
  _decimal (/usr/lib/python3.6/lib-dynload/_decimal.cpython-36m-x86_64-linux-gnu.so) = 1.70
  blinker (/srv/paws/lib/python3.6/site-packages/blinker/) = 1.4
  brotli (/srv/paws/lib/python3.6/site-packages/brotli/) = ??
  bs4 (/srv/paws/lib/python3.6/site-packages/bs4/) = 4.9.1
  certifi (/srv/paws/lib/python3.6/site-packages/certifi/) = 2020.06.20
  chardet (/srv/paws/lib/python3.6/site-packages/chardet/) = 3.0.4
  cryptography (/srv/paws/lib/python3.6/site-packages/cryptography/) = 3.0
  ctypes (/usr/lib/python3.6/ctypes/) = 1.1.0
  cython_runtime ([path unknown]) = ??
  decimal (/usr/lib/python3.6/decimal.py) = 1.70
  distutils (/usr/lib/python3.6/distutils/) = 3.6.9
  idna (/srv/paws/lib/python3.6/site-packages/idna/) = 2.10
  ipaddress (/usr/lib/python3.6/ipaddress.py) = 1.0
  json (/usr/lib/python3.6/json/) = 2.0.9
  jwt (/srv/paws/lib/python3.6/site-packages/jwt/) = 1.7.1
  logging (/usr/lib/python3.6/logging/) = 0.5.1.2
  lxml (/srv/paws/lib/python3.6/site-packages/lxml/) = 4.5.2
  mpl_toolkits ([path unknown]) = ??
  mwoauth (/srv/paws/lib/python3.6/site-packages/mwoauth/) = 0.3.7
  mwparserfromhell (/srv/paws/lib/python3.6/site-packages/mwparserfromhell/) = 0.5.4
  oauthlib (/srv/paws/lib/python3.6/site-packages/oauthlib/) = 3.1.0
  pkg_resources (/srv/paws/lib/python3.6/site-packages/pkg_resources/) = ??
  platform (/usr/lib/python3.6/platform.py) = 1.0.8
  re (/usr/lib/python3.6/re.py) = 2.2.1
  requests (/srv/paws/lib/python3.6/site-packages/requests/) = 2.24.0
  requests_oauthlib (/srv/paws/lib/python3.6/site-packages/requests_oauthlib/) = 1.3.0
  ruamel ([path unknown]) = ??
  setuptools (/srv/paws/lib/python3.6/site-packages/setuptools/) = 49.2.0
  six (/srv/paws/lib/python3.6/site-packages/six.py) = 1.15.0
  soupsieve (/srv/paws/lib/python3.6/site-packages/soupsieve/) = 2.0.1
  urllib3 (/srv/paws/lib/python3.6/site-packages/urllib3/) = 1.25.10
MODULES:
  2020-10-21 10:24:51 /srv/paws/pwb/setup.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/__metadata__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/_wbtypes.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/bot.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/config2.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/logging.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/_unidata.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/daemonize.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/i18n.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/exceptions.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/plural.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/version.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/comms/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/comms/http.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/comms/threadedhttp.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/bot_choice.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/_logging.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/formatter.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/userinterfaces/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface_base.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/userinterfaces/transliteration.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface_unix.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/data/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/data/api.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/login.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/family.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/diff.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/chars.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/site/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/echo.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/site/_decorators.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/site/_siteinfo.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/throttle.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/page/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/textlib.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/pagegenerators.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/date.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/xmlreader.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/proofreadpage.py
  2020-07-30 19:46:59 /srv/paws/pwb/scripts/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/scripts/noreferences.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/families/wikipedia_family.py
=========================================================

=== Pywikibot framework v5.0.0 -- Logging header ===
COMMAND: ['/home/paws/reflinks2.py', '-start:Solariola_zoiai', '-v', '-debug']
DATE: 2020-10-21 14:19:12.618340 UTC
VERSION: [https] r-pywikibot-core.git (5ebba28, g13454, 2020/10/19, 16:57:22, n/a)
SYSTEM: posix.uname_result(sysname='Linux', nodename='jupyter--52ubinbot', release='4.19.0-11-amd64', version='#1 SMP Debian 4.19.146-1 (2020-09-17)', machine='x86_64')
CONFIG FILE DIR: /srv/paws
PACKAGES:
  __main__ (/home/paws/reflinks2.py) = ??
  _brotli (/srv/paws/lib/python3.6/site-packages/brotli/_brotli.abi3.so) = ??
  _cffi_backend (/srv/paws/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so) = 1.14.1
  _ctypes (/usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so) = 1.1.0
  _cython_0_29_21 ([path unknown]) = ??
  _decimal (/usr/lib/python3.6/lib-dynload/_decimal.cpython-36m-x86_64-linux-gnu.so) = 1.70
  blinker (/srv/paws/lib/python3.6/site-packages/blinker/) = 1.4
  brotli (/srv/paws/lib/python3.6/site-packages/brotli/) = ??
  bs4 (/srv/paws/lib/python3.6/site-packages/bs4/) = 4.9.1
  certifi (/srv/paws/lib/python3.6/site-packages/certifi/) = 2020.06.20
  chardet (/srv/paws/lib/python3.6/site-packages/chardet/) = 3.0.4
  cryptography (/srv/paws/lib/python3.6/site-packages/cryptography/) = 3.0
  ctypes (/usr/lib/python3.6/ctypes/) = 1.1.0
  cython_runtime ([path unknown]) = ??
  decimal (/usr/lib/python3.6/decimal.py) = 1.70
  distutils (/usr/lib/python3.6/distutils/) = 3.6.9
  idna (/srv/paws/lib/python3.6/site-packages/idna/) = 2.10
  ipaddress (/usr/lib/python3.6/ipaddress.py) = 1.0
  json (/usr/lib/python3.6/json/) = 2.0.9
  jwt (/srv/paws/lib/python3.6/site-packages/jwt/) = 1.7.1
  logging (/usr/lib/python3.6/logging/) = 0.5.1.2
  lxml (/srv/paws/lib/python3.6/site-packages/lxml/) = 4.5.2
  mpl_toolkits ([path unknown]) = ??
  mwoauth (/srv/paws/lib/python3.6/site-packages/mwoauth/) = 0.3.7
  mwparserfromhell (/srv/paws/lib/python3.6/site-packages/mwparserfromhell/) = 0.5.4
  oauthlib (/srv/paws/lib/python3.6/site-packages/oauthlib/) = 3.1.0
  pkg_resources (/srv/paws/lib/python3.6/site-packages/pkg_resources/) = ??
  platform (/usr/lib/python3.6/platform.py) = 1.0.8
  re (/usr/lib/python3.6/re.py) = 2.2.1
  requests (/srv/paws/lib/python3.6/site-packages/requests/) = 2.24.0
  requests_oauthlib (/srv/paws/lib/python3.6/site-packages/requests_oauthlib/) = 1.3.0
  ruamel ([path unknown]) = ??
  setuptools (/srv/paws/lib/python3.6/site-packages/setuptools/) = 49.2.0
  six (/srv/paws/lib/python3.6/site-packages/six.py) = 1.15.0
  soupsieve (/srv/paws/lib/python3.6/site-packages/soupsieve/) = 2.0.1
  urllib3 (/srv/paws/lib/python3.6/site-packages/urllib3/) = 1.25.10
MODULES:
  2020-10-21 10:24:51 /srv/paws/pwb/setup.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/__metadata__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/_wbtypes.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/bot.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/config2.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/logging.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/_unidata.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/daemonize.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/i18n.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/exceptions.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/plural.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/version.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/comms/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/comms/http.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/comms/threadedhttp.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/bot_choice.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/_logging.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/formatter.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/userinterfaces/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface_base.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/userinterfaces/transliteration.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/userinterfaces/terminal_interface_unix.py
  2020-07-30 19:46:59 /srv/paws/pwb/pywikibot/data/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/data/api.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/login.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/family.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/diff.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/tools/chars.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/site/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/echo.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/site/_decorators.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/site/_siteinfo.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/throttle.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/page/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/textlib.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/pagegenerators.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/date.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/xmlreader.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/proofreadpage.py
  2020-07-30 19:46:59 /srv/paws/pwb/scripts/__init__.py
  2020-10-21 10:24:51 /srv/paws/pwb/scripts/noreferences.py
  2020-10-21 10:24:51 /srv/paws/pwb/pywikibot/families/wikipedia_family.py
=========================================================
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0]
LOADING SITE wikipedia:ru VERSION: 1.36.0-wmf.13
Found 1 wikipedia:ru processes running, including this one.
WARNING: /home/paws/reflinks2.py:498: ResourceWarning: unclosed file <_io.BufferedReader name='404-links.txt'>
  listof404pages, 'r', 'latin_1').read()

Retrieving 50 pages from wikipedia:ru.
No changes were needed on [[Solariola zoiai]]
https://docs.oracle.com/cd/E19253-01/817-0547/ : No title found...
https://docs.oracle.com/cd/E19253-01/821-0382/ : No title found...
PDF file.
Traceback (most recent call last):
  File "/srv/paws/pwb/pwb.py", line 363, in <module>
    if not main():
  File "/srv/paws/pwb/pwb.py", line 358, in main
    file_package)
  File "/srv/paws/pwb/pwb.py", line 75, in run_python_file
    main_mod.__dict__)
  File "/home/paws/reflinks2.py", line 820, in <module>
    main()
  File "/home/paws/reflinks2.py", line 816, in main
    bot.run()
  File "/home/paws/reflinks2.py", line 550, in run
    self.getPDFTitle(ref, f)
  File "/home/paws/reflinks2.py", line 468, in getPDFTitle
    urlobj.write(f.text)
  File "/srv/paws/pwb/pywikibot/comms/threadedhttp.py", line 190, in text
    return self.decode(self.encoding)
  File "/srv/paws/pwb/pywikibot/comms/threadedhttp.py", line 155, in encoding
    self._encoding = self._try_decode(self.header_encoding)
  File "/srv/paws/pwb/pywikibot/comms/threadedhttp.py", line 170, in _try_decode
    self.raw.decode(encoding)
LookupError: unknown encoding:
Dropped throttle(s).
Closing network session.
CRITICAL: Exiting due to uncaught exception <class 'LookupError'>
Network session closed.
WARNING: sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=11 mode='w+' encoding='UTF-8'>

Related Objects

Event Timeline

Rubin16 created this task.Oct 21 2020, 2:28 PM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptOct 21 2020, 2:28 PM
Rubin16 triaged this task as Lowest priority.Oct 21 2020, 2:28 PM
Xqt closed this task as Resolved.Oct 21 2020, 4:12 PM
Xqt claimed this task.
Xqt subscribed.

Already solved in 5.0.1.dev0 either with rPWBC238855690 or rPWBCc64c9068

Xqt reassigned this task from Xqt to Mpaa.Oct 21 2020, 4:13 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL