Python 3 fails with UnicodeDecodeError
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Xqt
	May 2 2016, 1:25 PM

Description

Python 3 fails with UnicodeDecodeError like

Could not load cache: UnicodeDecodeError('ascii', b'\x07\xe0\x05\x02\r\t6\n\x92\xfd', 1, 2, 'ordinal not in range(128)')

Some unitttest fails for that reason, see https://integration.wikimedia.org/ci/job/tox-jessie/7513/console, also getversion_svn_setuptools call is broken (see T132292)

The problem in inside api.py code: CachedRequest.__load_cache() when printing the Exception message:

except Exception as e:
    pywikibot.output("Could not load cache: %r" % e)
    return False

The problem might be that the exception is catched anywhere in upper level.

Related Objects
Search...

Status	Assigned	Task
Invalid	None	T72936 Important tasks to be solved (tracking)
Resolved	Xqt	T60053 Pywikibot Python 3 compatibility (tracking)
Resolved	Xqt	T134172 Python 3 fails with UnicodeDecodeError

Event Timeline

Xqt created this task.May 2 2016, 1:25 PM

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptMay 2 2016, 1:25 PM

Xqt triaged this task as Unbreak Now! priority.May 2 2016, 1:25 PM

Restricted Application added subscribers: Luke081515, TerraCodes, Urbanecm. · View Herald TranscriptMay 2 2016, 1:25 PM

Xqt added a parent task: T132292: do not deprecate getversion_svn and svn_rev_info in version.py.May 2 2016, 1:26 PM

We can either clear apicache between test runs, or change this .output() into .debug(); the issue is Python 3 trying to load a Python 2 pickle file, and failing on a Python 2 bytestring. (Python 3 always tries to convert those to unicode, which, not surprisingly, fails)

I saw that there are both apicache-py2 and apicache-py3.
Shouldn't it be that the corresponding cache is used when using py2 or py3? So that such incompatibility is avoided?
I remember a similar discussion but cannot remember when it happened and the conclusions :-(

That message is not showing something a failing. It is just a message output , and all tests pass.
It occurs when py3 tests are run after py2 , as occurs in tox-jessie job, and the caching layer finds suitable filenames but cant load them.

jayvdb removed a parent task: T132292: do not deprecate getversion_svn and svn_rev_info in version.py.May 3 2016, 4:26 PM

AbdealiJK moved this task from Backlog to Upstream issues on the Pywikibot-tests board.May 5 2016, 3:46 AM

AbdealiJK moved this task from Upstream issues to Backlog on the Pywikibot-tests board.May 19 2016, 2:13 PM

Strainu subscribed.Nov 10 2017, 10:06 AM

Dvorapa added a parent task: T60053: Pywikibot Python 3 compatibility (tracking).Feb 1 2018, 11:17 AM