Page MenuHomePhabricator

Python 3 fails with UnicodeDecodeError
Closed, ResolvedPublic

Description

Python 3 fails with UnicodeDecodeError like

Could not load cache: UnicodeDecodeError('ascii', b'\x07\xe0\x05\x02\r\t6\n\x92\xfd', 1, 2, 'ordinal not in range(128)')

Some unitttest fails for that reason, see https://integration.wikimedia.org/ci/job/tox-jessie/7513/console, also getversion_svn_setuptools call is broken (see T132292)

The problem in inside api.py code: CachedRequest.__load_cache() when printing the Exception message:

except Exception as e:
    pywikibot.output("Could not load cache: %r" % e)
    return False

The problem might be that the exception is catched anywhere in upper level.

Event Timeline

Xqt created this task.May 2 2016, 1:25 PM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptMay 2 2016, 1:25 PM
Xqt triaged this task as Unbreak Now! priority.May 2 2016, 1:25 PM
Restricted Application added subscribers: Luke081515, TerraCodes, Urbanecm. · View Herald TranscriptMay 2 2016, 1:25 PM

We can either clear apicache between test runs, or change this .output() into .debug(); the issue is Python 3 trying to load a Python 2 pickle file, and failing on a Python 2 bytestring. (Python 3 always tries to convert those to unicode, which, not surprisingly, fails)

Mpaa added a subscriber: Mpaa.EditedMay 2 2016, 7:34 PM

I saw that there are both apicache-py2 and apicache-py3.
Shouldn't it be that the corresponding cache is used when using py2 or py3? So that such incompatibility is avoided?
I remember a similar discussion but cannot remember when it happened and the conclusions :-(

jayvdb lowered the priority of this task from Unbreak Now! to Lowest.May 3 2016, 4:20 PM
jayvdb added a subscriber: jayvdb.

That message is not showing something a failing. It is just a message output , and all tests pass.
It occurs when py3 tests are run after py2 , as occurs in tox-jessie job, and the caching layer finds suitable filenames but cant load them.

AbdealiJK moved this task from Backlog to Upstream on the Pywikibot-tests board.May 5 2016, 3:46 AM
AbdealiJK moved this task from Upstream to Backlog on the Pywikibot-tests board.May 19 2016, 2:13 PM
Dalba assigned this task to Xqt.Jul 21 2018, 9:40 AM
Dalba added a subscriber: Dalba.

I could not find any occurrence of that error message in recent logs. I guess 2be2bbda8de16408723074a4ebcdf1d85e403fb0 has resolved it.

Xqt closed this task as Resolved.Jul 23 2018, 8:51 AM

Me too.