Page MenuHomePhabricator

Can't login. TypeError: must be str, not bytes.
Closed, ResolvedPublic

Description

Python 3.4.3 win32 while using IDLE and an up-to-date version of pywikibot:

>>> import pywikibot as pwb
>>> pwb.Site('fa').login()
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    pwb.Site('fa').login()
  File "E:\core\pywikibot\site.py", line 1676, in login
    if loginMan.login(retry=True):
  File "E:\core\pywikibot\login.py", line 232, in login
    password=True)
  File "E:\core\pywikibot\bot.py", line 540, in input
    data = ui.input(question, password)
  File "E:\core\pywikibot\userinterfaces\terminal_interface_base.py", line 212, in input
    self.output(question + ' ')
  File "E:\core\pywikibot\userinterfaces\terminal_interface_base.py", line 191, in output
    self._print(text, targetStream)
  File "E:\core\pywikibot\userinterfaces\terminal_interface_base.py", line 132, in _print
    self.printColorized(text, targetStream)
  File "E:\core\pywikibot\userinterfaces\terminal_interface_win32.py", line 91, in printColorized
    targetStream.write(text.encode(self.encoding, 'replace'))
  File "C:\Python34\lib\idlelib\PyShell.py", line 1344, in write
    raise TypeError('must be str, not ' + type(s).__name__)
TypeError: must be str, not bytes
>>>

Event Timeline

Dalba raised the priority of this task from to Needs Triage.
Dalba updated the task description. (Show Details)
Dalba subscribed.
Restricted Application added subscribers: Aklapper, Unknown Object (MLST). · View Herald TranscriptMar 8 2015, 4:50 AM

Was also reported here.

@Dalba could you try using the command prompt instead of IDLE?

With Python 3 the sys.stdout etc. streams are now using str (which is basically unicode in Python 2). But in Python 2 it requires bytes so it's encoding the string as UTF-8 bytes.

Now the Unicode support in Windows is always a bit problematic. I don't have a Windows OS available at the moment to test that quickly, but one way to fix it would be not to encode it. I'll provide a patch shortly and would appreciate if you test it (especially with non ASCII characters).

Apart from that (referencing to the discussion @Sitic linked to) I don't think that Python 3 is only partially supported. I basically only use it with Python 3.4.2 and got not problems (but on Linux so I can't encounter this particular bug).

Change 195086 had a related patch set uploaded (by XZise):
[FIX] Win32 UI: Don't encode in Python 3

https://gerrit.wikimedia.org/r/195086

So the

exception UnsupportedOperation('fileno',) while fixing up sys.stdout and sys.stderr

from the report I linked is unrelated? I was wondering about that (but also misread one of them as stdin).

I didn't want to say there that Python 3 support is "partial", but rather that the support for is isn't as matured/in longtime use as for 2.7.

Oh I usually look at the end of a stack trace and work my way up… overlooked the first line. That error might be related but only in a sense that Python 3 changed how str works. Unfortunately that error line isn't really helpful (except that it occurred somewhere else).

Sorry, It seems that I missed this -- seemingly related -- exception when importing pywikibot:

>>> import sys
>>> sys.path.append('c:/core')
>>> import pywikibot as pwb
WARNING: Running on Windows and transliteration_target is not set.
Please see https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Pywikibot/Windows
exception UnsupportedOperation('fileno',) while fixing up sys.stdout and sys.stderr

@Sitic: It works fine in command prompt.

@XZise: I applied the patch and now it logs in successfully. But still got the following error when trying to save a test edit:

>>> p= pwb.Page(pwb.Site('fa'), 'ویکی‌پدیا:صفحه تمرین')
>>> p.put('تمرين', ' ')
Page [[fa:وyکy‌پدyا:صفحه تمرyن]] saved
Traceback (most recent call last):
  File "<pyshell#12>", line 1, in <module>
    p.put('تمرين', ' ')
  File "c:/core\pywikibot\page.py", line 1117, in put
    **kwargs)
  File "c:/core\pywikibot\tools\__init__.py", line 930, in wrapper
    return obj(*__args, **__kw)
  File "c:/core\pywikibot\page.py", line 1036, in save
    **kwargs)
  File "c:/core\pywikibot\page.py", line 1052, in _save
    pywikibot.output(u"Page %s saved" % link)
  File "c:/core\pywikibot\bot.py", line 457, in output
    logoutput(text, decoder, newline, INFO, **kwargs)
  File "c:/core\pywikibot\bot.py", line 427, in logoutput
    logger.log(_level, text, extra=context, **kwargs)
  File "C:\Python34\lib\logging\__init__.py", line 1341, in log
    self._log(level, msg, args, **kwargs)
  File "C:\Python34\lib\logging\__init__.py", line 1409, in _log
    self.handle(record)
  File "C:\Python34\lib\logging\__init__.py", line 1419, in handle
    self.callHandlers(record)
  File "C:\Python34\lib\logging\__init__.py", line 1481, in callHandlers
    hdlr.handle(record)
  File "C:\Python34\lib\logging\__init__.py", line 853, in handle
    self.emit(record)
  File "c:/core\pywikibot\userinterfaces\terminal_interface_base.py", line 434, in emit
    return self.UI.output(text, targetStream=self.stream)
  File "c:/core\pywikibot\userinterfaces\terminal_interface_base.py", line 191, in output
    self._print(text, targetStream)
  File "c:/core\pywikibot\userinterfaces\terminal_interface_base.py", line 132, in _print
    self.printColorized(text, targetStream)
  File "c:/core\pywikibot\userinterfaces\terminal_interface_win32.py", line 99, in printColorized
    targetStream.write(text.encode(self.encoding, 'replace'))
  File "C:\Python34\lib\idlelib\PyShell.py", line 1344, in write
    raise TypeError('must be str, not ' + type(s).__name__)
TypeError: must be str, not bytes
>>>

(Although the edit was saved.)
Should I open another task for that?

This seems like the exact same error message. But I'm confused, line 99 in pywikibot.userinterfaces.terminal_interface_win32 does not have the line shown in the stack trace. Neither in the most current version or one of the revisions of my patch.

By the way https://gerrit.wikimedia.org/r/195089 might fix the message with the UnsupportedOperation error (which afaik unrelated to the warning above. Unfortunately can't really test it :(

I don't know, probably my mistake in applying the patch. Sorry about that. Anyway I reapplied both of your patches and now everything is OK. Both log-in and page saving were done without a single error. Thanks XZise!

Change 195086 merged by jenkins-bot:
[FIX] Win32 UI: Don't encode in Python 3

https://gerrit.wikimedia.org/r/195086

Okay both patches have been merged… We'll see if now other Windows users without IDLE complain. By the way you don't need to be sorry ;) mistakes happen and in the end it worked \o/

jayvdb claimed this task.
jayvdb subscribed.

Appears to be resolved.

XZise set Security to None.