windows unicode directory names fail on py2
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	jayvdb
	Apr 15 2015, 9:05 AM

Description

Unicode directory names cause problems whether invoked via pwb.py or directly.

C:\pywikibot\Häuser-core>python pwb.py shell
Traceback (most recent call last):
  File "pwb.py", line 117, in <module>
    os.path.join(rewrite_path, 'pywikibot', 'compat'),
  File "C:\Python279\lib\ntpath.py", line 84, in join
    result_path = result_path + p_path
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 12: ordinal not in range(128)

C:\pywikibot\Häuser-core>set PYTHONPATH=.
C:\pywikibot\Häuser-core>python scripts\shell.py
Traceback (most recent call last):
  File "scripts\shell.py", line 44, in <module>
    main(*args)
  File "scripts\shell.py", line 30, in main
    import pywikibot
  File "C:\pywikibot\Hõuser-core\pywikibot\__init__.py", line 32, in <module>
    from pywikibot import config2 as config
  File "C:\pywikibot\Hõuser-core\pywikibot\config2.py", line 307, in <module>
    register_families_folder(os.path.join(os.path.dirname(__file__), 'families')
)
  File "C:\Python279\lib\ntpath.py", line 84, in join
    result_path = result_path + p_path
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 12: ordinal not in range(128)

Even if we can solve those, we then have the problem of paths in the environment:

The solution doesnt look good
https://measureofchaos.wordpress.com/2011/03/04/python-on-windows-unicode-environment-variables/

Related Objects
Search...

Status	Assigned	Task
Resolved	Xover	T265640 phe-tools: Match&Split bot is not running because of python2 deprecation in pywikibot
Resolved	Xqt	T213287 Drop support of Python 2.7
Resolved	Xqt	T243770 Pywikibot Python 2 compatibility (tracking)
Declined	None	T96127 windows unicode directory names fail on py2

Event Timeline

jayvdb created this task.Apr 15 2015, 9:05 AM

jayvdb raised the priority of this task from to Needs Triage.

jayvdb updated the task description. (Show Details)

jayvdb added a project: Pywikibot.

jayvdb subscribed.

Restricted Application added subscribers: Aklapper, Unknown Object (MLST). · View Herald TranscriptApr 15 2015, 9:05 AM

Wow… that is a serious shortcoming on Python 2. That you have to patch your way into using Unicode environment :/ I mean supporting putenv() with unicode should simply use the Unicode functionality (as described in the blog you mention). Okay getenv() is a bit more problematic as it can't simply return unicode :(

So I guess we need some logic in tools or so which allows us to set and get unicode environment variables and use that instead of os.environ.

Xqt triaged this task as High priority.Jul 29 2018, 5:03 PM

Decoding sys.argv[0] would solve this issue above
absolute_path = abspath(os.path.dirname(sys.argv[0].decode('utf-8')))
but the same problem occurs when importing:

C:\pwb\GIT\Häuser\core>py -2 pwb.py shell
Traceback (most recent call last):
  File "pwb.py", line 174, in <module>
    import pywikibot
  File "C:\pwb\GIT\Hõuser\core\pywikibot\__init__.py", line 25, in <module>
    from pywikibot.bot import (
  File "C:\pwb\GIT\Hõuser\core\pywikibot\bot.py", line 101, in <module>
    from pywikibot import config2 as config
  File "C:\pwb\GIT\Hõuser\core\pywikibot\config2.py", line 404, in <module>
    register_families_folder(os.path.join(os.path.dirname(__file__), 'families')
)
  File "C:\Python27\lib\ntpath.py", line 85, in join
    result_path = result_path + p_path
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 10: ordinal
 not in range(128)

C:\pwb\GIT\Häuser\core>

Xqt added a parent task: T243770: Pywikibot Python 2 compatibility (tracking).Jan 27 2020, 3:01 PM

Xqt lowered the priority of this task from High to Lowest.Mar 10 2020, 10:50 AM

Declined in favour of Python 3. Python 2 will be dropped at the end of this month.

windows unicode directory names fail on py2Closed, DeclinedPublicActions

Description

Related ObjectsSearch...

Event Timeline

windows unicode directory names fail on py2
Closed, DeclinedPublic
Actions

Related Objects
Search...