unicodedata2 doesnt support all python environments supported by Pywikibot
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	VcamX
	Aug 5 2015, 5:55 PM

Description

unicodedata2 originally support Python 2 very reliably, and not not support Python 3

Details

Subject	Repo	Branch	Lines +/-
[cleanup] remove unicodedata2 dependency	pywikibot/core	master	+1 -19
Use unicodedata2>=7.0.0-2	pywikibot/core	master	+2 -4
unicodedata2 support for Python 2 only added	pywikibot/core	master	+2 -3

Customize query in gerrit

Related Objects

Mentioned In: rPWBCe9b4a0bc6855: Use unicodedata2>=7.0.0-2
rPWBC0bb6009a523a: unicodedata2 support for Python 2 only added
Mentioned Here: rPWBC1db58dbb763f: Python issue #10254
T106512: Appveyor builds failing to build requests[security]

Event Timeline

VcamX created this task.Aug 5 2015, 5:55 PM

VcamX raised the priority of this task from to Needs Triage.

VcamX updated the task description. (Show Details)

VcamX added a project: Pywikibot-OAuth.

VcamX subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 5 2015, 5:55 PM

VcamX set Security to None.Aug 5 2015, 5:56 PM

VcamX added subscribers: jayvdb, XZise.

jayvdb claimed this task.Aug 5 2015, 6:01 PM

jayvdb added a project: Pywikibot.Aug 6 2015, 4:13 AM

Restricted Application added a subscriber: pywikibot-bugs-list. · View Herald TranscriptAug 6 2015, 4:13 AM

jayvdb moved this task from Backlog to Upstream issues on the Pywikibot board.Aug 6 2015, 4:13 AM

unicodedata2 is only initially intended for Python 2.
There were problems with Python 2.7.0-2.7.2 , which are explained at https://github.com/mikekap/unicodedata2/issues/2

I've added Win32 and Linux tests for lots of Python 2.6 & 2.7 versions:
https://travis-ci.org/jayvdb/unicodedata2/builds/74420595
https://ci.appveyor.com/project/jayvdb/unicodedata2/build/1.0.27

I've created a bug for Python 3 support https://github.com/mikekap/unicodedata2/issues/6

Change 229731 had a related patch set uploaded (by John Vandenberg):
unicodedata2 support for Python 2 only added

https://gerrit.wikimedia.org/r/229731

gerritbot added a project: Patch-For-Review.Aug 6 2015, 3:11 PM

Change 229731 merged by jenkins-bot:
unicodedata2 support for Python 2 only added

https://gerrit.wikimedia.org/r/229731

jayvdb mentioned this in rPWBC0bb6009a523a: unicodedata2 support for Python 2 only added.Aug 6 2015, 4:11 PM

jayvdb triaged this task as Low priority.Aug 6 2015, 4:11 PM

jayvdb removed a project: Patch-For-Review.

jayvdb removed a project: Pywikibot-OAuth.

Ugh… it didn't actually fix the 2.7 problems. The 3.3+ problems obviously have gone as it won't use unicodedata2 now anymore. Unfortunately it won't compile unicodedata2 without fixing requests[security] (T106512). Now the most recent build (which included John's patch and a fix for T106512) on 2.7.2 is https://ci.appveyor.com/project/VcamX/pywikibot-core/build/1.0.137/job/xjtoddmnp4nfd7gy but I can't compare it with a build of it without John's patch (to see if it improved anything).

In T108068#1515373, @XZise wrote:

Ugh… it didn't actually fix the 2.7 problems. The 3.3+ problems obviously have gone as it won't use unicodedata2 now anymore. Unfortunately it won't compile unicodedata2 without fixing requests[security] (T106512). Now the most recent build (which included John's patch and a fix for T106512) on 2.7.2 is https://ci.appveyor.com/project/VcamX/pywikibot-core/build/1.0.137/job/xjtoddmnp4nfd7gy but I can't compare it with a build of it without John's patch (to see if it improved anything).

In that build, it is downloading unicodedata2 from pypi. Fixing...

Change 230026 had a related patch set uploaded (by John Vandenberg):
Use patched unicodedata2 for < Python 2.7.3

https://gerrit.wikimedia.org/r/230026

gerritbot added a project: Patch-For-Review.Aug 7 2015, 1:07 AM

My pull requests have been merged, so we can use master now

Change 230026 merged by jenkins-bot:
Use unicodedata2>=7.0.0-2

https://gerrit.wikimedia.org/r/230026

jayvdb mentioned this in rPWBCe9b4a0bc6855: Use unicodedata2>=7.0.0-2.Aug 8 2015, 2:05 PM

jayvdb renamed this task from unicodedata2 doesn't work reliably on Python 2.7+ for windows platform to unicodedata2 doesnt support all python environments supported by Pywikibot .Aug 9 2015, 5:46 AM

jayvdb updated the task description. (Show Details)

jayvdb removed a project: Patch-For-Review.

jayvdb added a project: Upstream.

jayvdb moved this task from Backlog to Reported Upstream on the Upstream board.

Upstream issue is marked as resolved. I guess we now just need to enable installation of unicodedata2 on Python 3 setups?

Dalba moved this task from Upstream issues to Backlog on the Pywikibot board.May 12 2018, 5:52 AM

Dalba removed a project: Upstream.

Why is this lib needed by pwb?

In T108068#4202162, @Xqt wrote:

Why is this lib needed by pwb?

It seems to first have been introduced in 1db58dbb763fa4687f30d69878d5dbb2598b6819 to resolve an NFC normalization issue in old Python versions (resolved in Python 2.7.2).

Now that Pywikibot only supports Python 2.7.2+ I'm not sure if having it in our dependencies is still necessary/useful.

Dalba removed jayvdb as the assignee of this task.May 12 2018, 6:18 AM

Change 432763 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [cleanup] remove unicodedata2 dependency

https://gerrit.wikimedia.org/r/432763

gerritbot added a project: Patch-For-Review.May 13 2018, 10:00 AM

Xqt claimed this task.May 13 2018, 10:03 AM

Dalba closed this task as Resolved.May 13 2018, 12:37 PM

Change 432763 merged by jenkins-bot:
[pywikibot/core@master] [cleanup] remove unicodedata2 dependency