Page MenuHomePhabricator
Create Task
Maniphest T263223

captcha.py issues due to change in `/` behaviour in python 3
Closed, ResolvedPublic
Actions

Description

I tried using captcha.py on python 3 and I get the following error:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "captcha.py", line 205, in run_in_thread
    for i in range(count):
TypeError: 'float' object cannot be interpreted as an integer
"""

This is probably because python3 division has changed: "/" now does true divide not floor divide. We need to replace with "//" in some places to do floor divide.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Make captcha.py work under python 3mediawiki/extensions/ConfirmEditmaster+2 -2
Make captcha.py work under python 3mediawiki/extensions/ConfirmEditREL1_35+2 -2
Make captcha.py work under python 3mediawiki/extensions/ConfirmEditREL1_31+2 -2
Use floor division (//) when calculating chunks in pythonmediawiki/extensions/ConfirmEditmaster+2 -3
Use floor division (//) when calculating chunks in pythonmediawiki/extensions/ConfirmEditREL1_35+2 -3
Use floor division (//) when calculating chunks in pythonmediawiki/extensions/ConfirmEditREL1_31+2 -3
Use floor division (//) when calculating chunks in pythonmediawiki/extensions/ConfirmEditREL1_34+2 -3
Customize query in gerrit

Related Objects

Event Timeline

TimSC created this task.Sep 18 2020, 4:46 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 18 2020, 4:46 AM
TimSC mentioned this in T141490: Deploy improved FancyCaptcha.Sep 18 2020, 4:46 AM
DannyS712 added a project: ConfirmEdit (CAPTCHA extension).Sep 18 2020, 6:30 AM
Aklapper renamed this task from captcha.py in FancyCaptcha doesn't support Python 3 to captcha.py in FancyCaptcha doesn't support Python 3: TypeError: 'float' object cannot be interpreted as an integer.Sep 18 2020, 7:40 AM
Aklapper added a project: Python3-Porting.
Reedy subscribed.Sep 18 2020, 3:28 PM

This is curious.. We did the python3 support back in T157888: Make captcha(-old)?.py python3 compatible, but obviously didn't necessarily test all paths through the code (and it's 3 years ago now)... Wikimedia production is still seemingly using 2.7 for "python", so probably haven't seen it there either

The only relevant divison I really see is in this block, which probably is where the count for the loop is coming from

	if count < threads:
		chunks = 1
		threads = 1
	else:
		chunks = (count / threads)

This is probably because python3 division has changed: "/" now does true divide not floor divide. We need to replace with "//" in some places to do floor divide.

Can you try that and see if it actually works? It's line 290 of captcha.py to be changed, I think

Also, what version of python, what paramters did you call the python (or more specifically, the php maintenance script) with? I guess the number of captchas and the threads parameters are probably the interesting ones here...

There are some other bits of division in the image generation, but your run hasn't got that far yet... Some use int() to wrap the divison

gerritbot added a comment.Sep 18 2020, 4:59 PM

Change 628406 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/ConfirmEdit@master] Use floor division (//) when calculating chunks

https://gerrit.wikimedia.org/r/628406

gerritbot added a project: Patch-For-Review.Sep 18 2020, 4:59 PM
Reedy added a comment.Sep 18 2020, 5:02 PM

Ok, so using a chunk of 100, and 3 threads, I can replicate your reported error. Patch just uploaded does fix that. But it then does expose an error in another part of the code, though, this time, it is in "vendor" code in the PIL library

Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/var/www/wiki/mediawiki/extensions/ConfirmEdit/captcha.py", line 216, in run_in_thread
    gen_captcha(word, font, fontsize, os.path.join(output, filename))
  File "/var/www/wiki/mediawiki/extensions/ConfirmEdit/captcha.py", line 112, in gen_captcha
    noise = Image.new('L', nsize, bgcolor)
  File "/usr/lib/python3/dist-packages/PIL/Image.py", line 2544, in new
    return im._new(core.fill(mode, size, color))
TypeError: integer argument expected, got float

Line 112 of captcha.py is the second of these two lines

	nsize = (im.size[0] / nblock, im.size[1] / nblock)
	noise = Image.new('L', nsize, bgcolor)

So I guess we probably want something similar there too

gerritbot added a comment.Sep 18 2020, 5:48 PM

Change 628372 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/ConfirmEdit@REL1_35] Use floor division (//) when calculating chunks in python

https://gerrit.wikimedia.org/r/628372

gerritbot added a comment.Sep 18 2020, 5:48 PM

Change 628373 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/ConfirmEdit@REL1_34] Use floor division (//) when calculating chunks in python

https://gerrit.wikimedia.org/r/628373

gerritbot added a comment.Sep 18 2020, 5:49 PM

Change 628374 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/ConfirmEdit@REL1_31] Use floor division (//) when calculating chunks in python

https://gerrit.wikimedia.org/r/628374

gerritbot added a comment.Sep 18 2020, 5:51 PM

Change 628373 merged by jenkins-bot:
[mediawiki/extensions/ConfirmEdit@REL1_34] Use floor division (//) when calculating chunks in python

https://gerrit.wikimedia.org/r/628373

gerritbot added a comment.Sep 18 2020, 5:52 PM

Change 628372 merged by jenkins-bot:
[mediawiki/extensions/ConfirmEdit@REL1_35] Use floor division (//) when calculating chunks in python

https://gerrit.wikimedia.org/r/628372

gerritbot added a comment.Sep 18 2020, 5:52 PM

Change 628374 merged by jenkins-bot:
[mediawiki/extensions/ConfirmEdit@REL1_31] Use floor division (//) when calculating chunks in python

https://gerrit.wikimedia.org/r/628374

gerritbot added a comment.Sep 18 2020, 6:02 PM

Change 628406 merged by jenkins-bot:
[mediawiki/extensions/ConfirmEdit@master] Use floor division (//) when calculating chunks in python

https://gerrit.wikimedia.org/r/628406

Maintenance_bot removed a project: Patch-For-Review.Sep 18 2020, 6:10 PM
Reedy renamed this task from captcha.py in FancyCaptcha doesn't support Python 3: TypeError: 'float' object cannot be interpreted as an integer to captcha.py issues due to change in / behaviour in python 3.Sep 18 2020, 6:17 PM
ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.10; 2020-09-22).Sep 18 2020, 7:00 PM
Tacsipacsi renamed this task from captcha.py issues due to change in / behaviour in python 3 to captcha.py issues due to change in `/` behaviour in python 3.Sep 20 2020, 12:31 AM
Tacsipacsi subscribed.
gerritbot added a comment.Oct 28 2020, 7:29 PM

Change 461505 had a related patch set uploaded (by Reedy; owner: Platonides):
[mediawiki/extensions/ConfirmEdit@master] Make captcha.py work under python 3

https://gerrit.wikimedia.org/r/461505

gerritbot added a project: Patch-For-Review.Oct 28 2020, 7:29 PM
gerritbot added a comment.Oct 28 2020, 7:49 PM

Change 636959 had a related patch set uploaded (by Reedy; owner: Platonides):
[mediawiki/extensions/ConfirmEdit@REL1_35] Make captcha.py work under python 3

https://gerrit.wikimedia.org/r/636959

gerritbot added a comment.Oct 28 2020, 7:50 PM

Change 636960 had a related patch set uploaded (by Reedy; owner: Platonides):
[mediawiki/extensions/ConfirmEdit@REL1_31] Make captcha.py work under python 3

https://gerrit.wikimedia.org/r/636960

gerritbot added a comment.Oct 28 2020, 7:54 PM

Change 636960 merged by jenkins-bot:
[mediawiki/extensions/ConfirmEdit@REL1_31] Make captcha.py work under python 3

https://gerrit.wikimedia.org/r/636960

gerritbot added a comment.Oct 28 2020, 7:56 PM

Change 636959 merged by jenkins-bot:
[mediawiki/extensions/ConfirmEdit@REL1_35] Make captcha.py work under python 3

https://gerrit.wikimedia.org/r/636959

gerritbot added a comment.Oct 28 2020, 8:58 PM

Change 461505 merged by jenkins-bot:
[mediawiki/extensions/ConfirmEdit@master] Make captcha.py work under python 3

https://gerrit.wikimedia.org/r/461505

Reedy closed this task as Resolved.Oct 28 2020, 9:00 PM
Reedy claimed this task.

Should be all fixed now..

ReleaseTaggerBot edited projects, added MW-1.36-notes (1.36.0-wmf.16; 2020-11-03); removed MW-1.36-notes (1.36.0-wmf.10; 2020-09-22).Oct 28 2020, 9:00 PM
Maintenance_bot removed a project: Patch-For-Review.Oct 28 2020, 9:10 PM
Reedy mentioned this in T268468: captcha.py needs to be ported to Python 3.Nov 23 2020, 4:50 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits