Pywikibot ImageTransfer script has features deprecated for 5 years
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	Vicarage
	Oct 12 2020, 8:52 AM

Description

pywikibot core_stable.zip August 19 2020

pwb.py imagetransfer -v File:happy.jpeg -tofamily:newwiki
Python 2.7.17 (default, Jul 20 2020, 15:37:01) 
[GCC 7.5.0]
------------------------------------------------------------
0. Found image: [[File:happy.jpeg]]
Found 1 oldwiki:en-gb processes running, including this one.

Found 1 newwiki:en-gb processes running, including this one.
============================================================
WARNING: /home/john/bin/pywikibot/core/scripts/imagetransfer.py:143: DeprecationWarning: pywikibot.page.FilePage.fileUrl is deprecated for 4 years and 4 months; use get_file_url instead.
  url = sourceImagePage.fileUrl().encode('utf-8')

URL should be: http://oldwiki/images/9/96/happy.jpeg
WARNING: /data/priory1/home/john/bin/pywikibot/core1/pywikibot/page.py:2605: DeprecationWarning: pywikibot.page.FilePage.getFileVersionHistory is deprecated for 5 years and 11 months; use FilePage.get_file_history() instead.
  for info in self.getFileVersionHistory():

WARNING: url as string is deprecated. Use an iterable instead.
Logging in to newwiki as John@johnbot
Should be logged in now
ERROR: Upload error: Local file uploads are disabled on newwiki:en-gb.

(File uploads are not disabled on newwiki, $wgEnableUploads = true; and uploads from the webpage work, so there is a real bug, but that's not the focus of this ticket)

Details

	Subject	Repo	Branch	Lines +/-
	[bugfix] do not encode str to bytes in imagetransfer.py	pywikibot/core	master	+1 -1

Customize query in gerrit

Related Objects

Mentioned In: rPWBC1432b0f8769d: [bugfix] do not encode str to bytes in imagetransfer.py

Event Timeline

Vicarage created this task.Oct 12 2020, 8:52 AM

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptOct 12 2020, 8:52 AM

python -V
Python 2.7.17
oldwiki and newwiki are 1.35.0

The deprecation warning is already solved with https://gerrit.wikimedia.org/r/c/pywikibot/core/+/633303

The remaining problem is the UploadWarning from api:
Upload error: Local file uploads are disabled on newwiki:en-gb.

Note that current releases does not support Python 2 anymore.

Could you please run the script with -verbose option please to get the api url passed to the site.

Mpaa closed this task as Resolved.Oct 12 2020, 9:47 AM

Mpaa claimed this task.

Mpaa subscribed.

This comment was removed by Mpaa.

Mpaa reopened this task as Open.Oct 12 2020, 9:48 AM

Mpaa removed Mpaa as the assignee of this task.

When I ran software ticket systems, I found it best to let the problem reporter to update the ticket before marking it closed.

My pwb.py starts

#!/usr/bin/env python

PYTHON_VERSION = sys.version_info[:3]
PY2 = (PYTHON_VERSION[0] == 2)

versions_required_message = """
Pywikibot is not available on:
{version}

This version of Pywikibot only supports Python 2.7.2+ or 3.4+.
"""

def python_is_supported():

"""Check that Python is supported."""
# Any change to this must be copied to setup.py
return PYTHON_VERSION >= (3, 4, 0) or PY2 and PYTHON_VERSION >= (2, 7, 2)

So I had no reason to try python3

python3 /home/john/bin/pywikibot/core/pwb.py imagetransfer -v File:Happy.jpeg -tofamily:newwiki
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0]

Found 1 oldwiki:en-gb processes running, including this one.

0. Found image: [[File:Happy.jpeg]]

Found 1 newwiki:en-gb processes running, including this one.

WARNING: /home/john/bin/pywikibot/core/scripts/imagetransfer.py:143: DeprecationWarning: pywikibot.page.FilePage.fileUrl is deprecated for 4 years and 4 months; use get_file_url instead. url = sourceImagePage.fileUrl().encode('utf-8')

Traceback (most recent call last):

File "/home/john/bin/pywikibot/core/pwb.py", line 257, in <module>
  if not main():
File "/home/john/bin/pywikibot/core/pwb.py", line 250, in main
  run_python_file(filename, [filename] + args, argvu, file_package)
File "/home/john/bin/pywikibot/core/pwb.py", line 119, in run_python_file
  main_mod.__dict__)
File "/home/john/bin/pywikibot/core/scripts/imagetransfer.py", line 336, in <module>
  main()
File "/home/john/bin/pywikibot/core/scripts/imagetransfer.py", line 332, in main
  bot.run()
File "/home/john/bin/pywikibot/core/scripts/imagetransfer.py", line 278, in run
  self.transferImage(imagelist[todo])
File "/home/john/bin/pywikibot/core/scripts/imagetransfer.py", line 144, in transferImage
  pywikibot.output('URL should be: ' + url)

TypeError: must be str, not bytes
Dropped throttle(s).
<class 'TypeError'>
CRITICAL: Closing network session.
Network session closed.
python3 -V
Python 3.6.9

So you still have a 4 year old deprecation, and it fails.

BTW, the wikis I'm using are private, so I can't give correct urls.

Peachey88 updated the task description. (Show Details)Oct 12 2020, 10:18 AM

The script version is neither python2 branch nor 4.0+. Anyway there is still a bug described above. url string is encoded to bytes (don’t know why) and cannot be combined to str in that way.

Xqt triaged this task as Medium priority.Oct 12 2020, 1:20 PM

Xqt changed the subtype of this task from "Task" to "Bug Report".

Change 633572 had a related patch set uploaded (by Mpaa; owner: Mpaa):
[pywikibot/core@master] [bugfix] do not encode str to bytes in imagetransfer.py

https://gerrit.wikimedia.org/r/633572

gerritbot added a project: Patch-For-Review.Oct 12 2020, 7:24 PM

Change 633572 merged by jenkins-bot:
[pywikibot/core@master] [bugfix] do not encode str to bytes in imagetransfer.py

https://gerrit.wikimedia.org/r/633572

Mpaa mentioned this in rPWBC1432b0f8769d: [bugfix] do not encode str to bytes in imagetransfer.py.Oct 13 2020, 8:48 AM

Maintenance_bot removed a project: Patch-For-Review.Oct 13 2020, 9:10 AM

@Vicarage, is this solved after the patch?

I found the maintenance script importImages.php more suitable for my needs, as it handled old and new wiki metadata better, so I've not returned to the pywikibot script which feels more clumsy

OK, I will mark this as Resolved, shall be reopened if needed.

Pywikibot ImageTransfer script has features deprecated for 5 yearsClosed, ResolvedPublicBUG REPORTActions