Page MenuHomePhabricator

Rewrite pywikibot scripts to support Python 3.5+ only
Closed, ResolvedPublic

Description

Pywikibot 2.7 and Pywikibot 3.4 has been dropped due to T239542 and T213287. Rewrite all scripts in scripts folder to support Python 3 code only.

Which scripts should be converted?

  • All rPWBC scripts inside the scripts folder except scripts inside following subfolders:
    • archive: archived scripts only which are desupported
    • i18n: json translation files only
    • maintenance: already done with https://gerrit.wikimedia.org/r/c/pywikibot/core/+/610076 which can be used as an example
    • userscripts: empty folder for bot owners scripts
    • pwb.py must kept Python 2 compatible
    • setup.py must kept Python 2 compatible
  • the generate_family_file.py script
  • the generate_user_files.py script
  • The remaining scripts can be found here

How to convert?

  • remove all __future__ imports
  • remove all future_builtins imports
  • remove all Python 2 related code. Most of them are something like:
if PY2:
    # Python 2 related code
else:
    # Python 3 code

(don't forget to remove the line from pywikibot.tools import PY2 in such case)

try:
    from pathlib import Path
     # Python 3 code
except ImportError:
    from pathlib2 import Path
     # Python 2 code
  • replace class MyClass(object): by class MyClass:. This looks like old style but is new style class.
  • replace super(MyClass, self).my_method() by super().my_method()
  • use contextlib.suppress context manager to suppress passed exceptions like
try:
    1/0
except ZeroDivisionError:
    pass

should become:

with suppress(ZeroDivisionError):
    1/0
  • replace unicode usage by str
  • replace tools.UnicodeType by str
  • replace tools.StringType by (str, bytes) but check whether it can be just str
  • replace tools.py2_encode_utf_8(func) by func
  • remove tools.UnicodeMixin from class inheritance and replace their __unicode__ method by __str__
  • replace for item in iterator: yield item by yield from iterator
  • use iterator unpacking for multiple ignoring values, e.g. x, _, _, _ = range(4) should become x, *_ = range(4)
  • add annotation type hints if appropriate
  • use re.fullmatch instead of re.match(r'...$') or re.search(r'^...$')
  • use str.format_map instead of str.format(**kwargs)
  • consider os.scandir instead of os.listdir
  • see also https://www.asmeurer.com/python3-presentation/slides.html

Other things to know?

Details

ProjectBranchLines +/-Subject
pywikibot/coremaster+10 -22
pywikibot/coremaster+22 -35
pywikibot/coremaster+16 -27
pywikibot/coremaster+6 -20
pywikibot/coremaster+11 -25
pywikibot/coremaster+3 -5
pywikibot/coremaster+2 -4
pywikibot/coremaster+1 -3
pywikibot/coremaster+1 -3
pywikibot/coremaster+0 -1
pywikibot/coremaster+1 -3
pywikibot/coremaster+0 -1
pywikibot/coremaster+0 -2
pywikibot/coremaster+18 -17
pywikibot/coremaster+44 -55
pywikibot/coremaster+4 -6
pywikibot/coremaster+3 -5
pywikibot/coremaster+12 -19
pywikibot/coremaster+1 -3
pywikibot/coremaster+10 -19
pywikibot/coremaster+44 -40
pywikibot/coremaster+8 -15
pywikibot/coremaster+51 -75
pywikibot/coremaster+14 -16
pywikibot/coremaster+8 -8
pywikibot/coremaster+44 -46
pywikibot/coremaster+10 -16
pywikibot/coremaster+12 -14
pywikibot/coremaster+7 -9
pywikibot/coremaster+11 -17
pywikibot/coremaster+4 -6
pywikibot/coremaster+7 -8
pywikibot/coremaster+5 -7
pywikibot/coremaster+17 -17
pywikibot/coremaster+13 -11
pywikibot/coremaster+5 -7
pywikibot/coremaster+9 -14
pywikibot/coremaster+5 -7
pywikibot/coremaster+6 -9
pywikibot/coremaster+5 -7
pywikibot/coremaster+45 -49
pywikibot/coremaster+39 -39
pywikibot/coremaster+7 -10
Show related patches Customize query in gerrit

Related Objects

StatusSubtypeAssignedTask
InvalidNone
ResolvedXqt

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 611831 merged by jenkins-bot:
[pywikibot/core@master] Update newitem for py3

https://gerrit.wikimedia.org/r/611831

Change 611834 merged by jenkins-bot:
[pywikibot/core@master] Update noreferences for py3

https://gerrit.wikimedia.org/r/611834

Change 611858 merged by jenkins-bot:
[pywikibot/core@master] Update redirect for py3

https://gerrit.wikimedia.org/r/611858

Change 611888 merged by jenkins-bot:
[pywikibot/core@master] Update revertbot for py3

https://gerrit.wikimedia.org/r/611888

Xqt added a subscriber: JJMC89.Jul 12 2020, 10:12 AM

Great work @JJMC89, thanks a lot!

Change 611936 had a related patch set uploaded (by JJMC89; owner: JJMC89):
[pywikibot/core@master] Update template to py3

https://gerrit.wikimedia.org/r/611936

Change 611836 merged by jenkins-bot:
[pywikibot/core@master] Update pagefromfile for py3

https://gerrit.wikimedia.org/r/611836

Change 611889 merged by jenkins-bot:
[pywikibot/core@master] Update solve_disambiguation for py3

https://gerrit.wikimedia.org/r/611889

Change 611936 merged by jenkins-bot:
[pywikibot/core@master] Update template to py3

https://gerrit.wikimedia.org/r/611936

Change 611652 merged by jenkins-bot:
[pywikibot/core@master] Update checkimages for py3

https://gerrit.wikimedia.org/r/611652

Change 611938 had a related patch set uploaded (by JJMC89; owner: JJMC89):
[pywikibot/core@master] Update templatecount to py3

https://gerrit.wikimedia.org/r/611938

Change 611940 had a related patch set uploaded (by JJMC89; owner: JJMC89):
[pywikibot/core@master] Update touch to py3

https://gerrit.wikimedia.org/r/611940

Change 611945 had a related patch set uploaded (by JJMC89; owner: JJMC89):
[pywikibot/core@master] Update transferbot to py3

https://gerrit.wikimedia.org/r/611945

Change 611946 had a related patch set uploaded (by JJMC89; owner: JJMC89):
[pywikibot/core@master] Update upload to py3

https://gerrit.wikimedia.org/r/611946

Change 611947 had a related patch set uploaded (by JJMC89; owner: JJMC89):
[pywikibot/core@master] Update version to py3

https://gerrit.wikimedia.org/r/611947

Change 611940 merged by jenkins-bot:
[pywikibot/core@master] Update touch to py3

https://gerrit.wikimedia.org/r/611940

Change 611945 merged by jenkins-bot:
[pywikibot/core@master] Update transferbot to py3

https://gerrit.wikimedia.org/r/611945

Change 611938 merged by jenkins-bot:
[pywikibot/core@master] Update templatecount to py3

https://gerrit.wikimedia.org/r/611938

Change 611947 merged by jenkins-bot:
[pywikibot/core@master] Update version to py3

https://gerrit.wikimedia.org/r/611947

Change 611946 merged by jenkins-bot:
[pywikibot/core@master] Update upload to py3

https://gerrit.wikimedia.org/r/611946

Change 611948 had a related patch set uploaded (by JJMC89; owner: JJMC89):
[pywikibot/core@master] Update welcome to py3

https://gerrit.wikimedia.org/r/611948

Change 611948 merged by jenkins-bot:
[pywikibot/core@master] Update welcome to py3

https://gerrit.wikimedia.org/r/611948

Xqt updated the task description. (Show Details)Sep 16 2020, 3:11 PM

is someone working on it?

Xqt added a comment.Sep 29 2020, 11:23 AM

is someone working on it?

Currently not. Maybe just create a subtask for the script you are working on and claim this subtask.

Xqt updated the task description. (Show Details)Oct 11 2020, 5:31 AM

Change 637879 had a related patch set uploaded (by Shubham656jain; owner: Shubham656jain):
[pywikibot/core@master] Modify __init__.py to support python 3.5+ only

https://gerrit.wikimedia.org/r/637879

Change 637879 merged by Xqt:
[pywikibot/core@master] Modify __init__.py to support python 3.5+ only

https://gerrit.wikimedia.org/r/637879

@Xqt I made a new branch called T257399-v2. When I write git review -R to push new changes I get an error that I have 2 outstanding commits but I have already submitted the first one. Can you tell me what I might be doing wrong?

Change 637883 had a related patch set uploaded (by Shubham656jain; owner: Shubham656jain):
[pywikibot/core@master] Modify checkimages_tests.py to support Python 3.5+ only

https://gerrit.wikimedia.org/r/637883

Change 637883 merged by jenkins-bot:
[pywikibot/core@master] Modify checkimages_tests.py to support Python 3.5+ only

https://gerrit.wikimedia.org/r/637883

Change 638024 had a related patch set uploaded (by Shubham656jain; owner: Shubham656jain):
[pywikibot/core@master] Modify conftest.py to support Python 3.5 only

https://gerrit.wikimedia.org/r/638024

Change 638099 had a related patch set uploaded (by Diwanshu885; owner: Diwanshu885):
[pywikibot/core@master] Modify pwb.py to support python 3.5+ only

https://gerrit.wikimedia.org/r/638099

Change 638101 had a related patch set uploaded (by Diwanshu885; owner: Diwanshu885):
[pywikibot/core@master] Modify disambredir.py to support python 3.5+ only

https://gerrit.wikimedia.org/r/638101

Change 638024 merged by jenkins-bot:
[pywikibot/core@master] Modify conftest.py to support Python 3.5 only

https://gerrit.wikimedia.org/r/638024

Xqt updated the task description. (Show Details)Mon, Nov 2, 3:29 PM

Change 638099 abandoned by Diwanshu885:
[pywikibot/core@master] Modify pwb.py to support python 3.5+ only

Reason:
changes not required

https://gerrit.wikimedia.org/r/638099

Change 638128 had a related patch set uploaded (by Diwanshu885; owner: Diwanshu885):
[pywikibot/core@master] Modify fixing_redirects.py to support python 3.5+ only

https://gerrit.wikimedia.org/r/638128

Change 638128 merged by jenkins-bot:
[pywikibot/core@master] Modify fixing_redirects.py to support python 3.5+ only

https://gerrit.wikimedia.org/r/638128

Change 638101 merged by jenkins-bot:
[pywikibot/core@master] Modify disambredir.py to support python 3.5+ only

https://gerrit.wikimedia.org/r/638101

Change 638150 had a related patch set uploaded (by Shubham656jain; owner: Shubham656jain):
[pywikibot/core@master] Modify echo_tests.py to support Python 3.5+ only

https://gerrit.wikimedia.org/r/638150

Change 638501 had a related patch set uploaded (by Diwanshu885; owner: Diwanshu885):
[pywikibot/core@master] Modify data_ingestion_tests.py to support python 3.5+ only

https://gerrit.wikimedia.org/r/638501

Change 638501 merged by jenkins-bot:
[pywikibot/core@master] Modify data_ingestion_tests.py to support python 3.5+ only

https://gerrit.wikimedia.org/r/638501

Change 638150 merged by jenkins-bot:
[pywikibot/core@master] Modify echo_tests.py to support Python 3.5+ only

https://gerrit.wikimedia.org/r/638150

Xqt updated the task description. (Show Details)Wed, Nov 4, 7:51 AM
Xqt removed a project: Patch-For-Review.

Hi @Xqt can I take up this task?

Xqt added a comment.Wed, Nov 4, 3:43 PM

Hi @Xqt can I take up this task?

Sure there are some remaining issues but only a few for each patch.

This comment was removed by Udoka_Ugo.

Change 639638 had a related patch set uploaded (by Udoka; owner: Udoka):
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/639638

Change 639880 had a related patch set uploaded (by Udoka; owner: Udoka):
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/639880

Change 640360 had a related patch set uploaded (by Udoka; owner: Udoka):
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/640360

Change 640361 had a related patch set uploaded (by Udoka; owner: Udoka):
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/640361

Change 640363 had a related patch set uploaded (by Udoka; owner: Udoka):
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/640363

Change 640361 merged by jenkins-bot:
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/640361

Change 640363 merged by jenkins-bot:
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/640363

Change 640360 merged by jenkins-bot:
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/640360

Change 639638 merged by jenkins-bot:
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/639638

Change 639880 merged by jenkins-bot:
[pywikibot/core@master] [IMPR] Rewrite pywikibot scripts to support Python 3.5+ only

https://gerrit.wikimedia.org/r/639880

Xqt closed this task as Resolved.Tue, Nov 10, 8:37 PM
Xqt claimed this task.

Thank you all