redirect.py is stuck (no longer works)
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Meno25
	Mar 9 2022, 7:11 AM

Description

What happens?:

When running the script redirect.py, it gets stuck and does not edit the wiki. The script used to work correctly up until a few days ago, so, the recent changes to the script should be checked:

Command line:

python pwb.py redirect double -lang:ary

Output:

Retrieving double redirect special page...
Retrieving 2 pages from wikipedia:ary.
 ***

>>> أحمد الذهبي بن إسماعيل <<< ***
   Links to: [[احمد بن اسماعيل]].
   Links to: [[حمد بن سماعيل]].
@@ -1 +1 @@ ***
- #تحويل [[احمد بن اسماعيل]] ***
+ #تحويل [[حمد بن سماعيل]] ***

Edit summary: بوت: تصليح تحويلة مزدوجة إلى [[حمد بن سماعيل]]

Output of version.py:

Pywikibot: [https] r-pywikibot-core (314b789, g16082, 2022/03/07, 18:21:17, master)
Release version: 7.1.0.dev0
setuptools version: 60.9.3
mwparserfromhell version: 0.6.4
wikitextparser version: n/a
requests version: 2.27.1
    certificate test: ok
Python: 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)]

Details

	Subject	Repo	Branch	Lines +/-
	[cleanup] no longer use win32_unicode for Python 3.6+	pywikibot/core	master	+47 -73

Customize query in gerrit

Related Objects

Mentioned In: T281042: replace.py: Exiting due to uncaught exception <class 'OSError'>
Mentioned Here: rPWBC13f59c7f1859: Revert "[cleanup] no longer use win32_unicode for Python 3.6+"
rPWBC65862799dd66: [cleanup] no longer use win32_unicode for Python 3.6+
rPWBCe59939aebd49: add None return type annotations
rPWBC5fc9e0f67c4a: add parameter types of int/float/str based on the default values
rPWBCb8f864b01e2a: add parameter type of bool based on the default value

Event Timeline

Meno25 created this task.Mar 9 2022, 7:11 AM

Restricted Application added subscribers: pywikibot-bugs-list, alaa, Aklapper. · View Herald TranscriptMar 9 2022, 7:11 AM

Meno25 triaged this task as High priority.Mar 9 2022, 7:12 AM

There is a wider problem here. Several scripts are either stuck or the output on the screen is different that what it should be. These are two examples:

cosmetic_changes.py: Stuck and does not edit the wiki.
- Command line: python pwb.py cosmetic_changes -page:"أسامة السروت" -lang:ary
- Output:

Retrieving 1 pages from wikipedia:ary.
 ***

>>> أسامة السروت <<< ***
@@ -6,3 +6,3 @@ ***
- *{{مرجع كتاب|title=لماشينا|year=2016|oclc=1044589168}}<ref name="biblio22016">{{Cite book|title=الببليوغرافيا الوطنية المغربية 2016 ج2|url=http://www.bnrm.ma/bnrm/images/Pages_de_bibliographie2016-arabe-livre2.pdf|year=2016}}</ref> ***
+ * {{مرجع كتاب|title=لماشينا|year=2016|oclc=1044589168}}<ref name="biblio22016">{{Cite book|title=الببليوغرافيا الوطنية المغربية 2016 ج2|url=http://www.bnrm.ma/bnrm/images/Pages_de_bibliographie2016-arabe-livre2.pdf|year=2016}}</ref> ***
- *{{مرجع كتاب|title=زهرة ليام|year=2018}}<ref name="biblio32018">{{Cite book|title=الببليوغرافيا الوطنية المغربية 2018 ج3|url=http://www.bnrm.ma/bnrm/images/Pages_de_Biblio-2018-arabe-livre3.pdf|year=2018}}</ref> ***
+ * {{مرجع كتاب|title=زهرة ليام|year=2018}}<ref name="biblio32018">{{Cite book|title=الببليوغرافيا الوطنية المغربية 2018 ج3|url=http://www.bnrm.ma/bnrm/images/Pages_de_Biblio-2018-arabe-livre3.pdf|year=2018}}</ref> ***
- *{{مرجع كتاب|title=خرافة إسمها الرياح|oclc=1044620221}} ***
+ * {{مرجع كتاب|title=خرافة إسمها الرياح|oclc=1044620221}} ***

@@ -11,2 +11,2 @@ ***
- *{{مرجع كتاب|title=في طقوس العرض المسرحي|year=2020}}<ref name="toqos">{{Cite web|url=http://kitab.bnrm.ma/results-details.php?subID=47998|title=في طقوس العرض المسرحي}}</ref> ***
+ * {{مرجع كتاب|title=في طقوس العرض المسرحي|year=2020}}<ref name="toqos">{{Cite web|url=http://kitab.bnrm.ma/results-details.php?subID=47998|title=في طقوس العرض المسرحي}}</ref> ***
- *{{مرجع كتاب|title=جمالية الاضاءة المسرحية|year=2021}}<ref name="jamal">{{Cite web|url=http://kitab.bnrm.ma/ar/results-details_ar.php?subID=56306|title=جمالية الاضاءة المسرحية}}</ref> ***
+ * {{مرجع كتاب|title=جمالية الاضاءة المسرحية|year=2021}}<ref name="jamal">{{Cite web|url=http://kitab.bnrm.ma/ar/results-details_ar.php?subID=56306|title=جمالية الاضاءة المسرحية}}</ref> ***

Edit summary: بوت: تغييرات تجميلية

welcome.py: Edits the wiki but there is an extra *** printed on screen on every line.
- Command line: python pwb.py welcome -edit:0 -nlog -sul -limit:5000 -lang:arz
- Output:

[Match]  ***Sei tenten has enough edits to be welcomed.
Page [[نقاش المستخدم:Sei tenten]] saved
[Match]  ***Ondra185 has enough edits to be welcomed.
Page [[نقاش المستخدم:Ondra185]] saved
[Match]  ***Phermos has enough edits to be welcomed.
Page [[نقاش المستخدم:Phermos]] saved
[Match]  ***PENO607 has enough edits to be welcomed.

The user welcomed is arz:نقاش المستخدم:Sei tenten (Notice the extra ***)

Windows issue? perhaps caused by {6586279}?

In T303373#7763051, @JJMC89 wrote:

Windows issue? perhaps caused by {6586279}?

You are totally right. I tried the same two commands on the Toolforge and the scripts worked correctly as intended. This is the output:

python3 pwb.py cosmetic_changes -page:"أسامة السروت" -lang:ary

Retrieving 1 pages from wikipedia:ary.


>>> أسامة السروت <<<
@@ -6,3 +6,3 @@
- *{{مرجع كتاب|title=لماشينا|year=2016|oclc=1044589168}}<ref name="biblio22016">{{Cite book|title=الببليوغرافيا الوطنية المغربية 2016 ج2|url=http://www.bnrm.ma/bnrm/images/Pages_de_bibliographie2016-arabe-livre2.pdf|year=2016}}</ref>
+ * {{مرجع كتاب|title=لماشينا|year=2016|oclc=1044589168}}<ref name="biblio22016">{{Cite book|title=الببليوغرافيا الوطنية المغربية 2016 ج2|url=http://www.bnrm.ma/bnrm/images/Pages_de_bibliographie2016-arabe-livre2.pdf|year=2016}}</ref>
- *{{مرجع كتاب|title=زهرة ليام|year=2018}}<ref name="biblio32018">{{Cite book|title=الببليوغرافيا الوطنية المغربية 2018 ج3|url=http://www.bnrm.ma/bnrm/images/Pages_de_Biblio-2018-arabe-livre3.pdf|year=2018}}</ref>
+ * {{مرجع كتاب|title=زهرة ليام|year=2018}}<ref name="biblio32018">{{Cite book|title=الببليوغرافيا الوطنية المغربية 2018 ج3|url=http://www.bnrm.ma/bnrm/images/Pages_de_Biblio-2018-arabe-livre3.pdf|year=2018}}</ref>
- *{{مرجع كتاب|title=خرافة إسمها الرياح|oclc=1044620221}}
+ * {{مرجع كتاب|title=خرافة إسمها الرياح|oclc=1044620221}}

@@ -11,2 +11,2 @@
- *{{مرجع كتاب|title=في طقوس العرض المسرحي|year=2020}}<ref name="toqos">{{Cite web|url=http://kitab.bnrm.ma/results-details.php?subID=47998|title=في طقوس العرض المسرحي}}</ref>
+ * {{مرجع كتاب|title=في طقوس العرض المسرحي|year=2020}}<ref name="toqos">{{Cite web|url=http://kitab.bnrm.ma/results-details.php?subID=47998|title=في طقوس العرض المسرحي}}</ref>
- *{{مرجع كتاب|title=جمالية الاضاءة المسرحية|year=2021}}<ref name="jamal">{{Cite web|url=http://kitab.bnrm.ma/ar/results-details_ar.php?subID=56306|title=جمالية الاضاءة المسرحية}}</ref>
+ * {{مرجع كتاب|title=جمالية الاضاءة المسرحية|year=2021}}<ref name="jamal">{{Cite web|url=http://kitab.bnrm.ma/ar/results-details_ar.php?subID=56306|title=جمالية الاضاءة المسرحية}}</ref>

Edit summary: بوت: تغييرات تجميلية
Do you want to accept these changes? ([y]es, [N]o, [a]ll, [q]uit): y
Page [[أسامة السروت]] saved

1 pages read
1 pages written
0 pages skipped
Execution time: 21 seconds
Read operation time: 21.0 seconds
Write operation time: 21.0 seconds
Script terminated successfully.

python3 pwb.py welcome -edit:0 -nlog -sul -limit:5000 -lang:arz

[Match] اوهریش has enough edits to be welcomed.
Sleeping for 8.7 seconds, 2022-03-09 08:20:14
^C
KeyboardInterrupt during WelcomeBot bot run...

0 pages read
0 pages written
0 pages skipped
Execution time: 7 seconds
Script terminated successfully.

In T303373#7763051, @JJMC89 wrote:

Windows issue? perhaps caused by {6586279}?

I reverted the commit {6586279} on my local Windows machine and it indeed solved the problem. It should be reverted in the Pywikibot repository. Pinging @Xqt

In T303373#7763130, @Meno25 wrote:

In T303373#7763051, @JJMC89 wrote:

Windows issue? perhaps caused by {6586279}?

I reverted the commit {6586279} on my local Windows machine and it indeed solved the problem. It should be reverted in the Pywikibot repository. Pinging @Xqt

Reverted in rPWBC13f59c7f18593601a6635d2f94ad3066b7bec0a3

Meno25 mentioned this in T281042: replace.py: Exiting due to uncaught exception <class 'OSError'>.Mar 9 2022, 12:01 PM

Xqt merged a task: T303382: basic.py does not show any confirm message with put_current().Mar 10 2022, 11:14 AM

Xqt added a subscriber: valhallasw.

Reopened because ther is a remaining problem in terminal_interface:
sys.stdout and sys.stderr are io.TextIOWrapper if not redefined with win32_unicode.unicodeOutput. A flush is needed after all UI._wite is done.

Xqt lowered the priority of this task from High to Medium.Mar 10 2022, 11:18 AM

Change 769693 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [cleanup] no longer use win32_unicode for Python 3.6+

https://gerrit.wikimedia.org/r/769693

gerritbot added a project: Patch-For-Review.Mar 10 2022, 12:50 PM

Xqt closed this task as Resolved.Mar 12 2022, 11:22 PM

Change 769693 merged by jenkins-bot:

[pywikibot/core@master] [cleanup] no longer use win32_unicode for Python 3.6+

https://gerrit.wikimedia.org/r/769693

Maintenance_bot removed a project: Patch-For-Review.Mar 13 2022, 12:10 AM