Page MenuHomePhabricator
Create Task
Maniphest T362103

TypeError: 'NoneType' object is not iterable from noreferences script
Closed, ResolvedPublicFeature
Actions

Description

Steps to replicate the issue (include links if applicable):

  • I have the code correct, and it does log into simple wiki
  • When it is run it comes back with a fatal exception
  • "CRITICAL: Exiting due to uncaught exception TypeError: 'NoneType' object is not iterable"
  • I am running it against https://simple.wikipedia.org/wiki/User:PotsdamLamb/test
  • I have references on the page, so the script should be adding the == References == header

toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb -family:wikipedia -lang:simple noreferences -page:User:PotsdamLamb/test" fix-34434

What happens?:

2024-04-08T18:26:41+00:00 [fix-34434-rkprm] family and mylang are not set.
2024-04-08T18:26:41+00:00 [fix-34434-rkprm] Defaulting to family='wikipedia' and mylang='test'.
2024-04-08T18:26:43+00:00 [fix-34434-rkprm] Retrieving 1 pages from wikipedia:simple.
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] Working on 'User:PotsdamLamb/test'
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] Found ref without references.
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] 1 read operation
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] Execution time:
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] Read operation time: 0.0 seconds
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] Script terminated by exception:
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] ERROR: 'NoneType' object is not iterable (TypeError)
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] Traceback (most recent call last):
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/pwb.py", line 40, in <module>
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     sys.exit(main())
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/pwb.py", line 36, in main
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     runpy.run_path(str(path), run_name='__main__')
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/layers/heroku_python/python/lib/python3.9/runpy.py", line 288, in run_path
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     return _run_module_code(code, init_globals, run_name,
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/layers/heroku_python/python/lib/python3.9/runpy.py", line 97, in _run_module_code
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     _run_code(code, mod_globals, init_globals,
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/layers/heroku_python/python/lib/python3.9/runpy.py", line 87, in _run_code
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     exec(code, run_globals)
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/pywikibot/scripts/wrapper.py", line 519, in <module>
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     main()
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/pywikibot/scripts/wrapper.py", line 503, in main
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     if not execute():
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/pywikibot/scripts/wrapper.py", line 490, in execute
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     run_python_file(filename, script_args, module)
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/pywikibot/scripts/wrapper.py", line 147, in run_python_file
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     exec(compile(source, filename, 'exec', dont_inherit=True),
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/scripts/noreferences.py", line 790, in <module>
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     main()
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/scripts/noreferences.py", line 784, in main
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     bot.run()
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/pywikibot/bot.py", line 1468, in run
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     self.treat(page)
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/pywikibot/bot.py", line 1721, in treat
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     self.treat_page()
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/scripts/noreferences.py", line 745, in treat_page
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     self.put_current(self.addReferences(text))
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]   File "/workspace/scripts/noreferences.py", line 614, in addReferences
2024-04-08T18:26:44+00:00 [fix-34434-rkprm]     for section in i18n.translate(self.site, referencesSections):
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] TypeError: 'NoneType' object is not iterable
2024-04-08T18:26:44+00:00 [fix-34434-rkprm] CRITICAL: Exiting due to uncaught exception TypeError: 'NoneType' object is not iterable

What should have happened instead?:

  • it should place the header of == References ==

Details

SubjectRepoBranchLines +/-
[L10N] localize noreferences.py for simple-wikipywikibot/coremaster+58 -18
[IMPR] Show an error message and leave if script is not localizedpywikibot/coremaster+39 -18
Customize query in gerrit

Related Objects

Event Timeline

PotsdamLamb created this task.Mon, Apr 8, 6:27 PM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptMon, Apr 8, 6:27 PM
JJMC89 renamed this task from noreferences.py - Bug to TypeError: 'NoneType' object is not iterable from noreferences script.Mon, Apr 8, 6:32 PM
JJMC89 added a project: Pywikibot-Scripts.
JJMC89 updated the task description. (Show Details)
Xqt subscribed.Tue, Apr 9, 1:01 PM

Seems noreferences is not localized for simple-wiki. The same traceback is shown e.g. for frr-wiki. Anyway the script should pass gracefully in such case.
@PotsdamLamb : Can we localize the bot with en-wiki entries or is there any difference?

PotsdamLamb added a comment.Tue, Apr 9, 11:46 PM

@Xqt I am running it on simple as I am banned on en-wiki. We use the same format though, so I am unsure what you are asking. I figured it should pass through without issue, but it dies when it hits that error.

Xqt added a comment.Wed, Apr 10, 7:25 AM

@PotsdamLamb: The references templates of simplewiki are not the same as at en-wiki. For example FootnotesSmall or Ref-list is missing but a Ref list is present. There are also some other differences. Therefore the script has to be localized first that it can be used. Could you please check the settings dictionaries of this script and provide valid 'simple' entries?

Xqt triaged this task as High priority.Wed, Apr 10, 8:08 AM
gerritbot added a comment.Wed, Apr 10, 8:49 AM

Change #1018643 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] Show an error message and leave if script is not localized

https://gerrit.wikimedia.org/r/1018643

gerritbot added a project: Patch-For-Review.Wed, Apr 10, 8:49 AM
Xqt lowered the priority of this task from High to Medium.Wed, Apr 10, 8:50 AM
Xqt changed the subtype of this task from "Bug Report" to "Feature Request".
Xqt moved this task from Backlog to Needs Review on the Pywikibot board.
Xqt moved this task from Backlog to L10N/i18n on the Pywikibot-Scripts board.
PotsdamLamb added a comment.EditedWed, Apr 10, 2:12 PM

@Xqt This is what we use

== Related Pages == (Our version of ENWPs see also)

Footnotes get used mostly in infoboxes where called. Otherwise, they are in the notes list below, which can use any of the same ones ENWP does

==  Notes == (above references)
{{notelist}}
{{Reflist|group=lower-alpha}}

== References ==
{{reflist}}

== Sources == (This is the one for any books cited via SFN, HARV, etc.)

== Other websites == (Our heading for ENWPs External Links)

Does this help you? This is our basic layout (https://simple.wikipedia.org/wiki/Wikipedia:Guide_to_layout), but it still lacks some things as they have not been added yet. We are in the process of doing it.

gerritbot added a comment.Wed, Apr 10, 6:24 PM

Change #1018643 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] Show an error message and leave if script is not localized

https://gerrit.wikimedia.org/r/1018643

gerritbot added a comment.Thu, Apr 11, 9:49 AM

Change #1018947 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [L10N] localize noreferences.py for simple-wiki

https://gerrit.wikimedia.org/r/1018947

Xqt added a comment.Thu, Apr 11, 9:51 AM

@PotsdamLamb: Could you please check the current L10N for simplewiki?

PotsdamLamb added a comment.Thu, Apr 11, 5:13 PM

@Xqt for some reason it keeps defaulting me to test, even though I have specified simple in my user config file.

# The family of sites to be working on.
# Pywikibot will import families/xxx_family.py so if you want to change
# this variable, you have to ensure that such a file exists. You may use
# generate_family_file to create one.
family = 'wikipedia'

# The site code (language) of the site to be working on.
mylang = 'simple'
JJMC89 subscribed.Thu, Apr 11, 5:18 PM

The tool-pywikibot/pywikibot-scripts-stable image does not use any config file you have stored on disk. There is one built into the image that you cannot change, so you must use the CLI arguments to specify the site and you will always see family and mylang are not set. and Defaulting to family='wikipedia' and mylang='test'. in the logs.

PotsdamLamb added a comment.Thu, Apr 11, 5:26 PM

@Xqt Sorry to be a pain, but can you specify what I need to change in the following, please? I am somewhat technically adept but not a programmer.

toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb -family:wikipedia -lang:simple noreferences -page:User:PotsdamLamb/test" fix-34434
Xqt added a comment.Fri, Apr 12, 5:53 AM

Please check whether you have pwb -family:wikipedia -lang:simple or pwb -site:Wikipedia:simple within your command line.

PotsdamLamb added a comment.Fri, Apr 12, 7:15 AM

@Xqt It is if you look at the code above, that is what I put in that sends me to the test wiki.

Xqt added a comment.Sun, Apr 14, 9:26 AM

@PotsdamLamb: No glue. @JJMC89: any ideas?

JJMC89 added a comment.Sun, Apr 14, 4:02 PM

AFAICT, nothing needs to change.

PotsdamLamb added a comment.Mon, Apr 15, 7:35 AM

@JJMC89 @Xqt Does it have to do with me running my command against the image maintained by WMF (toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest). Do I need to import everything into my tool again and use the local to run —command against the local tool version like I had to with my bot?

PotsdamLamb added a comment.Mon, Apr 15, 10:57 PM

@Xqt I did not realize you wanted me to review the patch until now. I looked at it, and I was confused. I want to ensure that notes will come before references (based on line 262). If that is the case, then we should be good to go with the push.

PotsdamLamb awarded a token.Tue, Apr 16, 11:58 PM
Xqt added a comment.Wed, Apr 17, 5:19 AM
In T362103#9715766, @PotsdamLamb wrote:

@Xqt I did not realize you wanted me to review the patch until now. I looked at it, and I was confused. I want to ensure that notes will come before references (based on line 262). If that is the case, then we should be good to go with the push.

Currently we cannot determine the references section to be placed after specified section but before others. Here it is before 'Sources' and 'Other websites'. The other way is not implemented yet but I understand it as a feature request. Can’t the references be placed inside 'Notes'? You did it in the example above (ok for groups).

PotsdamLamb added a comment.Wed, Apr 17, 10:50 AM

@Xqt as long as it is before those two sections then I think we will be alright as it should default to being under notes due to placement. Let’s go ahead and push it and I’ll test it out. Thanks!

gerritbot added a comment.Wed, Apr 17, 4:43 PM

Change #1018947 merged by jenkins-bot:

[pywikibot/core@master] [L10N] localize noreferences.py for simple-wiki

https://gerrit.wikimedia.org/r/1018947

Xqt claimed this task.Wed, Apr 17, 4:47 PM
Xqt closed this task as Resolved.Wed, Apr 17, 5:04 PM
Xqt mentioned this in T362806: Implement a PLACE_AFTER_SECTIONS in noreferences.py.
PotsdamLamb reopened this task as Open.Wed, Apr 17, 7:12 PM

@Xqt I just tried again, and I get the same errors. Here is the log:
(Also still defaults me to test instead of logging into simple)

Notice also that these are in different lines from the first error log I sent you.

2024-04-17T19:05:13+00:00 [fix-34434-hq7hm] family and mylang are not set.
2024-04-17T19:05:13+00:00 [fix-34434-hq7hm] Defaulting to family='wikipedia' and mylang='test'.
2024-04-17T19:05:15+00:00 [fix-34434-hq7hm] Retrieving 1 pages from wikipedia:simple.
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] Working on 'User:PotsdamLamb/test'
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] Found ref without references.
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] 1 read operation
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] Execution time:
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] Read operation time: 0.0 seconds
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] Script terminated by exception:
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] ERROR: 'NoneType' object is not iterable (TypeError)
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] Traceback (most recent call last):
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/pwb.py", line 40, in <module>
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     sys.exit(main())
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/pwb.py", line 36, in main
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     runpy.run_path(str(path), run_name='__main__')
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/layers/heroku_python/python/lib/python3.9/runpy.py", line 288, in run_path
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     return _run_module_code(code, init_globals, run_name,
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/layers/heroku_python/python/lib/python3.9/runpy.py", line 97, in _run_module_code
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     _run_code(code, mod_globals, init_globals,
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/layers/heroku_python/python/lib/python3.9/runpy.py", line 87, in _run_code
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     exec(code, run_globals)
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/pywikibot/scripts/wrapper.py", line 519, in <module>
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     main()
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/pywikibot/scripts/wrapper.py", line 503, in main
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     if not execute():
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/pywikibot/scripts/wrapper.py", line 490, in execute
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     run_python_file(filename, script_args, module)
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/pywikibot/scripts/wrapper.py", line 147, in run_python_file
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     exec(compile(source, filename, 'exec', dont_inherit=True),
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/scripts/noreferences.py", line 790, in <module>
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     main()
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/scripts/noreferences.py", line 784, in main
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     bot.run()
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/pywikibot/bot.py", line 1468, in run
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     self.treat(page)
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/pywikibot/bot.py", line 1721, in treat
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     self.treat_page()
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/scripts/noreferences.py", line 745, in treat_page
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     self.put_current(self.addReferences(text))
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]   File "/workspace/scripts/noreferences.py", line 614, in addReferences
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm]     for section in i18n.translate(self.site, referencesSections):
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] TypeError: 'NoneType' object is not iterable
2024-04-17T19:05:16+00:00 [fix-34434-hq7hm] CRITICAL: Exiting due to uncaught exception TypeError: 'NoneType' object is not iterable
JJMC89 closed this task as Resolved.Wed, Apr 17, 7:18 PM

You must wait for a release and then for the image to be updated.

Maintenance_bot removed a project: Patch-For-Review.Fri, Apr 26, 6:48 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL