Page MenuHomePhabricator

several scripts fails with RuntimeError('dictionary changed size during iteration')
Closed, ResolvedPublicBUG REPORT

Description

py -3 pwb.py login -all

C:\pwb\GIT\core>py -3 pwb.py login -all
WARNING: No user is logged in on site commons:commons
Password for user Xqbot on commons:commons (no characters will be shown):
Logging in to commons:commons as Xqbot
Logged in on commons:commons as Xqbot.
Traceback (most recent call last):
  File "pwb.py", line 390, in <module>
    if not main():
  File "pwb.py", line 382, in main
    run_python_file(filename,
  File "pwb.py", line 100, in run_python_file
    exec(compile(source, filename, 'exec', dont_inherit=True),
  File ".\scripts\login.py", line 187, in <module>
    main()
  File ".\scripts\login.py", line 159, in main
    for family_name in namedict:
RuntimeError: dictionary changed size during iteration
CRITICAL: Exiting due to uncaught exception <class 'RuntimeError'>

py -3 pwb.py watchlist -new

C:\pwb\GIT\core>py -3 pwb.py watchlist -new
Downloading all watchlists for your accounts in user-config.py
Retrieving watchlist for commons:commons.
Traceback (most recent call last):
  File "pwb.py", line 390, in <module>
    if not main():
  File "pwb.py", line 382, in main
    run_python_file(filename,
  File "pwb.py", line 100, in run_python_file
    exec(compile(source, filename, 'exec', dont_inherit=True),
  File ".\scripts\watchlist.py", line 111, in <module>
    main()
  File ".\scripts\watchlist.py", line 98, in main
    refresh_new()
  File ".\scripts\watchlist.py", line 73, in refresh_new
    for family in config.usernames:
RuntimeError: dictionary changed size during iteration
CRITICAL: Exiting due to uncaught exception <class 'RuntimeError'>

This occurres for both: Python 2 and Python 3

Event Timeline

Xqt triaged this task as High priority.May 19 2020, 3:55 PM
Xqt added a subscriber: Dvorapa.

Exactly! This also happens. Cloning keys with list() will solve the issue for both PY2 and PY3.

Xqt renamed this task from login -all fails with RuntimeError('dictionary changed size during iteration') to several scripts fails with RuntimeError('dictionary changed size during iteration').May 19 2020, 3:59 PM
Xqt updated the task description. (Show Details)

This has to do with instantiating a Site:

>>> import pwb, pywikibot
>>> from pywikibot.config2 import usernames
>>> for fam in usernames:
	for lang in usernames[fam]:
		pywikibot.Site(lang, fam)

		
APISite("commons", "commons")
Traceback (most recent call last):
  File "<pyshell#115>", line 1, in <module>
    for fam in usernames:
RuntimeError: dictionary changed size during iteration

The first site was created but the second fails.

The Problem is in line 1246 inside pywikibot.__init__.py:

code_to_user = config.usernames['*'].copy()

usernames is a defaultdict. If '*' key is not in that dict this statement creates it and changes the usernames before the copy will be done.

Change 597313 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [bugfix] Do not change usernames when creating a Site

https://gerrit.wikimedia.org/r/597313

The Problem is in line 1246 inside pywikibot.__init__.py:

code_to_user = config.usernames['*'].copy()

usernames is a defaultdict. If '*' key is not in that dict this statement creates it and changes the usernames before the copy will be done.

I was digging and found that. Thanks very much! I was thinking along the same lines :)

Change 597313 merged by jenkins-bot:
[pywikibot/core@master] [bugfix] Do not change usernames when creating a Site

https://gerrit.wikimedia.org/r/597313

xSavitar lowered the priority of this task from High to Low.May 19 2020, 5:27 PM

I'll leave this for @Xqt to close but I do confirm that the solution does fixes the problem. Thank you!

Thanks for review

Xqt changed the subtype of this task from "Task" to "Bug Report".