Page MenuHomePhabricator

78 datasets fail harvest because `"title() takes 1 positional argument but 2 were given`
Closed, ResolvedPublicBUG REPORT

Description

pywikibot was upgraded with T295238: Upgrade pywikibot to latest version. But now we are reaping the fruits of it...

The harvest of 78 datasets failed with:

"title() takes 1 positional argument but 2 were given"

Presumably this was not caught during the upgrade because I always use ge:ka as end-to-end functional test, and it is not one of the failures

Reproducing locally with no:no:

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/code/erfgoedbot/update_database.py", line 709, in <module>
    main()
  File "/code/erfgoedbot/update_database.py", line 678, in main
    days_back)
  File "/code/erfgoedbot/update_database.py", line 470, in process_country
    countryconfig, conn, cursor, full_update, days_back)
  File "/code/erfgoedbot/update_database.py", line 508, in process_country_list
    conn, cursor, unknown_fields=unknown_fields)
  File "/code/erfgoedbot/update_database.py", line 459, in process_page
    primkey_exceptions, page.title(True), countryconfig.get('table')))
  File "/usr/local/lib/python3.5/dist-packages/pywikibot/tools/_deprecate.py", line 404, in wrapper
    return obj(*__args, **__kw)
TypeError: title() takes 1 positional argument but 2 were given
CRITICAL: Exiting due to uncaught exception <class 'TypeError'>

Also explains the drop from 1.7M monuments to 420K in the database.

Complete list of datasets which failed:

ERROR: Unknown error occurred when processing country ph in lang en
ERROR: Unknown error occurred when processing country gb-nir in lang en
ERROR: Unknown error occurred when processing country no in lang no
ERROR: Unknown error occurred when processing country fr in lang fr
ERROR: Unknown error occurred when processing country pl in lang pl
ERROR: Unknown error occurred when processing country nl in lang nl
ERROR: Unknown error occurred when processing country be-wal in lang en
ERROR: Unknown error occurred when processing country wlpa-at in lang de
ERROR: Unknown error occurred when processing country il-npa in lang he
ERROR: Unknown error occurred when processing country ch2 in lang de
ERROR: Unknown error occurred when processing country es in lang ca
ERROR: Unknown error occurred when processing country ca-prov in lang en
ERROR: Unknown error occurred when processing country it-88 in lang ca
ERROR: Unknown error occurred when processing country dk-fortids in lang da
ERROR: Unknown error occurred when processing country ca in lang fr
ERROR: Unknown error occurred when processing country it-bz in lang de
ERROR: Unknown error occurred when processing country uy in lang es
ERROR: Unknown error occurred when processing country de-by in lang de
ERROR: Unknown error occurred when processing country it in lang it
ERROR: Unknown error occurred when processing country at in lang de
ERROR: Unknown error occurred when processing country de-he in lang de
ERROR: Unknown error occurred when processing country gb-eng in lang en
ERROR: Unknown error occurred when processing country il in lang he
ERROR: Unknown error occurred when processing country es in lang es
ERROR: Unknown error occurred when processing country se-fornmin in lang sv
ERROR: Unknown error occurred when processing country be-bru in lang nl
ERROR: Unknown error occurred when processing country ie in lang en
ERROR: Unknown error occurred when processing country az in lang az
ERROR: Unknown error occurred when processing country nl-aw in lang en
ERROR: Unknown error occurred when processing country be-wal in lang nl
ERROR: Unknown error occurred when processing country mx in lang es
ERROR: Unknown error occurred when processing country wlpa-il in lang en
ERROR: Unknown error occurred when processing country za in lang en
ERROR: Unknown error occurred when processing country nl-prov in lang nl
ERROR: Unknown error occurred when processing country es-gl in lang gl
ERROR: Unknown error occurred when processing country pt in lang pt
ERROR: Unknown error occurred when processing country co in lang es
ERROR: Unknown error occurred when processing country cn in lang en
ERROR: Unknown error occurred when processing country dk-bygning in lang da
ERROR: Unknown error occurred when processing country wlpa-fi in lang fi
ERROR: Unknown error occurred when processing country rs in lang sr
ERROR: Unknown error occurred when processing country ch in lang de
ERROR: Unknown error occurred when processing country gb-sct in lang en
ERROR: Unknown error occurred when processing country hu in lang hu
ERROR: Unknown error occurred when processing country ve in lang es
ERROR: Unknown error occurred when processing country jp-nhs in lang en
ERROR: Unknown error occurred when processing country pe in lang es
ERROR: Unknown error occurred when processing country cz in lang cs
ERROR: Unknown error occurred when processing country bo in lang es
ERROR: Unknown error occurred when processing country se-bbr in lang sv
ERROR: Unknown error occurred when processing country se-ship in lang sv
ERROR: Unknown error occurred when processing country nl-aw in lang nl
ERROR: Unknown error occurred when processing country be-vlg in lang nl
ERROR: Unknown error occurred when processing country es-vc in lang ca
ERROR: Unknown error occurred when processing country es-ct in lang ca
ERROR: Unknown error occurred when processing country de-nrw-bm in lang de
ERROR: Unknown error occurred when processing country pa in lang es
ERROR: Unknown error occurred when processing country th in lang th
ERROR: Unknown error occurred when processing country ua in lang uk
ERROR: Unknown error occurred when processing country ru in lang ru
ERROR: Unknown error occurred when processing country ca-fed in lang en
ERROR: Unknown error occurred when processing country wlpa-es-ct in lang ca
ERROR: Unknown error occurred when processing country ch-old in lang en
ERROR: Unknown error occurred when processing country ca-muni in lang en
ERROR: Unknown error occurred when processing country nl-gem in lang nl
ERROR: Unknown error occurred when processing country sk in lang de
ERROR: Unknown error occurred when processing country ar in lang es
ERROR: Unknown error occurred when processing country eg in lang ar
ERROR: Unknown error occurred when processing country se-arbetsl in lang sv
ERROR: Unknown error occurred when processing country fr in lang ca
ERROR: Unknown error occurred when processing country by in lang be-tarask
ERROR: Unknown error occurred when processing country ro in lang ro
ERROR: Unknown error occurred when processing country de-nrw-k in lang de
ERROR: Unknown error occurred when processing country us in lang en
ERROR: Unknown error occurred when processing country cl in lang es
ERROR: Unknown error occurred when processing country am in lang hy
ERROR: Unknown error occurred when processing country ch3 in lang de
ERROR: Unknown error occurred when processing country ir in lang fa

Event Timeline

FYI this caused https://map-of-monuments.toolforge.org to go almost empty. It's causing an issue, as we just started a contest in writing articles about monuments, and this tool is where contest participants are directed.

Change 759229 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[labs/tools/heritage@master] update_database: Fix title() takes 1 positional argument but 2 were given

https://gerrit.wikimedia.org/r/759229

Mentioned in SAL (#wikimedia-cloud) [2022-02-02T11:42:10Z] <urbanecm> Run python erfgoedbot/update_database.py -countrycode:cz -langcode:cs with 759229 applied to ensure fix for T300252 works

Mentioned in SAL (#wikimedia-cloud) [2022-02-02T11:42:10Z] <urbanecm> Run python erfgoedbot/update_database.py -countrycode:cz -langcode:cs with 759229 applied to ensure fix for T300252 works

This livehacked ran worked fine as far as I can see:

MariaDB [s51138__heritage_p]> select count(*) from `monuments_cz_(cs)`;
+----------+
| count(*) |
+----------+
|    41482 |
+----------+
1 row in set (0.02 sec)

MariaDB [s51138__heritage_p]>

Looking forward to the commit above being merged.

Mentioned in SAL (#wikimedia-cloud) [2022-02-02T12:30:46Z] <urbanecm> Run P19935 in s51138__heritage_p to workaround T300252 for Czech

Change 759229 merged by jenkins-bot:

[labs/tools/heritage@master] update_database: Fix title() takes 1 positional argument but 2 were given

https://gerrit.wikimedia.org/r/759229

Mentioned in SAL (#wikimedia-cloud) [2022-02-02T12:49:48Z] <wm-bot> <jeanfred> Trigger full update_monuments.sh run now that T300252 is fixed

Lokal_Profil assigned this task to Urbanecm.