Page MenuHomePhabricator

Operator should be warned if he has not sufficient rights to reupload existing files with upload script
Open, LowPublic

Description

Greetings!
I'm trying to mass re-upload new versions of existing files to Fandom using Pywikibot, but get an error. Here is a part of output from one attempt

Uploading file to ageofempires:en...
WARNING: /home/user/pywikibot/core/pywikibot/page/__init__.py:2565: FutureWarning: "ignore_warnings" as a boolean and "report_success" is True or None is deprecated for 5 years and 4 months; use "report_success=False" or define "ignore_warnings" as callable/iterable instead.
return self.site.upload(self, source_filename=filename, source_url=url,

WARNING: API error fileexists-forbidden: A file with this name already exists, and cannot be overwritten. If you still want to upload your file, please go back and use a new name. [[File:Britons_King_Move_3_AoE2.ogg|thumb|center|Britons_King_Move_3_AoE2.ogg]]
API Error: query=
("{'action': ['upload'], 'token': "
"['here_was_token'], 'text': ['New version"
'with higher quality of a dialogue file for a Briton unit in Age of Empires'
"2'], 'filename': ['Britons King Move 3 AoE2.ogg'], 'comment': ['New version"
'with higher quality of a dialogue file for a Briton unit in Age of Empires'
"2'], 'assert': ['user'], 'watch': [False], 'ignorewarnings': [True],"
"'maxlag': ['5'], 'format':['json']}")
response=
{'error': {'code': 'fileexists-forbidden', 'info': 'A file with this name already exists, and cannot be overwritten. If you still want to upload your file, please go back and use a new name. [[File:Britons_King_Move_3_AoE2.ogg|thumb|center|Britons_King_Move_3_AoE2.ogg]]', 'filekey': '181qqtpbrbxc.p9l6p4.47783138.ogx', 'sessionkey': '181qqtpbrbxc.p9l6p4.47783138.ogx', 'invalidparameter': 'filename', 'help': 'See https://ageofempires.fandom.com/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes.'}}
ERROR: Upload error:

I contacted Fandom support and they pointed out that according to here ignore_warnings must be set to reupload.

Event Timeline

The version result is required to determine your used Pywikibot version because it is not the current one. Please run

pwb version

In addition we have to know which script are you unsing and the options you have given.

Btw. if you are using upload.py there is a -ignorewarn option which can be set. Refer pwb upload -help for further Information about available Options.

Xqt renamed this task from Can't reupload exisitng files with upload script to Can't reupload existing files with upload script.Jan 13 2021, 9:49 AM

Sorry, forgot about those

Pywikibot: [https] r-pywikibot-core.git (bb333d2, g14032, 2021/01/12, 13:32:59, stable)
Release version: 5.5.0
requests version: 2.22.0
  cacerts: /etc/ssl/certs/ca-certificates.crt
    certificate test: ok
Python: 3.8.5 (default, Jul 28 2020, 12:59:40)
[GCC 9.3.0]
PYWIKIBOT_DIR: Not set
PYWIKIBOT_DIR_PWB:
PYWIKIBOT_NO_USER_CONFIG: Not set
Config base dir: /home/user/pywikibot/core
Usernames for family "ageofempires":
        en: Red Khan Bot

I used -ignorewarn option. The command I'm running is
python3 pwb.py upload -v -ignorewarn '/path/to/file/Britons King Move 3 AoE2.ogg' -descfile:/path/to/file/description.txt

I also tried -ignorewarn:exists, -ignorewarn:fileexists and -ignorewarn:fileexists-forbidden with the same result.

This comment was removed by Xqt.

Isn't there a traceback after that ERROR: Upload error:? That might be important.
Anyway 'ignorewarnings' was set to True. Probably you cannot override a given file and there is a restriction made by the site: 'fileexists-forbidden' is an error not just a warning.

Are you sure you have a 'reupload'/'reupload-shared' permission?

Here are all the log entries for this command


Just in case you are wondering - I'm running pywikibot from Windows Subsystem for Linux.

As for the restrictions made by the site. That's what I initially thought and here is their answer after I sent them the output in my initial post

From what I am reading from here: https://www.mediawiki.org/wiki/API:Upload#Upload_warnings ignore_warnings must be set to reupload. I do see, however, that pybot is giving an error in reference to ignore_warnings. As far as I am aware, nothing has changed in reference to our API that would differ it from core MediaWiki 1.33. If you have the latest version of pybot and are ignoring warnings, I would suggest filing a bug with pybot here: https://phabricator.wikimedia.org/maniphest/task/create/?projects=PHID-PROJ-orw42whe2lepxc7gghdq (Or hop on their IRC server for consult/triage/bug reporting: https://webchat.freenode.net/?channels=pywikibot)

Let's have a look to user rights:

>>> import pywikibot
>>> site = pywikibot.Site('ageofempires:en')
>>> sorted(site.userinfo['rights'])

What is the result of it?

['abusefilter-log', 'abusefilter-view', 'applychangetags', 'autoconfirmed', 'autopatrol', 'changetags', 'createpage', 'createtalk', 'edit', 'editsemiprotected', 'edittemplates', 'minoredit', 'move', 'move-categorypages', 'move-rootuserpages', 'move-subpages', 'nominornewtalk', 'purge', 'read', 'skipcaptcha', 'upload', 'writeapi']

Also I can upload files with new names (i.e. not re-uploading) using pywikibot.

Xqt triaged this task as Low priority.Jan 13 2021, 12:40 PM
Xqt added a project: Documentation.
['abusefilter-log', 'abusefilter-view', 'applychangetags', 'autoconfirmed', 'autopatrol', 'changetags', 'createpage', 'createtalk', 'edit', 'editsemiprotected', 'edittemplates', 'minoredit', 'move', 'move-categorypages', 'move-rootuserpages', 'move-subpages', 'nominornewtalk', 'purge', 'read', 'skipcaptcha', 'upload', 'writeapi']

Ok I see: You or your bot haven't needed rights to overwrite existing files. The needed permission is 'reupload' or specially for own files 'reupload-own'.

I would close this task as invalid but indeed there should be some informative hints in such case.

Xqt renamed this task from Can't reupload existing files with upload script to Operator should be warned if he has not sufficient rights to reupload existing files with upload script.Jan 13 2021, 12:42 PM

Thanks! I'll return to Fandom support then.

Still, unfortunately, no luck.
https://ageofempires.fandom.com/wiki/Special:ListGroupRights specifically shows that bots do have the 'reupload' right but I'm still getting no reupload in the rights list

['abusefilter-log', 'abusefilter-view', 'applychangetags', 'autoconfirmed', 'autopatrol', 'changetags', 'createpage', 'createtalk', 'edit', 'editsemiprotected', 'edittemplates', 'minoredit', 'move', 'move-categorypages', 'move-rootuserpages', 'move-subpages', 'nominornewtalk', 'purge', 'read', 'skipcaptcha', 'upload', 'writeapi']

>>> sorted(site.userinfo['groups']) returns this
['*', 'autoconfirmed', 'bot', 'user']

So we are unsure where it is breaking down.

Are you using a BotPassword? If so, make sure you have a grant with the right.

@JJMC89, thanks for help. It looks like the uploadeditmovefile is needed?

Still, unfortunately, no luck.
https://ageofempires.fandom.com/wiki/Special:ListGroupRights specifically shows that bots do have the 'reupload' right but I'm still getting no reupload in the rights list

I assume you are either not logged in or you are not logged in with your bot account. To verify the following snippet might be usefull:

>>> import pywikibot
>>> site = pywikibot.Site('ageofempires:en')
>>> site.username()
YourUsername
>>> site.logged_in()
True
>>> site.isBot(site.username())
True

What's your result then?

>>> import pywikibot
>>> site = pywikibot.Site('ageofempires:en')
>>> site.username()
'Red Khan Bot'
>>> site.logged_in()
False
>>> site.isBot(site.username())
True

Isn't site = pywikibot.Site('ageofempires:en') supposed to log me in?

@JJMC89, thanks for help. It looks like the uploadeditmovefile is needed?

Yes

I assume you are either not logged in or you are not logged in with your bot account.

Since they show as in the bot group, that's not the issue.

Did some experimenting. This sequence works

>>> import pywikibot
>>> site = pywikibot.Site('ageofempires:en')
>>> sorted(site.userinfo['rights'])
['abusefilter-log', 'abusefilter-view', 'applychangetags', 'autoconfirmed', 'autopatrol', 'changetags', 'createpage', 'createtalk', 'edit', 'editsemiprotected', 'edittemplates', 'minoredit', 'move', 'move-categorypages', 'move-rootuserpages', 'move-subpages', 'nominornewtalk', 'purge', 'read', 'skipcaptcha', 'upload', 'writeapi']
>>> site.username()
'Red Khan Bot'
>>> site.logged_in()
True
>>> site.isBot(site.username())
True

Are you using a BotPassword?

Sorry, forgot to answer. Yes indeed, I'm using a BotPassword.
https://ageofempires.fandom.com/wiki/Special:BotPasswords/Red_Khan_Bot

OK, feeling stupid but figure it out now. I had to grant Upload, replace, and move files right on the Special:BotPasswords/Red_Khan_Bot page.
Thanks for help everybody, sorry to bother you.