Page MenuHomePhabricator

interwiki.py gives warning instead of editing and saving pages
Closed, ResolvedPublic

Description

When I try to use interwiki.py, it collects all the pages that it'll edit, but at the end, when (I assume so) it wants to save the pages, it fails to do so.

What I expected to happen

It saves the selected pages.

What happend

It gives me a couple of warnings which states that interwiki X is not linked to interwiki Y, that interwiki Z is not linked to interwiki Y, that interwiki Y is not linked to interwiki X, etc.

How to reproduce

  1. Get the Pywikibot version from http://tools.wmflabs.org/pywikibot/core_stable.zip
  2. Run the script for creating user-config.py and maybe to create a family file (only when reproducing doesn't work with the families that are included in the download).
  3. Login using login.py.
  4. Use interwiki.py (adding a page name is optional).

You'll see that, at the end, under ====== Post-processing [[interwiki]] ======, it gives a couple of warnings and won't try to edit and save the page.

Some code stuff

It looks like it gets stuck on this piece of code which starts at line 2071 and ends at line 2087 (not sometimes, but every time):

linkedPagesDict = {}
for linkedPage in linkedPages:
    linkedPagesDict[linkedPage.site] = linkedPage
for expectedPage in expectedPages - linkedPages:
    if expectedPage != page:
        try:
            linkedPage = linkedPagesDict[expectedPage.site]
            pywikibot.output(
                u"WARNING: %s: %s does not link to %s but to %s"
                % (page.site.family.name,
                   page, expectedPage, linkedPage))
        except KeyError:
            if not expectedPage.site.is_data_repository():
                pywikibot.output(
                    u"WARNING: %s: %s does not link to %s"
                    % (page.site.family.name,
                       page, expectedPage))

When this happens, I get in Command Prompt this output (<username> stands for the account I'm currently using, don't want to put my name here):

C:\Users\<username>\Documents\pywikibot\core_stable>interwiki.py Alpha_1.0.1
Retrieving 1 pages from minecraft:nl.
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[en:Alpha 1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[ru:Alpha 1.0.1]]
Retrieving 1 pages from minecraft:ru.
Retrieving 1 pages from minecraft:en.
======Post-processing [[nl:Alpha 1.0.1]]======
WARNING: minecraft: [[ru:Alpha 1.0.1]] does not link to [[nl:Alpha 1.0.1]]
WARNING: minecraft: [[en:Alpha 1.0.1]] does not link to [[ru:Alpha 1.0.1]]

Details

  • The Wiki I try to edit is http://minecraft-nl.gamepedia.com and all the other languages which are part of that Wiki group. As you might've seen in the output, not all the Wikis have every page the English Wiki has (which is also the base Wiki).
    • Even though it's a third-party Wiki, some users use Pywikibot as bot software. It looks like they don't have this issue (maybe an older version of Pywikibot?).
  • Python version 2.7.11.
  • The bot is logged in.
    • Doesn't have bot rights yet. I want to test it first before even asking for those rights.
  • I tried to use a (newer?) version from Github (link: https://github.com/wikimedia/pywikibot-core/), but that failed because it needs a so-called 'ListOption' and 'StandardOption' which are, according to the code, stored in pywikibot\bot.
    • I tried after that to search for the folder 'bot' in the Github of Pywikibot. I haven't found it (yet).
  • Pywikibot is downloaded from http://tools.wmflabs.org/pywikibot/core_stable.zip
  • Operating system: Windows 10, version 10.0.10586.

Event Timeline

Framawiki renamed this task from [Bug] Pywikibot gives warning instead of editing and saving pages when using interwiki.py to interwiki.py gives warning instead of editing and saving pages.Apr 25 2017, 10:39 PM
Xqt subscribed.

I found a section error raising:

C:\pwb\GIT\core>py -3.13 -m pwb interwiki -site:minecraft:nl Alpha_1.0.1
Retrieving 1 pages from minecraft:nl.
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[ru:Alpha v1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[en:Java Edition Alpha v1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[zh:Java版Alpha v1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[fr:Historique des versions Java/Alpha#1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[ko:자바 에디션 알파 v1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[cs:Java edice Alpha v1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[de:Versionen/Alpha#1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[tr:Java Edition Alpha v1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[ja:Java Edition Alpha v1.0.1]]
[[nl:Alpha 1.0.1]]: [[nl:Alpha 1.0.1]] gives new interwiki [[pt:Edição Java Alpha v1.0.1]]
Retrieving 1 pages from minecraft:ru.
NOTE: [[ru:Alpha v1.0.1]] is redirect to [[ru:Alpha v1.0.1 (Java Edition)]]
[[nl:Alpha 1.0.1]]: [[ru:Alpha v1.0.1]] gives new redirect [[ru:Alpha v1.0.1 (Java Edition)]]
Retrieving 1 pages from minecraft:en.
[[nl:Alpha 1.0.1]]: [[en:Java Edition Alpha v1.0.1]] gives new interwiki [[es:Java Edition Alpha v1.0.1]]
NOTE: [[nl:Alpha 1.0.1]]: [[en:Java Edition Alpha v1.0.1]] gives duplicate interwiki on same site [[ko:Java Edition 알파 v1.0.1]]
Retrieving 2 pages from minecraft:ko.
NOTE: [[ko:자바 에디션 알파 v1.0.1]] is redirect to [[ko:Java Edition 알파 v1.0.1]]
Retrieving 1 pages from minecraft:zh.
Retrieving 1 pages from minecraft:fr.
ERROR: '1.0.1' is not a valid section of Historique des versions Java/Alpha
Traceback (most recent call last):
  File "C:\pwb\GIT\core\scripts\interwiki.py", line 2383, in main
    bot.run()
    ~~~~~~~^^
  File "C:\pwb\GIT\core\scripts\interwiki.py", line 2023, in run
    self.queryStep()
    ~~~~~~~~~~~~~~^^
  File "C:\pwb\GIT\core\scripts\interwiki.py", line 2000, in queryStep
    self.oneQuery()
    ~~~~~~~~~~~~~^^
  File "C:\pwb\GIT\core\scripts\interwiki.py", line 1995, in oneQuery
    subject.batchLoaded(self)
    ~~~~~~~~~~~~~~~~~~~^^^^^^
  File "C:\pwb\GIT\core\scripts\interwiki.py", line 1279, in batchLoaded
    self.check_page(page, counter)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "C:\pwb\GIT\core\scripts\interwiki.py", line 1132, in check_page
    if page_empty_check(page):
       ~~~~~~~~~~~~~~~~^^^^^^
  File "C:\pwb\GIT\core\scripts\interwiki.py", line 2111, in page_empty_check
    txt = page.text
          ^^^^^^^^^
  File "C:\pwb\GIT\core\pywikibot\page\_basepage.py", line 581, in text
    return self.get(get_redirect=True)
           ~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "C:\pwb\GIT\core\pywikibot\page\_basepage.py", line 396, in get
    raise SectionError(f'{page_section!r} is not a valid section '
                       f'of {self.title(with_section=False)}')
pywikibot.exceptions.SectionError: '1.0.1' is not a valid section of Historique des versions Java/AlphaDump nl (minecraft) appended.

This should be solved first.

Pretty sure these restrictions prohibit the change:

# if we have an account for this site
if site.family.name in config.usernames \
   and site.code in config.usernames[site.family.name] \
   and not site.has_data_repository:
    # Try to do the changes

But wildcards aren't respected here!

Otherwise the bot works as expected:

======Post-processing [[nl:Alpha 1.0.1]]======
Updating links on page [[cs:Java edice Alpha v1.0.1]].
Changes to be made: Bot: Adding [[es:Java Edition Alpha v1.0.1]]; modifying [[ko:Java Edition 알파 v1.0.1]], [[ru:Alpha v1.0.1 (Java Edition)]]
@@ -85,0 +86 @@
+ [[es:Java Edition Alpha v1.0.1]]

@@ -88 +89 @@
- [[ko:자바 에디션 알파 v1.0.1]]
+ [[ko:Java Edition 알파 v1.0.1]]

@@ -91 +92 @@
- [[ru:Alpha v1.0.1]]
+ [[ru:Alpha v1.0.1 (Java Edition)]]

NOTE: Updating live wiki...
ERROR putting page: User "None" does not have required user right "edit" on site minecraft:cs.
Updating links on page [[en:Java Edition Alpha v1.0.1]].
Changes to be made: Bot: Modifying [[ru:Alpha v1.0.1]] to [[ru:Alpha v1.0.1 (Java Edition)]]
@@ -112 +112 @@
- [[ru:Alpha v1.0.1]]
+ [[ru:Alpha v1.0.1 (Java Edition)]]

NOTE: Updating live wiki...
Xqt triaged this task as Low priority.Jul 8 2025, 4:39 PM
Xqt removed a project: TestMe.

Change #1167523 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] IMPR: Add a warning in process_unlimited if a username is not configured for site

https://gerrit.wikimedia.org/r/1167523

Change #1167523 merged by jenkins-bot:

[pywikibot/core@master] IMPR: process_unlimited: warn if a username is not configured for site

https://gerrit.wikimedia.org/r/1167523