Page MenuHomePhabricator

PyWikiBot in -auto mode does not create graphs if the links contain pages of disambig or pages in different namespaces.
Closed, ResolvedPublicFeature

Description

Steps to replicate the issue (include links if applicable):

  • py pwb.py interwiki -lang:de -auto -ns:0,14,10000,10002,10004,10006, -recentchanges:0,70 -pt:1 -async
  • in user-config.py interwiki_graph = True interwiki_graph_formats = ['svg']
  • During the work process, the autonomous_problems.dat file was filled with 500 records, while graphs were created for only two cases.

In manual start all work

What happens?:

What should have happened instead?:
Graphs will be built for all problem cases, not only in manual mode.

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):

Other information (browser name/version, screenshots, etc.):

Event Timeline

Xqt triaged this task as Low priority.Feb 1 2026, 12:37 PM
Xqt changed the subtype of this task from "Bug Report" to "Feature Request".

@Xqt Why isn't this a bug? I don't see this behavior being expected. In addition, as I already pointed out, in automatic mode, the bot still makes graphs, but only in a combination of not very frequent situations.

@Xqt Why isn't this a bug? I don't see this behavior being expected. In addition, as I already pointed out, in automatic mode, the bot still makes graphs, but only in a combination of not very frequent situations.

It works as documented in https://doc.wikimedia.org/pywikibot/master/scripts_ref/scripts.html#module-scripts.interwiki:
-autonomous, -auto:
Run automatically, do not ask any questions. If a question to an operator is needed, write the name of the page to autonomous_problems.dat and continue on the next page.

See also:

I know that the interwiki script is a bit out of scope, since Wikimedia uses a Wikibase for that purpose. For testing, it would be great to know the site (or family) you are working on.

It works as documented in https://doc.wikimedia.org/pywikibot/master/scripts_ref/scripts.html#module-scripts.interwiki:
-autonomous, -auto:
Run automatically, do not ask any questions. If a question to an operator is needed, write the name of the page to autonomous_problems.dat and continue on the next page.

This does not explain why, if there is a conflict between pages in the same namespace and/or non-disambiguation pages, the script draws the graph.

I know that the interwiki script is a bit out of scope, since Wikimedia uses a Wikibase for that purpose. For testing, it would be great to know the site (or family) you are working on.

I work on minecraft.wiki
Draft of family.py placed here: https://ru.minecraft.wiki/w/Участник:Ivan-r/Черновик3

This does not explain why, if there is a conflict between pages in the same namespace and/or non-disambiguation pages, the script draws the graph.

interwiki.py assumes that interwiki links form a strict 1:1 relationship. The interwiki graph is a supporting feature to visualize and detect multiple relations between sites.

Conflicts caused by namespace mismatches or non-disambiguation pages do not violate the 1:1 assumption itself; they indicate issues in the meta content, not in the interwiki link structure. Therefore, they are not included in the Subject structure (yet).

That said, I may be missing something — the script is complex enough that I cannot claim to fully understand all of its internals. Therefore, online tests would be helpful.

Yes, I may have considered the situation in my project to be natural, which may not be true.

Our situation with namespaces is due to the fact that articles are located in different namespaces, all of which are marked as content. That is, politically, despite the differences in namespaces, they are all articles. https://minecraft.wiki/api.php?action=query&meta=siteinfo&siprop=namespaces&formatversion=2

And the situation with disambiguity is due to the fact that in some cases, instead of a simple list of terms, a full-fledged article is used, which serves similar purposes, and therefore must be linked.

The namespace mismatch can be solved by a cossnamespace attribute in your family file (see the example in the linked documentation). The previous wikipedia/wikisource usage can be seen in rPWBC77d1546. The CrossnamespaceType is defined as CrossnamespaceType = collections.defaultdict[str, dict[str, list[int]]] where str literals are site codes or '_default' and int values are the namespace numbers. The trick is this code

# Allow for a mapping between different namespaces
crossFrom = self.origin.site.family.crossnamespace.get(
    self.origin.namespace(), {})
crossTo = crossFrom.get(self.origin.site.code,
                        crossFrom.get('_default', {}))
nsmatch = crossTo.get(linkedPage.site.code,
                      crossTo.get('_default', []))
if linkedPage.namespace() in nsmatch:
    return False

I'm having trouble getting this feature to work. In manual mode, the bot complains about inappropriate namespace names instead number. There are too many problems there. When I finish solving problems on the host side, I will monitor how the bot processes it and possibly create a separate issue if the problem is entirely on the bot's side.

In any case, if the page is not located where it should be located, it is much easier to see it in the picture.

We could implement an option or an easier crossnamespace setting to ignore namespace mismatch if this is appropriate for that wiki.

I think the crossnamespace attribute has to be implemented as follows in your family file:

crossnamespace = dict.fromkeys(
    keys := [0, 10000, 10002, 10004, 10006],
    {'_default': {'default': keys}}
)

So. I'm done setting up the namespaces. Now the bot does not fall when processing links between valid namespaces.
https://ru.minecraft.wiki/w/Справка:PyWikiBot#mcw_family.py

But if it again encounters problems in namespaces, will it again write to autonomous_problems.dat without creating graphs?

I would also like to add that if you specify interwiki_graph_url in user-config.py, the bot adds the corresponding entries to autonomous_problems.dat, as if the files were made, but for a hundred such entries, only one corresponds to the actually created file.

It looks like the graph is called only in one strange place

elif self.conf.autonomous and duplicate and not skip:

It looks like the graph is called only in one strange place

elif self.conf.autonomous and duplicate and not skip:

No, it's line 1371:

if self.conf.graph:
    graphDrawer = interwiki_graph.GraphDrawer(self)
    graphDrawer.createGraph()

calling the Drawner with self as interwiki_graph.Subject instance. The other lines writes additional informations to the autonomous_problems.dat file.

Change #1323835 had a related patch set uploaded (by Ivan-r; author: Ivan-r):

[pywikibot/core@master] [bugfix] interwiki.py: Graph is often not created in -auto mode

https://gerrit.wikimedia.org/r/1323835

Xqt assigned this task to Ivan-r.

@Ivan-r: Great, thank you.

Change #1323835 merged by Xqt:

[pywikibot/core@master] [bugfix] interwiki.py: Graph is often not created in -auto mode

https://gerrit.wikimedia.org/r/1323835