Page MenuHomePhabricator

Bot should always add CFD template when moving category
Open, MediumPublic

Description

Steps to reproduce

  1. Run python pwb.py category move -from:"category1" -to:"category2"
  2. Bot moves category page to a new name
  3. Bot only sometimes adds CFD template to page "category1" on cswiki

Expected behavior
Bot should always add CFD template when renaming a category.

Current behavior
The CFD template is added only in cca 1 of 5 cases. I can't figure out what is the key. It seems like bot is doing this randomly. I thought it is caused by links to the category from other pages, but it isn't.

Configuration
last commit of pwb core from Gerrit, Python v3.5.2–3.6.1; the bug is present in master branch at least since Jan 2016

Event Timeline

Dvorapa created this task.May 14 2016, 10:30 AM
Restricted Application added subscribers: pywikibot-bugs-list, Zppix, Aklapper. · View Herald TranscriptMay 14 2016, 10:30 AM
Dvorapa renamed this task from Template for deletion after move transcluded only sometimes to Template for deletion after category move is added only sometimes.Jul 11 2016, 8:16 PM
Dvorapa renamed this task from Template for deletion after category move is added only sometimes to Template for deletion after category move is added only sometimesremove CFR template when moving category.Jul 11 2016, 8:27 PM
Dvorapa renamed this task from Template for deletion after category move is added only sometimesremove CFR template when moving category to Bot should always add CFD template when moving category.
Dvorapa updated the task description. (Show Details)
Dvorapa updated the task description. (Show Details)Jul 11 2016, 8:32 PM
Dvorapa updated the task description. (Show Details)Jul 14 2016, 11:23 AM
Dvorapa updated the task description. (Show Details)Jul 14 2016, 11:29 AM
Dvorapa updated the task description. (Show Details)May 18 2017, 11:15 AM
Dvorapa updated the task description. (Show Details)May 18 2017, 11:17 AM

Hello @Dvorapa.
It's hard to figure out the problem. Can you add many diffs where template has been added and diffs where it has not been ?

Hello @Dvorapa.
It's hard to figure out the problem. Can you add many diffs where template has been added and diffs where it has not been ?

I'll ask admins on cswiki for help, pages are deleted. Where CFD template was not added by pwb, I've always added it by myself.

In most cases bot doesn't add CFD template to the old page title (on the other hand removing CFR template from new page title works always), therefore I'm always surprised a bit when bot does add a CFD template to the old page title.

Perhaps you can copy the page's wikitext is a Phabricator past, before it is deleted ?

Dvorapa added a comment.EditedMay 24 2017, 9:07 AM

Last move:

original content on former title; with move proposal (CFR template)

Kategorie:FC Internazionale Milano
{{přejmenovat kategorii|Kategorie:Inter Milán}}
{{Hlavní článek|Inter Milán}} 
* {{Commonscat}}

[[Kategorie:Italské fotbalové kluby|Internazionale]]
[[Kategorie:Sport v Miláně]]

content moved to proposed title; move proposal (CFR template) removed automatically by pwb

Kategorie:Inter Milán
{{Hlavní článek|Inter Milán}}
* {{Commonscat}}

[[Kategorie:Italské fotbalové kluby|Internazionale]]
[[Kategorie:Sport v Miláně]]

redirect created automatically by pwb on the former title; CFD template NOT added

Kategorie:FC Internazionale Milano
#PŘESMĚRUJ [[:Kategorie:Inter Milán]]

CFD template added by hand

Kategorie:FC Internazionale Milano
{{Smazat|přejmenováno}}
#PŘESMĚRUJ [[:Kategorie:Inter Milán]]

Version: pwb core latest master for users (commit 05e0f916f280448f6ca27a7cd39f9c2652cd8aaf) obtained from gerrit and updated before move by git pull, logged in by password, python 3.6.1

Only 10/500 last category moves of my bot added CFD template like this:

original content on former title; with move proposal (CFR template)

Kategorie:Jidiš písně
{{Přejmenovat kategorii|Písně v jidiš}}
[[Kategorie:Písně podle jazyka]]
[[Kategorie:Jidiš]]
[[Kategorie:Židovská hudba]]

content moved to proposed title; move proposal (CFR template) removed automatically by pwb

Kategorie:Písně v jidiš
[[Kategorie:Písně podle jazyka]]
[[Kategorie:Jidiš]]
[[Kategorie:Židovská hudba]]

redirect created automatically by pwb on the former title

Kategorie:Jidiš písně
#PŘESMĚRUJ [[:Kategorie:Písně v jidiš]]

CFD template added correctly, automatically by pwb

Kategorie:Jidiš písně
{{delete|1=Robot: nahrazení kategorie Jidiš písně za Písně v jidiš}}
#PŘESMĚRUJ [[:Kategorie:Písně v jidiš]]
Dvorapa added a comment.EditedMay 24 2017, 11:21 AM

But if I look into the history of deleted categories moved by bot - those 10 of last 500, where bot added CFD template, it still looks like this is a random selection.

My code always look like python pwb.py category move. I sometimes use python pwb.py category move -inplace and sometimes not, but there is no evidence of this parameter affecting the behavior

I'd like to help, my python knowledge is really good, but code for adding CFD template to the redirect is commented poorly.

Dvorapa added a comment.EditedMay 24 2017, 11:35 AM

Maybe category move refuses to edit the redirect always, but sometimes the check if page is redirect fails somehow? (just a guess)

Dvorapa added a comment.EditedMay 24 2017, 11:40 AM

Or maybe the check if category is empty fails due to server cache somehow? Because sometimes the category doesn't look empty immediately in a browser after it is emptied by bot.

Thanks for your comments @Dvorapa !

JAnD added a subscriber: JAnD.EditedMay 24 2017, 8:17 PM

Maybe category move refuses to edit the redirect always, but sometimes the check if page is redirect fails somehow? (just a guess)

Maybe the same problem with recognizing category redirects like in T115091?

I've got special time-limited rights on Czech Wikipedia to be able to see deleted edits of my bot (ping @Framawiki). Are there any additional information I could provide using this advantage?

scripts/category.py
if self.oldcat.isEmptyCategory() and self.delete_oldcat and \
    ((self.newcat and self.move_oldcat) or not self.newcat):
  self._delete(can_move_page, can_move_talk)

Or maybe the check if category is empty fails due to server cache somehow? Because sometimes the category doesn't look empty immediately in a browser after it is emptied by bot.

Independently, I have made it to the same finding.

Change 355791 had a related patch set uploaded (by Matěj Suchánek; owner: Matěj Suchánek):
[pywikibot/core@master] When moving a category, track number of members locally as well

https://gerrit.wikimedia.org/r/355791

Dvorapa added a comment.EditedMay 26 2017, 3:19 PM
scripts/category.py
if self.oldcat.isEmptyCategory() and self.delete_oldcat and \
    ((self.newcat and self.move_oldcat) or not self.newcat):
  self._delete(can_move_page, can_move_talk)

I examined the same part of code when I was writing that comment, but I can not figure out a solution. Setting some time to wait is nonsense and could not solve the problem. My only thought was about some possibility to call touch on the category redirect right before this if-condition.

Xqt triaged this task as Medium priority.May 26 2017, 4:01 PM

@matej_suchanek Your solution looks better than my thoughts, please don't forget to track exceptions like doc pages (for templates categorized through doc), skipped pages, redirects and so.

Could you please add an option to localize CFD template name too? I know {{delete}} works on most of Wikipedias, but localized one would be better maybe.

@matej_suchanek Your solution looks better than my thoughts, please don't forget to track exceptions like doc pages (for templates categorized through doc), skipped pages, redirects and so.

In the newest patch, I improved tracking of doc subpages.

Could you please add an option to localize CFD template name too? I know {{delete}} works on most of Wikipedias, but localized one would be better maybe.

This option will add nothing since the result will be same. Localizing {{delete}} in pywikibot/page.py is considerable, though.

Dvorapa updated the task description. (Show Details)May 28 2017, 8:47 AM
Dvorapa updated the task description. (Show Details)
Dvorapa added a comment.EditedMay 28 2017, 9:49 AM

My only thought was about some possibility to call touch on the category redirect right before this if-condition.

Btw there is a -rebuild parameter, maybe just calling this functionality before checking if category is empty could work as well

My only thought was about some possibility to call touch on the category redirect right before this if-condition.

Btw there is a -rebuild parameter, maybe just calling this functionality before checking if category is empty could work as well

Actually irrelevant.

Could you please add an option to localize CFD template name too? I know {{delete}} works on most of Wikipedias, but localized one would be better maybe.

This option will add nothing since the result will be same. Localizing {{delete}} in pywikibot/page.py is considerable, though.

PS: Maybe this is already tracked in T72653

xSavitar moved this task from Backlog to Needs Review on the Pywikibot board.Nov 5 2018, 11:29 AM
Restricted Application added a subscriber: Urbanecm. · View Herald TranscriptNov 5 2018, 11:29 AM

Change 355791 abandoned by Matěj Suchánek:
When moving a category, track number of members locally as well

Reason:
Unclear how to proceed

https://gerrit.wikimedia.org/r/355791

matej_suchanek removed matej_suchanek as the assignee of this task.Jun 15 2019, 2:59 PM
matej_suchanek edited projects, added TestMe; removed Patch-For-Review.