Page MenuHomePhabricator

pywikibot-touch.py needs to operate in bot mode
Closed, ResolvedPublic

Description

When trying to undertake a large scale touch of Wikisource Page: namespace pages, I found that it was unable to be undertaken in bot mode. Discovered this when the bot found a page that was resaved with a difference due to a change made in the code elsewhere in either MediaWiki or ProofreadPage, and the edit was saved without the 'b' flag, which for large scale touch operations is problematic.

Event Timeline

Billinghurst raised the priority of this task from to Needs Triage.
Billinghurst updated the task description. (Show Details)
Billinghurst added a project: Pywikibot.
Billinghurst added a subscriber: Billinghurst.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptSep 23 2015, 7:02 AM
jayvdb added subscribers: Tpt, jayvdb.

Can see this is needed where a touch really does alter the page, and it is a big job.

@Tpt has confirmed that these page changes are unavoidable in the PHP?

Should be easy to add as an optional not-default flag. Most other scripts support this.

Mpaa set Security to None.

Change 240570 had a related patch set uploaded (by Mpaa):
touch.py: allow botflag tagging from cli

https://gerrit.wikimedia.org/r/240570

(replied by email, though that has been rejected after a couple of days)

Re the fix that touch applies ... it seems to relate to a version of
MediaWiki (PrP?) and it's ability to dispose of trailing white space. It
is usually a batch of pages in the footer section of a Page:s of a work, so
to me it seems to be a per user component, rather than at a point of time.
I have seen groups of pages with nothing then groups with some changes then
groups with nothing, so that means user-related if at same time in space
cleaning, OR batches done over time.

Change 240570 merged by jenkins-bot:
touch.py: allow botflag tagging from cli

https://gerrit.wikimedia.org/r/240570

Mpaa closed this task as Resolved.Oct 11 2015, 8:25 AM
Mpaa added a subscriber: Mpaa.

@Billinghurst, feedback welcome.

Billinghurst reopened this task as Open.Oct 12 2015, 11:47 AM

No luck, still recording as standard edits rather than as bot edits
https://da.wikisource.org/w/index.php?title=Speciel:Seneste_%C3%A6ndringer&limit=20&hidebots=0

12 October 2015
  11:38 	Side:Breve fra Helvede, udgivne af M. Rowel.djvu/66‎ (diff | hist) . . (-1)‎ . . Wikisource-bot (Talk | contribs | block) (Pywikibot touch edit) [rollback]
  11:23 	Side:Almindelig Handelsvidenskab.djvu/95‎ (diff | hist) . . (-1)‎ . . Wikisource-bot (Talk | contribs | block) (Pywikibot touch edit) [rollback]
  11:23 	Side:Almindelig Handelsvidenskab.djvu/93‎ (diff | hist) . . (-1)‎ . . Wikisource-bot (Talk | contribs | block) (Pywikibot touch edit) [rollback]
  11:23 	Side:Almindelig Handelsvidenskab.djvu/91‎ (diff | hist) . . (-1)‎ . . Wikisource-bot (Talk | contribs | block) (Pywikibot touch edit) [rollback]

https://meta.wikimedia.org/w/index.php?title=Special%3ACentralAuth&target=wikisource-bot

da.wikisource.org	12:31, 13 August 2015	created on login(?)	—	46	bot
XZise added a subscriber: XZise.Oct 12 2015, 11:49 AM

How did you use it and have you verified that you use @Mpaa's change?

XZise assigned this task to Mpaa.Oct 12 2015, 11:51 AM

and the version of touch.py looks to be the most recent

-rwxrwxr-x  1 tools.pywikibot tools.pywikibot   3135 Oct 12 01:02 touch.py
Billinghurst triaged this task as Medium priority.Oct 12 2015, 11:56 AM

Okay I can verify that the size is 3135 bytes with @Mpaa's change and 2936 Bytes without. But did you also use the -botflag option?

Billinghurst closed this task as Resolved.EditedOct 12 2015, 12:02 PM

running with -botflag and that works

12 October 2015
b   12:01 	Side:Danmark-Norges indre historie under enevælden 1.djvu/264‎ (diff | hist) . . (-1)‎ . . Wikisource-bot (Talk | contribs | block) (Pywikibot touch edit) [rollback]
b   12:01 	Side:Danmark-Norges indre historie under enevælden 1.djvu/263‎ (diff | hist) . . (-1)‎ . . Wikisource-bot (Talk | contribs | block) (Pywikibot touch edit) [rollback]
b   12:01 	Side:Danmark-Norges indre historie under enevælden 1.djvu/262‎ (diff | hist) . . (-1)‎ . . Wikisource-bot (Talk | contribs | block) (Pywikibot touch edit) [rollback]
JAnD reopened this task as Open.EditedAug 14 2018, 8:37 PM
JAnD added a subscriber: JAnD.
JAnD added a subscriber: Danny_B.Aug 14 2018, 8:38 PM
JAnD added a comment.Aug 17 2018, 9:38 PM

did you try -botflag per the help page?

No, this should be default beahvior. Usual syntax is touch -pagegenerator, no need for additional parameter.

Dvorapa closed this task as Declined.EditedAug 18 2018, 7:58 AM
Dvorapa added a subscriber: Dvorapa.

did you try -botflag per the help page?

No, this should be default beahvior. Usual syntax is touch -pagegenerator, no need for additional parameter.

The default behavior is no edit is ever made. An edit is an unexpected behavior and therefore it should never have bot flag or minor flag as it suggests bot might be broken. The -botflag parameter can workaround this if you are 100 % sure the bot is not broken and the edits made are error free (the same applies to the default summary). Closing as intentional.

JAnD reopened this task as Open.Aug 19 2018, 8:57 PM

I am afraid, this is not because bot is broken, but because database is broken.
https://cs.wikisource.org/w/index.php?title=Index:Žák,_František_-_Ohlasy_žalmů.pdf&diff=prev&oldid=168733
you can see, there was nothing changed in page, but its size decreased about 124 b

matej_suchanek added a subscriber: matej_suchanek.
> python -c "import this" | grep pass

Errors should never pass silently.

rPWBC456fd62f5a83: touch.py: allow botflag tagging from cli is wrong.

zhuyifei1999 closed this task as Declined.Aug 21 2018, 8:53 AM
zhuyifei1999 added a subscriber: zhuyifei1999.

I am afraid, this is not because bot is broken, but because database is broken.

File a bug against Wikimedia-Rdbms. Pywikibot itself has nothing to do with this.

Mpaa added a comment.Aug 25 2018, 12:33 PM

I am afraid, this is not because bot is broken, but because database is broken.

File a bug against Wikimedia-Rdbms. Pywikibot itself has nothing to do with this.

It is a known/justified behavior for wikisource Index/Page ns (see referenced bug).

Xqt changed the task status from Declined to Resolved.Feb 3 2019, 10:00 AM