Page MenuHomePhabricator

replace.py async does not seem to be async at all
Open, MediumPublic

Description

With T135992 and since T73646 partially fixed, the run of replace.py seems not to be really async at all. The page save makes the other API requests throttle after (wait to) it and also the queue of saved pages messages finally makes the whole process slow down for a while when outputting.

Event Timeline

Dvorapa created this task.May 31 2018, 6:01 AM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptMay 31 2018, 6:01 AM
Xqt added a subscriber: Xqt.May 31 2018, 6:05 AM

Could you explain a bit or make a sample to reproduce?

@Xqt Try to use replace.py and confirm/reject/modify all diffs without having to wait on the "save" thread. It is impossible, so the replace.py logic is not (at least not fully) async.

Dvorapa triaged this task as Medium priority.May 31 2018, 7:21 PM

Well, according to my experience, saving using solve_disambiguation works more smoothly than using replace.py. But maybe I'm not right?

Dvorapa added a comment.EditedMay 31 2018, 8:14 PM

What I would expect from Pwb:

  • manual mode (y/n) => async
  • always mode (a/-always) => sync

How Pwb behaves mostly:

  • manual mode => sync (omg why?)
  • always mode => async

How Pwb behaves sometimes (replace.py):

  • manual mode => async working not so async as it should
  • always mode => sync
Vvjjkkii renamed this task from replace.py async does not seem to be async at all to sxbaaaaaaa.Jul 1 2018, 1:06 AM
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot renamed this task from sxbaaaaaaa to replace.py async does not seem to be async at all.Jul 2 2018, 1:50 PM
CommunityTechBot lowered the priority of this task from High to Medium.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added a subscriber: Aklapper.
Xqt added a comment.Jun 23 2020, 7:30 AM

@Xqt Try to use replace.py and confirm/reject/modify all diffs without having to wait on the "save" thread. It is impossible, so the replace.py logic is not (at least not fully) async.

Cannot really reproduce it. But I re-implemented cached output from compat in https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/282818/ and discarded all these callback changes in replace.py (which does not work properly especially for other messages than saving hints) to use the default BaseBot's behaviour. In addition I introduce a saving delay for -simulate option with https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/607072/ and awaiting put threads beeing done during BaseBot.exit() in https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/607170/. With these concepts I do not find a sticking async behaviour. A good simulating command could be:

py -3 pwb.py replace -simulate a b -start:! -summary:"Bot: Tests"

Confirm several changes with [y]es and stop with [q]uit then.