Page MenuHomePhabricator

pywikibot._putthread is not alive anymore after KeyboardInterrupt
Closed, ResolvedPublicBUG REPORT

Description

List of steps to reproduce:
Run a script with changes pages asynchronously. In this case the command line was pwb -simulate:10 newitem -start:a -lang:da -touch. To ensure there is anything left to do the newitem.py script was modified in this way:

def _callback(self, page, exc) -> None:
    if True or exc is None and self.opt.touch:
        ....

def treat_page_and_item(self, page, item) -> None
    if False and item and item.exists():
        ....

Run this script for a while, e.g. one minute and enforce a KeyboardInterrupt by pressing Strg-C. A blue message occurs:
Waiting for 5 pages to be put. Estimated time remaining: 0:00:25

Again press Strg-C. Now this message and question will be printed:

There are 5 pages remaining in the queue. Estimated time remaining: 0:00:25
Really exit? ([y]es, [N]o):

Now type n for no.

What happens?:

The script terminates immediately:

11 skip operations
6 read operations
6 write operations
Execution time: 24 seconds
Read operation time: 2.8 seconds
Skip operation time: 2.2 seconds
Write operation time: 4.0 seconds
Script terminated successfully.

Digging deeper in the code _putthread.is_alive() beomes False imediately after Keyboardinterrupt:

if _putthread is not threading.current_thread():
    while (_putthread.is_alive()
           and (page_put_queue.qsize() > 0
                or page_put_queue_busy.qsize() > 0)):
        try:
            _putthread.join(1)
        except KeyboardInterrupt:
            print('???', _putthread.is_alive())   # This gives False which is not an expected behaviour
            if input_yn('There are {} pages remaining in the queue. '
                        'Estimated time remaining: {}\nReally exit?'
                        .format(*remaining()),
                        default=False, automatic_quit=False):

What should have happened instead?:

_putthread.is_alive() should be True and the async calls should be processed further until the queue in done .

Software version:
Python 3.9+, see https://github.com/python/cpython/issues/89437

Details

Event Timeline

Xqt triaged this task as High priority.Jun 21 2022, 4:19 PM

I also tested pwb add_text -text:"#### Foo" -start:A -limit:50 -simulate:10
This is a Python 3.9+ issue due to https://bugs.python.org/issue45274
The Thread is closed with KeyboardInterrupt within Thread._wait_for_tstate_lock()

Xqt updated the task description. (Show Details)
Xqt renamed this task from pywikibot._putthread is not alive anymore after KeyboardIntertupt to pywikibot._putthread is not alive anymore after KeyboardInterrupt.Jun 21 2022, 7:49 PM
Xqt added subscribers: valhallasw, Mpaa, matej_suchanek, JJMC89.

Change 807423 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [WIP] Renew _putthread after KeyboardInterrupt

https://gerrit.wikimedia.org/r/807423

Change 807423 merged by jenkins-bot:

[pywikibot/core@master] [bugfix] Handle page_put_queue after KeyboardInterrupt

https://gerrit.wikimedia.org/r/807423