pwb deadlocks when quitting
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Mineo
	Apr 16 2015, 10:14 AM

Description

I'm running https://github.com/mineo/mb2wikidatabot with Python 2.6.5 (I know, I know, but the server is not under my control and https://www.mediawiki.org/wiki/Manual:Pywikibot/Installation#Initial_setup claims this should work). Every now and then I have to kill the bot manually because it seems to run into a deadlock when quitting. The last messages in the log file are

2015-04-14 12:20:34 __init__.py, 702 in stopme: VERBOSE Dropped throttle(s).
2015-04-14 12:20:34 http.py, 122 in _flush: VERBOSE Waiting for 1 network thread(s) to finish. Press ctrl-c to abort
2015-04-14 12:20:34 http.py, 127 in _flush: VERBOSE All threads finished.

The process has not shut down since then and I can't press ctrl-c because the processes are being run from cron.

gdb output:

(gdb) info threads
2 Thread 0x7fe117339700 (LWP 7870) sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:86

1 Thread 0x7fe11b895700 (LWP 7869) sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:86

This looks like both threads are waiting for something, possible blocking each other.

The revision of pwb currently running is 297089be218a5528150ba1fffaa42773d291c683.

Related Objects

Mentioned Here: rPWBC297089be218a: Merge "Lua support only exists for Linux"

Event Timeline

Mineo created this task.Apr 16 2015, 10:14 AM

Mineo raised the priority of this task from to Needs Triage.

Mineo updated the task description. (Show Details)

Mineo added a project: Pywikibot.

Mineo subscribed.

Restricted Application added subscribers: Aklapper, Unknown Object (MLST). · View Herald TranscriptApr 16 2015, 10:14 AM

Maybe the process is waiting for a password input?

Could you try the following: https://wiki.python.org/moin/DebuggingWithGdb

Specifically the 'To see Python code positions for all threads, use' part. If we know where pywikibot is hanging, it's easier to understand what's going on.

Another option is doing this in pure python:

import sys,threading,time,traceback

class RampLogger(threading.Thread):
def run(self):
    print >> sys.stderr, "\n STACKTRACE - START \n"
    code = []
    for threadId, stack in sys._current_frames().items():
        code.append("\n# ThreadID: %s" % threadId)
        for filename, lineno, name, line in traceback.extract_stack(stack):
            code.append('File: "%s", line %d, in %s' % (filename, lineno, name))
            if line:
                code.append(" %s" % (line.strip()))

    for line in code:
        print >> sys.stderr, line
    print >> sys.stderr, "\n*** STACKTRACE - END ***\n"
    time.sleep(10)

RampLogger().start()

If you add that to the beginning of the script, you will get a full stacktrace (for all threads) every 10s. (the indentation of the code might be off in some spaces, but I think it's correct)

Xqt triaged this task as Low priority.Jun 28 2017, 8:18 AM

@Mineo: is this task still valid?

Probably solved with https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/525179/
Otherwise reopen this task.

pwb deadlocks when quittingClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

pwb deadlocks when quitting
Closed, ResolvedPublic
Actions