(another unexpected error filled in T93471)
Isn't this resolved already? (T187009)
KMLexport's maintainer is long-term inactive, therefore this tool looks for a maintainer who could use abandoned tool policy and fix e.g. T187625
Fri, Mar 16
Thu, Mar 15
Wed, Mar 14
Fri, Mar 9
Please try to update Pywikibot instance in PAWS and confirm whether this bug was solved by update or not.
On Pywikibot side this was solved in T183466, so this is just an issue of Pywikibot instance in PAWS not updated to the post-fix version.
Thu, Mar 8
Mon, Mar 5
There were support issues with Python 3.5+ in PyMySql in history, but I think they are solved and I agree with your proposal
Sun, Mar 4
OK, I run python3 pwb.py weblinkchecker -start:! -ns:0 on Ubuntu 17.10 (with Python 3.6). I created 3 core dumps. The script run with session replaced by requests in http.py#L384 and logged off. The script run more than 3x faster than on Arch, with being logged in and with session. Core dumps were obtained by sudo gdb python3 -p 1781 -> (gdb) generate-core-file.
offtopic, but I agree with @jayvdb: We should port only those someone requested or [are/want to be used] on some project. I ported ndashredir (and rewritten it a little bit), because I need it for some tasks on Czech Wikipedia. And I would suggest the same to others.
@zhuyifei1999 Thank you very much for the analysis! Yeah, I should mentioned at the beginning, that I install requests and requirements.txt (where the memento client is contained) every time after cloning Pywikibot. I forgot to install memento client when installing Python 3.5 on Arch, so I was confused by Python version (which has nothing to do with the issue finally), I apologize. Untill now we have found the original issue is made by 3 subissues, 1 solved, 1 reported upstream, 1 is the memory leak.
Sat, Mar 3
↑ (missing comment:) I reproduced the bug on Ubuntu 17.10. This was captured in freeze ([[.lu]] article currently processed). In addition I also tried (gdb) generate-core-file (found a suggestion somewhere in python forums) which gave me some type of binary file as an output (do you want to see it?)
I installed Python 3.5 and repeat worked as expected, so this is definitelly a python 3.6 issue!
Fri, Mar 2
Or I can use pdb if needed (but it needs to log from the beginning too)
On Arch Linux there is no support for py-bt and also there are missing things like symbols completely. Arch's python package does not contain any debug things. I use gdb the first time in my life so I even don't know how to use it correctly. I can also run it from beginning till end (interactive or automatic) if it helps
That would confirm the original deadlocking problem fixed :)
I will, but I think this can be some error in the data file.
For example and testing purposes: https://cs.wikipedia.org/wiki/%C5%A0ablona:Monokarboxylov%C3%A9_kyseliny contains many similar buggy previews
For me purge, blank edit or page reload didn't help. And this looks broken for so many chemical articles across Czech Wikipedia
Finally fixed by T182321
On many projects there already is some list of common fixes (see e.g. frwiki, cswiki or hewiki in https://www.wikidata.org/wiki/Q10957404 for WPCleaner or https://www.wikidata.org/wiki/Q6585066 for AWB). Pywikibot could read their syntax as well. I imagine something like (pseudo code):
Thu, Mar 1
@Dalba No, it was on default value of 5 seconds, but the -repeat should not make requests to MW API at all, or am I wrong?
Here: http://mysharegadget.com/743636238 (stored for 33 days). Unzip into deadlinks folder in pwb root folder.
PS: Data were collected using a workaround to this issue: timeout --signal=SIGINT 20m python pwb.py weblinkchecker -lang:cs -ns:0 -start:"Last article from last run" (it took me me than 3 weeks to collect the data by this 20 minutes blocks between start and KeyboardInterrupt thrown by SIGINT). The same workaround should work for -repeat too, but this issue with 49 forever openned threads should be solved somehow. I can send the collected data if needed to identify links, which create neverending threads
$ python pwb.py version Pywikibot: [https] r-pywikibot-core.git (6866469, g9129, 2018/03/01, 09:07:44, ok) Release version: 3.0-dev requests version: 2.18.4 cacerts: /etc/ssl/certs/ca-certificates.crt certificate test: ok Python: 3.6.4 (default, Jan 5 2018, 02:35:40) [GCC 7.2.1 20171224] PYWIKIBOT2_DIR: Not set PYWIKIBOT2_DIR_PWB: PYWIKIBOT2_NO_USER_CONFIG: Not set Config base dir: /home/pavel/pywikibot Usernames for family "wikipedia": cs: DvorapaBot (no sysop configured)
@Xqt Currently I'm AFK, I'll post version result and -repeat result (quite short as you'll see) at evening..
This is even worse on -repeat as weblinkchecker don't need to load page content from wiki (which slows it down a bit but in general it helps to postpone link checking a little). So within a few seconds of running, the pool is exhausted really quickly (49 remaining threads) and the checking sometimes freezes at this point.