Wed, Jan 10
Wed, Jan 3
This is from the "wikidata-terminator" tool (which you could probably have found out quicker than I).
@Niharika I have turned off notifications as per suggesestion
Dec 15 2017
Yes, ListeriaBot (which edits Wikipedia) does not cause those (so far), so only Wikidata.
As this is my first patch accept in Differential, how do I merge this here? No "merge" button that I can see on this page. Probably not "commandeer", whatever that is?
Both bots use the Addwiki lib, not sure if that uses OAuth:
Dec 14 2017
Dec 12 2017
Several of my tools appear to be affected as well, example:
Dec 11 2017
Dec 8 2017
Nov 28 2017
@jcrespo OK, but that only means the rows were missing since the last edit, presumably, on 5 August 2017.
I did ?action=purge on the item before I filed the bug.
MariaDB [wikidatawiki_p]> select @@hostname; +------------+ | @@hostname | +------------+ | labsdb1011 | +------------+ 1 row in set (0.01 sec)
"sql" is the recommended shell command to connect to the replicas on Toolforge, correct?
MariaDB [wikidatawiki_p]> select @@hostname; +------------+ | @@hostname | +------------+ | labsdb1003 | +------------+ 1 row in set (0.00 sec)
Please note that, while https://tools.wmflabs.org/admin/tools indeed lists all Toolforge "registered tools", many of them are containers for two or more "actual", distinct tools.
Nov 21 2017
I will look into this once there is a realistic test site. QS2 is designed for multiple sites, see https://phabricator.wikimedia.org/source/tool-quickstatements/browse/master/public_html/sites.json
Nov 16 2017
And now it's back. Something weird is going on. Is there a load record for that tools' webservice?
@chasemp The "missing animal audio" thing should be fixed now.
And it's stuck again...
Nov 15 2017
It came back after the bastion reboot. Maybe slow bastion => slow filesystem => slow tool/webservice?
I have now set up phpunit version 4 in a single tool. Support for that version ended February 3, 2017. However, it is the latest version that can tolerate the ancient PHP 5.59 on Toolforge.
Nov 14 2017
Thanks @Reedy . Is an OS upgrade planned any time soon?
Unless one has dozens of tools to install that one-at-a-time...
Update: I have added some more "re-run if failed" and "check connection/reconnect" code, and the GLAM data now seem to process more fluently. Unknown if that's because of the new code, or if upstream was fixed.
I will list some source below, just let me emphasize this is not limited to a specific query or type of query, programming language, tool, or webservice/shell. I even see this in PetScan, which is C++ and runs on its own VM, though it uses the DB replicas as well. Also, it does not seem to be reproducible in general, but tends to "cluster" time-wise, as in, there are some time periods where there is a lot of this, some where it's a rare occurrence. I have kept no log of those periods, just thought of it as the cost of doing business, especially after my initial bug report was declined.
Nov 13 2017
And yes, I know the disconnect happened before the *** line. In all the ~2sec it took me to type the command.
I am sure my client can handle ~800 rows, which is what I get when I use DISTINCT, as shown...
Nov 10 2017
Trying to decompile all these into statements and/or checks:
Oct 23 2017
Oct 22 2017
OK, these are my old workaround for the removal of a wikidata DB table. (I did day the removal of the table will cause problems, but does anyone ever listen to me??!?)
Oct 16 2017
The problem appears to have resolved itself. I have switched everything back to normal.
Oct 14 2017
Oct 13 2017
Sep 28 2017
Sep 27 2017
Sep 8 2017
I'm not sure what causes the error, but the authentication itself seems to be successful. Can you confirm that?
Aug 30 2017
Pull request merged.
Aug 25 2017
I just started this manually, I do this once a month for the GLAM community. It should subside quickly now.
Aug 23 2017
Everything works now after renaming/removing of .my.cnf
Aug 22 2017
Daily list to work through:
It was written to do single edits, so it does log in before each one. I can try and change it if it's a problem?
Aug 21 2017
@jcrespo But I am working as the respective "tool-user"!
Addendum: With some tools, I can connect to "local", but not create a new tool database:
@jcrespo But shouldn't the sql script automatically use the replica file? There's a "proper" user name in there, and 'sql dewiki' works fine.
Also for "flickr2commons" tool.
Jul 27 2017
Jul 26 2017
Jul 25 2017
@ Paucabot No. The alternative would be to use all images on a page, but that would remove too many pages because of navbar icons and irrelevant images, which cannot be filtered out automatically.
Jul 24 2017
Tried action=purge, didn't help
Jul 21 2017
I wrote the tool. It uses the "page image" information from the respective Wikipedia. For your example, there is no such entry. Ergo, this needs to be fixed in MediaWiki core, or the respective extension.
Jul 14 2017
Last page had a "\" in the title, maintenance script didn't remove it. Killed on DB prompt now.
Jul 13 2017
I have removed the webproxy and the instance from the wdq project.
Jul 11 2017
Flickr2Commons issue tracker:
Jul 6 2017
All done now, thanks :-)
One left, but es.wp won't let me change it:
Jul 5 2017
@Multichill AFAICT it's only wdq-mm-01 left in the wdq-mm project. Switched off since April 6.
Jul 4 2017
And I can't change that usage because I get caught in some spam filter. Any es admin around?
Jun 29 2017
Spam deleted, database locked. The "actual" tool at https://tools.wmflabs.org/comprende/ still works in read-only mode.
Jun 28 2017
Running cleanup now, will lock afterwards.
Jun 21 2017
Jun 20 2017
I have rewritten the query, should work better now I hope
Jun 19 2017
ssh email@example.com Permission denied (publickey). Killed by signal 1.
Jun 16 2017
Yes, I have been restarting at least a dozen tools because of this over the last week or two (?).