Page MenuHomePhabricator

"Lost connection to MySQL server during query" using pymysql on toolforge connecting to db replica
Closed, ResolvedPublic

Description

I'm rewriting a tool and the new Python version is getting "(2013, 'Lost connection to MySQL server during query')" while the old tool has no issues. The query is very large, probably half a million rows

This is the code in question for the new Python version: https://pastebin.com/i5H0pAd6

And the corresponding Perl code: https://github.com/openzim/wikimedia_wp1_bot/blob/master/backend/toolserver_api.pl#L168

The error happens on line 16 of the paste of the Python code: cursor.execute. It doesn't even get to the fetchmany call. Here is the stack trace of the actual call (the line numbers don't line up with the paste because of imports): https://pastebin.com/tyL39FH1

The database connection happens here: https://github.com/openzim/wikimedia_wp1_bot/blob/master/lucky/lucky/wiki_db.py

According to this mysql help article (https://dev.mysql.com/doc/refman/5.7/en/error-lost-connection.html) it's most likely the net_read_timeout variable, but I know that we can't adjust that globally.

Also the error is repeatable, on every run of the new tool, so it's not a network fluke or maintenance.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 18 2019, 5:24 PM

I'm going to try replacing DictCursor with SSDictCursor and see if that helps.

Audiodude closed this task as Resolved.Jun 18 2019, 11:30 PM
Audiodude claimed this task.

That seems to have resolved the issue!