Page MenuHomePhabricator

Replace MySQL-python with pymysql in fundraising python scripts
Closed, ResolvedPublic2 Story Points

Description

MySQL-python is not supported under python 3

Event Timeline

Ejegg triaged this task as Normal priority.Jan 14 2019, 8:24 PM
Ejegg created this task.

Change 520352 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[wikimedia/fundraising/tools@master] Switch to pymysql

https://gerrit.wikimedia.org/r/520352

Ejegg claimed this task.Jul 4 2019, 3:40 AM
Ejegg moved this task from Backlog to Review on the Fundraising Sprint Men In Slack board.
mepps added a subscriber: mepps.Jul 5 2019, 4:50 PM

@Ejegg What is the best way to test this change?

Ejegg added a comment.Jul 5 2019, 6:29 PM

The silverpop unit tests exercise the database connection. The only trick there is that you need a db named 'test' open to a user 'test' with no password.

Ejegg added a comment.Jul 5 2019, 7:19 PM

You can also just plain run the silverpop update and export. Copy the silverpop_export.yaml.example to ~/.fundraising/silverpop_export.yaml and fix the values, then run silverpop_export/update.py, then silverpop_export/export.py

Change 520352 merged by jenkins-bot:
[wikimedia/fundraising/tools@master] Switch to pymysql

https://gerrit.wikimedia.org/r/520352

Ejegg added a comment.Thu, Jul 25, 3:28 PM

Verified that pymysql is already installed on the cluster where we'll need it.

I included this in the deploy just now - seemed ok from looking & from comment above

OK - reversing this out of the deploy -

iles/silverpop_emails_build_export_files-20190730-220453.log
2019-07-30 22:04:54,101 INFO ===========
2019-07-30 22:04:54,101 INFO /usr/bin/python /srv/tools/silverpop_export/update.py (22444)
2019-07-30 22:04:54,101 INFO -----------
2019-07-30 22:04:54,395 ERROR Traceback (most recent call last):
2019-07-30 22:04:54,395 ERROR File "/srv/tools/silverpop_export/update.py", line 9, in <module>
2019-07-30 22:04:54,395 ERROR from database.db import Connection as DbConnection
2019-07-30 22:04:54,395 ERROR File "/srv/tools/database/db.py", line 4, in <module>
2019-07-30 22:04:54,396 ERROR import pymysql
2019-07-30 22:04:54,396 ERROR ImportError: No module named pymysql
2019-07-30 22:04:54,408 INFO ----------- end command output

Change 526548 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/tools@master] Revert "Switch to pymysql"

https://gerrit.wikimedia.org/r/526548

Change 526548 merged by jenkins-bot:
[wikimedia/fundraising/tools@master] Revert "Switch to pymysql"

https://gerrit.wikimedia.org/r/526548

Ejegg added a subscriber: Jgreen.Wed, Jul 31, 3:21 PM

Oh shoot, I forgot '--installed' when I ran 'apt list' on civi1001. The package is not actually installed up there. @Jgreen, can we install the python-pymysql deb package on hosts configured to get the 'tools' project deployed?

this is rolled back atm

Change 530232 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[wikimedia/fundraising/tools@master] Re-apply: Switch to pymysql

https://gerrit.wikimedia.org/r/530232

Change 530232 merged by jenkins-bot:
[wikimedia/fundraising/tools@master] Re-apply: Switch to pymysql

https://gerrit.wikimedia.org/r/530232

Ejegg closed this task as Resolved.Thu, Aug 15, 8:50 PM