Page MenuHomePhabricator

Import of wmfdata fails while trying to access analytics-store
Closed, ResolvedPublic

Description

I'm getting this error in a fresh Python 3 notebook on SWAP (JupyterLab, notebook1003), apparently because wmfdata tries to connect to analytics-store which is being decommissioned (T216491):

In [1]: 
# import stuff for accessing databases:
import wmfdata

---------------------------------------------------------------------------
MySQLInterfaceError                       Traceback (most recent call last)
~/venv/lib/python3.5/site-packages/mysql/connector/connection_cext.py in _open_connection(self)
    175         try:
--> 176             self._cmysql.connect(**cnx_kwargs)
    177         except MySQLInterfaceError as exc:

MySQLInterfaceError: Unknown MySQL server host 'analytics-store.eqiad.wmnet' (0)

During handling of the above exception, another exception occurred:

DatabaseError                             Traceback (most recent call last)
/srv/home/tbayer/venv/src/wmfdata/wmfdata/mariadb.py in run(cmds, fmt, host)
     55             charset = 'binary',
---> 56             autocommit = True
     57         )

~/venv/lib/python3.5/site-packages/mysql/connector/__init__.py in connect(*args, **kwargs)
    147         new_config = read_option_files(**kwargs)
--> 148         return connect(**new_config)
    149 

~/venv/lib/python3.5/site-packages/mysql/connector/__init__.py in connect(*args, **kwargs)
    181     if HAVE_CEXT and not use_pure:
--> 182         return CMySQLConnection(*args, **kwargs)
    183     return MySQLConnection(*args, **kwargs)

~/venv/lib/python3.5/site-packages/mysql/connector/connection_cext.py in __init__(self, **kwargs)
     77         if kwargs:
---> 78             self.connect(**kwargs)
     79 

~/venv/lib/python3.5/site-packages/mysql/connector/abstracts.py in connect(self, **kwargs)
    735         self.disconnect()
--> 736         self._open_connection()
    737         self._post_connection()

~/venv/lib/python3.5/site-packages/mysql/connector/connection_cext.py in _open_connection(self)
    178             raise errors.get_mysql_exception(msg=exc.msg, errno=exc.errno,
--> 179                                              sqlstate=exc.sqlstate)
    180         self._do_handshake()

DatabaseError: 2005 (HY000): Unknown MySQL server host 'analytics-store.eqiad.wmnet' (0)

During handling of the above exception, another exception occurred:

UnboundLocalError                         Traceback (most recent call last)
<ipython-input-1-7837621aa18b> in <module>()
      1 # import stuff for accessing databases:
----> 2 import wmfdata

/srv/home/tbayer/venv/src/wmfdata/wmfdata/__init__.py in <module>()
      1 # Import all submodules so all are accessible after `import wmfdata`
----> 2 from wmfdata import charting, hive, mariadb, utils
      3 
      4 # Direct reusers to package
      5 utils.print_err("You can find the source for `wmfdata` at https://github.com/neilpquinn/wmfdata")

/srv/home/tbayer/venv/src/wmfdata/wmfdata/hive.py in <module>()
      5 from impala.dbapi import connect as impala_conn
      6 from impala.util import as_pandas as impala_as_pd
----> 7 from wmfdata.utils import print_err, mediawiki_dt
      8 
      9 def run(cmds, fmt = "pandas"):

/srv/home/tbayer/venv/src/wmfdata/wmfdata/utils.py in <module>()
      2 from math import log10, floor
      3 
----> 4 from wmfdata import mariadb
      5 import pandas as pd
      6 

/srv/home/tbayer/venv/src/wmfdata/wmfdata/mariadb.py in <module>()
    116     return [row[0] for row in wikis]
    117 
--> 118 def multirun(cmds, wikis = list_wikis()):
    119     if type(cmds) == str:
    120         cmds = [cmds]

/srv/home/tbayer/venv/src/wmfdata/wmfdata/mariadb.py in list_wikis(groups)
    111         order by site_global_key asc
    112         """.format(groups = groups_list), 
--> 113         fmt = "raw"
    114     )
    115 

/srv/home/tbayer/venv/src/wmfdata/wmfdata/mariadb.py in run(cmds, fmt, host)
     84 
     85     finally:
---> 86         conn.close()
     87 
     88 def list_wikis(groups=["all"]):

UnboundLocalError: local variable 'conn' referenced before assignment

Event Timeline

Resolved after updating (!pip install --upgrade git+https://github.com/neilpquinn/wmfdata.git). I guess this can be closed in favor of T216634?

kzimmerman claimed this task.
kzimmerman added subscribers: nshahquinn, kzimmerman.

Per @Tbayer 's comment above, I'm closing this task. CC @neilpquinn