Page MenuHomePhabricator

proxy api crashing in codfw1dev
Closed, ResolvedPublic

Description

On proxy-02.proxy-codfw1dev.codfw1dev.wikimedia.cloud:

Jun 30 17:44:46 proxy-02 uwsgi-invisible-unicorn[411180]: mounting /usr/local/bin/invisible-unicorn.py on /dynamicproxy-api
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]: Traceback (most recent call last):
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]:   File "/usr/local/bin/invisible-unicorn.py", line 80, in <module>
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]:     db = SQLAlchemy(app)
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]:   File "/usr/lib/python3/dist-packages/flask_sqlalchemy/__init__.py", line 738, in __in>
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]:     session_options.setdefault('scopefunc', connection_stack.__ident_func__)
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]: AttributeError: '_FakeStack' object has no attribute '__ident_func__'

This crash seems to be interfering with cleaning up dns records for deleted VMs. So we have two problems :/

Event Timeline

This seems to be the same issue as https://stackoverflow.com/questions/72256625/why-am-i-getting-this-error-attributeerror-localstack-object-has-no-attribu -- something about incompatible package versions. So far I haven't found a combination that works.

Change 935039 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] dynamicproxy: api: install newer version of python3-flask-sqlalchemy

https://gerrit.wikimedia.org/r/935039

Change 935039 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] dynamicproxy: api: install newer version of python3-flask-sqlalchemy

https://gerrit.wikimedia.org/r/935039