Page MenuHomePhabricator
Create Task
Maniphest T340881

proxy api crashing in codfw1dev
Closed, ResolvedPublic
Actions

Description

On proxy-02.proxy-codfw1dev.codfw1dev.wikimedia.cloud:

Jun 30 17:44:46 proxy-02 uwsgi-invisible-unicorn[411180]: mounting /usr/local/bin/invisible-unicorn.py on /dynamicproxy-api
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]: Traceback (most recent call last):
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]:   File "/usr/local/bin/invisible-unicorn.py", line 80, in <module>
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]:     db = SQLAlchemy(app)
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]:   File "/usr/lib/python3/dist-packages/flask_sqlalchemy/__init__.py", line 738, in __in>
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]:     session_options.setdefault('scopefunc', connection_stack.__ident_func__)
Jun 30 17:44:48 proxy-02 uwsgi-invisible-unicorn[411180]: AttributeError: '_FakeStack' object has no attribute '__ident_func__'

This crash seems to be interfering with cleaning up dns records for deleted VMs. So we have two problems :/

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
dynamicproxy: api: install newer version of python3-flask-sqlalchemyoperations/puppetproduction+17 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

Andrew created this task.Jun 30 2023, 5:47 PM
Restricted Application edited projects, added cloud-services-team; removed cloud-services-team (FY2022/2023-Q4). · View Herald TranscriptJun 30 2023, 5:47 PM
Andrew updated the task description. (Show Details)Jun 30 2023, 5:48 PM
Andrew added a comment.Jun 30 2023, 10:19 PM

This seems to be the same issue as https://stackoverflow.com/questions/72256625/why-am-i-getting-this-error-attributeerror-localstack-object-has-no-attribu -- something about incompatible package versions. So far I haven't found a combination that works.

ayounsi unsubscribed.Jul 1 2023, 1:49 PM
• aborrero added a project: User-aborrero.Jul 3 2023, 11:44 AM
gerritbot added a comment.Jul 3 2023, 12:00 PM

Change 935039 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] dynamicproxy: api: install newer version of python3-flask-sqlalchemy

https://gerrit.wikimedia.org/r/935039

gerritbot added a project: Patch-For-Review.Jul 3 2023, 12:00 PM
gerritbot added a comment.Jul 3 2023, 3:31 PM

Change 935039 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] dynamicproxy: api: install newer version of python3-flask-sqlalchemy

https://gerrit.wikimedia.org/r/935039

Maintenance_bot removed a project: Patch-For-Review.Jul 3 2023, 4:10 PM
taavi closed this task as Resolved.Jul 3 2023, 4:11 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits