Page MenuHomePhabricator
Create Task
Maniphest T228332

Fatal error when adding new maintainer to a tool
Closed, ResolvedPublic
Actions

Description

In T227521#5341349, @MF-Warburg wrote:

Tried it several times now today.

500 Internal Server Error

A generic error message, given when an unexpected condition was encountered and no more specific message is suitable.
"List of HTTP status codes." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 26 August 2016. Web. 27 August 2016.

An unexpected technical problem has occured while handing the requested URI /tools/id/robin/maintainers/. This is probably temporary and should be fixed soon. Please try again later.

You may be able to get further information in the #wikimedia-cloud channel on the Freenode IRC network.

If you report this error, please include the details below.

Request ID

169831cc0e8942f98d5a4c0911e6220a

https://toolsadmin.wikimedia.org/tools/id/robin/maintainers/

Details

SubjectRepoBranchLines +/-
Bump striker and wheels sub moduleslabs/striker/deploymaster+4 -4
Fork django-ldapdb to workaround T228332labs/striker/wheelsmaster+0 -0
Fork django-ldapdb to workaround T228332labs/strikermaster+4 -1
Customize query in gerrit

Related Objects

Event Timeline

bd808 created this task.Jul 17 2019, 8:23 PM
bd808 claimed this task.Jul 17 2019, 8:30 PM
bd808 triaged this task as High priority.
Jul 17 20:22:32 labweb1001 uwsgi-striker[26620]: @cee: {"logger_name": "django.request", "lineno": 228, "stack_info": null, "process": 14831, "message": "Internal Server Error: /tools/id/robin/maintainers/", "@timestamp": "2019-07-17T20:22:32.992Z", "@version": "1", "thread_name": "b'uWSGIWorker48Core0'", "host": "labweb1001", "request": "<WSGIRequest: POST '/tools/id/robin/maintainers/'>", "level": "ERROR", "type": "striker", "tags": [], "stack_trace": "Traceback (most recent call last):\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/core/handlers/exception.py\", line 34, in inner\n    response = get_response(request)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 115, in _get_response\n    response = self.process_exception_by_middleware(e, request)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 113, in _get_response\n    response = wrapped_callback(request, *callback_args, **callback_
kwargs)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/contrib/auth/decorators.py\", line 21, in _wrapped_view\n    return view_func(request, *args, **kwargs)\n  File \"./striker/tools/views/decorators.py\", line
48, in decorated\n    return f(*args, **kwargs)\n  File \"./striker/tools/views/tool.py\", line 181, in maintainers\n    tool.save()\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/db/models/base.py\", line 741, in save\n    force_update=force_update, update_fields=update_fields)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/db/models/base.py\", line 779, in save_base\n    force_update, using, update_fields,\n  File \"/srv
/deployment/striker/venv/lib/python3.5/site-packages/ldapdb/models/base.py\", line 92, in _save_table\n    old = cls.objects.using(using).get(dn=self._saved_dn)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/db/models/query.py\", line 408, in get\n    self.model._meta.object_name\nstriker.tools.models.Tool.DoesNotExist: Tool matching query does not exist.\n", "status_code": 500, "path": "/srv/deployment/striker/venv/lib/python3.5/site-packages/django/utils/log.py"}

I can recreate this with other tools as well, so it seems to be a regression likely related to the recent rLSTRf3c62f141b02: Bump to Django 2.2.3 + other library upgrades.

bd808 renamed this task from Fatal error when adding new maintainer to tools.robin to Fatal error when adding new maintainer to a tool.Jul 17 2019, 8:30 PM
Legoktm subscribed.Jul 18 2019, 5:17 AM
bd808 added a project: cloud-services-team (Kanban).Jul 18 2019, 3:05 PM

I have tracked the root cause of this to a change in the django-ldapdb library and filed a bug report with them upstream. I have a patch to their library that works locally against the Striker codebase, but I'm having some troubles getting the change to pass tests in django-ldapdb's CI system. I'm not sure at the moment if the CI problem is with the test I wrote or with the change I'm proposing itself.

I think I described the bug pretty well upstream, but the TL;DR for folks watching here is that our Tool model applies a filter to all ORM generated queries which breaks the special case handling for dn based LDAP lookups in the django-ldapdb library. This in turn breaks save() for an existing Tool model, but not initial creation of new Tool models.

bd808 moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.Jul 18 2019, 5:30 PM
bd808 added a comment.Jul 18 2019, 8:17 PM

Tests on the django-ldapdb repo are passing now!

gerritbot added a comment.Jul 18 2019, 8:26 PM

Change 524325 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[labs/striker@master] Fork django-ldapdb to workaround T228332

https://gerrit.wikimedia.org/r/524325

gerritbot added a project: Patch-For-Review.Jul 18 2019, 8:26 PM
gerritbot added a comment.Jul 18 2019, 8:28 PM

Change 524325 merged by jenkins-bot:
[labs/striker@master] Fork django-ldapdb to workaround T228332

https://gerrit.wikimedia.org/r/524325

gerritbot added a comment.Jul 18 2019, 9:19 PM

Change 524346 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[labs/striker/wheels@master] Fork django-ldapdb to workaround T228332

https://gerrit.wikimedia.org/r/524346

gerritbot added a comment.Jul 18 2019, 9:23 PM

Change 524346 merged by jenkins-bot:
[labs/striker/wheels@master] Fork django-ldapdb to workaround T228332

https://gerrit.wikimedia.org/r/524346

gerritbot added a comment.Jul 18 2019, 9:24 PM

Change 524350 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[labs/striker/deploy@master] Bump striker and wheels sub modules

https://gerrit.wikimedia.org/r/524350

gerritbot added a comment.Jul 18 2019, 9:28 PM

Change 524350 merged by jenkins-bot:
[labs/striker/deploy@master] Bump striker and wheels sub modules

https://gerrit.wikimedia.org/r/524350

bd808 mentioned this in rLSTWb93e9eb37587: Fork django-ldapdb to workaround T228332.Jul 18 2019, 9:30 PM
Stashbot mentioned this in T228222: Django deprecation warning for "{% load staticfiles %}".Jul 18 2019, 9:34 PM

Mentioned in SAL (#wikimedia-operations) [2019-07-18T21:34:46Z] <bd808@deploy1001> Started deploy [striker/deploy@91594df]: Fixes for deprecation warnings and editing Tool models (T228222, T228332)

Stashbot added a comment.Jul 18 2019, 9:36 PM

Mentioned in SAL (#wikimedia-operations) [2019-07-18T21:35:59Z] <bd808@deploy1001> Finished deploy [striker/deploy@91594df]: Fixes for deprecation warnings and editing Tool models (T228222, T228332) (duration: 01m 13s)

bd808 closed this task as Resolved.Jul 18 2019, 9:38 PM
bd808 added a subscriber: MarcoAurelio.

Verified fix by adding @MarcoAurelio as a co-maintainer of https://toolsadmin.wikimedia.org/tools/id/robin per T227521#5341349

bd808 mentioned this in T227521: [robin] Webservice down (and possibly other issues).Jul 18 2019, 9:38 PM
MarcoAurelio added a comment.Jul 18 2019, 9:49 PM

Thank you!

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits