Page MenuHomePhabricator

Fatal error when adding new maintainer to a tool
Closed, ResolvedPublic

Description

Tried it several times now today.

500 Internal Server Error
A generic error message, given when an unexpected condition was encountered and no more specific message is suitable.
"List of HTTP status codes." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 26 August 2016. Web. 27 August 2016.

An unexpected technical problem has occured while handing the requested URI /tools/id/robin/maintainers/. This is probably temporary and should be fixed soon. Please try again later.
You may be able to get further information in the #wikimedia-cloud channel on the Freenode IRC network.
If you report this error, please include the details below.
Request ID

169831cc0e8942f98d5a4c0911e6220a

https://toolsadmin.wikimedia.org/tools/id/robin/maintainers/

Details

Related Gerrit Patches:
labs/striker/deploy : masterBump striker and wheels sub modules
labs/striker/wheels : masterFork django-ldapdb to workaround T228332
labs/striker : masterFork django-ldapdb to workaround T228332

Event Timeline

bd808 triaged this task as High priority.
Jul 17 20:22:32 labweb1001 uwsgi-striker[26620]: @cee: {"logger_name": "django.request", "lineno": 228, "stack_info": null, "process": 14831, "message": "Internal Server Error: /tools/id/robin/maintainers/", "@timestamp": "2019-07-17T20:22:32.992Z", "@version": "1", "thread_name": "b'uWSGIWorker48Core0'", "host": "labweb1001", "request": "<WSGIRequest: POST '/tools/id/robin/maintainers/'>", "level": "ERROR", "type": "striker", "tags": [], "stack_trace": "Traceback (most recent call last):\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/core/handlers/exception.py\", line 34, in inner\n    response = get_response(request)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 115, in _get_response\n    response = self.process_exception_by_middleware(e, request)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/core/handlers/base.py\", line 113, in _get_response\n    response = wrapped_callback(request, *callback_args, **callback_
kwargs)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/contrib/auth/decorators.py\", line 21, in _wrapped_view\n    return view_func(request, *args, **kwargs)\n  File \"./striker/tools/views/decorators.py\", line
48, in decorated\n    return f(*args, **kwargs)\n  File \"./striker/tools/views/tool.py\", line 181, in maintainers\n    tool.save()\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/db/models/base.py\", line 741, in save\n    force_update=force_update, update_fields=update_fields)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/db/models/base.py\", line 779, in save_base\n    force_update, using, update_fields,\n  File \"/srv
/deployment/striker/venv/lib/python3.5/site-packages/ldapdb/models/base.py\", line 92, in _save_table\n    old = cls.objects.using(using).get(dn=self._saved_dn)\n  File \"/srv/deployment/striker/venv/lib/python3.5/site-packages/django/db/models/query.py\", line 408, in get\n    self.model._meta.object_name\nstriker.tools.models.Tool.DoesNotExist: Tool matching query does not exist.\n", "status_code": 500, "path": "/srv/deployment/striker/venv/lib/python3.5/site-packages/django/utils/log.py"}

I can recreate this with other tools as well, so it seems to be a regression likely related to the recent rLSTRf3c62f141b02: Bump to Django 2.2.3 + other library upgrades.

bd808 renamed this task from Fatal error when adding new maintainer to tools.robin to Fatal error when adding new maintainer to a tool.Jul 17 2019, 8:30 PM

I have tracked the root cause of this to a change in the django-ldapdb library and filed a bug report with them upstream. I have a patch to their library that works locally against the Striker codebase, but I'm having some troubles getting the change to pass tests in django-ldapdb's CI system. I'm not sure at the moment if the CI problem is with the test I wrote or with the change I'm proposing itself.

I think I described the bug pretty well upstream, but the TL;DR for folks watching here is that our Tool model applies a filter to all ORM generated queries which breaks the special case handling for dn based LDAP lookups in the django-ldapdb library. This in turn breaks save() for an existing Tool model, but not initial creation of new Tool models.

Tests on the django-ldapdb repo are passing now!

Change 524325 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[labs/striker@master] Fork django-ldapdb to workaround T228332

https://gerrit.wikimedia.org/r/524325

Change 524325 merged by jenkins-bot:
[labs/striker@master] Fork django-ldapdb to workaround T228332

https://gerrit.wikimedia.org/r/524325

Change 524346 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[labs/striker/wheels@master] Fork django-ldapdb to workaround T228332

https://gerrit.wikimedia.org/r/524346

Change 524346 merged by jenkins-bot:
[labs/striker/wheels@master] Fork django-ldapdb to workaround T228332

https://gerrit.wikimedia.org/r/524346

Change 524350 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[labs/striker/deploy@master] Bump striker and wheels sub modules

https://gerrit.wikimedia.org/r/524350

Change 524350 merged by jenkins-bot:
[labs/striker/deploy@master] Bump striker and wheels sub modules

https://gerrit.wikimedia.org/r/524350

Mentioned in SAL (#wikimedia-operations) [2019-07-18T21:34:46Z] <bd808@deploy1001> Started deploy [striker/deploy@91594df]: Fixes for deprecation warnings and editing Tool models (T228222, T228332)

Mentioned in SAL (#wikimedia-operations) [2019-07-18T21:35:59Z] <bd808@deploy1001> Finished deploy [striker/deploy@91594df]: Fixes for deprecation warnings and editing Tool models (T228222, T228332) (duration: 01m 13s)

bd808 closed this task as Resolved.Jul 18 2019, 9:38 PM
bd808 added a subscriber: MarcoAurelio.

Verified fix by adding @MarcoAurelio as a co-maintainer of https://toolsadmin.wikimedia.org/tools/id/robin per T227521#5341349