Page MenuHomePhabricator
Create Task
Maniphest T257794

Server error on /users/my_library/
Closed, ResolvedPublic
Actions

Description

Over the weekend we saw a number of server errors on the My Library page. It looks like the source of the error is the Authorization.get_latest_app() function, which appears to be finding more than one latest application for a user's authorization.

Looking at one user who received the error, Jaydayal, they have two authorizations: A Library Bundle one, and one to the American Psychological Association. They have a total of 3 applications to APA; one fresh application and two renewals. The latest renewal was not approved.

Traceback: 

File "/venv/lib/python3.8/site-packages/django/db/backends/utils.py" in execute
  64.                 return self.cursor.execute(sql, params)

File "/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py" in execute
  101.             return self.cursor.execute(query, args)

File "/venv/lib/python3.8/site-packages/MySQLdb/cursors.py" in execute
  209.         res = self._query(query)

File "/venv/lib/python3.8/site-packages/MySQLdb/cursors.py" in _query
  315.         db.query(q)

File "/venv/lib/python3.8/site-packages/MySQLdb/connections.py" in query
  239.         _mysql.connection.query(self, query)


      The above exception ((1242, 'Subquery returns more than 1 row')) was the direct cause of the following exception:



File "/venv/lib/python3.8/site-packages/django/core/handlers/exception.py" in inner
  41.             response = get_response(request)

File "/venv/lib/python3.8/site-packages/django/core/handlers/base.py" in _legacy_get_response
  249.             response = self._get_response(request)

File "/venv/lib/python3.8/site-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/venv/lib/python3.8/site-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/venv/lib/python3.8/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
  23.                 return view_func(request, *args, **kwargs)

File "/venv/lib/python3.8/site-packages/django/views/generic/base.py" in view
  68.             return self.dispatch(request, *args, **kwargs)

File "/app/TWLight/view_mixins.py" in dispatch
  227.         return super(SelfOnly, self).dispatch(request, *args, **kwargs)

File "/venv/lib/python3.8/site-packages/django/views/generic/base.py" in dispatch
  88.         return handler(request, *args, **kwargs)

File "/venv/lib/python3.8/site-packages/django/views/generic/list.py" in get
  175.         context = self.get_context_data()

File "/app/TWLight/users/views.py" in get_context_data
  725.                     latest_app = each_authorization.get_latest_app()

File "/app/TWLight/users/models.py" in get_latest_app
  641.                 return Application.objects.filter(

File "/venv/lib/python3.8/site-packages/django/db/models/query.py" in latest
  561.         return self._earliest_or_latest(field_name=field_name, direction="-")

File "/venv/lib/python3.8/site-packages/django/db/models/query.py" in _earliest_or_latest
  555.         return obj.get()

File "/venv/lib/python3.8/site-packages/django/db/models/query.py" in get
  374.         num = len(clone)

File "/venv/lib/python3.8/site-packages/django/db/models/query.py" in __len__
  232.         self._fetch_all()

File "/venv/lib/python3.8/site-packages/django/db/models/query.py" in _fetch_all
  1121.             self._result_cache = list(self._iterable_class(self))

File "/venv/lib/python3.8/site-packages/django/db/models/query.py" in __iter__
  53.         results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)

File "/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py" in execute_sql
  899.             raise original_exception

File "/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py" in execute_sql
  889.             cursor.execute(sql, params)

File "/venv/lib/python3.8/site-packages/django/db/backends/utils.py" in execute
  64.                 return self.cursor.execute(sql, params)

File "/venv/lib/python3.8/site-packages/django/db/utils.py" in __exit__
  94.                 six.reraise(dj_exc_type, dj_exc_value, traceback)

File "/venv/lib/python3.8/site-packages/django/utils/six.py" in reraise
  685.             raise value.with_traceback(tb)

File "/venv/lib/python3.8/site-packages/django/db/backends/utils.py" in execute
  64.                 return self.cursor.execute(sql, params)

File "/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py" in execute
  101.             return self.cursor.execute(query, args)

File "/venv/lib/python3.8/site-packages/MySQLdb/cursors.py" in execute
  209.         res = self._query(query)

File "/venv/lib/python3.8/site-packages/MySQLdb/cursors.py" in _query
  315.         db.query(q)

File "/venv/lib/python3.8/site-packages/MySQLdb/connections.py" in query
  239.         _mysql.connection.query(self, query)

Exception Type: OperationalError at /users/my_library/
Exception Value: (1242, 'Subquery returns more than 1 row')

This started happening around the same time as T257667, and therefore may be related.

Related Objects

Event Timeline

Samwalton9-WMF created this task.Jul 13 2020, 9:00 AM
Restricted Application added subscribers: Sadads, Aklapper. · View Herald TranscriptJul 13 2020, 9:00 AM
Samwalton9-WMF triaged this task as High priority.Jul 13 2020, 9:23 AM
Samwalton9-WMF updated the task description. (Show Details)Jul 13 2020, 9:33 AM

My assumption here is that latest() (https://docs.djangoproject.com/en/1.11/ref/models/querysets/#latest) is supposed to be used on a date field, but here it's used on id, so it returns both the original application and its renewal. It should instead be using the date_closed field, I think.

jsn.sherman claimed this task.Jul 13 2020, 1:56 PM
jsn.sherman added a comment.Jul 13 2020, 3:32 PM

Yeah, Authorization.get_latest_app() looks a little strange to me.

I extended one of our existing tests to cover this scenario (app: approved, renewal 1: approved, renewal 2: denied), and it's not failing in the way I'd expect based on this report. My initial thought is that we have some tests that aren't catching all of the app revision state as desired.

I'm going to pull down a backup and suss out exactly what's going on and get this scenario to fail in tests.

jsn.sherman added a comment.Jul 13 2020, 4:55 PM

hmm, when I try to manually create this problem

  • apply, approve
  • return access, renew, approve
  • return access again, renew again ...

I'm not able to create that second renewal from my_library. It fails with the following error:

This object cannot be renewed. (This probably means that you have already requested that it be renewed.)

The apply button is disabled on the partner page.

As you suggest, there are clearly issues with latest_app, but the reported errors can't be reproduced (through the user interface) under the current code because in the current code the second renewal couldn't have been created which is a bug in itself

The not_approved renewals with this issue seem to predate the breaking commit, so I'll

  • add test for multiple renewals (which should currently fail) and fix that issue
  • verify that the fix also resolves our errors by manually changing my local copy of these apps and auths from Jaydayal to myself
Samwalton9-WMF moved this task from Ready to In Progress on the The-Wikipedia-Library (Kanban) board.Jul 14 2020, 11:38 AM
jsn.sherman added a comment.Jul 14 2020, 5:32 PM

Hotfix pushed, awaiting deployment.

jsn.sherman added a comment.EditedJul 14 2020, 6:07 PM

Hotfix deployed. If we're in communication with impacted users, let's have them verify the fix.

Samwalton9-WMF added a comment.Jul 15 2020, 11:21 AM

Have asked a user to check if they still receive the error.

jsn.sherman added a comment.Jul 15 2020, 12:56 PM
This comment was removed by jsn.sherman.
Samwalton9-WMF moved this task from In Progress to Done on the The-Wikipedia-Library (Kanban) board.Jul 15 2020, 1:04 PM

Confirmed that the library page is working again for that user.

Samwalton9-WMF closed this task as Resolved.Jul 17 2020, 9:05 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL