Page MenuHomePhabricator

Fix Flaky TWL Oauth test in CICD
Closed, ResolvedPublic5 Estimated Story PointsBUG REPORT

Description

One of our OAuth tests is flaky in CI, though it passes locally
https://github.com/WikipediaLibrary/TWLight/blob/82cfe0f1ba45ff2e262d3080202d8318f965829c/TWLight/users/tests.py#L1659-L1684

def test_create_user_and_editor(self, mock_urlopen):
    """
    OAuthBackend._create_user_and_editor() should:
    * create a user
        * with a suitable username and email
        * without a password
    * And a matching editor
    """
    oauth_backend = OAuthBackend()
    oauth_data = FAKE_IDENTITY_DATA
    identity = FAKE_IDENTITY


    mock_response = Mock()
    mock_response.read.side_effect = [json.dumps(oauth_data)] * 7
    mock_urlopen.return_value = mock_response


    user, editor = oauth_backend._create_user_and_editor(identity)


    self.assertEqual(user.email, "alice@example.com")
    self.assertEqual(user.username, "567823")
    self.assertFalse(user.has_usable_password())


    self.assertEqual(editor.user, user)
    self.assertEqual(editor.wp_sub, 567823)
    # We won't test the fields set by update_from_wikipedia, as they are
    # tested elsewhere.

For now, we're commenting out this test
https://github.com/WikipediaLibrary/TWLight/commit/3d2b8ba9a67c30e1d2b26f71798087ce00bcd171
but we should fix it

Acceptance criteria

  • The OAuth test above is enabled and runs
  • The test does not fail when run locally
  • The test does not fail intermittently in CI

Related Objects

StatusSubtypeAssignedTask
ResolvedTheresNoTime
ResolvedBUG REPORTTheresNoTime

Event Timeline

Thank you for tagging this task with good first task for Wikimedia newcomers!

Newcomers often may not be aware of things that may seem obvious to seasoned contributors, so please take a moment to reflect on how this task might look to somebody who has never contributed to Wikimedia projects.

A good first task is a self-contained, non-controversial task with a clear approach. It should be well-described with pointers to help a completely new contributor, for example it should clearly pointed to the codebase URL and provide clear steps to help a contributor get setup for success. We've included some guidelines at https://phabricator.wikimedia.org/tag/good_first_task/ !

Thank you for helping us drive new contributions to our projects <3

Dillon subscribed.

@TheresNoTime, would you be interested in taking this on as part of your onboarding path?

@TheresNoTime, would you be interested in taking this on as part of your onboarding path?

Sounds good!

I think there are more flaky tests that affect our CI/CD builds. Let me try to find them, and I'll edit the task. Thanks for taking this on!

Got ValueError: Model instances passed to related filters must be saved.

Run docker compose exec --no-TTY twlight bin/virtualenv_test.sh
[Thu Jul  2 02:48:52 UTC 2026]
Django version: 5.2.15
black --target-version py311 --check TWLight
All done! ✨ 🍰 ✨
241 files would be left unchanged.
/app/tests/shunit/twlight_i18n_lint_test.sh
testBadPyNewlines
testGoodPy
testBadPyComments

Ran 3 tests.

OK
Checking for localization issues
No localization issues found
Found 324 test(s).
Using existing test database for alias 'default'...
Using existing clone for alias 'default'...
Using existing clone for alias 'default'...
Using existing clone for alias 'default'...
Using existing clone for alias 'default'...
System check identified no issues (0 silenced).
........................................................................................................................................................................................................................................................E...........................................................................
======================================================================
ERROR: test_create_user_and_editor (TWLight.users.tests.OAuthTestCase.test_create_user_and_editor)
OAuthBackend._create_user_and_editor() should:
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/unittest/case.py", line 57, in testPartExecutor
    yield
  File "/usr/local/lib/python3.11/unittest/case.py", line 623, in run
    self._callTestMethod(testMethod)
    ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/unittest/case.py", line 579, in _callTestMethod
    if method() is not None:
    ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/unittest/mock.py", line 1378, in patched
    return func(*newargs, **newkeywargs)
    ^^^^^^^^^^^^^^^^^
  File "/app/TWLight/users/tests.py", line 1761, in test_create_user_and_editor
    user, editor = oauth_backend._create_user_and_editor(identity)
    ^^^^^^^^^^^^^^^^^
  File "/app/TWLight/users/oauth.py", line 171, in _create_user_and_editor
    editor = self._create_editor(user, identity)
    ^^^^^^^^^^^^^^^^^
  File "/app/TWLight/users/oauth.py", line 164, in _create_editor
    editor.update_from_wikipedia(identity, lang)  # This call also saves the editor
    ^^^^^^^^^^^^^^^^^
  File "/app/TWLight/users/models.py", line 885, in update_from_wikipedia
    self.wp_enough_edits = editor_enough_edits(self.wp_editcount)
    ^^^^^^^^^^^^^^^^^
  File "/app/TWLight/users/models.py", line 383, in wp_editcount
    return (EditorLog.objects.filter(editor=self).latest("timestamp")).editcount
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1495, in filter
    return self._filter_or_exclude(False, args, kwargs)
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1513, in _filter_or_exclude
    clone._filter_or_exclude_inplace(negate, args, kwargs)
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1523, in _filter_or_exclude_inplace
    self._query.add_q(Q(*args, **kwargs))
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/sql/query.py", line 1648, in add_q
    clause, _ = self._add_q(q_object, can_reuse)
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/sql/query.py", line 1680, in _add_q
    child_clause, needed_inner = self.build_filter(
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/sql/query.py", line 1590, in build_filter
    condition = self.build_lookup(lookups, col, value)
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/sql/query.py", line 1417, in build_lookup
    lookup = lookup_class(lhs, rhs)
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/lookups.py", line 38, in __init__
    self.rhs = self.get_prep_lookup()
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/fields/related_lookups.py", line 103, in get_prep_lookup
    self.rhs = get_normalized_value(self.rhs, self.lhs)[0]
    ^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.11/site-packages/django/db/models/fields/related_lookups.py", line 20, in get_normalized_value
    raise ValueError("Model instances passed to related filters must be saved.")
    ^^^^^^^^^^^^^^^^^
ValueError: Model instances passed to related filters must be saved.

----------------------------------------------------------------------
Ran 324 tests in 13.852s

FAILED (errors=1)

Slow tests (>1.5s):
jsn.sherman moved this task from QA to Done on the Moderator-Tools-Team (Kanban) board.

ci seems to be not angry; will reopen if it looks like we didn't actually solve. Thanks @TheresNoTime