Page MenuHomePhabricator

Diffusion repository creation fails via toolsadmin
Closed, ResolvedPublic

Description

An unexpected technical problem has occured while handing the requested URI /tools/id/coverage/repos/create. This is probably temporary and should be fixed soon. Please try again later.

If you report this error, please include the details below.

Request ID
    ef4ed5f882bd4e00933f3e5c3ff2e993

Event Timeline

2017-12-05T19:22:45Z [ef4ed5f882bd4e00933f3e5c3ff2e993] django.request ERROR: Internal Server Error: /tools/id/coverage/repos/create
Traceback (most recent call last):
  File "/srv/deployment/striker/venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/srv/deployment/striker/venv/lib/python3.4/site-packages/django/contrib/auth/decorators.py", line 22, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "./striker/tools/views/decorators.py", line 48, in decorated
    return f(*args, **kwargs)
  File "./striker/tools/views/repo.py", line 57, in create
    if form.is_valid():
  File "/srv/deployment/striker/venv/lib/python3.4/site-packages/django/forms/forms.py", line 184, in is_valid
    return self.is_bound and not self.errors
  File "/srv/deployment/striker/venv/lib/python3.4/site-packages/django/forms/forms.py", line 176, in errors
    self.full_clean()
  File "/srv/deployment/striker/venv/lib/python3.4/site-packages/django/forms/forms.py", line 392, in full_clean
    self._clean_fields()
  File "/srv/deployment/striker/venv/lib/python3.4/site-packages/django/forms/forms.py", line 410, in _clean_fields
    value = getattr(self, 'clean_%s' % name)()
  File "./striker/tools/forms.py", line 61, in clean_repo_name
    phab.get_repository(name)
  File "./striker/phabricator.py", line 165, in get_repository
    'order': 'name',
  File "./striker/phabricator.py", line 75, in post
    resp.get('result', None))
striker.phabricator.APIError: Constraint "name" is not a valid constraint for this query. (ERR-CONDUIT-CORE)

@20after4 did something change on the Phabricator side with this API?

@Legoktm: I think a bunch of conduit APIs switched to using "query" instead of "name"...

Change 395924 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[labs/striker@master] Update for diffusion.repository.search API change

https://gerrit.wikimedia.org/r/395924

Change 395924 merged by jenkins-bot:
[labs/striker@master] Update for diffusion.repository.search API change

https://gerrit.wikimedia.org/r/395924

Change 401810 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[labs/striker/deploy@master] Bump striker submodule and add ddl changes

https://gerrit.wikimedia.org/r/401810

Change 401810 merged by jenkins-bot:
[labs/striker/deploy@master] Bump striker submodule and add ddl changes

https://gerrit.wikimedia.org/r/401810

Stashbot subscribed.

Mentioned in SAL (#wikimedia-operations) [2018-01-03T22:31:18Z] <bd808@tin> Started deploy [striker/deploy@69f1b15]: Enhance membership request workflow and fix Diffusion repo creation (T168027, T182142)

Mentioned in SAL (#wikimedia-operations) [2018-01-03T22:31:48Z] <bd808@tin> Finished deploy [striker/deploy@69f1b15]: Enhance membership request workflow and fix Diffusion repo creation (T168027, T182142) (duration: 00m 31s)

Repo creation works again, but there is a bug of some sort that is breaking the display of newly created repos. Reloading the page fixes it, so it should be something I can track down and fix relatively easy.

Page after creation:

Screen Shot 2018-01-03 at 15.46.39.png (394×523 px, 33 KB)

Reload fixes it:

Screen Shot 2018-01-03 at 15.46.55.png (656×654 px, 87 KB)

It seems that the creation of new repositories failed again. A reload does not solve the issue:

screenshot.png (569×818 px, 32 KB)

I have done a bunch of local testing on this issue and determined that it is functionally a race. When Phabricator is asked to create a new repo via the conduit diffusion.repository.edit api endpoint it is immediately available for lookup using the diffusion.repository.search api endpoint when the lookup is done using the new repo's PHID attribute. The same diffusion.repository.search api however will fail to return a result for a lookup done using the new repo's name. This feels like it is directly related to the original bug report here. There used to be a 'name' constraint that could be used for the search. This was replaced in the conduit api at some point with the 'query' constraint. The 'query' constraint is documented as "Find objects matching a fulltext search query. See "Search User Guide" in the documentation for details." In my local testing it appears to take "a while" (1-5 seconds) before whatever index backs the 'query' lookup to actually be populated and able to match the repo.

Luckily, subsequent Phabricator updates have added a 'shortNames' constraint that we can switch to. This seems to be indexed immediately.

Change 483673 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[labs/striker@master] Switch diffusion.repository.search search field

https://gerrit.wikimedia.org/r/483673

Change 483673 merged by jenkins-bot:
[labs/striker@master] Switch diffusion.repository.search search field

https://gerrit.wikimedia.org/r/483673

Change 498258 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[labs/striker/deploy@master] Upgrade to Django 2.1 and other small improvments

https://gerrit.wikimedia.org/r/498258

Change 498258 merged by jenkins-bot:
[labs/striker/deploy@master] Upgrade to Django 2.1 and other small improvments

https://gerrit.wikimedia.org/r/498258

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:34:41Z] <bd808@deploy1001> Started deploy [striker/deploy@c4726e3]: Django upgrade and various bug fixes (T192487, T182142, T176325, T217932)