Page MenuHomePhabricator

`search_index --rebuild` exits with non-zero status
Closed, ResolvedPublicBUG REPORT

Description

Seen in the demo server following an update to rWTHU723c3bc0c6bd: dev: Upgrade Django from version 2.2 to 3.2. The demo server uses an entrypoint which includes rebuilding the search index as part of a shell pipeline. When the newly deployed build started returning a error status for the call to poetry run python3 manage.py search_index --rebuild -f the demo server would fail the entrypoint entirely and the container would restart.

The non-zero exit can be recreated in a dev environment with something like:

$ make web-shell
$ poetry run python3 manage.py search_index --rebuild -f
... (lots of logs) ...
$ echo $?
1

Event Timeline

bd808 triaged this task as High priority.
bd808 moved this task from Backlog to In Progress on the Toolhub board.

In my dev environment, this error is logged to explain the exit status:

elasticsearch.helpers.errors.BulkIndexError: ('9 document(s) failed to index.', [
  {'index':
    {'_index': 'toolhub_lists',
     '_type': 'doc',
     '_id': '4',
     'status': 400,
     'error': {
       'type': 'mapper_parsing_exception',
       'reason': 'failed to parse',
       'caused_by': {
         'type': 'illegal_state_exception',
         'reason': "Can't handle serializing a dynamic type with content token [VALUE_NULL] and field name [x_merged_type]"
       }
     }
     'data': {
       'tools': [
         {
           'name': 'toolforge-anticompositetools-hyphenator',
           'title': 'Hyphenator',
           'description': 'Adds hyphens to ISBNs and converts 10-digit ISBNs to 13-digit ISBNs.',
           'url': 'https://anticompositetools.toolforge.org/hyphenator',
           'keywords': ['books', 'citation'],
           'author': [{'name': 'AntiCompositeNumber', 'wiki_username': None, 'developer_username': None, 'email': None, 'url': None}],
           'icon': None,
           'tool_type': None,
           'annotations': {'icon': None, 'tool_type': None}
         },
        {
           'name': 'toolforge-stashbot',
...(snip)...

I think this is more likely caused by the changes from rWTHUa46fbb44b86d: api: Add annotations to SummaryToolSerializer than the Django version update.

bd808 renamed this task from `search_index --rebuild` exits with non-zero status following Django 3.2 upgrade to `search_index --rebuild` exits with non-zero status.Apr 11 2022, 5:37 PM

Change 779136 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[wikimedia/toolhub@main] bug(search): add missing type for x_merged_type in ListDocument

https://gerrit.wikimedia.org/r/779136

Change 779136 merged by jenkins-bot:

[wikimedia/toolhub@main] bug(search): add missing type for x_merged_type in ListDocument

https://gerrit.wikimedia.org/r/779136

Change 786342 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/deployment-charts@master] toolhub: Bump container version to 2022-04-21-215651-production

https://gerrit.wikimedia.org/r/786342

Change 786342 merged by jenkins-bot:

[operations/deployment-charts@master] toolhub: Bump container version to 2022-04-21-215651-production

https://gerrit.wikimedia.org/r/786342