Page MenuHomePhabricator

Rework how source_regex timeout is done in Cirrus
Closed, ResolvedPublic

Description

The way search timeouts are managed by elastic has completely changed in elastic 6.x and this affects how we limit the source_regex query from consuming all the cluster resource.

The timeout check in elastic is now active with a specific thread that inspect timeouts and will respond before the search thread finish.
The timeout thread will then mark the lucene collector as cancelled and the search thread will then throw an exception when it reaches a new segment.

The timeout strategy in the source_regex has not changed and do a timeout check on every docs but we need to make sure that the timeout is reached by the source_regex plugin only after the timeout done by elastic is reached.
Failing to do this will into search failures instead of partial results.

The task can be seen as: find a way to tune timeouts properly so that the source_regex query does not run too long after the partial response is sent.

Event Timeline

dcausse triaged this task as Medium priority.Jul 3 2018, 5:25 PM
dcausse created this task.
Restricted Application added a project: Discovery. · View Herald TranscriptJul 3 2018, 5:25 PM
dcausse claimed this task.Feb 6 2019, 8:19 AM

Change 488954 had a related patch set uploaded (by DCausse; owner: DCausse):
[search/extra@master] Remove source_regex timeout

https://gerrit.wikimedia.org/r/488954

Change 488990 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@es6] Drop support for SourceRegex timeout param

https://gerrit.wikimedia.org/r/488990

Change 488954 merged by jenkins-bot:
[search/extra@master] Remove source_regex timeout

https://gerrit.wikimedia.org/r/488954

Change 488990 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@es6] Drop support for SourceRegex timeout param

https://gerrit.wikimedia.org/r/488990

debt closed this task as Resolved.Feb 15 2019, 7:04 PM