Page MenuHomePhabricator

Completion suggester: fine-tune thread pools
Closed, ResolvedPublic

Description

The completion suggester uses a new threadpool equals to the number of proc (32 in codfw). This value needs to be changed because we could run 32+49 queries.
A value of 10 seems to work well on codfw but maybe we should decrease the search thread pool from 49 to 39.
It's even harder in eqiad since the number of proc is not the same on all machines.

This is the purpose of this ticket: find the best strategy for thread pools.

Event Timeline

dcausse raised the priority of this task from to Needs Triage.
dcausse updated the task description. (Show Details)
dcausse added a subscriber: dcausse.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald Transcript
Deskana triaged this task as Medium priority.Feb 9 2016, 11:05 PM
Deskana added a subscriber: Deskana.
Deskana raised the priority of this task from Medium to High.Feb 9 2016, 11:09 PM

Bumping up priority since this relates to the rollout of the completion suggester (T121616), which is a priority this quarter.

Is this really blocking T121616? If so, it needs to be resolved soon, because the timeline for full production release has been set.

This has to happen as part of the march 16th rollout, it's hard to predict what these need to be before we start putting real load on the cluster.

This has to happen as part of the march 16th rollout, it's hard to predict what these need to be before we start putting real load on the cluster.

Understood. Thanks!

Everything ended up working fine with the defaults, we didn't change anything

I don't think it's related but I see short but huge spikes in tp99... I had a quick look but found nothing obvious, I created T130719 to track this problem.

Deskana claimed this task.

Calling this one resolved then!