Increase cpu and disk quota for the 'search' group
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	EBernhardson
	Oct 22 2015, 4:55 PM

Description

Within search we are starting to build out a relevancy lab (T115615). We have tested the elasticsearch half of this by setting up 2xlarge instances in labs (16GB memory, 140G disk, 8 cpus). This works for our initial small testing, but we can't import many different languages. With these two machines we are able to import enwiki_content, dewiki_content, itwiki_content and half of ruwiki. While importing ruwiki it ran out of disk space.

Ideally we would like to add one or two new VM's, and double the available disk space. Is this possible? There is no need to maintain the existing VM's, those can be destroyed and recreated if that makes the transition easier.

The relevant group on wikitech is 'search'

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• Deskana	T114673 [EPIC] Create a Relevance Lab which helps us measure the effect on search result relevance of proposed changes to search
		Resolved		yuvipanda	T116292 Increase cpu and disk quota for the 'search' group

Event Timeline

EBernhardson created this task.Oct 22 2015, 4:55 PM

EBernhardson raised the priority of this task from to Needs Triage.

EBernhardson updated the task description. (Show Details)

EBernhardson subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 22 2015, 4:55 PM

EBernhardson added a project: Cloud-Services.Oct 22 2015, 4:55 PM

EBernhardson set Security to None.

based on current vm sizes, of 8 cores and 16G memory with 160G of disk, we can only fit two of those in a single default project quota (bumps up against the 20 core maximum before we use up the memory quota), so i suppose technically we already have most (but not all) of the memory we need for a third large instance available via existing quota, we just can't boot it because we don't have the vcpu's available to boot another large instance.

If that's a big ask we could try on the current 2 instances with more disk, or with a 3rd instance and more disk. The only real strict requirement is we need at least twice as much disk as currently available to the two large instances (160G*2). It would be nice to have more than that, due to the current 320G limitations we are only importing the content namespaces and completely ignoring that people also search in the non-content namespaces (partially because its not part of the default search).

More memory/vcpu's for a third instance is mostly just an idea to relieve iops, i think the project would still work fine without it.

EBernhardson added a parent task: T114673: [EPIC] Create a Relevance Lab which helps us measure the effect on search result relevance of proposed changes to search.Oct 22 2015, 5:17 PM

EBernhardson merged a task: T109377: Increase quota's for search project in labs.Oct 22 2015, 6:01 PM

EBernhardson added a project: Scrum-of-Scrums.Oct 26 2015, 5:08 PM

@yuvipanda this is the ticket we talked about last thursday night