Page MenuHomePhabricator

Define the details of the hardware we need to run WDQS
Closed, ResolvedPublic

Description

We need to determine what we are looking for hardware-wise in WDQS server.

Requirements (from a comment below)

  1. Running updates fast - i.e., much faster than the update stream - for comparison, labs machines do between 1 and 3 s of update stream per 1 s of clock time, but I'd like to have it on the level of 10s or so.
  1. Running simple queries under 30 s. Yes, I know "simple" is not defined, so we'd have to kind of invent definition for it.
  1. Being able to run several queries without noticeable performance degradation. I think for now since we don't have a lot of load values of several can be small, i.e. under 10, but in the future we may have higher reqs here.
  1. Being able to run common WDQ queries within reasonable time (see T101528)

Event Timeline

Smalyshev assigned this task to Joe.
Smalyshev raised the priority of this task from to Medium.
Smalyshev updated the task description. (Show Details)

Initial estimate:

  • 64G RAM at least
  • SSD drive. DB size is about 70G now, if we have a spare server setup then anything over 100-150G should be sufficient
  • Since queries are somewhat parallelized, as many cores as possible (dual 4-core?)

What are the requirements that this hardware is needed to satisfy?

Smalyshev set Security to None.
  1. Running updates fast - i.e., much faster than the update stream - for comparison, labs machines do between 1 and 3 s of update stream per 1 s of clock time, but I'd like to have it on the level of 10s or so.
  1. Running simple queries under 30 s. Yes, I know "simple" is not defined, so we'd have to kind of invent definition for it.
  1. Being able to run several queries without noticeable performance degradation. I think for now since we don't have a lot of load values of several can be small, i.e. under 10, but in the future we may have higher reqs here.
  1. Being able to run common WDQ queries within reasonable time (see T101528)

Awesome, thanks! I'll copy them into the task description.

Just as a note - labs instances are incredibly slower than production hardware. IOPS are horrible, in particular, so if we just need 3x as an IO performance, we should be able to pull it off with non-ssd disks as well.

Based on the above, I think what we need is:

  • 64G memory
  • 300 G SSD
  • 4-8 cores with 2.5 GHz min