Page MenuHomePhabricator

Define the details of the hardware we need to run WDQS
Closed, ResolvedPublic

Description

We need to determine what we are looking for hardware-wise in WDQS server.

Requirements (from a comment below)

  1. Running updates fast - i.e., much faster than the update stream - for comparison, labs machines do between 1 and 3 s of update stream per 1 s of clock time, but I'd like to have it on the level of 10s or so.
  1. Running simple queries under 30 s. Yes, I know "simple" is not defined, so we'd have to kind of invent definition for it.
  1. Being able to run several queries without noticeable performance degradation. I think for now since we don't have a lot of load values of several can be small, i.e. under 10, but in the future we may have higher reqs here.
  1. Being able to run common WDQ queries within reasonable time (see T101528)

Event Timeline

Smalyshev created this task.Jul 6 2015, 5:25 PM
Smalyshev assigned this task to Joe.
Smalyshev raised the priority of this task from to Normal.
Smalyshev updated the task description. (Show Details)
Restricted Application added a subscriber: Matanya. · View Herald TranscriptJul 6 2015, 5:25 PM

Initial estimate:

  • 64G RAM at least
  • SSD drive. DB size is about 70G now, if we have a spare server setup then anything over 100-150G should be sufficient
  • Since queries are somewhat parallelized, as many cores as possible (dual 4-core?)

What are the requirements that this hardware is needed to satisfy?

Smalyshev set Security to None.
  1. Running updates fast - i.e., much faster than the update stream - for comparison, labs machines do between 1 and 3 s of update stream per 1 s of clock time, but I'd like to have it on the level of 10s or so.
  1. Running simple queries under 30 s. Yes, I know "simple" is not defined, so we'd have to kind of invent definition for it.
  1. Being able to run several queries without noticeable performance degradation. I think for now since we don't have a lot of load values of several can be small, i.e. under 10, but in the future we may have higher reqs here.
  1. Being able to run common WDQ queries within reasonable time (see T101528)

Awesome, thanks! I'll copy them into the task description.

Jdouglas updated the task description. (Show Details)Jul 7 2015, 3:51 PM
Joe added a comment.Jul 8 2015, 11:33 AM

Just as a note - labs instances are incredibly slower than production hardware. IOPS are horrible, in particular, so if we just need 3x as an IO performance, we should be able to pull it off with non-ssd disks as well.

Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.Jul 12 2015, 1:09 PM
Smalyshev added a comment.EditedJul 13 2015, 6:37 AM

Based on the above, I think what we need is:

  • 64G memory
  • 300 G SSD
  • 4-8 cores with 2.5 GHz min
Smalyshev closed this task as Resolved.Jul 13 2015, 6:08 PM
Deskana moved this task from On Sprint Board to Done on the Discovery board.Aug 12 2015, 12:58 AM