Page MenuHomePhabricator

New Cloud VPS project 'cloud-analytics'
Closed, ResolvedPublic


In T207321: Figure out networking details for new cloud-analytics-eqiad Hadoop/Presto cluster we determined that we will be setting up a Hadoop & Presto cluster in Cloud VPS under a new project cloud-analytics. We'll be adding some new hardware to Cloud VPS networks for worker nodes, but we also need some virt resources for various master and coordination nodes.

We need 3 more VMs for HA Hadoop masters and Hive etc. 'coordinator' nodes. These should be:
Each should have 16G RAM and 8 cores. (If this is too highly speced, we can be flexible here).

Event Timeline

GTirloni triaged this task as Medium priority.

I've created this project -- of course we'll be waiting for the hardware to get racked before it can do much.

Regarding the usage of the support nodes: you currently have default new-project quotas, which are 8 instances, 8 cores, 16Gb RAM. I'd encourage you to do some testing with the stock VM sizes that are available; if you turn out to be maxing out your existing specs then we can adjust your quotas as needed.

(And, of course, your quotas will have to be increased immensely to permit scheduling on the dedicated hardware. I'll cross that bridge when I come to it.)

In the meantime, @Ottomata is the project admin of this project. You can add other team members as you see fit via Horizon.

PERFECT, thanks Andrew! We'll try to fit what we need in the stock VMs first. Sounds good.