When this task is completed we will have a 3 node presto cluster in labs with tests data. This data is been pushed from one of the mediawiki reconstruction snapshots we hold in hadoop, the test data is available to be queried by users of our cloud environment. The task is completed when infrastructure is setup and useable.
Description
Details
Event Timeline
Change 467766 merged by Ottomata:
[operations/puppet/cdh@master] Use net_topology script content rather than erb path
Change 467815 merged by Ottomata:
[operations/puppet@production] Move Hive profile settings to role hiera
Change 467769 merged by Ottomata:
[operations/puppet@production] Move Hadoop net topology to hiera
Change 468004 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Temporarily revert the change to net-topology.py.erb
Change 468004 merged by Ottomata:
[operations/puppet@production] Temporarily revert the change to net-topology.py.erb
Change 468046 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet/cdh@master] Use net_topology_script_path to configure net.topology.script.file.name
Change 468046 merged by Ottomata:
[operations/puppet/cdh@master] Use net_topology_script_path to configure net.topology.script.file.name
Change 468049 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Fix net-topology.py.erb script to render proper python dict
Change 468049 merged by Ottomata:
[operations/puppet@production] Fix net-topology.py.erb script to render proper python dict
Change 467821 merged by Ottomata:
[operations/puppet@production] Label Hadoop prometheus metrics with the hadoop_cluster_name
Change 468056 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add more labels to Hadoop daemon JMX prometheus metrics
Change 468068 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use hadoop_cluster label in icinga alerts
Change 468068 merged by Ottomata:
[operations/puppet@production] Use hadoop_cluster label in icinga alerts
Change 468070 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] [WIP] Configure cloud-analytics-eqiad Hadoop cluster
Change 468071 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Conditionally pass in zookeeper_hosts to cdh::hive
Change 468071 merged by Ottomata:
[operations/puppet@production] Conditionally pass in zookeeper_hosts to cdh::hive
Change 468056 abandoned by Ottomata:
Add more labels to Hadoop daemon JMX prometheus metrics
Change 478746 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add hieradata/labs/cloud-analytics/common.yaml
Change 478746 merged by Ottomata:
[operations/puppet@production] Add hieradata/labs/cloud-analytics/common.yaml
Change 478748 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add cloud-analytics zookeeper settings
Change 478748 merged by Ottomata:
[operations/puppet@production] Add cloud-analytics zookeeper settings
Change 478751 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Update zookeeper package version for cloud-analytics
Change 478751 merged by Ottomata:
[operations/puppet@production] Update zookeeper package version for cloud-analytics
Change 478755 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Allow configuration of $hadoop_var_directory in profile::hadoop::commmon
Change 478755 merged by Ottomata:
[operations/puppet@production] Allow configuration of $hadoop_var_directory in profile::hadoop::commmon
Change 478758 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use subdir of hadoop data path for datanode_mounts in cloud-analytics
Change 478758 merged by Ottomata:
[operations/puppet@production] Use subdir of hadoop data path for datanode_mounts in cloud-analytics
Change 478767 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Set monitoring_enabled to true for cloud-analytics
Change 478767 merged by Ottomata:
[operations/puppet@production] Set monitoring_enabled to true for cloud-analytics
Change 478772 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Bump zookeeper submodule version
Change 478772 merged by Ottomata:
[operations/puppet@production] Bump zookeeper submodule version
Change 478778 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Attempt to get around prometheus jmx exporter race condition on new cluster
Change 478778 merged by Ottomata:
[operations/puppet@production] Attempt to get around prometheus jmx exporter race condition on new cluster
Change 478779 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Undo last change and temporarily disable JMX prometheus in cloud-analytics
Change 478779 merged by Ottomata:
[operations/puppet@production] Undo last change and temporarily disable JMX prometheus in cloud-analytics
Change 478782 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Bump cdh submodule to vary zkCli.sh path
Change 478782 merged by Ottomata:
[operations/puppet@production] Bump cdh submodule to vary zkCli.sh path
Change 478789 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Move cloud-analytics zookeeper to ca-conf-*
Change 478789 merged by Ottomata:
[operations/puppet@production] Move cloud-analytics zookeeper to ca-conf-*
Change 478798 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Set yarn and hadoop heapsize for cloud-analytics
Change 478798 merged by Ottomata:
[operations/puppet@production] Set yarn and hadoop heapsize for cloud-analytics
Change 478800 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Set Xmx for datanode and nodemanager in cloud-analytics
Change 478800 merged by Ottomata:
[operations/puppet@production] Set Xmx for datanode and nodemanager in cloud-analytics
Change 478801 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Reenable hadoop prometheus jmx exporters in cloud-analytics
Change 478801 merged by Ottomata:
[operations/puppet@production] Reenable hadoop prometheus jmx exporters in cloud-analytics
Status update!
cloud-analytics Hadoop cluster is up and running! Tomorrow will be Presto work... :)
\o/ !!! That's super great :)
Let's discuss a one-off way to copy data over for tests.
Is there another another way than through public-endpot?
If not, we shold start the copy :)
Change 478962 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Fix comment in cloud-analytics common.yaml
Change 478963 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add cloud_analytics roles
Change 478962 merged by Ottomata:
[operations/puppet@production] Fix comment in cloud-analytics common.yaml
Change 478963 merged by Ottomata:
[operations/puppet@production] Add cloud_analytics roles
Change 468070 abandoned by Ottomata:
[WIP] Configure cloud-analytics-eqiad Hadoop cluster
Reason:
doing this in Cloud VPS
Change 479324 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use fqdn instead of hostname in nodes id
Change 479324 merged by Ottomata:
[operations/puppet@production] Use fqdn instead of hostname in nodes id
Change 479339 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Set presto hive connector name to hive-hadoop2
Change 479339 merged by Ottomata:
[operations/puppet@production] Set presto hive connector name to hive-hadoop2
Change 479343 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Use thrift:// uri for hive http:// uri for discovery uri
Change 479343 merged by Ottomata:
[operations/puppet@production] Use thrift:// uri for hive http:// uri for discovery uri
Change 479508 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Run Presto coordinator on ca-master-2 in cloud-analytics
Change 479508 merged by Ottomata:
[operations/puppet@production] Run Presto coordinator on ca-master-2 in cloud-analytics
Change 479509 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add ::profile::presto::server to ca-master-2 to run Presto coordinator
Change 479509 merged by Ottomata:
[operations/puppet@production] Add ::profile::presto::server to ca-master-2 to run Presto coordinator
Change 479525 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Make presto module and profile smarter
Change 479525 merged by Ottomata:
[operations/puppet@production] Make presto module and profile smarter
Change 480243 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Fix Hadoop settings for presto in cloud-analytics
Change 480243 merged by Ottomata:
[operations/puppet@production] Fix Hadoop settings for presto in cloud-analytics
After our talk with ops and cloud looks like we are going to need to move this work to prod due to lack of monitoring in labs. Project blocked
Deleting all instances because T216707: CloudVPS: cloudvirtan1002 puppet failures due to memory allocation issues?.
Presto may not work here, unless we just poke a hole to a production cluster. We'll have it for reference.