Page MenuHomePhabricator

Find deployment host to deploy refinery from that has neither refinery-hive, nor passwords in hive-site.xml
Closed, ResolvedPublic

Description

Since the addition of refinery-hive.jar in 29e1364de763a5cd4b2c56ab15268a794dd9ea1a, stat1002's /etc/hive/conf/hive-site.xml comes with

file:///srv/deployment/analytics/refinery/artifacts/refinery-hive.jar

in hive.aux.jars.path.
When deploying refinery from stat1002 this setting makes it into the hive configuration that Oozie uses, and Oozie jobs fail, due to that file not being available
on worker nodes.

Hence, we cannot deploy from stat1002.

We also cannot deploy from analytics1027, as it's /etc/hive/conf/hive-site.xml contains passwords that we would not want to leak into the cluster.

Event Timeline

QChris raised the priority of this task from to Needs Triage.
QChris updated the task description. (Show Details)
QChris added a project: Analytics-Clusters.
QChris changed Security from none to None.
QChris subscribed.

Change 177714 had a related patch set uploaded (by QChris):
Abort deployment, if Oozie's Hive config seems to contain passwords

https://gerrit.wikimedia.org/r/177714

Patch-For-Review

Change 177716 had a related patch set uploaded (by QChris):
Drop jars that are not on all worker nodes from Oozie's Hive config

https://gerrit.wikimedia.org/r/177716

Patch-For-Review

Change 177714 merged by Ottomata:
Abort deployment, if Oozie's Hive config seems to contain passwords

https://gerrit.wikimedia.org/r/177714

Change 177716 merged by Ottomata:
Drop jars that are not on all worker nodes from Oozie's Hive config

https://gerrit.wikimedia.org/r/177716

QChris claimed this task.