The druid100[4-6] servers are due to be decommissioned.
{T336042} has been completed we shall now proceed with the decommissioning of the druid100[4-6] hosts. steps as per [[ https://wikitech.wikimedia.org/wiki/Data_Engineering/Systems/Druid#Removing_hosts/_taking_hosts_out_of_service_from_cluster | Removing hosts/ taking hosts out of service from cluster ]] as below:
- Change the druid-public endpoint used by services namely
- The druid host required to index [[ https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/blob/main/analytics/dags/mediawiki/mediawiki_history_reduced_dag.py#L53 | mediawiki_history_reduced]] to the druid-public cluster.
- The host and port configured for [[ https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/blob/main/analytics/dags/datahub/ingestion/configs/druid_public.yaml#L4 | datahub injestion ]]
- The druid host configured for [[ https://github.com/wikimedia/operations-puppet/blob/production/hieradata/role/common/analytics_cluster/launcher.yaml#L25 | profile::analytics::refinery::job::data_purge::public_druid_host ]]
- There are some more mentions for druid1004 but mainly related to [[ https://codesearch.wmcloud.org/search/?q=druid1004&files=&excludeFiles=&repos= | oozie ]]
[x] druid1004
- Depool the hosts.
[x] druid1004
[x] druid1005
[x] druid1006
- Using the coordinator web interface set nodes into decommissioningNodes mode. Once the historical disk cache is drained, the middlemanager is not running any jobs, and the overlord is not targeted by any scheduled jobs, it is safe to stop the services.
[x] druid1004
[x] druid1005
[x] druid1006
- Remove the hosts from LVS
[] druid1004
[] druid1005
[] druid1006
- Decommission hosts
[] druid1004
[] druid1005
[] druid1006
- Remove mention of hosts from site.pp
[] druid1004
[] druid1005
[] druid1006
- Remove keytabs and dummy-keytabs
[] druid1004
[] druid1005
[] druid1006