Page MenuHomePhabricator
Create Task
Maniphest T218343

Upgrade analytics cluster to Cloudera CDH 5.16.1
Closed, ResolvedPublic13 Estimated Story Points
Actions

Description

We should first evaluate wether this distro provides enough value for us to upgrade, also work is a bit contingent on other testing being done for kerberos, we want to test upgrades on testing cluster once cluster is stable.

Current version: 5.15.0

Issue fixed up to 5.16.1:

What's new up to 5.16.1:

Overall nothing really juicy from what I can see, but there are a couple of security fixes that are surely good to have.

Last upgrade: T204759
Old etherpad: https://etherpad.wikimedia.org/p/analytics-cdh5.15
New etherpad: https://etherpad.wikimedia.org/p/analytics-cdh5.16.1

Details

SubjectRepoBranchLines +/-
cumin: update hadoop aliasoperations/puppetproduction+1 -1
profile::hadoop::spark2: auto upload of spark2-assembly.zipoperations/puppetproduction+47 -2
oozie: override the oozie-setup scriptoperations/puppet/cdhmaster+43 -1
aptrepo: update cloudera-jessie to 5.16.1operations/puppetproduction+1 -1
cumin: add hadoop-hdfs-backup aliasesoperations/puppetproduction+2 -1
cumin: add more hadoop-related aliasesoperations/puppetproduction+2 -0
cumin: add aliases for Hadoop HDFS journalnodesoperations/puppetproduction+1 -0
Customize query in gerrit

Related Objects

Event Timeline

• Nuria created this task.Mar 14 2019, 7:05 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 14 2019, 7:05 PM
Milimetric triaged this task as High priority.Mar 18 2019, 3:20 PM
Milimetric moved this task from Incoming to Operational Excellence on the Analytics board.
elukey renamed this task from Upgrade analytics cluster to cloudera distro CDH 5.16 to Upgrade analytics cluster to Cloudera CDH 5.16.1.Mar 27 2019, 2:39 PM
elukey claimed this task.
elukey lowered the priority of this task from High to Medium.
elukey updated the task description. (Show Details)
elukey added a project: User-Elukey.
elukey added subscribers: JAllemandou, Ottomata.
elukey moved this task from Backlog to In Progress on the User-Elukey board.Apr 1 2019, 11:28 AM
elukey updated the task description. (Show Details)Apr 1 2019, 1:43 PM
gerritbot added a comment.Apr 1 2019, 1:58 PM

Change 500453 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] aptrepo: update cloudera-jessie to 5.16.1

https://gerrit.wikimedia.org/r/500453

gerritbot added a project: Patch-For-Review.Apr 1 2019, 1:58 PM
gerritbot added a comment.Apr 1 2019, 2:43 PM

Change 500465 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] cumin: add aliases for Hadoop HDFS journalnodes

https://gerrit.wikimedia.org/r/500465

gerritbot added a comment.Apr 1 2019, 3:17 PM

Change 500465 merged by Elukey:
[operations/puppet@production] cumin: add aliases for Hadoop HDFS journalnodes

https://gerrit.wikimedia.org/r/500465

gerritbot added a comment.Apr 3 2019, 2:23 PM

Change 500967 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] cumin: add more hadoop-related aliases

https://gerrit.wikimedia.org/r/500967

gerritbot added a comment.Apr 3 2019, 6:28 PM

Change 500967 merged by Elukey:
[operations/puppet@production] cumin: add more hadoop-related aliases

https://gerrit.wikimedia.org/r/500967

gerritbot added a comment.Apr 4 2019, 9:50 AM

Change 501162 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] cumin: add hadoop-hdfs-backup aliases

https://gerrit.wikimedia.org/r/501162

gerritbot added a comment.Apr 4 2019, 10:06 AM

Change 501162 merged by Elukey:
[operations/puppet@production] cumin: add hadoop-hdfs-backup aliases

https://gerrit.wikimedia.org/r/501162

elukey updated the task description. (Show Details)Apr 10 2019, 7:59 AM
gerritbot added a comment.Apr 10 2019, 8:03 AM

Change 500453 merged by Elukey:
[operations/puppet@production] aptrepo: update cloudera-jessie to 5.16.1

https://gerrit.wikimedia.org/r/500453

Stashbot added a comment.Apr 10 2019, 8:36 AM

Mentioned in SAL (#wikimedia-operations) [2019-04-10T08:36:50Z] <elukey> update thirdparty/cloudera packages to cdh 5.16.1 for jessie/stretch-wikimedia - T218343

gerritbot added a comment.Apr 12 2019, 7:55 AM

Change 503266 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet/cdh@master] oozie: override the oozie-setup script

https://gerrit.wikimedia.org/r/503266

elukey added a comment.Apr 12 2019, 9:02 AM

I have tested https://etherpad.wikimedia.org/p/analytics-cdh5.16.1 upgrading the Hadoop test cluster, all good!

https://gerrit.wikimedia.org/r/503266 needs to be merged/deployed before upgrading but it is a minor nit. I have also removed all the spark 1 packages and rechecked that the packages that we are installing are already present on the hosts of the Production cluster.

I can confirm that the refine webrequest job in the testing cluster works fine, we only need to test Spark 2 but I don't foresee anything weird on that front. After this, ready to upgrade production!

elukey added a project: Analytics-Kanban.Apr 12 2019, 9:02 AM
elukey moved this task from Next Up to In Progress on the Analytics-Kanban board.
elukey moved this task from In Progress to Ready to Deploy on the Analytics-Kanban board.Apr 12 2019, 2:08 PM
gerritbot added a comment.Apr 15 2019, 1:32 PM

Change 503266 merged by Elukey:
[operations/puppet/cdh@master] oozie: override the oozie-setup script

https://gerrit.wikimedia.org/r/503266

gerritbot added a comment.Apr 16 2019, 8:36 AM

Change 504268 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::hadoop::spark2: auto upload of spark2-assembly.zip

https://gerrit.wikimedia.org/r/504268

gerritbot added a comment.Apr 16 2019, 1:06 PM

Change 504268 merged by Elukey:
[operations/puppet@production] profile::hadoop::spark2: auto upload of spark2-assembly.zip

https://gerrit.wikimedia.org/r/504268

Stashbot added a comment.Apr 17 2019, 1:52 PM

Mentioned in SAL (#wikimedia-operations) [2019-04-17T13:52:43Z] <elukey> upgrading hadoop cdh distrubition to 5.16.1 on all the Hadoop-related nodes - T218343

gerritbot added a comment.Apr 17 2019, 3:26 PM

Change 504585 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] cumin: update hadoop alias

https://gerrit.wikimedia.org/r/504585

gerritbot added a comment.Apr 17 2019, 4:01 PM

Change 504585 merged by Elukey:
[operations/puppet@production] cumin: update hadoop alias

https://gerrit.wikimedia.org/r/504585

elukey moved this task from In Progress to Done on the User-Elukey board.Apr 18 2019, 8:32 AM
elukey set the point value for this task to 13.

Upgrade done, no errors reported!

elukey moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Apr 18 2019, 8:32 AM
• Nuria closed this task as Resolved.May 14 2019, 9:14 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL