Page MenuHomePhabricator
Create Task
Maniphest T253553

Create a profile to standardize the deployment of JVM packages and configurations
Closed, ResolvedPublic
Actions

Description

In https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/597219/ we are discussing the creation of profile::java, a single profile to deploy one or more jvm versions to the same host. The idea is to avoid copy/paste of puppet code across operations/puppet, re-using the same profile in multiple places.

This is also needed since a lot of Java-based daemons (like hadoop, kafka, etc..) don't support Java 11 yet, so we'll have to use 8 when migrating to Buster.

The goal of this task is to reach consensus for profile::java, and to port all roles in need of Java to it.

Details

SubjectRepoBranchLines +/-
Switch CI to profile::javaoperations/puppetproduction+16 -15
alternatives::java: fix java pathsoperations/puppetproduction+9 -3
profile::java: ensure /etc/environment.doperations/puppetproduction+5 -0
role::search::airflow: use profile::javaoperations/puppetproduction+6 -0
Swap profile::java::analytics with profile::javaoperations/puppetproduction+23 -20
Switch the IDPs to profile::javaoperations/puppetproduction+2 -1
role::archiva: move to profile::javaoperations/puppetproduction+7 -1
profile::java: set defaults correctly accoring to their typeoperations/puppetproduction+2 -2
profile::java: set extra_args with type Optional[String]operations/puppetproduction+1 -1
Customize query in gerrit

Related Objects

Event Timeline

elukey triaged this task as Medium priority.May 25 2020, 11:43 AM
elukey created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 25 2020, 11:43 AM
elukey added a subscriber: Eevans.May 25 2020, 11:44 AM
gerritbot added a comment.May 25 2020, 2:04 PM

Change 598488 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Switch the IDPs to profile::java

https://gerrit.wikimedia.org/r/598488

gerritbot added a project: Patch-For-Review.May 25 2020, 2:04 PM
gerritbot added a comment.May 25 2020, 5:12 PM

Change 598505 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::java: set extra_args with type Optional[String]

https://gerrit.wikimedia.org/r/598505

gerritbot added a comment.May 25 2020, 5:14 PM

Change 598505 merged by Elukey:
[operations/puppet@production] profile::java: set extra_args with type Optional[String]

https://gerrit.wikimedia.org/r/598505

gerritbot added a comment.May 25 2020, 5:18 PM

Change 598507 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::java: set defaults correctly accoring to their type

https://gerrit.wikimedia.org/r/598507

gerritbot added a comment.May 25 2020, 5:20 PM

Change 598507 merged by Elukey:
[operations/puppet@production] profile::java: set defaults correctly accoring to their type

https://gerrit.wikimedia.org/r/598507

gerritbot added a comment.May 26 2020, 6:45 AM

Change 598672 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] role::archiva: move to profile::java

https://gerrit.wikimedia.org/r/598672

gerritbot added a comment.May 26 2020, 6:59 AM

Change 598672 merged by Elukey:
[operations/puppet@production] role::archiva: move to profile::java

https://gerrit.wikimedia.org/r/598672

gerritbot added a comment.May 26 2020, 7:21 AM

Change 598675 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Swap profile::java::analytics with profile::java

https://gerrit.wikimedia.org/r/598675

elukey moved this task from Next Up to In Progress on the Analytics-Kanban board.May 27 2020, 7:05 AM
gerritbot added a comment.May 27 2020, 10:30 AM

Change 598488 merged by Muehlenhoff:
[operations/puppet@production] Switch the IDPs to profile::java

https://gerrit.wikimedia.org/r/598488

Milimetric raised the priority of this task from Medium to High.May 28 2020, 3:30 PM
Milimetric moved this task from Incoming to Operational Excellence on the Analytics board.
gerritbot added a comment.May 28 2020, 4:07 PM

Change 598675 merged by Elukey:
[operations/puppet@production] Swap profile::java::analytics with profile::java

https://gerrit.wikimedia.org/r/598675

Maintenance_bot removed a project: Patch-For-Review.May 28 2020, 4:10 PM
elukey added a comment.Jun 4 2020, 8:56 AM

https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/602009/

gerritbot added a comment.Jun 4 2020, 8:56 AM

Change 602291 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] role::search::airflow: use profile::java

https://gerrit.wikimedia.org/r/602291

gerritbot added a project: Patch-For-Review.Jun 4 2020, 8:56 AM
gerritbot added a comment.Jun 4 2020, 9:00 AM

Change 602293 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::java: ensure /etc/environment.d

https://gerrit.wikimedia.org/r/602293

gerritbot added a comment.Jun 4 2020, 9:04 AM

Change 602291 merged by Elukey:
[operations/puppet@production] role::search::airflow: use profile::java

https://gerrit.wikimedia.org/r/602291

gerritbot added a comment.Jun 4 2020, 9:38 AM

Change 602293 merged by Elukey:
[operations/puppet@production] profile::java: ensure /etc/environment.d

https://gerrit.wikimedia.org/r/602293

Maintenance_bot removed a project: Patch-For-Review.Jun 4 2020, 10:10 AM
gerritbot added a comment.Jun 5 2020, 1:42 PM

Change 602689 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] alternatives::java: fix java paths

https://gerrit.wikimedia.org/r/602689

gerritbot added a project: Patch-For-Review.Jun 5 2020, 1:42 PM
gerritbot added a comment.Jun 5 2020, 2:36 PM

Change 602689 merged by Elukey:
[operations/puppet@production] alternatives::java: fix java paths

https://gerrit.wikimedia.org/r/602689

Maintenance_bot removed a project: Patch-For-Review.Jun 5 2020, 3:11 PM
elukey added a project: Analytics-Clusters.Jun 10 2020, 2:41 PM
elukey moved this task from Backlog to Q1 2020/2021 on the Analytics-Clusters board.
elukey moved this task from Q1 2020/2021 to Q4 2019/2020 on the Analytics-Clusters board.
elukey added a comment.Jun 16 2020, 10:42 AM

In T252913 Keith is working on moving ES and Kafka to profile::java, so the one missing is Cassandra plus few others.

gerritbot added a comment.Jun 16 2020, 12:14 PM

Change 605886 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Switch CI to profile::java

https://gerrit.wikimedia.org/r/605886

gerritbot added a project: Patch-For-Review.Jun 16 2020, 12:14 PM
Aklapper removed a project: Analytics.Jul 4 2020, 7:59 AM
elukey added a comment.Jul 6 2020, 7:11 AM

The goal of this task has been reached, there are some remaining systems to be migrated (CI, cassandra, etc.) but work is in progress.

elukey set Final Story Points to 8.Jul 6 2020, 7:12 AM
elukey moved this task from In Progress to Done on the Analytics-Kanban board.
• Nuria closed this task as Resolved.Jul 6 2020, 5:50 PM
hashar added a comment.Jul 6 2020, 9:37 PM

Thank you @elukey for the new Puppet Java profile and for taking in accounts suggestions for the weird use case I had.

tech-barnstar

^^ better seen with the web interface.

elukey mentioned this in T261966: Move cassandra puppet code (used by Restbase, Sessionstore, AQS, maps) to profile::java.Sep 3 2020, 3:59 PM
gerritbot added a comment.Sep 28 2020, 8:40 AM

Change 605886 merged by Muehlenhoff:
[operations/puppet@production] Switch CI to profile::java

https://gerrit.wikimedia.org/r/605886

MoritzMuehlenhoff mentioned this in T264174: Migrate remaining services using Java to profile::java .Sep 30 2020, 11:10 AM
MoritzMuehlenhoff mentioned this in T264176: Switch Zookeeper to profile::java.Sep 30 2020, 11:22 AM
MoritzMuehlenhoff mentioned this in T264177: Switch cergen to profile::java.
MoritzMuehlenhoff mentioned this in T264178: Switch puppetdb to profile::java.
MoritzMuehlenhoff mentioned this in T264181: Migrate WDQS to profile::java.Sep 30 2020, 12:16 PM
MoritzMuehlenhoff mentioned this in T264182: Migrate Gerrit to profile::java.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL