Page MenuHomePhabricator
Create Task
Maniphest T186619

Upload cassandra package(s) to wikimedia apt repository
Closed, ResolvedPublic
Actions

Description

The RESTBase cluster is using a custom build of the Cassandra 3.11.0 release. Currently tagged in our fork as wmf/3.11.0-wmf5 (see: https://github.com/wikimedia/cassandra/compare/cassandra-3.11.0...wmf/3.11.0-wmf5).

Debian packages are on rutherfordium:

https://people.wikimedia.org/~eevans/debian/cassandra_3.11.0-wmf5_all.deb
https://people.wikimedia.org/~eevans/debian/cassandra-tools_3.11.0-wmf5_all.deb
https://people.wikimedia.org/~eevans/debian/cassandra_3.11.0-wmf5.dsc
https://people.wikimedia.org/~eevans/debian/cassandra_3.11.0-wmf5.diff.gz
https://people.wikimedia.org/~eevans/debian/cassandra_3.11.0-wmf5_amd64.changes
https://people.wikimedia.org/~eevans/debian/cassandra_3.11.0.orig.tar.gz

These packages are signed by my key (same one in used in the Debian keyring):

pub   rsa4096 2009-07-12 [SC]
      C4965EE9E3015D192CCCF2B6F758CE318D77295D
uid           [  full  ] Eric Evans <eevans@sym-link.com>
uid           [  full  ] Eric Evans <eevans@apache.org>
uid           [  full  ] Eric Evans <eevans@debian.org>
uid           [  full  ] Eric Evans <eevans@opennms.org>
uid           [ unknown] Eric Evans <eevans@opennms.com>
sub   rsa4096 2009-07-12 [E]

The current version in the repository is 2.2.6-wmf1, which is the version in use by both the AQS and maps clusters. Simply upgrading the version to this one would be problematic for them. While RESTBase is deployed on jessie, these packages are arch independent and should be compatible with stretch; Would it be possible to have these added under stretch using the new structure, component/cassandra311 perhaps?

Details

SubjectRepoBranchLines +/-
cassandra: enable component/cassandra33 where applicableoperations/puppetproduction+13 -1
cassandra: actually enable component/cassandra311 (not component/cassandra33)operations/puppetproduction+2 -2
Add a component for Cassandra 3.11 packages for stretch-wikimediaoperations/puppetproduction+1 -1
Customize query in gerrit

Event Timeline

Eevans created this task.Feb 6 2018, 5:13 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 6 2018, 5:13 PM
fgiunchedi added a project: User-fgiunchedi.Feb 7 2018, 5:32 PM
MoritzMuehlenhoff added a comment.Feb 7 2018, 6:10 PM

Is there a specific reason for calling the repo component cassandra311? That's very specific and adding/removing components requires some puppet churn. IOW do we have reason to believe that Cassandra 3.12 will be incompatible with a 3.11 cluster?

If this is merely about providing separate packages for migrating between Cassandra 2 and 3 let's rather call this component/cassandra3?

Copying between releases is okay, usually we rebuild but if the packages just provide a few jars without many run-time deps it's fine to copy them as well. I can take of the import tomorrow.

Eevans added a comment.Feb 7 2018, 11:04 PM
In T186619#3953499, @MoritzMuehlenhoff wrote:

Is there a specific reason for calling the repo component cassandra311? That's very specific and adding/removing components requires some puppet churn. IOW do we have reason to believe that Cassandra 3.12 will be incompatible with a 3.11 cluster?

If this is merely about providing separate packages for migrating between Cassandra 2 and 3 let's rather call this component/cassandra3?

Even as of right now we have versions 2.1.13 and 2.2.6, (in addition to 3.11.0) in play. Version 2.1.13 is used by maps (which depending on who you talk to Doesn't Matter(tm)), and AQS uses 2.2.6, and probably will for the foreseeable future (they have no plans to upgrade). Even if you disqualify maps, we did keep AQS on a 2.1.x release for a considerable period of time (months?) after we'd moved RESTBase to a 2.2 release.

MoritzMuehlenhoff added a comment.Feb 9 2018, 3:02 PM

@Eevans wrote:

Even as of right now we have versions 2.1.13 and 2.2.6, (in addition to 3.11.0) in play. Version 2.1.13 is used by maps (which depending on who you talk to Doesn't Matter(tm)), and AQS uses 2.2.6, and probably will for the foreseeable future (they have no plans to upgrade). Even if you disqualify maps, we did keep AQS on a 2.1.x release for a considerable period of time (months?) after we'd moved RESTBase to a 2.2 release.

Ok, let's use component/cassandra311, then.

fgiunchedi triaged this task as Medium priority.Feb 12 2018, 10:06 AM
gerritbot subscribed.Feb 12 2018, 1:00 PM

Change 409877 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Add a component for Cassandra 3.11 packages for stretch-wikimedia

https://gerrit.wikimedia.org/r/409877

gerritbot added a project: Patch-For-Review.Feb 12 2018, 1:00 PM
gerritbot added a comment.Feb 12 2018, 2:09 PM

Change 409877 merged by Muehlenhoff:
[operations/puppet@production] Add a component for Cassandra 3.11 packages for stretch-wikimedia

https://gerrit.wikimedia.org/r/409877

Stashbot subscribed.Feb 12 2018, 2:38 PM

Mentioned in SAL (#wikimedia-operations) [2018-02-12T14:38:38Z] <moritzm> uploading cassandra 3.11.0-wmf5 to component/cassandra311 for stretch-wikimedia/apt.wikimedia.org (T186619)

MoritzMuehlenhoff closed this task as Resolved.Feb 12 2018, 2:41 PM
MoritzMuehlenhoff claimed this task.

Uploaded to apt.wikimedia.org. To add it to a server you can use

apt::repository { 'cassandra-3.11':
    uri        => 'http://apt.wikimedia.org/wikimedia',
    dist       => 'stretch-wikimedia',
    components => 'component/cassandra311',
}
Eevans edited projects, added Services (done); removed Services (doing).Feb 12 2018, 2:42 PM
Eevans awarded a token.
gerritbot added a comment.Feb 13 2018, 8:23 PM

Change 410252 had a related patch set uploaded (by Eevans; owner: Eevans):
[operations/puppet@production] cassandra: enable component/cassandra33 where applicable

https://gerrit.wikimedia.org/r/410252

gerritbot added a comment.Feb 13 2018, 9:40 PM

Change 410252 merged by Dzahn:
[operations/puppet@production] cassandra: enable component/cassandra33 where applicable

https://gerrit.wikimedia.org/r/410252

gerritbot added a comment.Feb 13 2018, 9:58 PM

Change 410342 had a related patch set uploaded (by Eevans; owner: Eevans):
[operations/puppet@production] cassandra: actually enable component/cassandra311 (not component/cassandra33)

https://gerrit.wikimedia.org/r/410342

gerritbot added a comment.Feb 13 2018, 10:07 PM

Change 410342 merged by Dzahn:
[operations/puppet@production] cassandra: actually enable component/cassandra311 (not component/cassandra33)

https://gerrit.wikimedia.org/r/410342

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits