wrote before I realized Logstash apparently uses JRuby 1.9 so should not be an issue

A gotcha is the gem binaries are compiled against a ruby version. If you deploy on different Linux distributions / vary ruby version, you end up with an incompatible ABI. So you would need to namespace the binaries modules by ABI version :-( Potentially has branches in operations/software/logstash/plugins.git.

In ruby world you could use rake-compiler ( https://github.com/rake-compiler/rake-compiler#generating-native-rubygems ) which creates native / pre compiled gems.

Or easier, bundle package package all your gems under vendor/cache.

Both have the same issue, you need to build the .gem on a staging machine that matches the ruby version of the deployment target.

The scap Deployments is being improved to deploy MediaWiki service backends, but that uses git fetch. Tin has a rsync server though, so maybe you can reuse it ?

The rsync common container serves /srv/mediawiki-staging setup via rsync::server::module. Maybe package { provider => Trebuchet } could be made to setup an appropriate rsync container or have the deployment role set it up via rsync::server::module.

Not sure how many Gems we're talking here and how often they change, if the numbers are low, it would also be an option to use gem2deb to create a Debian source package and build that for trusty/jessie:
https://wiki.debian.org/Teams/Ruby/Packaging
https://packages.debian.org/sid/gem2deb

One problem is that logstash moves quickly when it comes to dependency and there is a lot to package or the distro versions would not match the requirements :(

For v1.5.3 see Gemfile and Gemfile.jruby-1.9.lock.

Interesting!

Rather than using git-fat + rsync, perhaps it would be better to do the deploy-repo module, where there is a git-submodule that includes committed dependencies, including binaries. This isn't ideal, but at least it keeps the binaries out of your main source repository.

Maybe package { provider => Trebuchet } could be made to setup an appropriate rsync container

It can already do this via 'gitfat_enabled' => true in the repo config in deployment.pp

In T107121#1487727, @MoritzMuehlenhoff wrote:

Not sure how many Gems we're talking here and how often they change, if the numbers are low, it would also be an option to use gem2deb to create a Debian source package and build that for trusty/jessie:
https://wiki.debian.org/Teams/Ruby/Packaging
https://packages.debian.org/sid/gem2deb

For the initial 1.5.3 deploy we only need one non-core plugin: https://rubygems.org/gems/logstash-filter-prune/versions/0.1.5. This is a pure ruby gem (https://github.com/logstash-plugins/logstash-filter-prune) so the ABI issue is moot at this time.

The tricky bit about deb packaging however is that logstash doesn't use the system ruby or any standard gem locations. It's actually a jruby app and their new plugin management system seems to rely on reading the Gemfile and Gemfile.lock in the deploy directory (https://github.com/elastic/logstash/issues/3540).

I can't say that I like how they are changing things upstream, but arguing with Jordan Sissel of fpm fame about package management practices has not been historically useful.