Page MenuHomePhabricator

Investigate using the rich_data option to support Binary and binary_file for binary data
Closed, DeclinedPublic

Description

Puppet is moving away from pson and is supporting json by default. As such it no longer supports sending binary data on the wire. This normally occures when a file resource has its source set to a binary file. In order to fix this we need to migrate to using the binary_file handler. for example given the following resource

class test {
  file {'/tmp/test.png':
    ensure => file,
    source => 'puppet:///modules/test/test.png,
}

We need to change this to

class test {
  file {'/tmp/test.png':
    ensure => file,
    content => binary_file('test/test.png'),
}

To support sending the binary type to the agent via the puppet catalogue we must use the rich_data option. This is option is on by default in puppet6

WARNING: it seems that in puppet < 6 the rich_data option only make a difference to puppet apply not puppet agent
WARNING: even though wmflib::secret exists it will not work untill we enable this configuration. If used the function will copy the base64 content to the destination instead of the binary file content

Event Timeline

jbond triaged this task as Medium priority.

Change 546181 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] apereo_case: use Binary for storing the keystore

https://gerrit.wikimedia.org/r/546181

Change 546181 merged by Jbond:
[operations/puppet@production] apereo_case: use Binary for storing the keystore

https://gerrit.wikimedia.org/r/546181

Change 546464 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib:㊙️ add a new secret function which supports binary files

https://gerrit.wikimedia.org/r/546464

Change 546465 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] apereo_cas: migrate keystor to wmflib::secret

https://gerrit.wikimedia.org/r/546465

Change 546464 merged by Jbond:
[operations/puppet@production] wmflib:㊙️ add a new secret function which supports binary files

https://gerrit.wikimedia.org/r/546464

Change 546465 merged by Jbond:
[operations/puppet@production] apereo_cas: migrate keystor to wmflib::secret

https://gerrit.wikimedia.org/r/546465

jbond renamed this task from puppet should utilise Binary and binary_file for binary data to Investigate using the rich_)data opsion to support Binary and binary_file for binary data.Oct 30 2019, 12:58 PM
jbond updated the task description. (Show Details)
jbond renamed this task from Investigate using the rich_)data opsion to support Binary and binary_file for binary data to Investigate using the rich_data opsion to support Binary and binary_file for binary data.Oct 30 2019, 1:01 PM
jbond updated the task description. (Show Details)

Change 557041 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] puppet-compiler: test binary_file function

https://gerrit.wikimedia.org/r/557041

Change 557050 had a related patch set uploaded (by Aklapper; owner: John Bond):
[operations/software/puppet-compiler@master] puppet_compiler: add rich_data support

https://gerrit.wikimedia.org/r/557050

Aklapper renamed this task from Investigate using the rich_data opsion to support Binary and binary_file for binary data to Investigate using the rich_data option to support Binary and binary_file for binary data.Feb 7 2020, 4:32 PM

Change 557050 merged by jenkins-bot:
[operations/software/puppet-compiler@master] puppet_compiler: add rich_data support

https://gerrit.wikimedia.org/r/557050

Its now possible to use rich_data in the mode field to check the catalogue compilation with rich_data

e.g: https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler/20789

jbond updated the task description. (Show Details)

Change 557041 abandoned by Jbond:
[operations/puppet@production] puppet-compiler: test binary_file function

Reason:
test complete

https://gerrit.wikimedia.org/r/557041

This is not usefull untill puppet6 where we get it by default