Page MenuHomePhabricator

Drop PSON support
Closed, ResolvedPublic

Description

Support for the PSON (Pure JSON) serialization format has been removed.
Puppet agents may fail if binary data is accidentally added to the
catalog without using the Binary data type or binary_file function.
For more information visit
https://www.puppet.com/docs/puppet/latest/lang_data_binary.html.

It is possible, but not recommended, to re-enable PSON support by
installing the puppet-pson gem on all agents and server hosts and
setting preferred_serialization_format=pson in puppet.conf on each
agent.

Evaluation

Puppet 7 defaults to json for serialization, but falls back to pson when
binary data is present, unfortunately this is occurring on many of nodes:

while IFS= read -r host; do
    if curl -s \
	-X GET localhost:8080/pdb/query/v4 \
	-d 'query=reports { certname = '"'${host}'"' and latest_report? = true }' |
	jq . |
	grep -q pson; then
        echo $host
    fi
done < <(curl -s -X GET http://localhost:8080/pdb/query/v4 --data-urlencode "query=nodes[certname] {}" | jq -r '.[].certname')

Sample hosts based on roles:

  • modules/profile/manifests/kerberos/keytabs.pp:36
    • Hosts: an-airflow1002.eqiad.wmnet
  • modules/aptrepo/manifests/common.pp:58
    • Hosts: apt1002.wikimedia.org
  • modules/profile/manifests/base/cuminunpriv.pp:23
    • Hosts: apt1002.wikimedia.org
  • modules/profile/manifests/dns/auth/config.pp:91
    • Hosts: dns1004.wikimedia.org
  • modules/gerrit/manifests/init.pp:245
    • Hosts: gerrit1003.wikimedia.org
  • modules/apereo_cas/manifests/init.pp:208
    • Hosts: idp1004.wikimedia.org

For these nodes we will need to ascertain where binary data
is being used and wrap the binary data in Puppet's Binary
type.

In addition we use the to_pson function in a few
erb files to serialize data structures, these will
need to be replaced.

  • modules/tcpircbot/templates/tcpircbot.json.erb:4
    • Hosts: alert1001.wikimedia.org
  • modules/librenms/templates/config.php.erb:11
    • Hosts: netmon1003.wikimedia.org
  • modules/httpd/templates/env.conf.erb:3
    • Hosts: mwdebug2001.codfw.wmnet

Unfortunately, pson serialization cannot be disabled on bookworm's Puppet 7.23. A flag to disable
pson was added to 7.25, if that version is packaged then we can flip the flag, https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1081696

Event Timeline

jhathaway raised the priority of this task from Low to Medium.
jhathaway updated the task description. (Show Details)

T236481 has some related background work that @jbond performed to prepare us for the pson deprecation. Also, in order to compile catalog's on the command line, you need to use --render-as=rich_data_json in order to serialize binary data into json.

Change #1065284 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: ensure type is binary

https://gerrit.wikimedia.org/r/1065284

Change #1065284 merged by JHathaway:

[operations/puppet@production] puppet8: ensure type is binary

https://gerrit.wikimedia.org/r/1065284

Change #1071959 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: replace to_pson with to_json

https://gerrit.wikimedia.org/r/1071959

Change #1071960 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: replace to_pson with to_json

https://gerrit.wikimedia.org/r/1071960

Change #1071959 merged by JHathaway:

[operations/puppet@production] puppet8: replace to_pson with to_json

https://gerrit.wikimedia.org/r/1071959

Change #1071960 merged by JHathaway:

[operations/puppet@production] puppet8: replace to_pson with to_json

https://gerrit.wikimedia.org/r/1071960

Change #1071962 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: replace to_pson with to_json

https://gerrit.wikimedia.org/r/1071962

jhathaway updated the task description. (Show Details)

The offending resource can be tracked down using commands of the form:

host=stat1008.eqiad.wmnet; curl http://localhost:8080/pdb/query/v4/catalogs/$host | grep -EC 10 '\\u' | v -

Change #1072593 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: ensure kerberos keytab type is binary

https://gerrit.wikimedia.org/r/1072593

Change #1072608 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/software/puppet-compiler@master] catalog: use rich_data_json, rather than pson

https://gerrit.wikimedia.org/r/1072608

Change #1072608 merged by JHathaway:

[operations/software/puppet-compiler@master] catalog: use rich_data_json, rather than pson

https://gerrit.wikimedia.org/r/1072608

Change #1072617 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] pcc: bump version on workers

https://gerrit.wikimedia.org/r/1072617

Change #1072617 merged by JHathaway:

[operations/puppet@production] pcc: bump version on workers

https://gerrit.wikimedia.org/r/1072617

Change #1072628 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/software/puppet-compiler@master] fix rich data keys

https://gerrit.wikimedia.org/r/1072628

Change #1072628 merged by JHathaway:

[operations/software/puppet-compiler@master] fix rich data keys

https://gerrit.wikimedia.org/r/1072628

Change #1072631 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] pcc: bump version on workers, again :(

https://gerrit.wikimedia.org/r/1072631

Change #1072631 merged by JHathaway:

[operations/puppet@production] pcc: bump version on workers, again :(

https://gerrit.wikimedia.org/r/1072631

Change #1071962 merged by JHathaway:

[operations/puppet@production] puppet8: replace to_pson with to_json

https://gerrit.wikimedia.org/r/1071962

Change #1072593 merged by JHathaway:

[operations/puppet@production] puppet8: ensure kerberos keytab type is binary

https://gerrit.wikimedia.org/r/1072593

Change #1072768 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: ensure gpg keyring type is binary

https://gerrit.wikimedia.org/r/1072768

Change #1072769 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: ensure kerberos keytab type is binary

https://gerrit.wikimedia.org/r/1072769

Change #1072770 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: ensure dns cookie type is binary

https://gerrit.wikimedia.org/r/1072770

Change #1072773 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: ensure java ssh key type is binary

https://gerrit.wikimedia.org/r/1072773

Change #1072777 had a related patch set uploaded (by JHathaway; author: JHathaway):

[operations/puppet@production] puppet8: ensure java keystore type is binary

https://gerrit.wikimedia.org/r/1072777

Change #1072768 merged by JHathaway:

[operations/puppet@production] puppet8: ensure gpg keyring type is binary

https://gerrit.wikimedia.org/r/1072768

Change #1072769 merged by JHathaway:

[operations/puppet@production] puppet8: ensure kerberos keytab type is binary

https://gerrit.wikimedia.org/r/1072769

Change #1072770 merged by JHathaway:

[operations/puppet@production] puppet8: ensure dns cookie type is binary

https://gerrit.wikimedia.org/r/1072770

Change #1072773 merged by JHathaway:

[operations/puppet@production] puppet8: ensure java ssh key type is binary

https://gerrit.wikimedia.org/r/1072773

Change #1072777 merged by JHathaway:

[operations/puppet@production] puppet8: ensure java keystore type is binary

https://gerrit.wikimedia.org/r/1072777

jhathaway claimed this task.

All known uses of pson have been removed. However, since we cannot disable support on 7.23, I don't think there is anything left to do at the moment, so resolving.