Page MenuHomePhabricator

Identify and upstream useful functions from wmflib
Closed, ResolvedPublic

Description

WARNING: puppetlabs has droped support for puppet < 6.0.0 as of puppetlabs-stdlib 7.0.0

We have a number of functions in wmflib which would be genrally usefull for inclusion in stdlib. We should identify theses and make an effort to upstream them

Types

`WmflibStdlibnotes
Wmflib::HttpStatusStdlib::HttpStatusPR 1132, Release 6.6.0
Wmflib::Http::Method
Wmflib::Enable_Service
Wmflib::Ensure
Wmflib::Php_version(move to php module)
Wmflib::EmailStdlib::EmailPR 1160

Functions

WmflibStdlibnotes
array_concatDropped
conflictsDroped
conftoolDroped
cron_splayIm not sure this would fit stdlib but perhaps puppet-ext, either way we should port it to the newer api or puppet DSL
ensure_directorystdlib::ensure($ensure, 'directory')PR 1150, Release 6.6.0
ensure_linkstdlib::ensure($ensure, 'link')PR 1150, Release 6.6.0
ensure_mounted
ensure_servicestdlib::ensure($ensure, 'service')PR 1150, Release 6.6.0
get_clustersDont think this is usefull to stdlib but we should port this to the newer puppet api
hash_deselect_re$hash.filter l$itemsl { $items[0] =~ /^a/ }Dropped - (if copy pasting his code the pipe is an lower case 'L')
hash_select_re$hash.filter l$itemsl { $items[0] !~ /^a/ }Dropped - (if copy pasting this code the pipe is an lower case 'L')
htpasswdapache::pw_hash allready in puppetlabs-apachewonder if we should import apache::pw_hash to our http module
iniDefinetly a candidate to upstrem probably as a ruby function in the newer api
init_templateCurrently used by systemd/systemv/upstrat_template
ipresolveThe initial plan for this was that it could cache for the life of the daemon however it currently only caches for the life of the catalouge compilation. we should look to fix this but also investigate how much performance boost this gives vs e.g. installing a dns cache on the puppet masters.
ordered_jsoncan probably use to_json may need an upstream patch to sort keys but i suspect the order is consodent among puppet runs
ordered_yamlcan probably use to_yaml may need an upstream patch to sort keys but i suspect the order is consodent among puppet runs
php_inishould move to php function
puppet_ssldir$facts['puppet_config']['ssldir']
require_packageensure_packages
requires_realmshould port to puppet dsl and possibly change to require_domain
secret
shell_exportsDropped
ssl_ciphersuiteWill drop or replace with ssh module
systemd_templatemove to systemd (along with base::service_unit)?
sysvinit_templateDroped
to_millisecondsDropped
to_secondsWill Dropped
upstart_templateDroped

Details

Related Changes in Gerrit:
Show related patches Customize query in gerrit

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
Duplicatejbond
Resolvedjbond
Resolvedjbond
DeclinedNone
Resolvedjbond
Resolvedjbond
ResolvedDzahn
ResolvedSLyngshede-WMF
ResolvedLadsgroup
Resolved Zabe
ResolvedNone
OpenNone
Resolvedelukey
Resolvedelukey
Resolvedfaidon
OpenNone
Resolvedfaidon
Resolvedherron
Resolvedherron
ResolvedAndrew
Resolvedfgiunchedi
DeclinedNone
OpenNone
Resolvedjcrespo
ResolvedNone
Resolvedelukey
ResolvedNone
ResolvedDzahn
Resolved ema
ResolvedMoritzMuehlenhoff
ResolvedPRODUCTION ERRORCatrope
ResolvedNone
Resolvedelukey
DuplicateNone
ResolvedNone
ResolvedNone
ResolvedDzahn
Resolvedfaidon
DuplicateNone
Resolvedfgiunchedi
DeclinedNone
DeclinedNone
DeclinedNone
ResolvedNone
Resolvedelukey
Resolvedandrea.denisse
ResolvedNone
Resolvedfgiunchedi
ResolvedJoe
ResolvedBBlack
Resolvedfgiunchedi
DuplicateNone
Resolvedelukey
Declinedfaidon
ResolvedMoritzMuehlenhoff
Resolvedfgiunchedi
Resolvedcolewhite
ResolvedPRODUCTION ERRORjcrespo
Resolved Gilles
Resolved Gilles
Resolvedfgiunchedi
ResolvedNone
DuplicateNone
Resolved chasemp
Resolvedjijiki
ResolvedJoe
ResolvedAndrew
Resolvedjcrespo
Resolved mmodell
ResolvedNone
Declinedaaron
ResolvedMarostegui
Resolvedjbond
Resolved GTirloni
Resolvedelukey
ResolvedMoritzMuehlenhoff
Resolvedjbond
Resolvedfgiunchedi
ResolvedVolans
ResolvedArielGlenn
Resolvedaaron
ResolvedJMeybohm
Resolved dpifke
DuplicateNone
Resolvedcolewhite
Resolved chasemp
DeclinedNone
ResolvedAndrew
Resolvedfgiunchedi
ResolvedBTullis
ResolvedJelto
ResolvedVgutierrez
OpenNone
OpenNone
DeclinedNone
OpenNone
OpenNone
OpenNone
ResolvedDzahn
Resolvedjbond
DuplicateNone
ResolvedBUG REPORTjhathaway
Resolvedjbond
Openjhathaway

Event Timeline

jbond triaged this task as Medium priority.

A new functions (stdlib::ensure) has now been added to sddlib as such we can replace our ensure_{service,directory} functions with this new interface

Stdlib now has Stdlib::HTTPStatus types as such we should update our code base to use them instad of Wmflib::HttpStatus

Aklapper renamed this task from Identify and upstream usefull fuinctions from wmflib to Identify and upstream useful functions from wmflib.Feb 3 2021, 12:43 PM

Change 661372 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib: drop ensure_link in favour of stdlib::ensure

https://gerrit.wikimedia.org/r/661372

Change 661368 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib: drop ensure_directory in favour of stdlib::ensure

https://gerrit.wikimedia.org/r/661368

Change 661367 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib: drop ensure_service in favour of stdlib::ensure

https://gerrit.wikimedia.org/r/661367

Change 661368 merged by Jbond:
[operations/puppet@production] wmflib: drop ensure_directory in favour of stdlib::ensure

https://gerrit.wikimedia.org/r/661368

Wmflib::Php_version is probably a bit specific to go to stdlib but we should move it to the php module.

Change 661414 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] nutcracker: drop use of to_milliseconds function

https://gerrit.wikimedia.org/r/661414

Change 661415 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib: drop to_seconds and to_milliseconds

https://gerrit.wikimedia.org/r/661415

Change 640480 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] (WIP) ssl: new ssl module intialy planned to replace ssl_ciphersuite()

https://gerrit.wikimedia.org/r/640480

Change 661689 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib: Email type

https://gerrit.wikimedia.org/r/661689

Change 661367 merged by Jbond:
[operations/puppet@production] wmflib: drop ensure_service in favour of stdlib::ensure

https://gerrit.wikimedia.org/r/661367

jbond updated the task description. (Show Details)
jbond updated the task description. (Show Details)
jbond updated the task description. (Show Details)
jbond updated the task description. (Show Details)

Change 661689 merged by Jbond:
[operations/puppet@production] wmflib: Email type

https://gerrit.wikimedia.org/r/661689

Change 661372 merged by Jbond:
[operations/puppet@production] wmflib: drop ensure_link in favour of stdlib::ensure

https://gerrit.wikimedia.org/r/661372

Change 661761 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] puppetmaster: drop shell_exports function

https://gerrit.wikimedia.org/r/661761

Change 661762 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib: drop shell_export function

https://gerrit.wikimedia.org/r/661762

Change 661761 merged by Jbond:
[operations/puppet@production] puppetmaster: drop shell_exports function

https://gerrit.wikimedia.org/r/661761

Change 661762 merged by Jbond:
[operations/puppet@production] wmflib: drop shell_export function

https://gerrit.wikimedia.org/r/661762

Change 661769 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] varnish::instance: drop use of array_concat

https://gerrit.wikimedia.org/r/661769

Change 661773 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] P:ntp: drop use of array_concat

https://gerrit.wikimedia.org/r/661773

Change 661780 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] scap::target: drop array_concat

https://gerrit.wikimedia.org/r/661780

Change 661785 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] prometheus: update profiles to drop array_concat

https://gerrit.wikimedia.org/r/661785

@BBlack im looking at the array_conact function and trying to decide if its worth porting to the newer puppet API or dropping. I have looked at the current use cases and think we can achieve the same functionality with puppet bulletins. I think there are se cases where this function would improve readability but not sure that applies currently and my gut feeling, as well as looking at the date this was committed, leads me to think the bulletins i have used to update the current use cases probably didn't exist when this function was introduced but wanted to ping you to make sure i wasn't missing something.

jbond updated the task description. (Show Details)

Change 661794 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib: drop conflicts method

https://gerrit.wikimedia.org/r/661794

Change 661795 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib: drop conftool funtion

https://gerrit.wikimedia.org/r/661795

Change 661780 merged by Jbond:
[operations/puppet@production] scap::target: drop array_concat

https://gerrit.wikimedia.org/r/661780

Change 661785 merged by Jbond:
[operations/puppet@production] prometheus: update profiles to drop array_concat

https://gerrit.wikimedia.org/r/661785

Change 661795 merged by Jbond:
[operations/puppet@production] wmflib: drop conftool funtion

https://gerrit.wikimedia.org/r/661795

Change 661794 merged by Jbond:
[operations/puppet@production] wmflib: drop conflicts method

https://gerrit.wikimedia.org/r/661794

Change 661909 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib:: function to replace get_clusters in puppet DSL

https://gerrit.wikimedia.org/r/661909

Change 661909 abandoned by Jbond:
[operations/puppet@production] wmflib:: function to replace get_clusters in puppet DSL

Reason:
Do to the comment below and the imutable nature of puppet i think this is still better to do in ruby

https://gerrit.wikimedia.org/r/661909

I had a look at porting get_clusters to the puppet DSL but i think its better to keep this one in ruby, we should still port it to the newer api though

Change 661910 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] wmflib: drop hash(de)select_re functions as puppet has filter now

https://gerrit.wikimedia.org/r/661910

Change 661910 merged by Jbond:
[operations/puppet@production] wmflib: drop hash(de)select_re functions as puppet has filter now

https://gerrit.wikimedia.org/r/661910

Change 661917 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] base::service_unit: drop support for sysV and upstart init scripts

https://gerrit.wikimedia.org/r/661917

Change 661917 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] base::service_unit: drop support for sysV and upstart init scripts

https://gerrit.wikimedia.org/r/661917

Change 661917 merged by Jbond:
[operations/puppet@production] base::service_unit: drop support for sysV and upstart init scripts

https://gerrit.wikimedia.org/r/661917

Change 661414 merged by Jbond:
[operations/puppet@production] nutcracker: drop use of to_milliseconds function

https://gerrit.wikimedia.org/r/661414

Change 661415 merged by Jbond:
[operations/puppet@production] wmflib: drop to_seconds and to_milliseconds

https://gerrit.wikimedia.org/r/661415

Change 661773 merged by Jbond:
[operations/puppet@production] P:ntp: drop use of array_concat

https://gerrit.wikimedia.org/r/661773

Change 661769 merged by Ema:
[operations/puppet@production] varnish::instance: drop use of array_concat

https://gerrit.wikimedia.org/r/661769

Change 675750 had a related patch set uploaded (by Jbond; author: John Bond):
[operations/puppet@production] wmflib: drop array_concat function

https://gerrit.wikimedia.org/r/675750

Change 675750 merged by Jbond:
[operations/puppet@production] wmflib: drop array_concat function

https://gerrit.wikimedia.org/r/675750

jbond updated the task description. (Show Details)
jbond lowered the priority of this task from Medium to Low.Jun 12 2023, 4:55 PM
jbond edited projects, added Puppet-Core; removed Puppet.

most of the useful functions and types have been upstream and we will try to include the style guide