We should have an authoritative list of ports that are used in our infrastructure, probably in Puppet.
Change 670788 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] service_definitions: add defined ports
this came up before as a minor ask and i used it to create a PoC which used custom types. Ultimately i abandoned that as i felt that custom types may be overkill. That said now i look at the implementation again I'm not sure its that bad and we/i could potentially simplify this by using the resource API. There is also this patch from cdanis which uses pure puppet however i think we should refresh that so that it has a similar type definition and structure to the first patch. regardless of if the implementation is in ruby or puppet
Change 670810 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] P:base: manage /etc/services file
Change 670917 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] netbase: add new module to manage /etc/services
When drafting the patch in https://gerrit.wikimedia.org/r/c/operations/puppet/+/670917/7 i came up with the following structure
type Netbase::Service = Struct[{ port => Stdlib::Port, protocols => Array[Enum['udp', 'tcp', 'ddp']], description => Optional[String[1]], aliases => Optional[Array[Pattern[/\w+/]]], }]
With this structure i made the assumption that any individual service regardless of the protocol will always be on a single port .i.e. dns and kerberos both work over udp/tcp however they use the same port number for both protocols (53 and 88 respectively). I think this assumption holds true for all IP tcp/udp protocols at least at the IANA level. however when you add in AppleTalk ddp we hit a problem with echo
$ grep ^echo /etc/services echo 7/tcp echo 7/udp echo 4/ddp # AppleTalk Echo Protocol
which in the current implementation renders a yaml object like the following
echo: proto: - tcp - udp - ddp port: 4 description: AppleTalk Echo Protocol
which is obviously wrong and im not sure what the best way to fix this is. some ideas
type Netbase::Service::IP = Struct[{ port => Stdlib::Port, protocols => Array[Enum['udp', 'tcp']], description => Optional[String[1]], aliases => Optional[Array[Pattern[/\w+/]]], }] type Netbase::Service::AppleTalk = Struct[{ port => Stdlib::Port, protocols => Enum['ddp'], description => Optional[String[1]], aliases => Optional[Array[Pattern[/\w+/]]], }] type Netbase::Service = Hash[String, Variant[Netbase::Service::AppleTalk, Netbase::Service::IP]]
For now (and as i just realised there are only 4) i think ill do the last option and we can consider dropping ddp at a later date. however i wanted to ask if we should consider supporting multiple ports per protocol as an options. From the https://wikitech.wikimedia.org/wiki/Service_ports page we already have eventgate-main, eventgate-analytics , recommendation-api and apertium which support multiple ports as such we would need to chaneg the port signature to ports => Array[Stdlib::Port]
This is a simple change however it will make other interface a bit more awkward. for instance if we had a functions netbase::service::resolve_port('ssh') it makes much more senses for this to just return Integer(22) rather then Array(Integer(22)) especially considering the 99% of all services will only ever have one port. Further we need to think how flexible we should make the data structure, should we consider supporting a protocol something like foo = 42/tcp 42/udp 4242/udp. if so we would need to ensure that the protocol was included in the ports array (now a hash) i.e. ports => Hash[Stdlib::Port, Enum['tcp,udp]' and netbase::service::resolve_port('foo') = {'foo => { ports => {42 => 'udp', 42 => 'tcp, 4242 => 'udp'}}}
Personally i think we should just decide now that we every port has to have a unique service name and the examples above get renamed to something more specific e.g. apertium 2737/tcp, apertium-tls 4737/tcp [no idea if theses make senses]. Further i would strongly urge against supporting the later foo use case as this will just make the interfaces messy.
For now (and as i just realised there are only 4) i think ill do the last option
implemented
just a note to self, wondered if it my be better to index names by port number but we have issues with the following conflicts. i think this works with the current patches however the netbase::service::port_keys probably dosn;t handle this (not sure if its something to worry about)
| udp | tcp |
| fsp | ftp |
| biff | exec |
| who | login |
| syslog | shell |
Just in case it's relevant, we use a range of ports for mariadb. Most (but not all) of them are here. Which ports are used on which hosts is determined by a combination of role + hiera values.
Change 670917 merged by Jbond:
[operations/puppet@production] netbase: add new module to manage /etc/services
Change 670788 abandoned by Jbond:
[operations/puppet@production] service_definitions: add defined ports
Reason:
imblemented with sysbase
Change 670810 abandoned by Jbond:
[operations/puppet@production] P:base: manage /etc/services file
Reason:
implemented with netbase