Page MenuHomePhabricator

Create a generic network performance profile
Open, Stalled, MediumPublic

Description

There are a few places in puppet where we updated various sysctl settings in an order to optimise the network stack and other performance features similar to what we have for the caching servers. We should create a generic policy with sane defaults based on the server HW which can be applied to all production roles

Event Timeline

ayounsi renamed this task from Create a generic network proformance profile to Create a generic network performance profile.Feb 9 2021, 9:23 AM
ayounsi updated the task description. (Show Details)

Other classes in our Puppet tree which already apply some of the generic settings:

  • swift
  • profile::mediawiki::api
  • base::mysterious_sysctl
  • profile::mediawiki::common
  • profile::restbase
  • profile::phabricator::performance

I have created a starting point profile using mostly based on cacheproxy::performance below ill note where the modules listed above have difference/additions from this profile.

  • swift
'net.ipv4.tcp_syncookies'             => '0',                                 # not present in cacheproxy::performance
'net.ipv4.tcp_tw_recycle'             => '1',                                 # not present in cacheproxy::performance (no longer available in modern kernels)
'net.ipv4.netfilter.ip_conntrack_max' => '262144',              # not present in cacheproxy::performance
'net.ipv4.ip_local_port_range'        =>  [ 1024, 65535 ],      # [4001, 65534] in cacheproxy::performance
'net.core.netdev_max_backlog'         =>  30000,                  # 60000 in cacheproxy::performance
'net.ipv4.tcp_max_orphans'            => 262144,                   #  524288 in cacheproxy::performance
  • profile::mediawiki::api
'net.ipv4.local_port_range' => '22500 65535',                  #   [4001, 65534] in cacheproxy::performance
  • base::mysterious_sysctl
 'net.ipv4.local_port_range' => '22500 65535',                 # [4001, 65534] in cacheproxy::performance
'net.core.netdev_max_backlog'         =>  30000,              # 60000 in cacheproxy::performance
'net.core.somaxconn'           => 4096,                                # 16384  in cacheproxy::performance
'net.ipv4.tcp_max_syn_backlog' => 262144,                    # 524288  in cacheproxy::performance
'net.ipv4.tcp_max_tw_buckets'  => 360000,                    # 524288  in cacheproxy::performance
'net.ipv4.tcp_max_orphans'     => 262144,                      # profile::performace::ipv4_tcp_max_orphans: 524288
  • profile::mediawiki::common
  • profile::restbase
  • profile::phabricator::performance

Change 662933 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] sysctl: reject undef values

https://gerrit.wikimedia.org/r/662933

Change 662932 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] profile::performance: add a new profile for tweaking sysctl parameters

https://gerrit.wikimedia.org/r/662932

Change 662933 merged by Jbond:
[operations/puppet@production] sysctl: reject undef values

https://gerrit.wikimedia.org/r/662933

BBlack subscribed.

The swap of Traffic for Traffic-Icebox in this ticket's set of tags was based on a bulk action for all such tickets that haven't been updated in 6 months or more. This does not imply any human judgement about the validity or importance of the task, and is simply the first step in a larger task cleanup effort. Further manual triage and/or requests for updates will happen this month for all such tickets. For more detail, have a look at the extended explanation on the main page of Traffic-Icebox . Thank you!

BCornwall changed the task status from Open to Stalled.Jan 20 2023, 9:16 PM
BCornwall subscribed.

@jbond is this still desirable? If so, was the failing test holding you back from poking the tagged reviewers? Other than the test, was there anything more needed to finish this ticket?

@BCornwall thanks for reviving this. i think that this ultimately stalled as there was a questions of wether it would be usefull. from memory the main questions where

  • is there a set of universal sysctl settings that are useful across the board
  • does it make senses to try have theses settings configured in a base profile or is it better to have the settings in the application specific profile

however it seems that even if my memory is right none of this is documented anywhere so its probably worth adding some more information to this task. i also took a quick look at the CR and its still quite rough and in need of review.

@MoritzMuehlenhoff do you have any more memory on why this stalled?

Change 662932 abandoned by Jbond:

[operations/puppet@production] profile::performance: add a new profile for tweaking sysctl parameters

Reason:

https://gerrit.wikimedia.org/r/662932