Welcome to my notebook! I’m a Site Reliability Engineer who is curious, loves to learn and discover the nature of things. Zola 2022-04-20T00:00:00+00:00 https://realmofchaos.xyz/tags/linux/atom.xml 2022-04-20T00:00:00+00:00 2022-04-20T00:00:00+00:00 Thomas Cuthbert https://realmofchaos.xyz/tech/networking/ <h2 id="linux-tcp-tuning">Linux TCP tuning</h2> <p>These are example Netflix TCP kernel parameters.</p> <pre class="giallo" style="color: #F8F8F2; background-color: #272822;"><code data-lang="plain"><span class="giallo-l"><span>net.core.default_qdisc = fq</span></span> <span class="giallo-l"><span>net.core.netdev_max_backlog = 5000</span></span> <span class="giallo-l"><span>net.core.rmem_max = 16777216</span></span> <span class="giallo-l"><span>net.core.somaxconn = 1024</span></span> <span class="giallo-l"><span>net.core.wmem_max = 16777216</span></span> <span class="giallo-l"><span>net.ipv4.ip_local_port_range = 10240 65535</span></span> <span class="giallo-l"><span>net.ipv4.tcp_abort_on_overflow = 1</span></span> <span class="giallo-l"><span>net.ipv4.tcp_congestion_control = bbr</span></span> <span class="giallo-l"><span>net.ipv4.tcp_max_syn_backlog = 8192</span></span> <span class="giallo-l"><span>net.ipv4.tcp_rmem = 4096 12582912 16777216</span></span> <span class="giallo-l"><span>net.ipv4.tcp_slow_start_after_idle = 0</span></span> <span class="giallo-l"><span>net.ipv4.tcp_syn_retries = 2</span></span> <span class="giallo-l"><span>net.ipv4.tcp_tw_reuse = 1</span></span> <span class="giallo-l"><span>net.ipv4.tcp_wmem = 4096 12582912 16777216</span></span></code></pre> <p>These are some examples I came up with for an apache2 site running moin via wsgi.</p> <pre class="giallo" style="color: #F8F8F2; background-color: #272822;"><code data-lang="plain"><span class="giallo-l"><span># WSGIDaemonProcess moin processes=12 threads=1 maximum-requests=5000 queue-timeout=20</span></span> <span class="giallo-l"><span>net.core.somaxconn = 256 # observed max number of requests on the scoreboard</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span># consider being more aggressive with flaky connections</span></span> <span class="giallo-l"><span># try this first</span></span> <span class="giallo-l"><span>#net.ipv4.tcp_slow_start_after_idle = 0</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span># aggressively reap flakey connections. avoid queue saturation from slow/flakey</span></span> <span class="giallo-l"><span># clients</span></span> <span class="giallo-l"><span>#net.ipv4.tcp_fin_timeout = 5</span></span> <span class="giallo-l"><span>#net.ipv4.tcp_syn_retries = 2</span></span> <span class="giallo-l"><span>#net.ipv4.tcp_synack_retries = 2</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span># then maybe this</span></span> <span class="giallo-l"><span>#net.ipv4.tcp_tw_reuse = 1 # this should only be enabled if we are seeing a high rate of time-wait connections</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span># needs further investigation, but potentially some of the below could be used</span></span> <span class="giallo-l"><span>#net.core.rmem_max = 16777216</span></span> <span class="giallo-l"><span>#net.core.wmem_max = 16777216</span></span> <span class="giallo-l"><span>#net.ipv4.tcp_rmem = 4096 12582912 16777216</span></span> <span class="giallo-l"><span>#net.ipv4.tcp_wmem = 4096 12582912 16777216</span></span> <span class="giallo-l"><span>#net.ipv4.tcp_mem = 1638400 1638400 1638400</span></span> <span class="giallo-l"><span># /etc/systemd/system/apache2.service.d/prlimits.conf</span></span> <span class="giallo-l"><span>#[Service]</span></span> <span class="giallo-l"><span>#LimitNOFILE=</span></span> <span class="giallo-l"><span>#LimitNOFILE=infinity</span></span></code></pre>