Difference between revisions of "EJFAT UDP General Performance Considerations"
Jump to navigation
Jump to search
| Line 1: | Line 1: | ||
<font size="+2"> | <font size="+2"> | ||
| − | === | + | === NIC queues on multi homed nodes === |
| − | [https://www.kernel.org/doc/html/latest/networking/scaling.html Scaling in the Linux Networking Stack] | + | : [https://www.kernel.org/doc/html/latest/networking/scaling.html Scaling in the Linux Networking Stack] |
: Contemporary NICs support multiple receive and transmit descriptor queues. On reception, a NIC can send different packets to different queues to distribute processing among CPUs. Find out how many NIC queues there are on your node by looking at the '''combined''' property: | : Contemporary NICs support multiple receive and transmit descriptor queues. On reception, a NIC can send different packets to different queues to distribute processing among CPUs. Find out how many NIC queues there are on your node by looking at the '''combined''' property: | ||
| Line 13: | Line 13: | ||
</pre> | </pre> | ||
</blockquote> | </blockquote> | ||
| + | |||
| + | === Effect of NIC queues on UDP transmission === | ||
| + | |||
: In the case of ejfat nodes, there are a max of 63 queues even though there are 128 cores. It seems odd to me that there isn't 1 queue per cpu, and it does not appear to be changeable so most likely it's built into the kernel when first created. | : In the case of ejfat nodes, there are a max of 63 queues even though there are 128 cores. It seems odd to me that there isn't 1 queue per cpu, and it does not appear to be changeable so most likely it's built into the kernel when first created. | ||
Revision as of 15:33, 21 December 2023
NIC queues on multi homed nodes
- Contemporary NICs support multiple receive and transmit descriptor queues. On reception, a NIC can send different packets to different queues to distribute processing among CPUs. Find out how many NIC queues there are on your node by looking at the combined property:
// See how many queues there are sudo ethtool -l enp193s0f1np1
Effect of NIC queues on UDP transmission
- In the case of ejfat nodes, there are a max of 63 queues even though there are 128 cores. It seems odd to me that there isn't 1 queue per cpu, and it does not appear to be changeable so most likely it's built into the kernel when first created.
Jumbo Frames