EJFAT UDP General Performance Considerations

From epsciwiki
Revision as of 15:33, 21 December 2023 by Timmer (talk | contribs)
Jump to navigation Jump to search

NIC queues on multi homed nodes

Scaling in the Linux Networking Stack
Contemporary NICs support multiple receive and transmit descriptor queues. On reception, a NIC can send different packets to different queues to distribute processing among CPUs. Find out how many NIC queues there are on your node by looking at the combined property:
// See how many queues there are 
sudo ethtool -l enp193s0f1np1

Effect of NIC queues on UDP transmission

In the case of ejfat nodes, there are a max of 63 queues even though there are 128 cores. It seems odd to me that there isn't 1 queue per cpu, and it does not appear to be changeable so most likely it's built into the kernel when first created.


Jumbo Frames

Retrieved from "https://wiki.jlab.org/epsciwiki/index.php?title=EJFAT_UDP_General_Performance_Considerations&oldid=4369"