Difference between revisions of "EJFAT UDP Single Thread Packet Sending and Receiving"
(Created page with "= Transmission between ejfat-2 and U280 on ejfat-1 (Sep 2022) = <font size="+1">We can find the NUMA node number of ejfat-2's NIC by looking at the output of:</font> <pre...") |
|||
| Line 39: | Line 39: | ||
| − | [[File: | + | [[File: UdpSingleThdSendAndReceive.png|1000px|link=|"Data Source Stream Processing"]] |
| − | |||
| − | |||
| − | |||
<font size="+2">Conclusions</font><br> | <font size="+2">Conclusions</font><br> | ||
<font size="+1">Notice how the data rate spikes when the core used to send is in the same NUMA node as the NIC itself. In addition, the faster rates always corresponded to the closer NUMA node. Setting the sender's core to get the best performance is a necessity as running the program without specifying it defaults to a low core # such as 8 or 10. The core must be specified to be in the range 80-87 in order to get the best performance. Note that running in realtime RR scheduling made the program considerably slower and uneven in performance. It would not run in the realtime FIFO mode at all.</font> | <font size="+1">Notice how the data rate spikes when the core used to send is in the same NUMA node as the NIC itself. In addition, the faster rates always corresponded to the closer NUMA node. Setting the sender's core to get the best performance is a necessity as running the program without specifying it defaults to a low core # such as 8 or 10. The core must be specified to be in the range 80-87 in order to get the best performance. Note that running in realtime RR scheduling made the program considerably slower and uneven in performance. It would not run in the realtime FIFO mode at all.</font> | ||
Revision as of 15:49, 12 September 2022
Transmission between ejfat-2 and U280 on ejfat-1 (Sep 2022)
We can find the NUMA node number of ejfat-2's NIC by looking at the output of:
cat /sys/class/net/enp193s0f1np1/device/numa_node
Which is:
10
To find out more info about the cores and NUMA node numbers of ejfat-2. Look at the output of:
numactl --hardware
Which is:
The following graphs were created by a single UDP packet sending application.
Data was sent from ejfat-2 to LB on ejfat-1 (172.19.22.241), using
./packetBlaster -p 19522 -host 172.19.22.241 -mtu 9000 -s 25000000 -b 100000 -cores <N>
in which the UDP Send buffer = 50MB and the app sent buffers of 100kB.
Conclusions
Notice how the data rate spikes when the core used to send is in the same NUMA node as the NIC itself. In addition, the faster rates always corresponded to the closer NUMA node. Setting the sender's core to get the best performance is a necessity as running the program without specifying it defaults to a low core # such as 8 or 10. The core must be specified to be in the range 80-87 in order to get the best performance. Note that running in realtime RR scheduling made the program considerably slower and uneven in performance. It would not run in the realtime FIFO mode at all.