Testing Load Balancer Bandwidth

Transmission between 4 senders on various ejfat nodes to the Load Balancer ejfat-1 and from there to 4 receivesr on ejfat-1(Sep 2022)

Here we test the data rate between 4 single threaded UDP senders and corresponding 4 single threaded receivers. In one test, data was sent from senders on ejfat-2 to LB on ejfat-1 (172.19.22.241). In another test, there was one sender on each of ejfat-2, 3, 4, and 5. Both showed the same behavior The following was used to send:

./packetBlaster -p 19522 -host 172.19.22.241 -mtu 9000 -s 25000000  -b 100000 -cores 80 (81,82,83

in which the UDP Send buffer = 50MB and the app sent buffers of 100kB. The receiver was run as:

./packetBlastee  -p 17750 -b 400000 -r 25000000 -cores 80

The sending and receiving threads were pinned to core #80. This is because cores 80-87 are on the same NUMA node as the NIC and perform by far the best when transferring data. When the same test was run pinning the threads to core #1 (on worst performing node) the max transfer rate was 1000 MB/s. After that packets were constantly being dropped.

The following graph shows the CPU usage of both sender and receiver as a function of the data rate.