Testing Load Balancer Bandwidth

Transmission between 4 senders on various ejfat nodes to the Load Balancer on ejfat-1 and from there to 4 receivers on ejfat-1 (Sep 2022)

Here we measure the data rate between 4 single threaded UDP senders and corresponding 4 single threaded receivers. In one test, data was sent from all senders on ejfat-2 to the LB on ejfat-1 (172.19.22.241). In another test, there was one sender on each of ejfat-2, 3, 4, and 5. Both showed the same behavior The following was used to send:

    ./packetBlaster -p 19522 -host 172.19.22.241 -mtu 9000 -s 25000000  -b 100000 -byterate 2940000000 -cores 80 (81,82,83) -e 0 (1,2,3) -id 0 (1,2,3)

in which the UDP Send buffer = 50MB and the app sent buffers of 100kB. Quantities in parenthesis are substituted for one of the 4 invocations. The receivers were all run on ejfat-1 as:

    ./packetBlastee  -p 17750 (1,2,3) -b 400000 -r 25000000 -cores 80 (81,82,83)

The sending and receiving threads were pinned to core numbers in the 80-87 range since they are on the same NUMA node as the NIC and therefore allow top performance. The result of this test is that we had the following average total byte rates for the receivers

    2933, 2929, 2929, 2929 MB/sec

Adding all rates and multiplying by 8 bits/byte gives:

    93.8 Gb/sec

This rate was sustainable without packet drops for about 1.5 minutes. Completely eliminating packet drops requires a lower throughput.