Difference between revisions of "How to run a simulation of a control plane with ERSAP backend"
Jump to navigation
Jump to search
(36 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
− | + | =='''Start with the gRPC installation directory and define some environmental variables'''== | |
− | |||
− | |||
− | |||
− | |||
− | |||
<blockquote> | <blockquote> | ||
<pre> | <pre> | ||
export GRPC_INSTALL_DIR=/daqfs/gRPC/installation | export GRPC_INSTALL_DIR=/daqfs/gRPC/installation | ||
+ | export GRPC_JAVA_INSTALL_DIR=/daqfs/gRPC/java_installation | ||
export PATH="$GRPC_INSTALL_DIR/bin:$PATH" | export PATH="$GRPC_INSTALL_DIR/bin:$PATH" | ||
export LD_LIBRARY_PATH="$GRPC_INSTALL_DIR/lib:$LD_LIBRARY_PATH" | export LD_LIBRARY_PATH="$GRPC_INSTALL_DIR/lib:$LD_LIBRARY_PATH" | ||
Line 17: | Line 13: | ||
</blockquote> | </blockquote> | ||
− | |||
+ | =='''Running the ET related programs'''== | ||
+ | |||
+ | |||
+ | : '''Access to ET system files has been made simple by placing them into the gRPC installation''' | ||
+ | |||
+ | :: ET libraries are in $GRPC_INSTALL_DIR/lib | ||
+ | :: ET includes are in $GRPC_INSTALL_DIR/include | ||
+ | :: ET executables are in $GRPC_INSTALL_DIR/bin | ||
+ | :: ET jar file is in $GRPC_JAVA_INSTALL_DIR/jars | ||
+ | |||
+ | |||
+ | : '''Run the ET system as a fifo. Use 1000, 150kB buffers''' | ||
<blockquote> | <blockquote> | ||
<pre> | <pre> | ||
− | + | cd $GRPC_INSTALL_DIR/bin | |
− | + | ./et_start_fifo -f /tmp/fifoEt -d -s 150000 -n 1 -e 1000 | |
− | + | </pre> | |
+ | </blockquote> | ||
+ | |||
+ | : '''On the same node, after running the ET system, run a java-based consumer of ET system buffers with a delay of 1 microsec between fifo gets (ERSAP simulator). Compiled with java 8.''' | ||
+ | <blockquote> | ||
+ | <pre> | ||
+ | java -cp $GRPC_JAVA_INSTALL_DIR/jars/* org.jlab.coda.et.apps.ErsapFifoConsumer -f /tmp/fifoEt -v -d 1 | ||
+ | </pre> | ||
+ | </blockquote> | ||
+ | |||
+ | |||
+ | =='''Running the ejfat repository's gRPC enabled programs:'''== | ||
+ | |||
+ | : '''Run the following programs in this order:''' | ||
+ | # reassembler of backend data and sender of info to control plane | ||
+ | # sender of data to the backend | ||
+ | # simulator of the control plane | ||
+ | |||
+ | |||
+ | : '''Compile the code in the ersap branch of the ejfat repository''' | ||
+ | <blockquote> | ||
+ | <pre> | ||
+ | cd <ejfat dir>/build | ||
+ | git pull | ||
+ | rm CMakeCache.txt | ||
+ | cmake .. -DBUILD_ET=1 -DBUILD_GRPC=1 | ||
+ | make | ||
</pre> | </pre> | ||
</blockquote> | </blockquote> | ||
− | : ''' | + | : '''Run the data reassembler with PID loop and communication back to control plane''' |
<blockquote> | <blockquote> | ||
<pre> | <pre> | ||
+ | cd <ejfat dir>/build/bin | ||
+ | ./packetBlasteeEtFifo -p 17750 -b 4000000 -r 25000000 -cores 82 -f /tmp/fifoEt | ||
+ | </pre> | ||
+ | </blockquote> | ||
+ | : '''Run the data sender''' | ||
+ | <blockquote> | ||
+ | <pre> | ||
+ | cd <ejfat dir>/build/bin | ||
+ | ./clasBlaster -f /daqfs/java/clas_005038.1231.hipo -host 172.19.22.244 -p 19522 -mtu 9000 -s 25000000 -cores 80 | ||
</pre> | </pre> | ||
</blockquote> | </blockquote> | ||
− | + | : '''Run the simulated control plane''' | |
− | < | + | <blockquote> |
+ | <pre> | ||
+ | cd <ejfat dir>/build/bin | ||
+ | ./control_plane -a 127.0.0.1 -p 50051 | ||
+ | </pre> | ||
+ | </blockquote> | ||
− | + | ||
+ | =='''The heart of the simulated control plane is a very simple loop'''== | ||
<blockquote> | <blockquote> | ||
<pre> | <pre> | ||
+ | LoadBalancerControlClient client(ipAddr, port, "carlsControlPlane"); | ||
+ | while (true) { | ||
+ | |||
+ | int32_t err = client.GetState(); | ||
+ | if (err != 0) { | ||
+ | std::cerr << "Error calling GetState()" << std::endl; | ||
+ | } | ||
+ | client.printBackendState(); | ||
+ | |||
+ | // Delay 2 seconds between printouts | ||
+ | std::this_thread::sleep_for(std::chrono::seconds(2)); | ||
+ | } | ||
</pre> | </pre> | ||
</blockquote> | </blockquote> | ||
+ | |||
+ | |||
</font> | </font> |
Latest revision as of 15:36, 4 January 2023
Start with the gRPC installation directory and define some environmental variables
export GRPC_INSTALL_DIR=/daqfs/gRPC/installation export GRPC_JAVA_INSTALL_DIR=/daqfs/gRPC/java_installation export PATH="$GRPC_INSTALL_DIR/bin:$PATH" export LD_LIBRARY_PATH="$GRPC_INSTALL_DIR/lib:$LD_LIBRARY_PATH"
- Access to ET system files has been made simple by placing them into the gRPC installation
- ET libraries are in $GRPC_INSTALL_DIR/lib
- ET includes are in $GRPC_INSTALL_DIR/include
- ET executables are in $GRPC_INSTALL_DIR/bin
- ET jar file is in $GRPC_JAVA_INSTALL_DIR/jars
- Run the ET system as a fifo. Use 1000, 150kB buffers
cd $GRPC_INSTALL_DIR/bin ./et_start_fifo -f /tmp/fifoEt -d -s 150000 -n 1 -e 1000
- On the same node, after running the ET system, run a java-based consumer of ET system buffers with a delay of 1 microsec between fifo gets (ERSAP simulator). Compiled with java 8.
java -cp $GRPC_JAVA_INSTALL_DIR/jars/* org.jlab.coda.et.apps.ErsapFifoConsumer -f /tmp/fifoEt -v -d 1
Running the ejfat repository's gRPC enabled programs:
- Run the following programs in this order:
- reassembler of backend data and sender of info to control plane
- sender of data to the backend
- simulator of the control plane
- Compile the code in the ersap branch of the ejfat repository
cd <ejfat dir>/build git pull rm CMakeCache.txt cmake .. -DBUILD_ET=1 -DBUILD_GRPC=1 make
- Run the data reassembler with PID loop and communication back to control plane
cd <ejfat dir>/build/bin ./packetBlasteeEtFifo -p 17750 -b 4000000 -r 25000000 -cores 82 -f /tmp/fifoEt
- Run the data sender
cd <ejfat dir>/build/bin ./clasBlaster -f /daqfs/java/clas_005038.1231.hipo -host 172.19.22.244 -p 19522 -mtu 9000 -s 25000000 -cores 80
- Run the simulated control plane
cd <ejfat dir>/build/bin ./control_plane -a 127.0.0.1 -p 50051
The heart of the simulated control plane is a very simple loop
LoadBalancerControlClient client(ipAddr, port, "carlsControlPlane"); while (true) { int32_t err = client.GetState(); if (err != 0) { std::cerr << "Error calling GetState()" << std::endl; } client.printBackendState(); // Delay 2 seconds between printouts std::this_thread::sleep_for(std::chrono::seconds(2)); }