How to setup ejfat nodes

From epsciwiki
Revision as of 18:59, 15 December 2023 by Timmer (talk | contribs)
Jump to navigation Jump to search

  1. Install BOOST with:
    sudo apt-get install lib boost-all-dev
  2. Install ZMQ with:
    sudo apt install libzmq3-dev
  3. For ERSAP install protobuf:
    sudo apt install libprotobuf-dev protobuf-compiler
  4. For pip (python installer for python) install:
    sudo apt install pip
  5. For prometheus:
    1. installation:
      sudo apt install prometheus
      sudo apt install prometheus-node-exporter
    2. python client libraries:
      pip install prometheus-client
      pip install prometheus-api-client
    3. exporter start up:
      sudo systemctl start prometheus-node-exporter
      sudo systemctl enable prometheus-node-exporter
  6. For increased UDP buffer & queue sizes:
    1. For the long term write the following lines into the sysctl.conf file:
      net.core.rmem_max=25000000
      net.core.wmem_max=25000000
      net.core.netdev_max_backlog=65536
    2. To make it immediately effective, execute:
      sudo sysctl -w net.core.rmem_max=25000000
      sudo sysctl -w net.core.wmem_max=25000000
      sudo sysctl -w net.core.netdev_max_backlog=65536
      ifconfig <ifname> txqueuelen 10000
  7. To mount /daqfs:
    1. sudo mkdir /daqfs
    2. edit /etc/fstab to include the line:
      dubhe:/raid/daqfs          nfs     rw,bg           0       0
    3. sudo apt-get install nfs-common
    4. sudo mount /daqfs
  8. To change one's home directory to /daqfs/home/myUserName:
    1. Edit the /etc/passwd file to include a line something like:
      myUserName:x:2101:1208:John Doe,,,:/daqfs/home/myUserName:/bin/bash
      where 2101 is user id and 1208 is epsci group. Hint: set your user id and group ids on the ejfat nodes to be the same as it is on the computer center machines or there will be an issue when you access files served by computer center like /daqfs/home or /group/...
  9. When sending UDP packets:
    • UDP Fragmentation
    • Unless you know your network can handle jumbo frames, break up your data into chunks of 1400 bytes. This is because you don't want to fragment your packets on high speed networks. On fast networks the 16-bit packet-id can overflow within the reassembly timeframe and - if the checksum matches or is not set - fragments of different packets could be reassembled.