Difference between revisions of "EJFAT Group Meeting Apr. 21, 2022"

From epsciwiki
Jump to navigation Jump to search
(Created page with " The meeting time is 11:00am. === Connection Info: === <div class="toccolours mw-collapsible mw-collapsed"> You can connect using [https://jlab-org.zoomgov.com/j/1610125238?p...")
 
 
(4 intermediate revisions by the same user not shown)
Line 32: Line 32:
 
<!-------------------------------------------------------------------------------------------------->
 
<!-------------------------------------------------------------------------------------------------->
 
=== Agenda: ===
 
=== Agenda: ===
−
* [[EJFAT Group Meeting Mar. 31, 2022 | Previous meeting]]
+
* [[EJFAT Group Meeting Apr. 7, 2022 | Previous meeting]]
 
*:
 
*:
 
* Situation:
 
* Situation:
Line 39: Line 39:
 
** Linux IP stack buffer size increased
 
** Linux IP stack buffer size increased
 
** Using script based LB Control Plane
 
** Using script based LB Control Plane
−
** Control Plane Gratuitous Arp cache updating (Scapy)
+
** ERSAP feed end bottleneck needs investigation; Timmer's blaster may provide relief
−
** Benchmarking
+
** EJFAT subnet
−
** Successfully Testing Calendar Epoch Advances
+
*** VLAN 937 172.19.22.0/24
−
** [https://indico.cern.ch/event/1109460/ RT2022 - April 08 submission - August 01-05 Conference]
+
*** 10Gbs NIC, <s>but '''need cables''' </s>(copper twinX will do)
 +
** DAQ Farm machines ''dafarm61 dafarm62 dafarm63 dafarm64'' currently on 129.57.29.0/24 subnet w/ additional i/f available for EJFAT subnet
 +
** [https://docs.google.com/document/d/1ssw8sye7jExtPCJVejloe8hNkyWOcxEQzVmm45xs5-w/edit#heading=h.mqilsqsmmpek Testing newly received FPGA load]
 +
*** arp, ping - working
 +
*** Port entropy field - currently broken, update pending
 +
*** MTU size restricted to max 1500, update pending
 +
*** Possibly incorrect control message responses being investigated
 
* Pending:
 
* Pending:
−
** Compute Equip.- ETA 1 June
+
** Compute Equip.- ETA 1 June - currently on track
−
** Networking Equip. - ETA 1 July
+
** Networking Equip. - ETA <s>1 July</s> 5 October
 
** Support C libraries for LB Host Control Plane
 
** Support C libraries for LB Host Control Plane
 
** ESnet smartnic open-source GitHub repo (April)
 
** ESnet smartnic open-source GitHub repo (April)
 
** ESnet private, forkable Jlab P4 and simulations GitHub repo (April)
 
** ESnet private, forkable Jlab P4 and simulations GitHub repo (April)
−
** Data_id <-> reassembly port mapping for powers-of-two ports per host
 
 
* To Do:
 
* To Do:
 
** Near Term:
 
** Near Term:
Line 63: Line 68:
 
***** Use EJFAT LB firmware that supports load balancing of multiple source/data_IDs.
 
***** Use EJFAT LB firmware that supports load balancing of multiple source/data_IDs.
 
***** EJFAT LB + RA engine performance as an event builder (i.e. source ID aggregator), a capability to send all available data_ID's with the same tick (time frame) to a single destination. NB Hall-B FT calorimeter and hodoscope setup is capable of sending up to 12 data streams.   
 
***** EJFAT LB + RA engine performance as an event builder (i.e. source ID aggregator), a capability to send all available data_ID's with the same tick (time frame) to a single destination. NB Hall-B FT calorimeter and hodoscope setup is capable of sending up to 12 data streams.   
−
*** C-based control plane
+
*** Control Plane
−
**** Feedback from Compute hosts design
+
**** Feedback from Compute hosts
 
**** Control Plane daemon for compute host
 
**** Control Plane daemon for compute host
 
** Downstream:
 
** Downstream:
 
*** [http://www.dpdk.org DPDK]
 
*** [http://www.dpdk.org DPDK]
−
*** P4 enhancements for ipv4/6 ping, arp
 
 
*** IPV6 testing
 
*** IPV6 testing
−
*** EJFAT Subnet
 
 
*** [https://www.epj-conferences.org/articles/epjconf/abs/2021/05/epjconf_chep2021_04005/epjconf_chep2021_04005.html HOSS Hall-D EJFAT  use case]
 
*** [https://www.epj-conferences.org/articles/epjconf/abs/2021/05/epjconf_chep2021_04005/epjconf_chep2021_04005.html HOSS Hall-D EJFAT  use case]
 
**** parallelize writing of raw data files
 
**** parallelize writing of raw data files
 
**** distribute raw data across multiple compute nodes for calibration skims
 
**** distribute raw data across multiple compute nodes for calibration skims
 +
**** [https://docs.google.com/presentation/d/1m3rFm-1GymYv8zGimlAjL1NmWtXVfyIQdGzhx_j_BKE/edit?usp=sharing Slides]
 +
**** [https://jeffersonlab-my.sharepoint.com/personal/bmorris_jlab_org/Documents/Microsoft%20Teams%20Chat%20Files/JLab%20Network%20-%20HallD-to-EJFAT.png Netwok Diagram]
 +
*** [https://indico.cern.ch/event/1109460/ RT2022 - August 01-05 Conference]
 
* AOT
 
* AOT
 +
<hr>
  
 
=== Minutes: ===
 
=== Minutes: ===
 
<ul>
 
<ul>
−
<li>EJFAT subnet: VLAN 937  172.19.22.0/24
+
<li>May be able to use Abbott's indra-s1 setup</li>
−
<li>DAQ Farm machines ''dafarm61 dafarm62 dafarm63 dafarm64'' currently on 129.57.29.0/24 subnet w/ additional i/f available for EJFAT subnet
+
<li>May be to use new VTP f/w with  Hall-B VTP's next week</li>
−
<ul>
+
<li>Hall-D 1 Gbs at hi-luminosity</li>
−
<li>Running CentOS 7.7
+
<li>Hall-D comms with indra-s2 (DAQ 109 subnet) require network customization</li>
−
<li>10Gbs NIC, but '''need cables''' (copper twinX will do)
 
−
</ul>
 
−
<li>New P4 to include
 
−
<ul>
 
−
<li>ARP, ping, iCMP ipv4/6
 
−
<li>ipv6 neighbor discovery TBD
 
−
</ul>
 
−
<li>ERSAP feed end bottleneck needs investigation; Timmer's blaster may provide relief
 
−
<li>Peter Nugent/LBNL: LDRD LBNL+JLAB combines JLAB streaming + LBNL compute
 
 
</ul>
 
</ul>
 
<hr>
 
<hr>

Latest revision as of 19:35, 26 April 2022

The meeting time is 11:00am.

Connection Info:

You can connect using ZoomGov Video conferencing (ID: 161 012 5238). (Click "Expand" to the right for details -->): 

Meeting URL
 https://jlab-org.zoomgov.com/j/1610125238?pwd=QnEvcjV6VFFndWZsQW15SmJKU0RJZz09&from=addon

Meeting ID
161 012 5238

Passcode
503371

Want to dial in from a phone?

Dial one of the following numbers:
US: +1 669 254 5252 or +1 646 828 7666 or +1 551 285 1373 or +1 669 216 1590 or 833 568 8864 (Toll Free)

Enter the meeting ID and passcode followed by #

Connecting from a room system?
Dial: bjn.vc or 199.48.152.152 and enter your meeting ID & passcode

Agenda:

  • Previous meeting
  • Situation:
    • Testing with End-to-end EJFAT ERSAP solution on FPGA LB
    • Jumbo Frames - indra-s2,s3, alkaid, fpga
    • Linux IP stack buffer size increased
    • Using script based LB Control Plane
    • ERSAP feed end bottleneck needs investigation; Timmer's blaster may provide relief
    • EJFAT subnet
      • VLAN 937 172.19.22.0/24
      • 10Gbs NIC, but need cables (copper twinX will do)
    • DAQ Farm machines dafarm61 dafarm62 dafarm63 dafarm64 currently on 129.57.29.0/24 subnet w/ additional i/f available for EJFAT subnet
    • Testing newly received FPGA load
      • arp, ping - working
      • Port entropy field - currently broken, update pending
      • MTU size restricted to max 1500, update pending
      • Possibly incorrect control message responses being investigated
  • Pending:
    • Compute Equip.- ETA 1 June - currently on track
    • Networking Equip. - ETA 1 July 5 October
    • Support C libraries for LB Host Control Plane
    • ESnet smartnic open-source GitHub repo (April)
    • ESnet private, forkable Jlab P4 and simulations GitHub repo (April)
  • To Do:
    • Near Term:
      • Hall-B FT calorimeter and hodoscope streaming readout test (scheduled for week of 04/10 or 04/17)
        • 2 crates for FT calorimeter, furnished with fADC250s. Total 332 channels.
        • 1 crate for the hodoscope that has 12 fADC (total 192 channels)
        • Update firmware on all 3 VTPs to send fADC aggregated data frames in the EVIO format, through UDP.
        • Design ERSAP FE application containing an actor that receives VTP frames through UDP, extracts information about the source/data ID and frame number (tick), and sends it to the EJFAT packetizing engine.
        • Stage 1 testing: single stream/data-ID
          • EJFAT LB + reassembly (RA) engine overall performance data-rate dependence.
          • EJFAT LB + RA engine performance dependence from the EJFAT packetizing engine MTU settings
        • Stage 2 testing: multi-stream/data_ID
          • Use EJFAT LB firmware that supports load balancing of multiple source/data_IDs.
          • EJFAT LB + RA engine performance as an event builder (i.e. source ID aggregator), a capability to send all available data_ID's with the same tick (time frame) to a single destination. NB Hall-B FT calorimeter and hodoscope setup is capable of sending up to 12 data streams.
      • Control Plane
        • Feedback from Compute hosts
        • Control Plane daemon for compute host
    • Downstream:
  • AOT

Minutes:

  • May be able to use Abbott's indra-s1 setup
  • May be to use new VTP f/w with Hall-B VTP's next week
  • Hall-D 1 Gbs at hi-luminosity
  • Hall-D comms with indra-s2 (DAQ 109 subnet) require network customization
Retrieved from "https://wiki.jlab.org/epsciwiki/index.php?title=EJFAT_Group_Meeting_Apr._21,_2022&oldid=1757"