Difference between revisions of "EJFAT Group Meeting Nov. 16, 2023"

From epsciwiki
Jump to navigation Jump to search
(Created page with "The meeting time is 11:00am Eastern/USA. === Connection Info: === <div class="toccolours mw-collapsible mw-collapsed"> You can connect using [ https://jlab-org.zoomgov.com/j/...")
 
 
(One intermediate revision by the same user not shown)
Line 59: Line 59:
 
## Data Compressibility Studies using Hall B/D sample data
 
## Data Compressibility Studies using Hall B/D sample data
 
## '''Ready to supply up to 200 Gbps to EJFAT switch'''
 
## '''Ready to supply up to 200 Gbps to EJFAT switch'''
−
# [https://ieeexplore.ieee.org/document/10046405 RT2022 Paper]: Submission Process Complete
+
# Demo Ready EJFAT Instance
−
# [https://jeffersonlab.sharepoint.com/:b:/r/sites/SciComp/Shared%20Documents/EPSCI/EJFAT/EJFAT_ACAT_2022_QL_sub.pdf?csf=1&web=1&e=dR566P ACAT2022 Paper]: - submitted
+
## Live Session?
−
# [https://jeffersonlab.sharepoint.com/:b:/r/sites/SciComp/Shared%20Documents/EPSCI/EJFAT/chep2023_proceedings-ejfat.pdf?csf=1&web=1&e=VpLBVA CHEP 2023 Conference Publication]
+
### Check with Wenji Wu on demo format at SC23 (Amitoj)
 +
## Emulator ?
 +
## '''Recorded Session'''
 +
# EJFAT Operational Status Board -> '''Prometheus Reporting'''
 
# EJFAT Phase II
 
# EJFAT Phase II
 
##  Implementation details in the DAOS gateway.   
 
##  Implementation details in the DAOS gateway.   
 
### Specially when to keep track of how the FPGA would DMA event data cells in the future if it was a SmartNIC card. ( Cissie )
 
### Specially when to keep track of how the FPGA would DMA event data cells in the future if it was a SmartNIC card. ( Cissie )
 
### Connection Strategy to DAOS - '''Infiniband  - no room for additional PCIe cards in ejfat machines'''
 
### Connection Strategy to DAOS - '''Infiniband  - no room for additional PCIe cards in ejfat machines'''
 +
### "We just '''pulled the ConnectX-6 NIC of daosfs02​''' out and insert it to daosfs01​ and rearrange the NVMe SSD slots, so now daosfs01​ has 2 physical IB cards and can run 2 true engines with each CPU socket hosting one engine. (Amitoj)"
 
##  Flow Control
 
##  Flow Control
 
##  Progress of multi FPGA and multi virtual LB control plane sw.  ( Derek ) '''currently:''' small features like authentication etc..
 
##  Progress of multi FPGA and multi virtual LB control plane sw.  ( Derek ) '''currently:''' small features like authentication etc..
Line 77: Line 81:
 
### In a pinch one can use the 2x A100 GPUs in the NVIDIA Bluefield2 DPU server (hostname: nvidarm)
 
### In a pinch one can use the 2x A100 GPUs in the NVIDIA Bluefield2 DPU server (hostname: nvidarm)
 
### Initially purchase one of each GPU: NVIDIA, INTEL, AMD so we can compare performance across all 3 flavors.
 
### Initially purchase one of each GPU: NVIDIA, INTEL, AMD so we can compare performance across all 3 flavors.
−
# Demo Ready EJFAT Instance
 
−
## Live Session?
 
−
### Check with Wenji Wu on demo format at SC23 (Amitoj)
 
−
## Emulator ?
 
−
## '''Recorded Session'''
 
−
# EJFAT Operational Status Board -> '''Prometheus Reporting'''
 
 
# Resources:
 
# Resources:
 
## [https://jeffersonlab.sharepoint.com/sites/HPDF HPDF]
 
## [https://jeffersonlab.sharepoint.com/sites/HPDF HPDF]
 
# AOT
 
# AOT
 
<hr>
 
<hr>

Latest revision as of 15:28, 16 November 2023

The meeting time is 11:00am Eastern/USA.

Connection Info:

You can connect using [ https://jlab-org.zoomgov.com/j/1611828967?pwd=UVVCS0pUVW5FMlphT0lRQXdoQ0o4Zz09&from=addon ZoomGov Video conferencing (ID: 161 012 5238)]. (Click "Expand" to the right for details -->): 

Meeting URL
 https://jlab-org.zoomgov.com/j/1611828967

Meeting ID
161 182 8967

Passcode
570041

Want to dial in from a phone?

Dial one of the following numbers:
US: +1 669 254 5252 or +1 646 828 7666 or +1 551 285 1373 or +1 669 216 1590 or 833 568 8864 (Toll Free)

Enter the meeting ID and passcode followed by #

Connecting from a room system?
Dial: bjn.vc or 199.48.152.152 and enter your meeting ID & passcode


Agenda:

  1. Previous meeting
  2. Announcements:
  3. NERSC Test Development:
    1. Data Source:
      1. JLAB, CLAS12, pre-triggered events - 1 channel
      2. Front End Packetizer pending mods for Tick-sync msg to CP - UDP packet to port on CP Host
    2. Data Sink:
      1. Perlmutter
      2. ERSAP
    3. Networking for Test
      1. Currently 2 x 10 Gbps for JLab/L3 VPN - will expand to 200Gbps in near fututure - LB FW capable of this?
    4. JLab Preps
      1. Standing up second JLab LB instance
      2. Currently debugging test-harness set-up at NESRC/Perlmutter
    5. Test Plans - JLab, ESnet, NERSC:
  4. Test with Oak Ridge (similar to NERSC) - Shankar, Mallikarjun (Arjun) <shankarm@ornl.gov>
  5. Hall B CLAS12 detector streaming test
    1. Switch 7050 is expected to arrive some time around October; we have already transceivers, short cables and patch panel to connect up to 32 VTPs to it using two 10GBit links per VTP
    2. Fiber installation between hallb forward carriage and hallb counting room should be done this summer, will be enough for 24 VTPs using two 10GBit links per VTP
    3. We have only one fiber between hallb counting room and counting house second floor available right now, will order more fibers installation, may take several months
    4. There are several available fibers between counting house second floor and computer center (like 6), we can use a couple of them for our test
    5. Summary: sometime in October, we should have 48 10GBit links from 24 VTPs connected to the switch in hallb counting room, with that switch connected to computer center by 2x100GBit links
    6. Need to develop CONOPS with Streaming group (Abbott)
    7. SRO RTDP LDRD - need configuration for Spring '24 test with EJFAT
    8. Data Compressibility Studies using Hall B/D sample data
    9. Ready to supply up to 200 Gbps to EJFAT switch
  6. Demo Ready EJFAT Instance
    1. Live Session?
      1. Check with Wenji Wu on demo format at SC23 (Amitoj)
    2. Emulator ?
    3. Recorded Session
  7. EJFAT Operational Status Board -> Prometheus Reporting
  8. EJFAT Phase II
    1. Implementation details in the DAOS gateway.
      1. Specially when to keep track of how the FPGA would DMA event data cells in the future if it was a SmartNIC card. ( Cissie )
      2. Connection Strategy to DAOS - Infiniband - no room for additional PCIe cards in ejfat machines
      3. "We just pulled the ConnectX-6 NIC of daosfs02​ out and insert it to daosfs01​ and rearrange the NVMe SSD slots, so now daosfs01​ has 2 physical IB cards and can run 2 true engines with each CPU socket hosting one engine. (Amitoj)"
    2. Flow Control
    3. Progress of multi FPGA and multi virtual LB control plane sw. ( Derek ) currently: small features like authentication etc..
    4. Progress of FPGA architecture ( Peter and Jonathan )
    5. Progress of finalizing a reassembly frame format (subordinate to 4.) ( Carl / Stacey )
    6. Progress on software development for NVIDIA Bluefield2 DPU data steering from NIC to GPU memory ( Amitoj/Cissie )
    7. Progress on DAOS file-server OS and filesystem installation ( Amitoj/Cissie )
      1. Install Infiniband NIC in EJFAT servers to connect to DAOS Infiniband Switch.
        1. Need to check available open PCIe slots on existing EJFAT servers (Michael)
    8. GPU purchase for EJFAT Test stand servers under IRIAD funds. The servers are capable of hosting 2 GPUs per server. ( Amitoj )
      1. In a pinch one can use the 2x A100 GPUs in the NVIDIA Bluefield2 DPU server (hostname: nvidarm)
      2. Initially purchase one of each GPU: NVIDIA, INTEL, AMD so we can compare performance across all 3 flavors.
  9. Resources:
    1. HPDF
  10. AOT
Retrieved from "https://wiki.jlab.org/epsciwiki/index.php?title=EJFAT_Group_Meeting_Nov._16,_2023&oldid=4182"