Difference between revisions of "SRO RTDP Jul. 15, 2024"

From epsciwiki
Jump to navigation Jump to search
 
(One intermediate revision by the same user not shown)
Line 46: Line 46:
 
# Announcements:
 
# Announcements:
 
#:
 
#:
−
# No meeting July 29th (David OoO)
+
#:* Renewal Proposal Submitted
 +
#:* No meeting July 29th (David OoO)
 
#:
 
#:
 
# R&D Projects
 
# R&D Projects
Line 86: Line 87:
  
 
=== Minutes ===
 
=== Minutes ===
−
<!-- Attendees: David L., Ayan R., Vardan G., Fabio R., Jeng T. -->
+
Attendees: David L., Ayan R., Vardan G., Fabio R., Jeng T., Cissie M.
 +
 
 +
* Fabio
 +
** Made is safely back to Italy (after some weather delays)
 +
** Spoke with Sergey Furletov about FPGA streaming. He confirmed that Alveo was not a good choice for SRO DAQ as PCIe transfers would dominate much of the rate.
 +
*** Need board with dedicated network connections (Sergey suggested a few boards)
 +
** One crate with a flash ADC exists in Hall-B as a shared resource for testing
 +
** Fabio will use over the coming weeks/months to become familiar with the electronics
 +
** Fabio spoke with Ben about FADC250 streaming readout and limitations on reading windows. It will be some time before window readout is supported for SRO.
 +
 
 +
* Vardan
 +
** No direct progress to report on DPPUI (Data Processing Pipeline User Interface)
 +
** Some low-level work being done on DPOL (Data Processing Ontology Language) which required some redesign to address performance issues.
 +
 
 +
* Jeng
 +
** Looking to deploy JIRIAF using FABRIC
 +
** Spoke some with Srinivas from HPDF to get started
 +
 
 +
* Cissie
 +
** Able to capture podman process status using the fabric python module
 +
*** [https://www.fabfile.org/ Fabric] is not to be confused with [https://portal.fabric-testbed.net/ FABRIC]
 +
*** Tool allows launching over ssh and monitoring process by retrieving JSON records.
 +
 
 +
* Ayan
 +
** Quarterly report for LDRD largely completed.
 +
*** Figures and status table are done but need to replace Q2 text with Q3 text
 +
** ACAT2024 paper almost done
 +
*** Would like to include more description of figures in the text as opposed to just what is in the captions.
 +
*** Will communicate with David offline to resolve remaining questions about figures.
 +
 
 +
* Discussion
 +
** David noted we now have 3 variations on stream processing representing 3 experiments:
 +
**# CLAS12
 +
**# GlueX
 +
**# ePIC/PODIO
 +
** Some discussion on how the JLab -> NERSC exercise Jeng did for ePIC/PODIO worked
 +
*** Jeng explained the system worked via two connected ssh tunnels with the NERSC login node as the intermediate point between the JIRIAF node at JLab and the slurm-allocated Perlmutter node.
 +
** David suggested Jeng and Ayan work together to document the three examples so any of us could run them
 +
*** EJFAT would likely be needed for high bandwidth communication, but will require some coding to make the GlueX and epic/PODIO systems use it.
 +
** Our next goal is to generalize these examples so RTDP could configure and run each of them.
 +
 
 +
 
 +
 
  
 
=== Action Items ===
 
=== Action Items ===

Latest revision as of 15:08, 15 July 2024

The current weekly meeting time is every other Monday at 10:00 US/Eastern

Connection Info:

You can connect using ZOOM Video conferencing (ID: 160 248 4178). (Click "Expand" to the right for details -->):

One tap mobile: US: +16692545252,,1602484178# or +16468287666,,1602484178#

Meeting URL: https://jlab-org.zoomgov.com/j/1602484178?pwd=UFViZUpyODYrZmsxdkVNbWZTMUR1Zz09&from=addon
Meeting ID: 160 248 4178
Passcode: 104412

Join by Telephone
For higher quality, dial a number based on your current location.
Dial: +1 669 254 5252 US (San Jose)
+1 646 828 7666 US (New York)
+1 646 964 1167 US (US Spanish Line)
+1 551 285 1373 US (New Jersey)
+1 669 216 1590 US (San Jose)
+1 415 449 4000 US (US Spanish Line)
833 568 8864 US Toll-free
Meeting ID: 160 248 4178
International numbers
Join from an H.323/SIP room system
H.323: 161.199.138.10 (US West)
161.199.136.10 (US East)
Meeting ID: 160 248 4178
Passcode: 104412

SIP: 1602484178@sip.zoomgov.com
Passcode: 104412

One tap mobile: US: +16692545252,,1601987443# or +16468287666,,1601987443#

Meeting URL: https://jlab-org.zoomgov.com/j/1601987443?pwd=bUVPV3puTHJPU0Fub2Q2Z1dsV0JjQT09&from=addon
Meeting ID: 160 198 7443
Passcode: 607906

Agenda:

  1. Previous Meeting
  2. Announcements:
    • Renewal Proposal Submitted
    • No meeting July 29th (David OoO)
  3. R&D Projects
    • BDX Experiment
    • Configuration
      • DPPUI
  4. Data Capture and Processing
    • Aggregation/Streaming
      • GlueX
      • CLAS12
      • ePIC
  5. Milestone Progress: (Github Project)
    • FY24Q2
    • M05: Create stream splitter program for EVIO or HIPO data formatted files
    • M06: Create stream splitter program for simulated data in PODIO for ePIC
    • M07: Create VTP emulator using files produced by stream splitter
    • M08: Create controller program to synchronize multiple VTP emulators
    • FY24Q3
    • M09: Determine appropriate schema for all aspects of monitoring system
    • M10: Establish databases for monitoring system using existing JLab servers.
    • M11: Integrate Hydra as monitoring component.
  6. AOT

Minutes

Attendees: David L., Ayan R., Vardan G., Fabio R., Jeng T., Cissie M.

  • Fabio
    • Made is safely back to Italy (after some weather delays)
    • Spoke with Sergey Furletov about FPGA streaming. He confirmed that Alveo was not a good choice for SRO DAQ as PCIe transfers would dominate much of the rate.
      • Need board with dedicated network connections (Sergey suggested a few boards)
    • One crate with a flash ADC exists in Hall-B as a shared resource for testing
    • Fabio will use over the coming weeks/months to become familiar with the electronics
    • Fabio spoke with Ben about FADC250 streaming readout and limitations on reading windows. It will be some time before window readout is supported for SRO.
  • Vardan
    • No direct progress to report on DPPUI (Data Processing Pipeline User Interface)
    • Some low-level work being done on DPOL (Data Processing Ontology Language) which required some redesign to address performance issues.
  • Jeng
    • Looking to deploy JIRIAF using FABRIC
    • Spoke some with Srinivas from HPDF to get started
  • Cissie
    • Able to capture podman process status using the fabric python module
      • Fabric is not to be confused with FABRIC
      • Tool allows launching over ssh and monitoring process by retrieving JSON records.
  • Ayan
    • Quarterly report for LDRD largely completed.
      • Figures and status table are done but need to replace Q2 text with Q3 text
    • ACAT2024 paper almost done
      • Would like to include more description of figures in the text as opposed to just what is in the captions.
      • Will communicate with David offline to resolve remaining questions about figures.
  • Discussion
    • David noted we now have 3 variations on stream processing representing 3 experiments:
      1. CLAS12
      2. GlueX
      3. ePIC/PODIO
    • Some discussion on how the JLab -> NERSC exercise Jeng did for ePIC/PODIO worked
      • Jeng explained the system worked via two connected ssh tunnels with the NERSC login node as the intermediate point between the JIRIAF node at JLab and the slurm-allocated Perlmutter node.
    • David suggested Jeng and Ayan work together to document the three examples so any of us could run them
      • EJFAT would likely be needed for high bandwidth communication, but will require some coding to make the GlueX and epic/PODIO systems use it.
    • Our next goal is to generalize these examples so RTDP could configure and run each of them.



Action Items

Retrieved from "https://wiki.jlab.org/epsciwiki/index.php?title=SRO_RTDP_Jul._15,_2024&oldid=5244"