Difference between revisions of "SRO RTDP Jul. 15, 2024"

From epsciwiki
Jump to navigation Jump to search
(Created page with "The current weekly meeting time is every other Monday at 10:00 US/Eastern === Connection Info: === <div class="toccolours mw-collapsible mw-collapsed"> You can connect using...")
 
 
(2 intermediate revisions by the same user not shown)
Line 46: Line 46:
 
# Announcements:
 
# Announcements:
 
#:
 
#:
−
# CLAS12 packet capture: May 16
+
#:* Renewal Proposal Submitted
−
#:* [https://docs.google.com/spreadsheets/d/1TnXKH3jdOVubZAX8sUD77rLFUk_TRn_mN65rX-oQgoU/edit?usp=sharing spreadsheet of runs]
+
#:* No meeting July 29th (David OoO)
 
#:
 
#:
 
# R&D Projects
 
# R&D Projects
Line 53: Line 53:
 
#:
 
#:
 
#* Configuration
 
#* Configuration
−
#** JCedit++
+
#** DPPUI
−
#** rtdp YAML
 
 
#:
 
#:
 
# Data Capture and Processing
 
# Data Capture and Processing
Line 88: Line 87:
  
 
=== Minutes ===
 
=== Minutes ===
−
<!-- Attendees: David L., Ayan R., Vardan G., Fabio R., Jeng T. -->
+
Attendees: David L., Ayan R., Vardan G., Fabio R., Jeng T., Cissie M.
 +
 
 +
* Fabio
 +
** Made is safely back to Italy (after some weather delays)
 +
** Spoke with Sergey Furletov about FPGA streaming. He confirmed that Alveo was not a good choice for SRO DAQ as PCIe transfers would dominate much of the rate.
 +
*** Need board with dedicated network connections (Sergey suggested a few boards)
 +
** One crate with a flash ADC exists in Hall-B as a shared resource for testing
 +
** Fabio will use over the coming weeks/months to become familiar with the electronics
 +
** Fabio spoke with Ben about FADC250 streaming readout and limitations on reading windows. It will be some time before window readout is supported for SRO.
 +
 
 +
* Vardan
 +
** No direct progress to report on DPPUI (Data Processing Pipeline User Interface)
 +
** Some low-level work being done on DPOL (Data Processing Ontology Language) which required some redesign to address performance issues.
 +
 
 +
* Jeng
 +
** Looking to deploy JIRIAF using FABRIC
 +
** Spoke some with Srinivas from HPDF to get started
 +
 
 +
* Cissie
 +
** Able to capture podman process status using the fabric python module
 +
*** [https://www.fabfile.org/ Fabric] is not to be confused with [https://portal.fabric-testbed.net/ FABRIC]
 +
*** Tool allows launching over ssh and monitoring process by retrieving JSON records.
 +
 
 +
* Ayan
 +
** Quarterly report for LDRD largely completed.
 +
*** Figures and status table are done but need to replace Q2 text with Q3 text
 +
** ACAT2024 paper almost done
 +
*** Would like to include more description of figures in the text as opposed to just what is in the captions.
 +
*** Will communicate with David offline to resolve remaining questions about figures.
 +
 
 +
* Discussion
 +
** David noted we now have 3 variations on stream processing representing 3 experiments:
 +
**# CLAS12
 +
**# GlueX
 +
**# ePIC/PODIO
 +
** Some discussion on how the JLab -> NERSC exercise Jeng did for ePIC/PODIO worked
 +
*** Jeng explained the system worked via two connected ssh tunnels with the NERSC login node as the intermediate point between the JIRIAF node at JLab and the slurm-allocated Perlmutter node.
 +
** David suggested Jeng and Ayan work together to document the three examples so any of us could run them
 +
*** EJFAT would likely be needed for high bandwidth communication, but will require some coding to make the GlueX and epic/PODIO systems use it.
 +
** Our next goal is to generalize these examples so RTDP could configure and run each of them.
 +
 
 +
 
 +
 
  
 
=== Action Items ===
 
=== Action Items ===

Latest revision as of 15:07, 15 July 2024

The current weekly meeting time is every other Monday at 10:00 US/Eastern

Connection Info:

You can connect using ZOOM Video conferencing (ID: 160 248 4178). (Click "Expand" to the right for details -->):

One tap mobile: US: +16692545252,,1602484178# or +16468287666,,1602484178#

Meeting URL: https://jlab-org.zoomgov.com/j/1602484178?pwd=UFViZUpyODYrZmsxdkVNbWZTMUR1Zz09&from=addon
Meeting ID: 160 248 4178
Passcode: 104412

Join by Telephone
For higher quality, dial a number based on your current location.
Dial: +1 669 254 5252 US (San Jose)
+1 646 828 7666 US (New York)
+1 646 964 1167 US (US Spanish Line)
+1 551 285 1373 US (New Jersey)
+1 669 216 1590 US (San Jose)
+1 415 449 4000 US (US Spanish Line)
833 568 8864 US Toll-free
Meeting ID: 160 248 4178
International numbers
Join from an H.323/SIP room system
H.323: 161.199.138.10 (US West)
161.199.136.10 (US East)
Meeting ID: 160 248 4178
Passcode: 104412

SIP: 1602484178@sip.zoomgov.com
Passcode: 104412

One tap mobile: US: +16692545252,,1601987443# or +16468287666,,1601987443#

Meeting URL: https://jlab-org.zoomgov.com/j/1601987443?pwd=bUVPV3puTHJPU0Fub2Q2Z1dsV0JjQT09&from=addon
Meeting ID: 160 198 7443
Passcode: 607906

Agenda:

  1. Previous Meeting
  2. Announcements:
    • Renewal Proposal Submitted
    • No meeting July 29th (David OoO)
  3. R&D Projects
    • BDX Experiment
    • Configuration
      • DPPUI
  4. Data Capture and Processing
    • Aggregation/Streaming
      • GlueX
      • CLAS12
      • ePIC
  5. Milestone Progress: (Github Project)
    • FY24Q2
    • M05: Create stream splitter program for EVIO or HIPO data formatted files
    • M06: Create stream splitter program for simulated data in PODIO for ePIC
    • M07: Create VTP emulator using files produced by stream splitter
    • M08: Create controller program to synchronize multiple VTP emulators
    • FY24Q3
    • M09: Determine appropriate schema for all aspects of monitoring system
    • M10: Establish databases for monitoring system using existing JLab servers.
    • M11: Integrate Hydra as monitoring component.
  6. AOT

Minutes

Attendees: David L., Ayan R., Vardan G., Fabio R., Jeng T., Cissie M.

  • Fabio
    • Made is safely back to Italy (after some weather delays)
    • Spoke with Sergey Furletov about FPGA streaming. He confirmed that Alveo was not a good choice for SRO DAQ as PCIe transfers would dominate much of the rate.
      • Need board with dedicated network connections (Sergey suggested a few boards)
    • One crate with a flash ADC exists in Hall-B as a shared resource for testing
    • Fabio will use over the coming weeks/months to become familiar with the electronics
    • Fabio spoke with Ben about FADC250 streaming readout and limitations on reading windows. It will be some time before window readout is supported for SRO.
  • Vardan
    • No direct progress to report on DPPUI (Data Processing Pipeline User Interface)
    • Some low-level work being done on DPOL (Data Processing Ontology Language) which required some redesign to address performance issues.
  • Jeng
    • Looking to deploy JIRIAF using FABRIC
    • Spoke some with Srinivas from HPDF to get started
  • Cissie
    • Able to capture podman process status using the fabric python module
      • Fabric is not to be confused with FABRIC
      • Tool allows launching over ssh and monitoring process by retrieving JSON records.
  • Ayan
    • Quarterly report for LDRD largely completed.
      • Figures and status table are done but need to replace Q2 text with Q3 text
    • ACAT2024 paper almost done
      • Would like to include more description of figures in the text as opposed to just what is in the captions.
      • Will communicate with David offline to resolve remaining questions about figures.
  • Discussion
    • David noted we now have 3 variations on stream processing representing 3 experiments:
      1. CLAS12
      2. GlueX
      3. ePIC/PODIO
    • Some discussion on how the JLab -> NERSC exercise Jeng did for ePIC/PODIO worked
      • Jeng explained the system worked via two connected ssh tunnels with the NERSC login node as the intermediate point between the JIRIAF node at JLab and the slurm-allocated Perlmutter node.
    • David suggested Jeng and Ayan work together to document the three examples so any of us could run them
      • EJFAT would likely be needed for high bandwidth communication, but will require some coding to make the GlueX and epic/PODIO systems use it.
    • Our next goal is to generalize these examples so RTDP could configure and run each of them.



Action Items

Retrieved from "https://wiki.jlab.org/epsciwiki/index.php?title=SRO_RTDP_Jul._15,_2024&oldid=5244"