SRO RTDP Sep. 9, 2024

From epsciwiki
Jump to navigation Jump to search

The current weekly meeting time is every other Monday at 10:00 US/Eastern

Connection Info:

You can connect using ZOOM Video conferencing (ID: 160 248 4178). (Click "Expand" to the right for details -->):

One tap mobile: US: +16692545252,,1602484178# or +16468287666,,1602484178#

Meeting URL: https://jlab-org.zoomgov.com/j/1602484178?pwd=UFViZUpyODYrZmsxdkVNbWZTMUR1Zz09&from=addon
Meeting ID: 160 248 4178
Passcode: 104412

Join by Telephone
For higher quality, dial a number based on your current location.
Dial: +1 669 254 5252 US (San Jose)
+1 646 828 7666 US (New York)
+1 646 964 1167 US (US Spanish Line)
+1 551 285 1373 US (New Jersey)
+1 669 216 1590 US (San Jose)
+1 415 449 4000 US (US Spanish Line)
833 568 8864 US Toll-free
Meeting ID: 160 248 4178
International numbers
Join from an H.323/SIP room system
H.323: 161.199.138.10 (US West)
161.199.136.10 (US East)
Meeting ID: 160 248 4178
Passcode: 104412

SIP: 1602484178@sip.zoomgov.com
Passcode: 104412

One tap mobile: US: +16692545252,,1601987443# or +16468287666,,1601987443#

Meeting URL: https://jlab-org.zoomgov.com/j/1601987443?pwd=bUVPV3puTHJPU0Fub2Q2Z1dsV0JjQT09&from=addon
Meeting ID: 160 198 7443
Passcode: 607906

Agenda:

  1. Previous Meeting
  2. Announcements:
  3. Project Progress: (Github Project)
    1. May 2024 Packet Capture Analysis
    2. RTDP workflow monitoring
    3. RTDP Configuration
  4. BDX Status + R&D
  5. AOT

Milestones

FY24Q2

  • M05: Create stream splitter program for EVIO or HIPO data formatted files
  • M06: Create stream splitter program for simulated data in PODIO for ePIC
  • M07: Create VTP emulator using files produced by stream splitter
  • M08: Create controller program to synchronize multiple VTP emulators

FY24Q3

  • M09: Determine appropriate schema for all aspects of monitoring system
  • M10: Establish databases for monitoring system using existing JLab servers.
  • M11: Integrate Hydra as monitoring component.

FY24Q4

  • M12: Integrate off-line data analysis framework into platform for CLAS12 data
  • M13: Integrate off-line data analysis framework into platform for ePIC or GlueX simulated data
  • M14: Integrate example JANA2 analysis into platform


Minutes

Attendees: David L., Vardan G., Jeng T., Cissie M., Nathan B.

  • Nathan reported on effort to process CLAS12 SRO packet data with JANA
  • Cissie reported on logging process telemetry on ifarm via podman/docker
    • While podman currently works on ifarm, it does not yet work on slurm farm nodes
    • Some flow charts of some data flow examples on JIRIAF were shown
    • Logged into clonfarm21 to better understand system there for streaming. Needs more input though on how streaming works from there.
    • Some discussion on setting up the streaming work flow using individual apptainers on farm as opposed to running all in one apptainer
      • Jeng noted that the process_exporter should work in this case
  • Vardan reported briefly on DPUX development
    • DPUX is in a git repository
      • Link to repository needed on RTDP main page
    • Jeng will work with Vardan on implementing configuration in issue 34
  • We had some discussion on how Prometheus works in regards to permanent and temporary storage
    • Question from David: Can the DB from running a particular workflow be saved easily to files for archival purposes and then easily resurrected by starting up a Prometheus container and pointing it to the DB files?
      • Answer: yes

Action Items

  1. Update checklist in Github issues (Nathan)
  2. Try running podio example using apptainer on slurm farm (Cissie)
  3. Add link to DPUX repository on RTDP wiki page (Vardan)
Retrieved from "https://wiki.jlab.org/epsciwiki/index.php?title=SRO_RTDP_Sep._9,_2024&oldid=5598"