SRO RTDP Sep. 9, 2024

The current weekly meeting time is every other Monday at 10:00 US/Eastern

Connection Info:

You can connect using ZOOM Video conferencing (ID: 160 248 4178). (Click "Expand" to the right for details -->):

One tap mobile: US: +16692545252,,1602484178# or +16468287666,,1602484178#

Meeting URL: https://jlab-org.zoomgov.com/j/1602484178?pwd=UFViZUpyODYrZmsxdkVNbWZTMUR1Zz09&from=addon
Meeting ID: 160 248 4178
Passcode: 104412

Join by Telephone
For higher quality, dial a number based on your current location.
Dial: +1 669 254 5252 US (San Jose)
+1 646 828 7666 US (New York)
+1 646 964 1167 US (US Spanish Line)
+1 551 285 1373 US (New Jersey)
+1 669 216 1590 US (San Jose)
+1 415 449 4000 US (US Spanish Line)
833 568 8864 US Toll-free
Meeting ID: 160 248 4178
International numbers
Join from an H.323/SIP room system
H.323: 161.199.138.10 (US West)
161.199.136.10 (US East)
Meeting ID: 160 248 4178
Passcode: 104412

SIP: 1602484178@sip.zoomgov.com
Passcode: 104412

One tap mobile: US: +16692545252,,1601987443# or +16468287666,,1601987443#

Meeting URL: https://jlab-org.zoomgov.com/j/1601987443?pwd=bUVPV3puTHJPU0Fub2Q2Z1dsV0JjQT09&from=addon
Meeting ID: 160 198 7443
Passcode: 607906

Agenda:

1. Previous Meeting

2. Announcements:

3. Project Progress: (Github Project)
   1. May 2024 Packet Capture Analysis
      • JANA2 configuration - Nathan (github issue)

   2. RTDP workflow monitoring
      • Prometheus server - Cissie (github issue)
      • Prometheus + ePIC - Cissie (github issue)

   3. RTDP Configuration
      • DPUX format spec. - Vardan
      • Graphical editor - Vardan (github issue)
      • DPUX example config. - Jeng (github issue)

4. BDX Status + R&D

5. AOT

---

Milestones

FY24Q2

• M05: Create stream splitter program for EVIO or HIPO data formatted files

• M06: Create stream splitter program for simulated data in PODIO for ePIC

• M07: Create VTP emulator using files produced by stream splitter

• M08: Create controller program to synchronize multiple VTP emulators

FY24Q3

• M09: Determine appropriate schema for all aspects of monitoring system

• M10: Establish databases for monitoring system using existing JLab servers.

• M11: Integrate Hydra as monitoring component.

FY24Q4

• M12: Integrate off-line data analysis framework into platform for CLAS12 data

• M13: Integrate off-line data analysis framework into platform for ePIC or GlueX simulated data

• M14: Integrate example JANA2 analysis into platform

---

Minutes

Attendees: David L., Vardan G., Jeng T., Cissie M., Nathan B.

• Nathan reported on effort to process CLAS12 SRO packet data with JANA
  • Stubs in place for filter components
  • integrated use official EVIO library
  • The checklist in Github issues needs to be updated

• Cissie reported on logging process telemetry on ifarm via podman/docker
  • While podman currently works on ifarm, it does not yet work on slurm farm nodes
  • Some flow charts of some data flow examples on JIRIAF were shown
  • Logged into clonfarm21 to better understand system and there for streaming. Needs more input though on how streaming works from there.
  • Some discussion on setting up the streaming work flow using individual apptainers on farm as opposed to running all in one apptainer
    • Jeng noted that the process_exporter should work in this case

• Vardan reported briefly on DPUX development
  • DPUX is in a git repository
    • Link to repository needed on RTDP main page

• We had some discussion on how Prometheus works in regards to permanent and temporary storage
  • Question from David: Can the DB from running a particular workflow be saved easily to files for archival purposes and then easily resurrected by starting up a Prometheus container and pointing it to the DB files?
    • Answer: yes

Action Items

1. Update checklist in Github issues (Nathan)
2. Try running podio example using apptainer on slurm farm (Cissie)
3. Add link to DPUX repository on RTDP wiki page (Vardan)