Difference between revisions of "EPSCI Group Meeting Aug. 3, 2020"

From epsciwiki
Jump to navigation Jump to search
(Created page with " The meeting time is 10:00am. === Connection Info: === <div class="toccolours mw-collapsible mw-collapsed"> You can connect using [https://bluejeans.com/253300597 BlueJeans V...")
 
 
Line 54: Line 54:
 
#** EIC SRO Meeting today @11:00
 
#** EIC SRO Meeting today @11:00
 
#** NO EPSCI SRO Meeting this week
 
#** NO EPSCI SRO Meeting this week
#** ERCAP
+
#** ERSAP
 
#** Hall-B/D TriDAS
 
#** Hall-B/D TriDAS
 
# Publications
 
# Publications
Line 64: Line 64:
 
<hr>
 
<hr>
 
=== Minutes: ===
 
=== Minutes: ===
<!-- Attendees: David L.(chair), Carl T., Nathan B., Thomas B., Vardan G., Kishan R., Graham H. -->
+
Attendees: David L.(chair), Carl T., Nathan B., Thomas B., Vardan G., Kishan R.
 +
 
 +
'''Announcements'''
 +
* Bluejeans now has virtual backgrounds
 +
 
 +
'''GUI For Calorimeter Calibration System in Hall-D'''
 +
* Thomas working on at request from Igal
 +
* Front end done
 +
* Gave 1 year time scale
 +
* Some evidence Igal's workflow is heavily burdening hallddb.jlab.org
 +
 
 +
'''NERSC'''
 +
* Processing of GlueX Spring 2020 dataset has started
 +
* Job throughput is very modest
 +
* May need to consider bundling jobs
 +
* Perlmutter will change situation dramatically in 1 year. (how much to invest?)
 +
 
 +
'''OSG'''
 +
* Multiple distribution technologies
 +
** cvmfs oasis - useful for code distributions - using both JLab VO and GlueX VO
 +
** stash cache - useful for larger files (e.g. random triggers) that could be used at multiple sites - federated storage that relies on xrootd
 +
** xrootd - used for transporting data (e.g. raw data)
 +
* Running OSG jobs onJLab SciComp farm
 +
** head node is being stood up now
 +
** still 1 month or so away from functioning
 +
* Collector at JLab still not fully functional due to network issues
 +
** UCSD still in use as primary for the moment
 +
 
 +
'''SPACK'''
 +
* Space on JLab CUE is being set up
 +
* Web server is process of being set up. (Sherman now in charge of web servers)
 +
* JLab CE replacement still about 1 month out. Oct. may be more realistic
 +
 
 +
'''EVIO-6'''
 +
* Full C++ port of Java code is complete
 +
* Testing phase has started
 +
 
 +
'''A.I.'''
 +
* Hydra deployed in Hall-D with [https://halldweb.jlab.org/hydra/HydraRun.html live web interface]
 +
 
 +
'''SRO'''
 +
* ERSAP Prototype
 +
** Multi-threaded I/O services working
 +
** Demo 10GB/s from 12 simulated streams through data lake (all on same node so no network hardware involved yet)
 +
** Redis data lake issue resolved by reconfiguring to use multiple threads
 +
** Vardan will report at EIC meeting immediately following this one
 +
 
 +
'''JANA2'''
 +
* GlueX Port
 +
** Merged numerous changes required of JANA2 into master
 +
** Working on test cases
 +
* Documentation
 +
** Investigated using ReadTheDocs. [https://breathe.readthedocs.io/en/latest Breathe] is tool that can help convert existing DOxygen documentation to ReadTheDocs (Sphinx) format.
 +
* A.I. support
 +
** Considering OpenCV over Tensorflow for C++ interface
 +
** Issues calling python interpreter from JFactory
 +
*** Python interface is one option for inferring on GPU from JANA
 +
*** GIL may be related to the issue

Latest revision as of 12:28, 10 August 2020

The meeting time is 10:00am.

Connection Info:

You can connect using BlueJeans Video conferencing (ID: 253 300 597). (Click "Expand" to the right for details -->):

Meeting URL
 https://bluejeans.com/253300597?src=join_info

Meeting ID
253 300 597

Want to dial in from a phone?

Dial one of the following numbers:
+1.888.240.2560 (US Toll Free)
(see all numbers - https://www.bluejeans.com/premium-numbers)

Enter the meeting ID and passcode followed by #

Connecting from a room system?
Dial: bjn.vc or 199.48.152.152 and enter your meeting ID & passcode

Agenda:

  1. Previous meeting
  2. Announcements
    • Beam delivery for physics starting now
      • David on shift 8/18, 8/19, 8/28, 8/29, 9/11, 9/12
      • Thomas on shift 7/29, 7/30, 8/20, 8/21, 8/30, 8/31, 9/9, 9/10
    • Streaming Data Scientist job posting (almost..?)
  3. Graham's Project
  4. GUI for Calorimeter calibration scripts (Hall-D Request)
  5. Ongoing Activities
    • Offsite Computing
      • NERSC, PSC
      • OSG
    • JLab Common Environment (CE) + SPACK
    • EVIO-6
    • A.I.
    • JANA2
      • GlueX port
      • A.I. support
      • ESCalate in SoLID
    • SRO
      • EIC SRO Meeting today @11:00
      • NO EPSCI SRO Meeting this week
      • ERSAP
      • Hall-B/D TriDAS
  6. Publications
  7. AOT



Minutes:

Attendees: David L.(chair), Carl T., Nathan B., Thomas B., Vardan G., Kishan R.

Announcements

  • Bluejeans now has virtual backgrounds

GUI For Calorimeter Calibration System in Hall-D

  • Thomas working on at request from Igal
  • Front end done
  • Gave 1 year time scale
  • Some evidence Igal's workflow is heavily burdening hallddb.jlab.org

NERSC

  • Processing of GlueX Spring 2020 dataset has started
  • Job throughput is very modest
  • May need to consider bundling jobs
  • Perlmutter will change situation dramatically in 1 year. (how much to invest?)

OSG

  • Multiple distribution technologies
    • cvmfs oasis - useful for code distributions - using both JLab VO and GlueX VO
    • stash cache - useful for larger files (e.g. random triggers) that could be used at multiple sites - federated storage that relies on xrootd
    • xrootd - used for transporting data (e.g. raw data)
  • Running OSG jobs onJLab SciComp farm
    • head node is being stood up now
    • still 1 month or so away from functioning
  • Collector at JLab still not fully functional due to network issues
    • UCSD still in use as primary for the moment

SPACK

  • Space on JLab CUE is being set up
  • Web server is process of being set up. (Sherman now in charge of web servers)
  • JLab CE replacement still about 1 month out. Oct. may be more realistic

EVIO-6

  • Full C++ port of Java code is complete
  • Testing phase has started

A.I.

SRO

  • ERSAP Prototype
    • Multi-threaded I/O services working
    • Demo 10GB/s from 12 simulated streams through data lake (all on same node so no network hardware involved yet)
    • Redis data lake issue resolved by reconfiguring to use multiple threads
    • Vardan will report at EIC meeting immediately following this one

JANA2

  • GlueX Port
    • Merged numerous changes required of JANA2 into master
    • Working on test cases
  • Documentation
    • Investigated using ReadTheDocs. Breathe is tool that can help convert existing DOxygen documentation to ReadTheDocs (Sphinx) format.
  • A.I. support
    • Considering OpenCV over Tensorflow for C++ interface
    • Issues calling python interpreter from JFactory
      • Python interface is one option for inferring on GPU from JANA
      • GIL may be related to the issue