JIRIAF Meeting Nov. 16 2023

From epsciwiki
Jump to navigation Jump to search


Connection Info:

You can connect using the following link (Meeting ID: 161 690 3130). (Click "Expand" to the right for details -->):

One tap mobile: US: +16692545252,,1608518798# or +16468287666,,1608518798#
Meeting URL: https://jlab-org.zoomgov.com/j/1616903130?pwd=cjg3U0Y4SndXL05SeFBmQjVHZkhrQT09&from=addon
Meeting ID: 161 690 3130
Passcode: 018094

Join by Telephone
For higher quality, dial a number based on your current location.
Dial:
US: +1 669 254 5252 or +1 646 828 7666 or +1 551 285 1373 or +1 669 216 1590 or 833 568 8864 (Toll Free)
Meeting ID: 161 690 3130

International numbers
Join by SIP
1616903130@sip.zoomgov.com
Join by H.323
161.199.138.10 (US West)
161.199.136.10 (US East)
Meeting ID: 160 851 8798
Passcode: 018094


Agenda:

  1. Announcements
    1. Successful deployment and data-stream processing from JLAB to NERSC Perlmutter.
    2. In the process of hiring a student from ODU.
  2. Summary of the project's undertakings and key achievements
    1. M1 (Complete)
      1. Multiple pods within the vk-cmd
    2. M2 (Complete)
      1. Prometheus scraper that uses UNIX perf
    3. M3
      1. Define mechanisms to act on user workflows, such as reducing previously allocated resources to the user workflow/application.
    4. M4
      1. JCS design and development
        1. Finalize and prototype JIRIAF central service database. Tables, such as
          1. available resource, user requests, and user workflow status.
        2. Examine the site resources database table (constantly updated by SWIF2) and submit SWIF2 requests to launch JRM and allocate/lease resources.
        3. Communicate with the k8s App server, ensuring submitted jobs are running, updating JIRIAF's available resource DB table.
        4. Develop a resource-request matching algorithm that compares user requests with the available resources.
          1. Define and suggest metadata structure for requests for accurate matching.
        5. Monitor/scrape Prometheus database process-related metrics and update active user workflows status DB table.
    5. M5 (Complete)
    6. JIRIAF k8s node/vk_cmd: Implement a function that can use ConfigMap configuration to write files in pods.
  3. Slides for upcoming presentations in preparation for the publications
    1. Slide describing JIRIAF virtual k8s cluster creation, emphasizing its dynamic nature.
    2. Slide showing Prometheus integration to monitor JIRIAF k8s cluster and pods.
    3. Start working on a paper describing JIRIAF resource acquisition and workflow deployment within a dynamic k8s cluster.
  4. Work requirement for the student to work on FE.
  5. AOT

Useful References

Minutes:

Retrieved from "https://wiki.jlab.org/epsciwiki/index.php?title=JIRIAF_Meeting_Nov._16_2023&oldid=4188"