ERSAP
(Environment for Real-time Streaming, Acquisition and Processing)
ERSAP is a reactive actor/micro-service based data-stream processing framework, a hardware/software system that is capable of processing unbounded streams of continuous data at scale over distributed heterogeneous resources in real-time. The goal of ERSAP is to design a framework that will provide a flexible environment to design and, most importantly, allow the future evolution of data processing applications. An environment that will encourage the implementation of new ideas and new technologies while preserving the integrity of existing data processing pipelines. ERSAP encourages to perform functional decomposition of the overall data processing application into small, mono-functional artifacts/actors. Artifacts that are easy to understand, develop, deploy, and debug. Fact that these artifacts/actors are programmatically independent, they can be scaled and optimized independently, which is impossible to do for components of the monolithic application. One of the important advantages is the ERSAP fault tolerance, where independent actors can come and go on the stream without forcing the entire application to crash, and it is easy to locate the faulty actor in the chain. Due to the fact that the actors are loosely coupled and that the message (the only coupling between actors) carries the context, they can run on heterogenous environments with different memory and/or processing power requirements. The active actors can be physically separated from each other, so the deployment of a new set of actors does not put the whole system on hold, and the updates are less intrusive. The development of the actors and actor engines handling specific areas of the system can be performed by independent development teams, with their own release cycles, and at their own pace. The ERSAP is not a ground-up design. It is based on our experience of micro-service-based, off-line data processing applications that are running for more than a decade in CLAS12, and CODA distributed, triggered-based DAQ that is running for more than a quarter of a century at the JLAB. Also, it will be utilizing the JANA framework to develop high-performance data processing engines.
| Projects | Meetings |
|
| Date | Presenter | Event | Link |
|---|---|---|---|
| 2023-04-08 | V. Gyurjyan | 26TH International Conference on Computing in High Energy & Nuclear Phusics | pptx |
| 2022-08-04 | V. Gyurjyan | 23rd IEEE Real TIme Conference | |
| 2021-12-08 | V. Gyurjyan | EIC Workshop IX on Streaming Readout | pptx |
| 2021-04-30 | V. Gyurjyan | EIC Workshop VIII on Streaming Readout | pptx |
| 2020-11-8 | V. Gyurjyan | EIC SRO Workshop VII | pptx |
| 2020-10-12 | V. Gyurjyan | 22nd IEEE Real Time Conference | poster |
| 2020-03-12 | V. Gyurjyan | ERSAP Design Requirements | docx |
| 2020-01-01 | G. Heyes | JLAB SRO paper |