Difference between revisions of "SAMPA SRO"

From epsciwiki
Jump to navigation Jump to search
Line 25: Line 25:
  
 
To program SAMP FECs, navigate to the node alkaid.jlab.org and cd to the user's home directory (~gurjyan/Devel/stream/exp-sampa). Then, run the following command to initiate the programming process:
 
To program SAMP FECs, navigate to the node alkaid.jlab.org and cd to the user's home directory (~gurjyan/Devel/stream/exp-sampa). Then, run the following command to initiate the programming process:
 +
 
```
 
```
 
python init/go-trorc.py -mode das -mask 0x1F -nbp 5 -c
 
python init/go-trorc.py -mode das -mask 0x1F -nbp 5 -c
 
```
 
```
 +
 
This will program the SAMP FECs with the specified settings.
 
This will program the SAMP FECs with the specified settings.
  
Line 33: Line 35:
  
 
To set up the environment for data acquisition, navigate to the /usr/local/trorc directory and run the following command:
 
To set up the environment for data acquisition, navigate to the /usr/local/trorc directory and run the following command:
 +
 
```
 
```
 
source trorc-operator/setenv.sh
 
source trorc-operator/setenv.sh
 
```
 
```
 +
 
This will setup the environment for data acquisition.
 
This will setup the environment for data acquisition.
  
Line 41: Line 45:
  
 
To start SAMPA data acquisition, navigate to the node alkaid.jlab.org and cd to the user's home directory (~gurjyan/Devel/stream/exp-sampa). Then, run the following command to start the acquisition process:
 
To start SAMPA data acquisition, navigate to the node alkaid.jlab.org and cd to the user's home directory (~gurjyan/Devel/stream/exp-sampa). Then, run the following command to start the acquisition process:
 +
 
```
 
```
 
treadout --data-type 1 --frames 4000 --mode das --mask 0x1F --nr 121 --output-dir /home/gurjyan/Devel
 
treadout --data-type 1 --frames 4000 --mode das --mask 0x1F --nr 121 --output-dir /home/gurjyan/Devel
 
```
 
```
 +
 
This will start the data acquisition process, recording the data in the specified output directory.
 
This will start the data acquisition process, recording the data in the specified output directory.
  
Line 49: Line 55:
  
 
After the data has been recorded, navigate to the ~gurjyan/Workspace/ersap/sampa directory and execute the following command to set up the ERSAP environment:
 
After the data has been recorded, navigate to the ~gurjyan/Workspace/ersap/sampa directory and execute the following command to set up the ERSAP environment:
 +
 
```
 
```
 
source setup-ersap.bash
 
source setup-ersap.bash
 
```
 
```
 +
 
This will set up the ERSAP environment for data processing.
 
This will set up the ERSAP environment for data processing.
  
Line 57: Line 65:
  
 
To run the ERSAP data processing pipeline, navigate to the ~gurjyan/Workspace/ersap/sampa directory and execute the following command:
 
To run the ERSAP data processing pipeline, navigate to the ~gurjyan/Workspace/ersap/sampa directory and execute the following command:
 +
 
```
 
```
 
$ERSAP_HOME/bin/ersap-shell
 
$ERSAP_HOME/bin/ersap-shell
 
```
 
```
 +
 
This will launch the ERSAP data processing pipeline, which will read the DAQ recorded file and process it.
 
This will launch the ERSAP data processing pipeline, which will read the DAQ recorded file and process it.
  
Line 65: Line 75:
  
 
To run the local ERSAP processing, navigate to the ~gurjyan/Workspace/ersap/sampa directory and execute the following command:
 
To run the local ERSAP processing, navigate to the ~gurjyan/Workspace/ersap/sampa directory and execute the following command:
 +
 
```
 
```
 
run-local
 
run-local
 
```
 
```
 +
 
This will run the local ERSAP processing on the recorded data.
 
This will run the local ERSAP processing on the recorded data.
  

Revision as of 21:58, 20 September 2023


What do we currently have?

The four software components (actors/microservices) that make up the ERSAP-based GEM detector data processing program are as follows:

  1. A SAMPA source actor with the capability described below
    1. Accepts an arbitrary and adjustable number of SAMA streams, often known as links. So that you know, each SAMPA front-end card has a capacity for two Links. It is necessary to set a number of streams equal to six to read out the present GEM setup.
    2. Decodes the raw SAMPA data coming from each stream.
    3. Aggregates the decoded data into a single array of byte buffers to transmit it to other actors in the application. (It is essential to remember that the array size is equivalent to the number of channels; for instance, in the case of six links and for a DAS mode, we have 480 channels)
  2. A statistical actor that:
    1. Receives decoded and aggregated arrays of byte buffers;
    2. Extracts data for each channel; and
    3. Determines the mean and sigma of the distribution in each channel.
  3. Histogram actor that:
    1. Receives decoded and aggregated arrays of byte buffers
    2. Extracts data for each channel
    3. Fills the histograms (selected channels) requested by the user
    4. Visualizes histograms in real-time on a configurable grid of a canvas
  4. File sink actor that
    1. Writes every 100-time slices decoded and aggregated data (array of byte buffers) into a file (Note: need to be activated file output in the application configuration/composition file: services.yaml)

Quick, two step DAS mode data acquisition and processing

1. Programming SAMP FECs:

To program SAMP FECs, navigate to the node alkaid.jlab.org and cd to the user's home directory (~gurjyan/Devel/stream/exp-sampa). Then, run the following command to initiate the programming process:

``` python init/go-trorc.py -mode das -mask 0x1F -nbp 5 -c ```

This will program the SAMP FECs with the specified settings.

2. Setting up the environment for data acquisition:

To set up the environment for data acquisition, navigate to the /usr/local/trorc directory and run the following command:

``` source trorc-operator/setenv.sh ```

This will setup the environment for data acquisition.

3. Starting SAMPA data acquisition:

To start SAMPA data acquisition, navigate to the node alkaid.jlab.org and cd to the user's home directory (~gurjyan/Devel/stream/exp-sampa). Then, run the following command to start the acquisition process:

``` treadout --data-type 1 --frames 4000 --mode das --mask 0x1F --nr 121 --output-dir /home/gurjyan/Devel ```

This will start the data acquisition process, recording the data in the specified output directory.

4. Preparing ERSAP data processing:

After the data has been recorded, navigate to the ~gurjyan/Workspace/ersap/sampa directory and execute the following command to set up the ERSAP environment:

``` source setup-ersap.bash ```

This will set up the ERSAP environment for data processing.

5. Running ERSAP data processing pipeline:

To run the ERSAP data processing pipeline, navigate to the ~gurjyan/Workspace/ersap/sampa directory and execute the following command:

``` $ERSAP_HOME/bin/ersap-shell ```

This will launch the ERSAP data processing pipeline, which will read the DAQ recorded file and process it.

6. Running the local ERSAP processing:

To run the local ERSAP processing, navigate to the ~gurjyan/Workspace/ersap/sampa directory and execute the following command:

``` run-local ```

This will run the local ERSAP processing on the recorded data.

That's it! With these steps, you should be able to set up and use the SAMPA and ERSAP data acquisition and processing systems on the node alkaid.jlab.org.

How to write your ERSAP processor engine?

Implementing the ERSAP interface is what you should do to make your processor engine. The user must put their code in the area of the execute method that has been left blank since it has only been partially completed.

NB. See already available engines to fill the rest of the interface methods (they are fairly similar)

public class TestProcEngine implements Engine {
   @Override
   public EngineData configure(EngineData engineData) {
              ByteBuffer bb = (ByteBuffer)input.getData();
       ByteBuffer[] data;
       try {
           data = DasDataType.deserialize(bb);
           int sampleLimit = data[0].limit()/2;
           for (int channel = 0; channel < chNum; channel++) {

                   short[] _sData = new short[sampleLimit];
                   for (int sample = 0; sample < sampleLimit; sample++) {
                      _sData[sample] =  data[channel].getShort(2 * sample);
                   }
                   USER CODE GOES HERE 
                   deals with _sData[] containing data for a single channel 
           }
       } catch (ErsapException e) {
           e.printStackTrace();
       }
       return input;
   }
   @Override
   public EngineData execute(EngineData engineData) {
       return null;
   }
   @Override
   public EngineData executeGroup(Set<EngineData> set) {
       return null;
   }
   @Override
   public Set<EngineDataType> getInputDataTypes() {
       return null;
   }
   @Override
   public Set<EngineDataType> getOutputDataTypes() {
       return null;
   }
   @Override
   public Set<String> getStates() {
       return null;
   }
   @Override
   public String getDescription() {
       return null;
   }
   @Override
   public String getVersion() {
       return null;
   }
   @Override
   public String getAuthor() {
       return null;
   }
   @Override
   public void reset() {
   }

Project dependencies

  1. ersap-java
  2. ersap-cpp
  3. ersap-actor

Installation

NB. For installation you should define ERSAP_HOME environmental variable.

  1. ersap-java instructions
  2. ersap-cpp instructions
  3. ersap-actor instructions

SAMPA SRO diagram

  1. diagram

Building SAMA DAQ codebase

NB. The SAMPA SRO package is kindly provided by the ALICE collaboration and is modified by the EPSCI SRO group to make it streaming. The modified package can be found at /home/gurjyan/Devel/stream/exp-sampa

  1. login into alkaid.jlab.org
  2. copy the ALICE modified package into your directory
  3. follow instructions in README to build the package

Configuration and running

NB: Keeping the order of instructions is important.

NB: On alkaid.jlab.org source setup_ersap.bash/tcsh from /home/gurjyan/Workspace/ersap/sampa. This script sets up necessary environmental variables pointing to a correct JAVA SDK.


NB. We recommend copying /home/gurjyan/Workspace/ersap/sampa dir, simplifying the installation process.

The CLI provides a high-level interface to configure and start the different ERSAP components required to run an application.

  1. Start the ERSAP shell:
    • $ERSAP_HOME/bin/ersap-shell
  2. Define the application within a services.yaml file. An example of the file can be found below. NB: The default location for the application definition file is in $ERSAP_USER_DATA/config dir
  3. Start the data processing. This will start the main Java DPE, a C++ DPE if the C++ service is listed in services.yaml, and it will run the streaming orchestrator to process the data stream.
    • ersap> run local
  4. Run SAMPA FE (on some other terminal. NB: use bash shell)
    • >source [modified ALICE code directory]/dist/trorc/trorc-operator/setenv.sh
    • >treadout --data-type 1 --frames 4000 --mode das --mask 0x7 --port 6000 --host_ip localhost --events 0

ERSAP application data-stream pipeline

The following is an ERSAP application composition file (services.yaml), describing SAMPA SRO and data-stream processing back-end.

--- io-services:

 reader:
   class: org.jlab.ersap.actor.sampa.engine.SampaDASSourceEngine
   name: SMPSource
 writer:
   class: org.jlab.ersap.actor.sampa.engine.SampaFileSinkEngine
   name: SMPWriter

services:

 - class: org.jlab.ersap.actor.sampa.engine.SampaStatProcEngine
   name: SMPStreamTest
 - class: org.jlab.ersap.actor.sampa.engine.SampaHistogramProcEngine
   name: SMPHistogram

configuration:

 io-services:
   reader:
     stream_count: 6
     port: 6000
   writer:
     file_output: "false"
 services:
   SMPStreamTest:
     verbose: "false"
   SMPHistogram:
     frame_title: "ERSAP"
     frame_width: 1400
     frame_height: 1200
     grid_size: 2
     #> hist_titles is a string containing the list of integers=channels separated by ,
     hist_titles: "1, 3, 7, 17"
     hist_bins: 100
     hist_min: 0
     hist_max: 500

mime-types:

 - binary/data-sampa