View source for Deploy ERSAP data pipelines on EJFAT nodes via JIRIAF


= Deploy ERSAP Data Pipelines Workflows on EJFAT Nodes =

This guide explains how to use the Helm charts in the <code>main/local-ejfat/job</code> folder to deploy ERSAP data pipeline workflows on EJFAT nodes.

== Prerequisites ==

* Helm 3 installed
* Kubernetes cluster access
* kubectl configured

== Chart Structure ==

The main chart is located in the <code>job/</code> directory. Key files include:

* <code>Chart.yaml</code>: Metadata for the chart
* <code>values.yaml</code>: Default configuration values
* Templates in the <code>templates/</code> directory

== Step-by-Step Usage ==

[[File:Local ejfat workflow flow chart.png|800px]]

This diagram provides a visual representation of the steps involved in setting up and deploying ERSAP workflows using the Helm charts in this repository.

=== 1. Setup Environment ===
* Ensure you have Helm 3 installed, access to a Kubernetes cluster, and kubectl configured.
* Clone the repository and navigate to the <code>local-ejfat</code> folder:
<pre>
git clone https://github.com/JeffersonLab/jiriaf-test-platform.git
cd jiriaf-test-platform/main/local-ejfat
</pre>

=== 2. Set Project ID ===
* Export your project ID as an environment variable:
<pre>
export ID=your-project-id # e.g. jlab-100g-nersc-ornl
</pre>

=== 3. Customize Deployment (Optional) ===
* Edit <code>job/values.yaml</code> to customize your deployment. Key settings include:
<pre>
Deployment:
  name: this-name-is-changing
  namespace: default
  replicas: 1
  serviceMonitorLabel: ersap-test4
  cpuUsage: "128"
  ejfatNode: "2"
ersapSettings:
  image: gurjyan/ersap:v0.1
  cmd: /ersap/run-pipeline.sh
  file: /x.ersap
</pre>
* For more advanced customization, consider modifying the template files in the <code>job/templates/</code> directory:
** <code>job-job.yaml</code>: Customize the Kubernetes Job specification
** <code>job-configmap.yaml</code>: Adjust the scripts for job containers
** <code>job-service.yaml</code>: Modify the Kubernetes Service configuration
** <code>prom-servicemonitor.yaml</code>: Fine-tune Prometheus monitoring settings

=== 4. Deploy Prometheus (If not already running) ===
* Refer to [[Deploy Prometheus Monitoring with Prometheus Operator]] or <code>main/prom/readme.md</code> for instructions on how to install and configure Prometheus.
* Check if the prometheus instance is running:
<pre>
helm ls | grep "$ID-prom"
</pre>
If the command returns no results, it means there's no Prometheus instance for your project ID.
* Install a Prometheus instance for your project if it's not already running:
<pre>
cd main/prom
helm install $ID-prom prom/ --set Deployment.name=$ID
</pre>

=== 5. Deploy EJFAT Workflow ===
* Use the <code>launch_job.sh</code> script to deploy the EJFAT workflow:
<pre>
cd main/local-ejfat
ID=jlab-100g-nersc-ornl
INDEX=2
./launch_job.sh $ID $INDEX
</pre>
* The script uses the following variables:
** <code>ID</code>: The project identifier (e.g., jlab-100g-nersc-ornl)
** <code>INDEX</code>: A unique index for the job (e.g., 2)

* This script will create two containers on EJFAT node (check <code>job-configmap.yaml</code>):
** jlabtsai/process-exporter:pgid-go
** gurjyan/ersap:v0.1

=== 6. Monitor Deployment ===
* Check Helm release: <code>helm ls | grep $ID-job-ejfat-$INDEX</code>
* Check pod status: <code>kubectl get pods</code>
* Describe a pod: <code>kubectl describe pod <pod-name></code>

=== 7. Manage Deployed Jobs and Clean Up Workflows ===
* List deployed jobs: <code>helm ls</code>
* Delete a deployed job:
<pre>
helm uninstall $ID-job-ejfat-<INDEX>
</pre>
* After uninstalling the Helm release, manually delete containers on EJFAT nodes:
*# Log in to each EJFAT node used in your deployment.
*# List all containers: <code>docker ps -a</code>
*# Identify containers related to your job.
*# Remove these containers: <code>docker rm -f <container-id></code>