Scientific Computing Systems
From Computer Center Documentation
The scientific computing environment for experimental physics is comprised of a Data Analysis facility and its accompanying Mass Storage System. The data analysis environment are batch farm compute nodes managed by the PBS/Maui scheduler via JLab's Auger software, with local /scratch and network-mounted /work disks. Auger's interface to the mass storage system is through tags provided by the user in the submission script. The mass storage environment includes a robotic tape library with staging pools, and accompanying copies of files cached on disk for immediate use.
Data Analysis Cluster
JLab Auger Software
JLab's Auger software manages the interface to the PBS/Maui batch farm.
Data Analysis Cluster
The batch farm is a 64-bit CentOS 6.2 environment.
The alias 'ifarm.jlab.org' (interactive farm node) is the primary user front-end to the batch computing farm. The farm is accessible via Auger's jsub command.
* ifarm1101: 32-core AMD Opteron, 64GB RAM, 1TB local /scratch; still running CentOS 5.3 through 9/30/13 * ifarm1102: 32-core AMD Opteron, 64GB RAM, 1TB local /scratch
Multithreaded code is encouraged!
Compute nodes contain the year the nodes were purchased (farm12* nodes were purchased in 2012): farm12001-032 32 Intel 16-core hyperthreaded, 32GB RAM, 1000GB disk farm11019-020 2 ADM Opteron 32-core, 64GB RAM, 1000GB disk farm11001-018 18 Intel 8-core hyperthreaded, 24GB RAM, 1000GB disk farm10001-024 24 Intel 8-core hyperthreaded, 24GB RAM, 500GB disk farm09001-024 24 Intel 8-core hyperthreaded, 24GB RAM, 500GB disk farm08*: 13 AMD Opteron(tm) Dual Quad-Core, 8GB RAM, 250GB disk
Work Disk (/work) servers provide ~100TB work space on SunFire X4500 series Thumper/Thor fileservers, for use in the data analysis process. Work areas are not backed up; some are allocated to a group and under its control while other areas are auto-managed to keep a pool of space available during data processing.
The Mass Storage System
JLab JASMine Software
- Jasmine : controls media access and transfer of data between tape (/mss) and the stage (/stage) pools. It also manages the cache servers (/cache), which contains online copies of files from offline tape storage.
Online disk storage in /cache consists of a mix of Linux and Sun X4500 (Thumper & Thor) fileservers, and by definition hold copied of data already stored on tape in the mass storage system. Some areas of cache are assgined to groups as DST (Data Summary Tape) storage, while others are automanaged.
The allocation and layout of the secondary mass storage can be found on the Cache Disk information pages.