Scientific Computing Systems

From Computer Center Documentation

Jump to: navigation, search



The scientific computing environment for experimental physics is comprised of a Data Analysis facility and its accompanying Mass Storage System. The data analysis environment are batch farm compute nodes managed by the PBS/Maui scheduler via JLab's Auger software, with local /scratch and network-mounted /work disks. Auger's interface to the mass storage system is through tags provided by the user in the submission script. The mass storage environment includes a robotic tape library with staging pools, and accompanying copies of files cached on disk for immediate use.

Data Analysis Cluster

JLab Auger Software

JLab's Auger software manages the interface to the PBS/Maui batch farm.

Data Analysis Cluster

The batch farm is a 64-bit CentOS 6.2 environment.

The alias '' (interactive farm node) is the primary user front-end to the batch computing farm. The farm is accessible via Auger's jsub command.

* ifarm1101: 32-core AMD Opteron, 64GB RAM, 1TB local /scratch; still running CentOS 5.3 through 9/30/13
* ifarm1102: 32-core AMD Opteron, 64GB RAM, 1TB local /scratch

Multithreaded code is encouraged!

Compute nodes contain the year the nodes were purchased (farm12* nodes were purchased in 2012): 

farm12001-032              32 Intel 16-core hyperthreaded, 32GB RAM, 1000GB disk
farm11019-020               2 ADM Opteron 32-core, 64GB RAM, 1000GB disk
farm11001-018              18 Intel 8-core hyperthreaded, 24GB RAM, 1000GB disk
farm10001-024              24 Intel 8-core hyperthreaded, 24GB RAM, 500GB disk
farm09001-024              24 Intel 8-core hyperthreaded, 24GB RAM, 500GB disk
farm08*:                   13 AMD Opteron(tm) Dual Quad-Core, 8GB RAM, 250GB disk

Work Disks

Work Disk (/work) servers provide ~100TB work space on SunFire X4500 series Thumper/Thor fileservers, for use in the data analysis process. Work areas are not backed up; some are allocated to a group and under its control while other areas are auto-managed to keep a pool of space available during data processing.

The Mass Storage System

JLab JASMine Software

  • Jasmine : controls media access and transfer of data between tape (/mss) and the stage (/stage) pools. It also manages the cache servers (/cache), which contains online copies of files from offline tape storage.

Cache Disks

Online disk storage in /cache consists of a mix of Linux and Sun X4500 (Thumper & Thor) fileservers, and by definition hold copied of data already stored on tape in the mass storage system. Some areas of cache are assgined to groups as DST (Data Summary Tape) storage, while others are automanaged.

The allocation and layout of the secondary mass storage can be found on the Cache Disk information pages.

IBM TS3500 Tape Library

Personal tools