PADS \\ Petascale Active Data Store

PADS Hardware Specifications

PADS Data Storage:

  • DataDirect Networks S2A9500 Disk SAN:
    • 460 1TB drives yielding a 335 TB GPFS filesystem
    • Capable of over 3.0 GB/s sustained throughput
  • 10 I/O Nodes:
    • Intel Xeon 5130 2.0 GHz (4 cores)
    • 4GB RAM
    • 2 x 1 Gb/s Ethernet
    • 4X QDR InfiniBand interconnect (32 Gb/s)
  • Spectra Logic T950 Tape Library:
    • 4 LTO-4 tape drives (4 Gb/s each)
    • 720 LTO-4 tape (704 TB assuming 2:1 compression)
    • IBM Tivoli Storage Manager v6.1

PADS Analysis Cluster:

  • 48 Analysis nodes (4.25 TFLOPs):
    • Nehalem 2.66 GHz processors (8 cores total)
    • 24 GB RAM
    • 12 300 GB SAS hard drives
    • 4X QDR InfiniBand interconnect (32 Gb/s)
  • 6 NVIDIA QuadPlex 2200 S4s (24 TFLOPs)
  • Voltaire 2004 4X QDR InfiniBand Switch

PADS Software Stack

Compilers & Development:

GCC 4.1 (Including a fortran 90/95 compiler)
GCC 3.4
Intel 11.0
perl 5.8
python 2.4 and 2.6
R 2.5
ruby 1.8
GNU Java 4.1
Sun Java 6
swig
open64 4.2

Debuggers:

ElectricFence
valgrind
gdb
strace
ltrace

Revision Control:

CVS
SVN
mercurial
RCS

Parallel & Distributed Computing:

MPICH ch_p4 1.2.7p1 (GCC 4.1 and Intel 11.0 versions)
MPICH ch_mx 1.2.7..7 (GCC 4.1 and Intel 11.0 versions)
MPICH2 1.0.7 (GCC 4.1 and Intel 11.0 versions)
OpenMPI 1.2.6 (GCC 4.1 and Intel 11.0 versions)
Globus Toolkit 4.2.1
OSG 1.0

Data Formats:

HDF4 4.1 (GCC 4.1 and Intel 11.0 versions)
HDF5 1.6 (GCC 4.1 and Intel 11.0 versions)
HDF5 1.8 (GCC 4.1 and Intel 11.0 versions)
netCDF 4.0 (GCC 4.1 and Intel 11.0 versions)
netCDF Operators 2.9.6 (GCC 4.1 version)
Parallel HDF5 1.6 (MPICH ch_p4 and MPICH ch_mx compiled with boht GCC 4.1 and Intel 11.0 versions)
Parallel HDF5 1.8 (MPICH ch_p4 and MPICH ch_mx compiled with boht GCC 4.1 and Intel 11.0 versions)
Parallel netCDF 1.0 (MPICH ch_p4 and MPICH ch_mx compiled with boht GCC 4.1 and Intel 11.0 versions)

Scientific Libraries:

fftw 3.2 (GCC 4.1 and Intel 11.0 versions)
gsl 1.12 (GCC 4.1 and Intel 11.0 versions)
Intel MKL 10.1
gmp
Netlib BLAS
Netlib LAPACK

Analysis & Visualization:

Mathematica
Matlab
Mesa 7.2
Paraview 3.4
povray 3.6 (GCC 4.1 and Intel 11.0 versions)
povray 3.7 b29 (GCC 4.1 version)
processing 1.0.1
VisIt 1.11.0
VisTrails 1.2.1
VMD 1.8.6
VTK 4.2.1 (Mesa and NVIDIA with non-parallel and parallel ch_p4 and ch_mx versions)