HPC Resources/PARADOX

From VI-SEEM Wiki
Jump to: navigation, search

Fourth major upgrade of PARADOX installation (Paradox IV) became operational during September 2013. This upgrade consists of 106 working nodes and 3 service nodes. Working nodes (HP ProLiant SL250s Gen8, 2U height) are configured with two Intel Xeon E5-2670 8-core Sandy Bridge processors, at a frequency of 2.6 GHz and 32 GB of RAM (2 GB per CPU-core). The total number of new processor-cores in the cluster is 1696. Each working node contains an additional GP-GPU card (NVIDIA Tesla M2090) with 6 GB of RAM. With a total of 106 NVIDIA Tesla M2090 graphics cards, PARADOX is a premier computer resource in the wider region, which provides access to a large production GPU cluster and new technology. The peak computing power of PARADOX is 105 TFlops.

One service node (HP DL380p Gen8), equipped with an uplink of 10 Gbps, is dedicated to cluster management and user access (gateway machine). All cluster nodes are interconnected via Infiniband QDR technology, through a non-blocking 144-port Mellanox QDR Infiniband switch. The communication speed of all nodes is 40 Gbps in both directions, which is a qualitative step forward over the previous (Gigabit Ethernet) PARADOX installation. The administration of the cluster is enabled by an independent network connection through the iLO (Integrated Lights-Out) interface integrated on motherboards of all nodes.

PARADOX cluster is installed in four water-cooled racks. The cooling system consists of 4 cooling modules (one within each rack), which are connected via a system of pipes with a large industrial chiller and configured so as to minimize power consumption.

Hpc-paradox.jpg

Technical Specification

Administrative Data
Name PARADOX
Short Description Serbian supercomputing cluster
Owner Institute of Physics Belgrade (IPB)
Country Serbia
Computational Power
Number of servers 106
Server specification HP ProLiant SL250s Gen8
CPU per server 2
RAM per server 32 GB
Total number of CPU-cores 1696
Max number of parallel processes With Hyper-Threading enabled 3392
Interconnect type QDR Infiniband
Interconnect latency 1.15 μs
Interconnect bandwidth 40 Gbps
Local filesystem type Lustre
Total storage (TB) 96
Accelerators type NVIDIA TESLA M2090
Number of cores 512
Accelerators per server 1
Servers equipped with accelerators 106
Peak performance CPU (Tflops) 35.3
Peak performance accelerators (Tflops) 70.5
Peak performance (Tflops) 105.8
Real performance (Tflops) N/A
Operating system Scientific Linux
Version 6.4 (Carbon)
Batch system/scheduler Torque/Maui
Development tools Intel Compilers (C/C++, FORTRAN), Portland Group Compilers (Fortran/C/C++ with accelaterator support for CUDA, OpenACC, OpenCL), GNU Compilers, OpenMPI, CUDA, TotalView, Scalasca, TAU, gprof, gdb, pgdbg, Program Database Toolkit, ANTLR3 C
Libraries Intel MKL, HDF5, FFTW, NetCDF, GSL, LAPACK, Boost, BLAS
Applications Gromacs, NAMD, Desmond, VMD, AgroTagger
Dedication to VI-SEEM
CPU (percent) 5%
Storage (percent) 10%
Accelerators (percent) 5%
CPU (core-hours per year) 742,848
Storage in TB 10
Accelerators (hours per year) 47,542,272
Integration
System operational since Sep 2013
Available to the project from PM01
Expected date system to be phased out N/A
Interfaces SSH, gridFTP