Researchers at
Southwest Research Institute's
Department of Space Studies
have implemented an Avalon-style
parallel computing cluster, Hercules,
for numerically intensive simulations
which would take impractically long (years) on a single workstation.
Pedigree
The cluster is modelled after the
Avalon cluster at Los Alamos
National Laboratory, which in turn is offspring of their
Loki cluster. Loki is an
early example of the Beowulf
implementation of commodity computers in a MIMD configuration to solve
super-computer-class problems.
Architecture
The cluster, hosted off-site in two 19" racks, consists of:
Fortran: Compaq, NDP, and egcs-1.1.1 g77 compilers
C: egcs-1.1.1 gcc
custom MPI port of
Swift/Symba-based
N-body dynamical integrator
custom MPI port of Smooth Particle Hydrodynamics (SPH) -->
Operating System
RedHatLinux, kernel versions
2.0.35, 2.1.125, and 2.2.5
Challenges
Overall, the implentation of this cluster was surprisingly easy thanks
to the well-packaged/documented efforts of the creators of Avalon, MPICH,
RedHat, and the larger Linux community.
Our key challenge was very poor ethernet performance with the initial 2.0.x
series Linux kernels. Later kernels vastly improved this performance
and we are currently experimenting with further reducing latencies based
on tips gleaned from the MPICH site. We have elected to boot the headless
nodes off an SRM boot floppy and are interested in migrating to network
boots for easier kernel maintenance. Of course, effective use of a
parallel cluster of this sort requires appropriate software; we are
currently porting several dynamics codes and have recently completed
porting an implementation of SPH. That code approaches Cray T3E performance
on a similar number of processors (see figure above),
proving that the lack of shared memory
architecture is not critical for workstation clusters of this scale.
Please check the sites listed in the Pedigree section.
We have also made extensive use of Peter S. Pacheco's book,
Parallel Programming with MPI
Hardware for this project was provided in part by the National Science
Foundation and NASA's Planetary Geology and Geophysics Program.
Hercules cluster logo based on Hercules Globular Star Cluster (M13) image from
NOAO
Peter Tamblyn /
ptamblyn@astro101.com
Last modified: December 22, 1999