Skip to main content

Latest Tweets

MPI on ECCO

Print Friendly

MPI programs

The cluster supports MPI. By default, openmpi is available (module load openmpi). To use Intel MPI, load the appropriate module

module load intel/intel-mpi-5.0.3

and compile your code.

Compiling with Intel MPI

A simple example (drawn from Intel website) might proceed as follows:

# create directory to work in
mkdir intel-mpi
cd intel-mpi/
# copy test code
cp /opt/intel/impi/5.0.3.048/test/test.c .
# compile the code using the Intel compiler - does this require a license?
module load intel/intel-compilers-15.0.3
module load intel/intel-mpi-5.0.3
mpiicc -o myprog test.c
# inspect the file
file myprog 

myprog: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, not stripped

# create the hostfile to run the MPI program on (hostnames are listed on the ECCO page)
cat "compute-0-1
compute-0-2" > hostfile

Note: At the time of writing, compute-0-1 and compute-0-2 each have 8 cores.

# run the program
iqsub
# test where mpirun is
which mpirun
/opt/openmpi/bin/mpirun
mpirun -n 16 ./myprog

Hello world: rank 0 of 16 running on compute-0-1.ecco
Hello world: rank 1 of 16 running on compute-0-1.ecco
Hello world: rank 2 of 16 running on compute-0-1.ecco
Hello world: rank 3 of 16 running on compute-0-1.ecco
Hello world: rank 4 of 16 running on compute-0-1.ecco
Hello world: rank 5 of 16 running on compute-0-1.ecco
Hello world: rank 6 of 16 running on compute-0-1.ecco
Hello world: rank 7 of 16 running on compute-0-1.ecco
Hello world: rank 8 of 16 running on compute-0-2.ecco
Hello world: rank 9 of 16 running on compute-0-2.ecco
Hello world: rank 10 of 16 running on compute-0-2.ecco
Hello world: rank 11 of 16 running on compute-0-2.ecco
Hello world: rank 12 of 16 running on compute-0-2.ecco
Hello world: rank 13 of 16 running on compute-0-2.ecco
Hello world: rank 14 of 16 running on compute-0-2.ecco
Hello world: rank 15 of 16 running on compute-0-2.ecco

So pulling it all together, the following minimal qsub should work for large-scale MPI runs:

Compiling with OpenMPI

Redoing the simple example, but with OpenMPI:

# create directory to work in
mkdir open-mpi
cd open-mpi/
# copy test code - we use the same
cp /opt/intel/impi/5.0.3.048/test/test.c .
# compile the code using the GNU compiler and OpenMPI
# note the difference here: mpicc vs. mpiicc
mpicc -o myprog test.c
# inspect the file
file myprog 

myprog: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, not stripped

Note: At the time of writing, compute-0-1 and compute-0-2 each have 8 cores.
# test run the program locally
mpirun -n 2  ./myprog

Hello world: rank 0 of 16 running on compute-0-1.ecco
Hello world: rank 1 of 16 running on compute-0-1.ecco

So pulling it all together, the following minimal qsub should work for large-scale MPI runs: