Developing Parallel Applications

The basics of your working environment are set up automatically by your system administrator during the installation of HP XC. However, your application development environment can be modified by means of modulefiles, as described in “Overview of Modules”.

There are modulefiles available that you can load yourself to further tailor your environment to your specific application development requirements. For example, the TotalView module is available for debugging applications. “Overview of Modules” provides instructions on how to list what modulefiles are available for you to load, and how load a modulefile.

If you encounter problems accessing tools or commands (and associated manpages), ensure that required modules are loaded on your system. If necessary, load required modules yourself, as described in “Overview of Modules”. Otherwise, contact your administrator.

With the HP XC system software, you can use the HP-MPI distributed memory programming model for building and running parallel applications. In this model, all data is private to each process. All intercore communication within a parallel application is performed through calls to the HP-MPI message passing library. Even though support for applications that use a shared-memory programming model is not available at this time, individual cores within an application node can be used in the same application as separate HP-MPI tasks. Applications that are MPI-based, and currently run on Linux (or Linux compatible) systems, can be easily migrated to an HP XC cluster.

See the HP-MPI documentation for more information.

The OpenMP specification is a set of compiler directives that can be used to specify shared-memory parallelism in Fortran and C/C++ programs. Both Intel and Portland Group Fortran and C/C++ compilers support OpenMP.

Although OpenMP is designed for use on shared-memory architectures, OpenMP can be used on an HP XC system within a node.

OpenMP can be used alone, or in conjunction with HP-MPI. For information about compiling programs using OpenMP, refer to OpenMP documentation.

POSIX Threads (Pthreads) is a standard library that programmers can use to develop portable threaded applications. Pthreads can be used in conjunction with HP-MPI on the HP XC system. Compilers from GNU, Intel and PGI provide a -pthread switch to allow compilation with the Pthread library.

Packages that link against Pthreads, such as MKL, require that the application is linked using the -pthread option. The Pthread option is invoked with the following compiler-specific switches:

For example:

$ mpicc object1.o ... -pthread -o myapp.exe

The Quadrics implementation of SHMEM runs on HP XC systems with Quadrics switches. SHMEM is a collection of high-performance routines (that support a distributed-memory model) for data passing between parallel executables.

To compile programs that use SHMEM, it is necessary to include the shmem.h file and to use the SHMEM and Elan libraries. For example:

$ gcc -o shping shping.c -lshmem -lelan

The MPI library supports MPI 1.2 as described in the 1997 release of MPI: A Message Passing Interface Standard. Users should note that the MPI specification describes the application programming interface, but does not specify the contents of the MPI header files, mpi.h and mpif.h, that are included in the source code. Therefore, an MPI application must be recompiled using the proper header files for the MPI library to which it is to be linked.

Parallel applications that use MPI for communication must include the HP XC infrastructure libraries. MPI applications must be built with mpicc, mpic++, mpif77, or mpif90 utilities.

When an MPI application is launched, the user environment, including any MPI environment variables that have been set, is passed to the application.

MPI profiling support is included in the HP XC MPI library, so you do not need to link with a separate library to access the PMPI_xxx() versions of the MPI routines.

The HP XC cluster comes with a modulefile for HP-MPI. The mpi modulefile is used to set up the necessary environment to use HP-MPI, such as the values of the search paths for header and library files.

You can use Intel Fortran compilers (Version 7.x and greater) on the HP XC cluster. However, the HP XC cluster does not supply a copy of Intel compilers. Intel compilers must be obtained directly from the vendor. Refer to Intel documentation for information about using these compilers.

You can use PGI Fortran 95, Fortran 77, and C/C++ compilers on the HP XC cluster. However, the HP XC cluster does not supply a copy of PGI compilers. PGI compilers must be obtained directly from the vendor. Refer to PGI documentation for information about using these compilers.

You can use the GNU C and C++ compilers on the HP XC cluster. The HP XC cluster supplies copies of the GNU C and C++ compilers.

You can use the Pathscale EKOPath Version 2.1 Compiler Suite on the CP4000 platform only. See the following Web site for more information:

http://www.pathscale.com/ekopath.html.

The GNU parallel Make command is used whenever the make command is invoked. GNU parallel Make provides the ability to do a parallel Make; however, all compiling takes place on the login node. Therefore, whether a parallel make improves build time depends upon how many cores are on the login node and the load on the login node.

Information about using the GNU parallel Make is provided in “Using the GNU Parallel Make Capability”.

For further information about using GNU parallel Make, refer to the make manpage. For additional sources of GNU information, refer to the references provided in the front of this manual, located in “About This Document”.

MKL is a math library that references pthreads, and in enabled environments, can use multiple threads. MKL can be linked in a single-threaded manner with your application by specifying the following in the link command:

You can use the AMD Core Math Library (ACML library) on the CP4000 platform.

Other libraries can be used as they would on any other system. However they must contain single core routines and have no dependencies on another message passing system.

If you are building non-MPI applications, such as an OpenMP application for example, you can compile and link them on an HP XC as you normally would, with standard header files and switches.

This section provides some general information about how to build an HP-MPI application in the HP XC environment.

Compiling and linking an MPI application on an HP XC system is performed by invoking the HP-MPI compiler utilities. HP-MPI compiler utilities are scripts supplied by HP-MPI to make it easier to invoke a compiler with the appropriate libraries and search paths. The HP-MPI compiler utilities add all the necessary path locations and library specifications to the compile and link steps that are required to build a HP XC parallel application. It is highly recommended that you use the HP-MPI compiler utilities to compile and link your MPI application on an HP XC cluster, rather than invoke a compiler directly.

The mpicc, mpic++, mpif90, and mpif77 MPI compiler commands are used to invoke the HP-MPI compiler utilities that compile and link an MPI application. The mpicc and mpic++ commands invoke the drivers of the C and C++ compilers. The mpif77 and mpif90 commands invoke the drivers of the Fortran 77 and Fortran 90 compilers.

Before you can compile and link an MPI program using the MPI compiler commands, the MPI compiler utilities module (mpi) must be loaded by using the module load mpi command, or you must arrange for them to be in your $PATH search list. The use of modules is described in “Overview of Modules”.

The following examples show how to compile and link your application code by invoking a compiler utility.

If you have not already loaded the mpi compiler utilities module , load it now as follows:

$ module load mpi

To compile and link a C application using the mpicc command:

$ mpicc -o mycode hello.c 

To compile and link a Fortran application using the mpif90 command:

$ mpif90 -o mycode hello.f 

In the above examples, the HP-MPI commands invoke compiler utilities which call the C and Fortran compilers with appropriate libraries and search paths specified to build the parallel application called hello. The -o specifies that the resulting program is called mycode.