Getting started using HP-UX or Linux

Configuring your environment

Setting PATH

If you move the HP-MPI installation directory from its default location in /opt/mpi for HP-UX, and /opt/hpmpi for Linux:

Set the MPI_ROOT environment variable to point to the location where MPI is installed.
Add $MPI_ROOT/bin to PATH.
Add $MPI_ROOT/share/man to MANPATH.

MPI must be installed in the same directory on every execution host.

Setting shell

By default, HP-MPI attempts to use ssh on Linux and remsh on HP-UX. On Linux, we recommend that ssh users set StrictHostKeyChecking=no in their ~/.ssh/config.

To use rsh on Linux instead, the following script needs to be run as root on each node in the cluster:

% /opt/hpmpi/etc/mpi.remsh.default

Or, to use rsh on Linux, use the alternative method of manually populating the files /etc/profile.d/hpmpi.csh and /etc/profile.d/hpmpi.sh with the following settings respectively:

% setenv MPI_REMSH rsh

% export MPI_REMSH=rsh

On HP-UX, MPI_REMSH specifies a command other than the default remsh to start remote processes. The mpirun, mpijob, and mpiclean utilities support MPI_REMSH. For example, you can set the environment variable to use a secure shell:

% setenv MPI_REMSH /bin/ssh

HP-MPI allows users to specify the remote execution tool to use when HP-MPI needs to start processes on remote hosts. The tool specified must have a call interface similar to that of the standard utilities: rsh, remsh and ssh. An alternate remote execution tool, such as ssh, can be used on HP-UX by setting the environment variable MPI_REMSH to the name or full path of the tool to use:

% export MPI_REMSH=ssh

% $MPI_ROOT/bin/mpirun <options> -f <appfile>

HP-MPI supports setting MPI_REMSH using the -e option to mpirun:

% $MPI_ROOT/bin/mpirun -e MPI_REMSH=ssh <options> -f \ <appfile>

HP-MPI also supports setting MPI_REMSH to a command which includes additional arguments:

% $MPI_ROOT/bin/mpirun -e MPI_REMSH="ssh -x" <options> \ -f <appfile>

When using ssh on HP-UX, first ensure that it is possible to use ssh from the host where mpirun is executed to the other nodes without ssh requiring any interaction from the user.

Compiling and running your first application

To quickly become familiar with compiling and running HP-MPI programs, start with the C version of a familiar hello_world program. This program is called hello_world.c and prints out the text string "Hello world! I’m r of s on host" where r is a process’s rank, s is the size of the communicator, and host is the host on which the program is run. The processor name is the host name for this implementation.

The source code for hello_world.c is stored in $MPI_ROOT/help and is shown below.

#include <stdio.h>
#include "mpi.h"
 
void main(argc, argv)
 
int                     argc;
char                    *argv[];
 
{
 
	        int             rank, size, len;
 
        char            name[MPI_MAX_PROCESSOR_NAME];
 
        MPI_Init(&argc, &argv);
 
        MPI_Comm_rank(MPI_COMM_WORLD, &rank);
 
        MPI_Comm_size(MPI_COMM_WORLD, &size);
 
        MPI_Get_processor_name(name, &len);
        
        printf("Hello world!I'm %d of %d on %s\n", rank, size,
                                        name);
 
        MPI_Finalize();
 
        exit(0);
 
}

Building and running on a single host

This example teaches you the basic compilation and run steps to execute hello_world.c on your local host with four-way parallelism. To build and run hello_world.c on a local host named jawbone:

Change to a writable directory.
Compile the hello_world executable file:
% $MPI_ROOT/bin/mpicc -o hello_world \ $MPI_ROOT/help/hello_world.c
Run the hello_world executable file:
% $MPI_ROOT/bin/mpirun -np 4 hello_world
where -np 4 specifies 4 as the number of processes to run.
Analyze hello_world output.
HP-MPI prints the output from running the hello_world executable in non-deterministic order. The following is an example of the output:
Hello world! I'm 1 of 4 on jawbone Hello world! I'm 3 of 4 on jawbone Hello world! I'm 0 of 4 on jawbone Hello world! I'm 2 of 4 on jawbone

For information on running more complex applications, refer to “Running applications on HP-UX and Linux”.

Building and running on a Linux cluster using appfiles

The following is an example of basic compilation and run steps to execute hello_world.c on a cluster with 4-way parallelism. To build and run hello_world.c on a cluster using an appfile:

Change to a writable directory.
Compile the hello_world executable file:
% $MPI_ROOT/bin/mpicc -o hello_world \ $MPI_ROOT/help/hello_world.c

Create a file "appfile" for running on nodes n01 and n02 as:

-h n01 -np 2 /path/to/hello_world
-h n02 -np 2 /path/to/hello_world

Run the hello_world executable file:
% $MPI_ROOT/bin/mpirun -f appfile
By default, mpirun will rsh/remsh to the remote machines n01 and n02. If desired, the environment variable MPI_REMSH can be used to specify a different command, such as /usr/bin/ssh or "ssh -x".
Analyze hello_world output.
HP-MPI prints the output from running the hello_world executable in non-deterministic order. The following is an example of the output:
Hello world! I'm 1 of 4 n01 Hello world! I'm 3 of 4 n02 Hello world! I'm 0 of 4 n01 Hello world! I'm 2 of 4 n02

Refer to “LSF on non-XC systems” for examples using LSF.

Building and running on an XC cluster using srun

The following is an example of basic compilation and run steps to execute hello_world.c on a XC cluster with 4-way parallelism. To build and run hello_world.c on a XC cluster (assuming LSF is not installed):

Change to a writable directory.
Compile the hello_world executable file:
% $MPI_ROOT/bin/mpicc -o hello_world \ $MPI_ROOT/help/hello_world.c
Run the hello_world executable file:
% $MPI_ROOT/bin/mpirun -srun -n4 hello_world
where -n4 specifies 4 as the number of processes to run from SLURM.
Analyze hello_world output.
HP-MPI prints the output from running the hello_world executable in non-deterministic order. The following is an example of the output:

I'm 1 of 4 n01 Hello world!
I'm 3 of 4 n02 Hello world!
I'm 0 of 4 n01 Hello world!
I'm 2 of 4 n02 Hello world!

Refer to “LSF on XC systems” for examples using LSF.

Directory structure for HP-UX and Linux

All HP-MPI files are stored in the /opt/mpi directory for HP-UX and the /opt/hpmpi directory for Linux. The directory structure is organized as described in Table 2-1 “Directory Structure for HP-UX and Linux”.

If you move the HP-MPI installation directory from its default location in /opt/mpi, set the MPI_ROOT environment variable to point to the new location. Refer to “Configuring your environment”.

Table 2-1 Directory Structure for HP-UX and Linux

Subdirectory	Contents
bin	Command files for the HP-MPI utilities gather_info script
help	Source files for the example programs
include	Header files
lib/pa2.0	HP-MPI PA-RISC 32-bit libraries
lib/pa20_64	HP-MPI PA-RISC 64-bit libraries
lib/hpux32	HP-MPI HP-UX Itanium 32-bit libraries
lib/hpux64	HP-MPI HP-UX Itanium 64-bit libraries
lib/linux_ia32	HP-MPI Linux 32-bit libraries
lib/linux_ia64	HP-MPI Linux 64-bit libraries for Itanium
lib/linux_amd64	HP-MPI Linux 64-bit libraries for Opteron and Intel®64
MPICH1.2/	MPICH compatibility wrapper libraries
newconfig/	Configuration files and release notes
share/man/man1*	man pages for the HP-MPI utilities
share/man/man3*	man pages for HP-MPI library
doc	Release notes

HP-UX and Linux man pages

The man pages are located in the $MPI_ROOT/share/man/man1* subdirectory for HP-UX and Linux. They can be grouped into three categories: general, compilation, and run time. There is one general man page, MPI.1, that is an overview describing general features of HP-MPI. The compilation and run-time man pages are those that describe HP-MPI utilities.

Table 2-2 “HP-UX and Linux man page categories” describes the three categories of man pages in the man1 subdirectory that comprise man pages for HP-MPI utilities.

Table 2-2 HP-UX and Linux man page categories

Category	man pages	Description
General	MPI.1	Describes the general features of HP-MPI
Compilation	mpicc.1 mpiCC.1 mpif77.1 mpif90.1	Describes the available compilation utilities. Refer to “Compiling applications” for more information
Runtime	mpiclean.1 mpidebug.1 mpienv.1 mpiexec.1 mpijob.1 mpimtsafe.1mpirun.1 mpirun.all.1 mpistdio.1 autodbl.1	Describes runtime utilities, environment variables, debugging, thread-safe and diagnostic libraries

Licensing Policy for Linux

HP-MPI for Linux uses FLEXlm licensing technology. A license is required to use HP-MPI for Linux. Licenses can be purchased from HP’s software depot at http://www.software.hp.com, or contact your HP representative.

Demo licenses for HP-MPI are also available from HP’s software depot.

If you’re running on an HP XC system, no license is required at runtime.

HP-MPI has an Independent Software Vendor (ISV) program that allows participating ISVs to freely distribute HP-MPI with their applications. When the application is part of the HP-MPI ISV program, there is no licensing requirement for the end user. The ISV provides a licensed copy of HP-MPI. Contact your application vendor to find out if they participate in the HP-MPI ISV program. The copy of HP-MPI distributed with a participating ISV will only work with that application. An HP-MPI license is required for all other applications.

Licensing for Linux

HP-MPI for Linux uses FLEXlm licensing technology. A license file can be named either as license.dat or any file name with an extension of .lic. The default location to place MPI license files is in the default installation directory /opt/hpmpi/licenses.

You will need to provide the hostname and hostid number of the system where the FLEXlm daemon for HP-MPI for Linux will run. The hostid, which is the MAC address of eth0, can be obtained by typing the following command if HP-MPI is already installed on the system:

% /opt/hpmpi/bin/licensing/<arch>/lmutil lmhostid

Or:

%/sbin/ifconfig | egrep "^eth0" | awk'{print $5}' | \ sed s/://g

The hostname can be obtained by entering the command hostname.

The default search path used to find an MPI license file is:

% $MPI_ROOT/licenses:/opt/hpmpi/licenses:.

For example, if MPI_ROOT=/home/hpmpi, license files will be searched in the following order:

/home/hpmpi/licenses/license.dat

/home/hpmpi/licenses/*.lic

/opt/hpmpi/licenses/license.dat

/opt/hpmpi/licenses/*.lic

./license.dat

./*.lic

If the license needs to be placed in another location which would not be found by the above search, the user may set the environment variable LM_LICENSE_FILE to explicitly specify the location of the license file.

For more information, see http://licensing.hp.com.

Installing License Files

A valid license file contains the system hostid and the associated license key. License files can be named either as license.dat or any name with extension of *.lic (like mpi.lic, for example). Copy the license file under the directory /opt/hpmpi/licenses.

The command to run the license server is:

% $MPI_ROOT/bin/licensing/<arch>/lmgrd -c mpi.lic

License Testing

Build and run the hello_world program in $MPI_ROOT/help/hello_world.c to check for a license. If your system is not properly licensed, you will receive the following error message:

(“MPI BUG: Valid MPI license not found in search path”)

Merging Licenses

Newer HP-MPI licenses use the INCREMENT feature which allows separate HP-MPI licenses to be used in combination by concatenating files. For example:

License 1: SERVER myserver 0014c2c1f34a
     DAEMON HPQ
     INCREMENT HP-MPI HPQ 1.0 permanent 8 9A40ECDE2A38 \
                 NOTICE="License Number = AAAABBBB1111" SIGN=E5CEDE3E5626

License 2: SERVER myserver 0014c2c1f34a
     DAEMON HPQ
     INCREMENT HP-MPI HPQ 1.0 permanent 16 BE468B74B592 \
                 NOTICE="License Number = AAAABBBB2222" SIGN=9AB4034C6CB2

Here, License 1 is for 8 ranks, and License 2 is for 16 ranks. The two licenses can be combined into a single file:

SERVER myserver 0014c2c1f34a
DAEMON HPQ
INCREMENT HP-MPI HPQ 1.0 permanent 8 9A40ECDE2A38 \
NOTICE="License Number = AAAABBBB1111" SIGN=E5CEDE3E5626

SERVER myserver 0014c2c1f34a
DAEMON HPQ
INCREMENT HP-MPI HPQ 1.0 permanent 16 BE468B74B592 \
NOTICE="License Number = AAAABBBB2222" SIGN=9AB4034C6CB2

The result is a valid license for 24 ranks.

Version identification

To determine the version of an HP-MPI installation, use the what command on HP-UX. Use the ident or rpm command on Linux.

For example:

% what $MPI_ROOT/bin/mpirun

% ident $MPI_ROOT/bin/mpirun

% rpm -qa | grep hpmpi