Configuring a Serviceguard NFS Package

Except for the variables listed below, use the default values for the variables in the package configuration file, or change them as needed.

For instructions on modifying the default values, see the Managing Serviceguard manual, or read the comments in the package configuration file.

Set the PACKAGE_NAME variable. For example:

PACKAGE_NAME pkg01

You can use the default package name if you are planning to run only one Serviceguard NFS package on your Serviceguard cluster. Each package must have a unique name.

Create a NODE_NAME variable for each node that will run the package. The first NODE_NAME should specify the primary node. All the NODE_NAME variables following the primary node should specify the adoptive nodes, in the order in which they will be tried. For example:

NODE_NAME thyme 
NODE_NAME basil
NODE_NAME sage

Set the RUN_SCRIPT and HALT_SCRIPT variables to the full path name of the control script. You do not have to specify a timeout for either script. For example:

RUN_SCRIPT /usr/local/cmcluster/pkg1/pkg1.cntl
RUN_SCRIPT_TIMEOUT NO_TIMEOUT
HALT_SCRIPT /usr/local/cmcluster/pkg1/pkg1.cntl
HALT_SCRIPT_TIMEOUT NO_TIMEOUT

If you want to run the NFS monitor script, set the SERVICE_NAME variable. For example:

SERVICE_NAME nfs1.monitor

Each package must have a unique service name. The SERVICE_NAME variable in the package configuration file must match the NFS_SERVICE_NAME variable in the NFS control script.

If you do not wish to run the NFS monitor script, comment out the SERVICE_NAME variable. For example:

SERVICE_NAME nfs1.monitor

If your NFS package configuration file specifies AUTO_RUN YES, the package switches to the next adoptive node in the event of a node or package failure. The NFS monitor script causes the package to fail over if any of the monitored NFS services fails.

Set the SUBNET variable to the subnet that is monitored for the package. For example:

SUBNET 192.100.112.0

Create a separate VG[n] variable for each volume group. For example:

VG[0]=vg01
VG[1]=vg02

Create a separate LV[n], FS[n], FS_TYPE[n], and FS_MOUNT_OPT[n] variable for each volume group and file system that will be mounted on the server. For example:

LV[0]=/dev/vg01/lvol1;FS[0]=/ha_root; FS_TYPE[0]=ext2;FS_MOUNT_OPT[0]=”-o rw”
LV[1]=/dev/vg01/lvol2;FS[1]=/users/scaf; FS_TYPE[1]=ext2;FS_MOUNT_OPT[1]=”-o rw”
LV[2]=/dev/vg02/lvol1;FS[2]=/ha_data; FS_TYPE[2]=ext2;FS_MOUNT_OPT[2]=”-o rw”

This example defines the variable for three NFS mounted file systems, ha_root, users/scaf, and ha_data.

Specify the IP address for the package and the address of the subnet to which the IP address belongs. For example:

IP[0]=15.13.114.243
SUBNET[0]=192.100.112.0

The IP address you specify is the relocatable IP address for the package. NFS clients that mount the file systems in the package will use this IP address to identify the server. You should configure a name for this address in the DNS or NIS database, or in the /etc/hosts file.

Specify that this package uses the high availability NFS server by uncommenting the HA_APP_SERVER variable. Uncomment the following line:

HA_APP_SERVER=”pre-IP”

If two packages have the same adoptive node, and you want to prevent a shared adoptive node from adopting both packages at once, specify the cmmodpkg command with the package control option (-d) in the customer_defined_run_cmds. For example:

function customer_defined_run_cmds
{
    cmmodpkg -d -n ‘hostname‘ pkg02 &
}

This package control function can prevent an adoptive node from becoming overloaded when multiple packages fail over. If an adoptive node becomes overloaded, it can fail.

In this example, if a host is an adoptive node for both pkg01 and pkg02, the above cmmodpkg -d command, in the control script for pkg01, would prevent the host that is running pkg01 from adopting pkg02. If pkg02 fails, it will failover to another adoptive node (if configured) where pkg01 is not running.

Add a similar line in the control script for pkg02 to prevent the host that is running pkg02 from adopting pkg01.

The ampersand (&) causes the cmmodpkg command to run in the background. The cmmodpkg command in the background allows the control script to complete and finish bringing up the package.

	NOTE: There is a small window of time, during which if one package has begun to fail over but the cmmodpkg command has not executed, the other package can fail over and the host will adopt it. In other words, if two packages fail over at approximately the same time, a host may adopt both packages, even though the package control option is specified.

See “Configuring One Adoptive Node to Support Failover of Multiple Packages” for a sample configuration using the package control option.

Use the default values for the rest of the variables in the control script, or change them as needed. For instructions on modifying the default values, see the Managing Serviceguard for Linux manual, or read the comments in the /usr/local/cmcluster/nfstoolkit/pkg.cntl template file.

Create a separate XFS[n] variable for each NFS directory to be exported. Specify the directory name and any export options. For example:

XFS[0]=”*:/ha_root”	XFS[1]="*:/users/scaf"	XFS[2]="-o ro *:/ha_data"		XFS[3]="-o fsid=23,rw *:/pkg3"

Do not configure these exported directories in the /etc/exports file. When an NFS server boots up, it attempts to export all file systems in its /etc/exports file. If those file systems are not currently present on the NFS server node, the node cannot boot properly. This happens if the server is an adoptive node for a file system, and the file system is available on the server only after failover of the primary node.

If you want to run the NFS monitor script:

If you want to start and monitor rpc.quotad daemon, set QUOTA_MON to YES. For example:

QUOTA_MON=YES

If you do not want to start and monitor rpc.quotad daemon, set QUOTA_MON to NO. For example:

QUOTA_MON=NO

In the package configuration file, set the SERVICE_FAIL_FAST_ENABLED to “YES” for the NFS monitoring service. Example:

SERVICE_NAME     nfs1.monitor	SERVICE_FAIL_FAST_ENABLED        YESSERVICE_HALT_TIMEOUT    300

	NOTE: In RedHat, there are times when sending SIGKILL to the kernel ‘lockd’ thread might not release all the file locks and cause the failure of the unmounting of filesystem. To force unmount of the filesystem, the machine has to be rebooted. In such cases, it is recommended to set SERVICE_FAIL_FAST_ENABLED to “YES” which reboots the machine upon service failure. In SLES, the sm directory does not consistenty update with the client entries. This is due to the client entry being made for the first time only after the system has booted. After a fail back of the package, the NFS fails to create sm directory entries. After a fail back, if the client attempts to reclaim his locks, fresh entries for the clients will not be made in the /var/lib/nfs/sm directory of the server. For SLES, it is mandatory to set SERVICE_FAIL_FAST_ENABLED to “YES”, so the server reboots in order to have lock migration feature work consistently. If you halt the package manually on any node configured for an NFS package, you must reboot the machine before the package is run again on the same node.

In the Package Control Script, edit the customer_defined_run_cmds function to execute the toolkit.sh script for file lock migration. In the example below, the following line was added, $HA_APP_SCRIPT lock_migration:<Package IP>.

	NOTE: HA_APP_SCRIPT is initialized to the toolkit.sh path in the package control script.

Example:

function customer_defined_run_cmds             {		$HA_APP_SCRIPT lock_migration:${IP[0]}		test_return 51

	NOTE: The argument to be passed to the HA_APP_SCRIPT for lock migration should contain the same name or IP address used by the clients while mounting the exported file system.

Example: If client mounts the NFS file system using the package name, specify the same in customer_defined_run_cmds as shown below:

function customer_defined_run_cmds             {		$HA_APP_SCRIPT lock_migration:<package name>		test_return 51

	NOTE: A Serviceguard package can support multiple relocatable IP addresses associated with it. However, if lock migration feature is enabled then only one IP is allowed to be specified in the NFS package control script since the NFS limits server reboot/failover notification is sent using only a single IP. In the package control script, only one IP/Subnet address pair is allowed to be specified under IP ADDRESS section.

Configure the following variables in hanfs.conf:

Use the cmapplyconf command to verify the content of your cluster and package configuration and to copy the binary configuration file to all the nodes in the cluster. In the following example, the cluster configuration file is /usr/local/cmcluster/cluster.conf. On your system, use the names of your own cluster configuration and package configuration files.

# cmapplyconf -v -C /usr/local/cmcluster/cluster.conf \

  -P /usr/local/cmcluster/pkg1/pkg1.conf

Use your favorite copy utility (for example, scp) to copy the package control, NFS control, and monitor scripts to the same path names on all the nodes in the cluster. For example, to copy the files from host thyme to host basil, issue the following command from host thyme:

# scp /usr/local/cmcluster/cluster/pkg1/* \

  basil:/usr/local/cmcluster/cluster/pkg1