Configuring One Adoptive Node to Support Failover of Multiple Packages

This section shows the package configuration file (pkg01.conf) for the package pkg01 in this sample configuration. The comments are not shown.

PACKAGE_NAME               pkg01
PACKAGE_TYPE               FAILOVER
FAILOVER_POLICY            CONFIGURED_NODE
FAILBACK_POLICY            MANUAL
NODE_NAME                  thyme
NODE_NAME                  sage
AUTO_RUN                   YES
NODE_FAIL_FAST_ENABLED     NO
RUN_SCRIPT                 /usr/local/cmcluster/pkg01/pkg01.cntl
RUN_SCRIPT_TIMEOUT         NO_TIMEOUT
HALT_SCRIPT                /usr/local/cmcluster/pkg01/pkg01.cntl
HALT_SCRIPT_TIMEOUT        NO_TIMEOUT
SERVICE_NAME               nfs1.monitor
SERVICE_FAIL_FAST_ENABLED  NO
SERVICE_HALT_TIMEOUT       300
MONITORED SUBNET         192.100.112.0
MONITORED_SUBNET_ACCESS	 FULL 

This section shows the package control script (pkg01.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted.

PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/
cmcluster/bin
VGCHANGE="vgchange -a y"                # Default
VG[0]="nfsu01"
LV[0]=/dev/nfsu01/lvol1; 
FS[0]=/hanfs/nfsu011; 
FS_TYPE[0]="ext3"; 
FS_MOUNT_OPT[0]="-o rw"
# FS_UMOUNT_COUNT=""
# FS_MOUNT_RETRY_COUNT=""
IP[0]="192.100.112.243"
SUBNET[0]="192.100.112.0"
HA_APP_SERVER="pre-IP"
SERVICE_NAME[0]="nfs1.monitor"
SERVICE_CMD[0]="/usr/local/cmcluster/pkg01/nfs.mon"
SERVICE_RESTART[0]="-r 0"

To prevent a shared adoptive node from adopting both packages together, disable the package by specifying the cmmodpkg command with the package control option (-d) in the customer_defined_run_cmds. For example:

function customer_defined_run_cmds
{
    cmmodpkg -d -n `hostname` pkg02 &
}

Also, add cmmodpkg command with package control option (-e ) in the customer_defined_halt_cmds for enabling the package pkg02 when the package pkg01 is halted. For example:

function customer_defined_halt_cmds
{
    cmmodpkg -e -n `hostname` pkg02 &
}

This package control function can prevent an adoptive node from becoming overloaded when multiple packages fail over. If an adoptive node becomes overloaded, it can fail. In this example, if a host is an adoptive node for both pkg01 and pkg02, disabling of pkg02 would prevent the host that is running pkg01 from adopting pkg02, and once the package pkg01 is halted in the node the pkg02 is enabled by the command cmmodpkg –e in the customer_defined_halt_cmds. The ampersand (&) causes the cmmodpkg command to run in the background.

The cmmodpkg command in the background allows the control script to complete and finish bringing up the package. There is a small window of time, during which if one package has begun to fail over but the cmmodpkg command has not executed, the other package can fail over and the host will adopt it.

In other words, if two packages fail over at approximately the same time, a host may adopt both packages, even though the package control option is specified. If you omit the cmmodpkg -d command from the NFS control script, host sage can adopt both pkg01 and pkg02 if their primary nodes fail.

	NOTE: The above changes in the customer_defined_halt_cmds and customer_defined_run_cmds must be done only in the package control scripts of the adoptive node and not on the package control script of the primary node. If there are more than two packages, repeat the cmmodpkg command as many times required with other package names. User may consider making FAILBACK_POLICY as AUTOMATIC to free the adoptive node as early as possible so that adoptive node is ready to accept pkg02 in case it fails.

This section shows the NFS Toolkit configuration file (hanfs.conf) for the package pkg01 on this sample configuration:

XFS[0]="-o rw *:/hanfs/nfsu011"

QUOTA_MON=YES

LOCK_MIGRATION=NO

This section shows the package configuration file (pkg02.conf) for the package pkg02 in this sample configuration. The comments are not shown.

PACKAGE_NAME               pkg02
PACKAGE_TYPE               FAILOVER
FAILOVER_POLICY            CONFIGURED_NODE
FAILBACK_POLICY            MANUAL
NODE_NAME                  basil
NODE_NAME                  sage
AUTO_RUN                   YES
NODE_FAIL_FAST_ENABLED     NO
RUN_SCRIPT                 /usr/local/cmcluster/pkg02/pkg02.cntl
RUN_SCRIPT_TIMEOUT         NO_TIMEOUT
HALT_SCRIPT                /usr/local/cmcluster/pkg02/pkg02.cntl
HALT_SCRIPT_TIMEOUT        NO_TIMEOUT
SERVICE_NAME               nfs2.monitor
SERVICE_FAIL_FAST_ENABLED  NO
SERVICE_HALT_TIMEOUT       300
SUBNET                     192.100.112.0

This section shows shows the package control script (pkg02.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted.

PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/
cmcluster/bin
VGCHANGE="vgchange -a y"                # Default
VG[0]="nfsu02"
LV[0]=/dev/nfsu02/lvol1; 
FS[0]=/hanfs/nfsu021; 
FS_TYPE[0]="ext3"; 
FS_MOUNT_OPT[0]="-o rw"
# FS_UMOUNT_COUNT=""
# FS_MOUNT_RETRY_COUNT=""
IP[0]="192.100.112.244"
SUBNET[0]="192.100.112.0"
HA_APP_SERVER="pre-IP"
NFS_SERVICE_NAME[0]="nfs2.monitor"
NFS_SERVICE_CMD[0]="/usr/local/cmcluster/pkg02/nfs.mon"
NFS_SERVICE_RESTART[0]="-r 0"

To prevent a shared adoptive node from adopting both packages together, disable the package by specifying the cmmodpkg command with the package control option (-d) in the customer_defined_run_cmds. For example:

function customer_defined_run_cmds
{
    cmmodpkg -d -n `hostname` pkg01 &
}

Also, add cmmodpkg command with package control option (-e ) in the customer_defined_halt_cmds for re-enabling the package pkg02 when the package pkg01 is halted. For example:

function customer_defined_halt_cmds
{
    cmmodpkg -e -n `hostname` pkg01 &
}

This package control function can prevent an adoptive node from becoming overloaded when multiple packages fail over. If an adoptive node becomes overloaded, it can fail. In this example, if a host is an adoptive node for both pkg01 and pkg02, disabling of pkg01 would prevent the host that is running pkg02 from adopting pkg01. Once the package pkg02 is halted in the node the pkg01 is enabled by the command cmmodpkg –e in the customer_defined_halt_cmds.

The ampersand (&) causes the cmmodpkg command to run in the background. The cmmodpkg command in the background allows the control script to complete and finish bringing up the package. There is a small window of time, during which if one package has begun to fail over but the cmmodpkg command has not executed, the other package can fail over and the host will adopt it.

In other words, if two packages fail over at approximately the same time, a host may adopt both packages, even though the package control option is specified. If you omit the cmmodpkg -d command from the NFS control script, host sage can adopt both pkg01 and pkg02 if their primary nodes fail.

	NOTE: The above changes in the customer_defined_halt_cmds and customer_defined_run_cmds must be done only in the package control scripts of the adoptive node and not on the package control script in the primary node. If there are more than 2 packages the user may repeat the cmmodpkg command as many times required with other package names. User may consider making FAILBACK_POLICY as AUTOMATIC to free the adoptive node as early as possible so that adoptive node is ready to accept pkg01 in case it fails.

This section shows the NFS Toolkit configuration file (hanfs.conf) for the package pkg02 on this sample configuration:

XFS[0]="-o rw *:/hanfs/nfsu021"
QUOTA_MON=YES
LOCK_MIGRATION=NO

This section shows the package configuration file (pkg01.conf) for the package pkg01 in this sample configuration. The comments are not shown.

package_name 			    pkg01
package_type      	 failover
failover_policy           	configured_node
failback_policy          	manual
node_name 			     thyme 
node_name 			     sage
auto_run                   yes
node_fail_fast_enabled     no
script_log_file 		/usr/local/cmcluster/pkg01/log 
TKIT_DIR 			     /usr/local/cmcluster/pkg01
XFS[0]				       "-o rw *:/hanfs/nfsu011"
QUOTA_MON				   YES 
LOCK_MIGRATION			NO
monitored_subnet 		192.100.112.0
monitored_subnet_access 	full 
service_name               nfs1.monitor
service_cmd     "$SGCONF/scripts/tkit/nfs/tkit_module.sh nfs_monitor"
service_restart            none
service_fail_fast_enabled  no
service_halt_timeout       300
external_script            $SGCONF/scripts/tkit/nfs/lock_migration.sh
ip_subnet 	    192.100.112.0
#ip_subnet_node
ip_address 	    192.100.112.243
vgchange_cmd 	"vgchange -a y"                
vg					         nfsu01
fs_name				     /dev/nfsu01/lvol1	
fs_directory			 /hanfs/nfsu011
fs_type				ext3
fs_mount_opt			"-o rw"
# fs_umount_opt
# fs_fsck_opt

If you want to prevent a shared adoptive node from adopting both packages together, specify the cmmodpkg command with the package control option (-d) in the external script. The external script template is provided at /usr/local/cmcluster/conf/examples/external_script.template in Red Hat and at /opt/cmcluster/conf/examples/external_script.template in SLES. Copy and rename the external_script.template to the package directory with the following command: # cp /usr/local/cmcluster/conf/examples/external_script.template \ /usr/local/cmcluster/pkg01/external_script.sh

Add the additional external script into the package configuration file. For Example:

external_script /usr/local/cmcluster/pkg01/external_script.sh

Modify the external script in the adoptive node only. Specify the cmmodpkg command with the package control option (-d) in the function start_command. As below:

function start_command
{
    sg_log 5 "start_command"
	   cmmodpkg –d –n `hostname` pkg02 &
    return $?
}

Also, add the cmmodpkg command with package control option (-e ) in the function stop_command for re-enabling the package (pkg02) when the package (pkg01) is halted. For example:

function stop_command
{
    sg_log 5 "stop_command"
	   cmmodpkg –e –n `hostname` pkg02 &
    return $?
}

This package control function can prevent an adoptive node from becoming overloaded when multiple packages fail over. If an adoptive node becomes overloaded, it can fail. In this example, if a host is an adoptive node for both pkg01 and pkg02, disabling of pkg02, in the external script for pkg01, would prevent the host that is running pkg01 from adopting pkg02, and once the package pkg01 is halted in the node the pkg02 is enabled by using the command cmmodpkg –e in the function stop_command. The ampersand (&) causes the cmmodpkg command to run in the background.

The cmmodpkg command in the background allows the control script to complete and finish bringing up the package. There is a small window of time, during which if one package has begun to fail over but the cmmodpkg command has not executed, the other package can fail over and the host will adopt it. In other words, if two packages fail over at approximately the same time, a host may adopt both packages, even though the package control option is specified. If you omit the cmmodpkg -d command from the NFS control script, host sage can adopt both pkg01 and pkg02 if their primary nodes fail.

	NOTE: The above changes in the external script must be done only on the adoptive node and NOT on the external script in the primary node. If there are more than 2 packages the user may repeat the cmmodpkg command as many times required with other package names. User may consider making FAILBACK_POLICY as AUTOMATIC to free the adoptive node as early as possible so that adoptive node is ready to accept pkg02 in case it fails.

This section shows the package configuration file (pkg02.conf) for the package pkg01 in this sample configuration. The comments are not shown.

package_name 			 pkg02
package_type      	failover
failover_policy    configured_node
failback_policy    manual
node_name 			basil 
node_name 			sage
auto_run                   yes
node_fail_fast_enabled     no
script_log_file 		/usr/local/cmcluster/pkg02/log 
TKIT_DIR 			/usr/local/cmcluster/pkg02
XFS[0]				"-o rw *:/hanfs/nfsu021"
QUOTA_MON				YES 
LOCK_MIGRATION			NO
monitored_subnet 		192.100.112.0
monitored_subnet_access 	full 
service_name               nfs2.monitor
service_cmd     "$SGCONF/scripts/tkit/nfs/tkit_module.sh nfs_monitor"
service_restart            none
service_fail_fast_enabled  no
service_halt_timeout       300
external_script            $SGCONF/scripts/tkit/nfs/lock_migration.sh
ip_subnet 	192.100.112.0
#ip_subnet_node
ip_address 	192.100.112.244
vgchange_cmd 	"vgchange -a y"                
vg					nfsu02
fs_name				    /dev/nfsu02/lvol1	
fs_directory			/hanfs/nfsu021
fs_type				ext3
fs_mount_opt			"-o rw"
# fs_umount_opt
# fs_fsck_opt

To prevent a shared adoptive node from adopting both packages together, specify the cmmodpkg command with the package control option (-d) in the external script. The external script template is provided at /usr/local/cmcluster/conf/examples/external_script.template in Red Hat and at /opt/cmcluster/conf/examples/external_script.template in SLES. Copy and rename the external_script.template to the package directory with the following command.

# cp /usr/local/cmcluster/conf/examples/external_script.template \ /usr/local/cmcluster/pkg02/external_script.sh

Add the additional external script into the package configuration file. For example:

external_script /usr/local/cmcluster/pkg02/external_script.sh

Modify the external script in the adoptive node. Specify the cmmodpkg command with the package control option (-d) in the function start_command. As below:

function start_command
{
 sg_log 5 "start_command"
	cmmodpkg –d –n `hostname` pkg01 &
 return $?
}

Also, add cmmodpkg command with package control option (-e ) in the function stop_command for re-enabling the package pkg0 when the package pkg02 is halted. For example:

function stop_command
{
 sg_log 5 "stop_command"
	cmmodpkg –e –n `hostname` pkg01 &
 return $?
}

This package control function can prevent an adoptive node from becoming overloaded when multiple packages fail over. If an adoptive node becomes overloaded, it can fail. In this example, if a host is an adoptive node for both pkg01 and pkg02, disabling of pkg01, would prevent the host that is running pkg02 from adopting pkg01, and once the package pkg02 is halted in the node the pkg01 is enabled by the command cmmodpkg –e in the function stop_command. The ampersand (&) causes the cmmodpkg command to run in the background.

The cmmodpkg command in the background allows the control script to complete and finish bringing up the package. There is a small window of time, during which if one package has begun to fail over but the cmmodpkg command has not executed, the other package can fail over and the host will adopt it. In other words, if two packages fail over at approximately the same time, a host may adopt both packages, even though the package control option is specified. If the cmmodpkg -d command is omitted from the NFS control script the host sage can adopt both pkg01 and pkg02, if their primary nodes fail.

	NOTE: The above changes in the external script must be done only in the adoptive node and not on the external script in the primary node. If there are more than 2 packages the user may repeat the cmmodpkg command as many times required with other package names. Consider making FAILBACK_POLICY as AUTOMATIC to free the adoptive node as early as possible so that adoptive node is ready to accept pkg02 in case it fails.