Preparing the Cluster for Data Replication

When the following procedures are completed, an adoptive node will be able to access the data belonging to a package after it fails over. Use the convenience scripts in the /opt/cmcluster/toolkits/SGSRDF/Samples to automate some of the tasks in the following sections:

mk3symgrps.nodename — to create EMC Symmetrix device groups.
mk4gatekpr.nodename — to create gatekeeper devices.
mk2imports — to import volume groups.
ftpit — to copy the configuration to other nodes in the cluster.
pre.cmquery — to split SRDF links before applying the package configuration.
post.cmapply — to restore SRDF links after applying the package configuration.

These scripts should be copied from /opt/cmcluster/toolkits/SGSRDF to another directory, such as /etc/cmcluster/SRDF.

Installing the Necessary Software

Before any configuration can begin, you need to make sure the following software is installed on all nodes:

Symmetrix SymCLI command line interface that allows you to manage the Symmetrix disks from the node.
Symmetrix PowerPath software if you are building an M by N configuration.
Metrocluster with Symmetrix SRDF should be installed on all nodes according to the instructions in the Metrocluster with EMC SRDF Release Notes.

Building the Symmetrix CLI Database

The Symmetrix CLI (Command Line Interface) should be installed on all nodes running packages that use data on the EMC Symmetrix disk arrays. Create the SymCLI database on each system using the following steps. For complete information, refer to the Symmetrix SymCLI manual.

Issue the following command on each node after the hardware is installed:

# symcfg discover

This builds the CLI database on the node.

You can display what is in the SymCLI database with the commands:

symdg list
symld -g symdevgrpname list
symgate list

If you have not configured the SymCLI database, you will see an error:

The Symmetrix configuration could not be loaded for a locally attached Symmetrix




	NOTE: Make sure that you do not set the `SYMCLI_SID` and `SYMCLI_DG` environment variables before running the symcfg command. These environment variables limit the amount of information gathered when the SymCLI database is created, and will not give you a complete database. Also, the `SYMCLI_OFFLINE` variable should not be set. This environment variable disables the command line interface.

In order to use SymCLI version V.5.1.0 with Metrocluser/SRDF A.04.10 and later, a compatibility variable needs to be set in the SRDF environment files. Add the following line to the environment file:

export SYMCLI_MODE=V50

Determining Symmetrix Device Names on Each Node

To correctly specify the device file names when creating Symmetrix device groups, you need to know how the HP-UX device files map to the R1 and R2 Symmetrix devices. Use the following steps to gather the necessary information.

Obtain a list of data for the Symmetrix devices available, using the following command on each node without any options:
# syminq
Sample output from both the R1 and R2 sides is shown in Figure 7-2 “Sample syminq Output from a Node on the R1 Side” and Figure 7-3 “Sample syminq Output from a Node on the R2 Side”.
Figure 7-2 Sample syminq Output from a Node on the R1 Side
Figure 7-3 Sample syminq Output from a Node on the R2 Side
You need the following information from these listings for each Symmetrix logical device:
- HP-UX device file name (e.g. /dev/rdsk/c3t3d2).
- device type (R1, R2, BCV, GK, or blank)
- Symmetrix serial number (e.g. 50006161), useful in matching the HP-UX device names to the actual devices in the Symmetrix configuration downloaded by EMC support staff. This number is further explained in Figure 7-4 “Parsing the Symmetrix Serial Number”.
  Figure 7-4 Parsing the Symmetrix Serial Number
  The Symmetrix ID is the same as the last two digits of the serial number of the Symmetrix frame, in this example 50.
  The next three hexadecimal digits are the unique Symmetrix device number that is seen in the output of the status command: symrdf -g symdevgrpname query
  This is used by the Metrocluster with Symmetrix SRDF control script and saved in the file /etc/cmcluster/package_name/symrdf.out. The contents of this file may be useful for debugging purposes.
  The next three digits indicate the Symmetrix host adapter (SA or FA) and port numbers; this is useful to see multiple host links to the same Symmetrix device. For example, PV links will show up as two HP-UX device file names with the same device number, but with different host adapter and port numbers.
Use the symrdf command on each Symmetrix disk array (that is, from both the R1 and the R2 side) to pair the logical device names for the R1 and R2 sides of each SRDF link:
# symrdf list
Sample output is shown in Figure 7-5 “Sample symrdf list Output from R1 Side” and Figure 7-6 “Sample symrdf list Output from R2 Side”.
NOTE: The format of output varies depending on the version of symrdf.
Figure 7-5 Sample symrdf list Output from R1 Side
Figure 7-6 Sample symrdf list Output from R2 Side

Match the logical device numbers in the symrdf listings with the HP-UX device file names in the output from the syminq command to see which devices are seen from each node to make sure that this node can see all necessary devices.

Use the Symmetrix ID to determine which Symmetrix array is connected to the node. Then use the Symmetrix device number to determine which devices are the same logical device seen by each node that is connected to the same Symmetrix unit. Record the HP-UX device file names in your table.

Table 7-2 “Partial Mapping for a 4 Node Cluster” show a partial mapping for a 4 node cluster connected to two Symmetrix arrays (95 and 50). Note that you may have many R1 and R2 devices and many gatekeepers for each package, so this table will be much larger for most clusters. Also, with M by N configurations, the number of devices increases according to the number of Symmetrix frames.

Table 7-2 Partial Mapping for a 4 Node Cluster

Symmetrix ID, device #, and type		Node 1 /dev/rdsk device file name	Node 2 /dev/rdsk device file name	Node 3 /dev/rdsk device file name	Nodes 4 /dev/rdsk device file name
ID	95	`c0t4d0`
Dev#	005		`c6t0d0`
Type	R1
ID	50			`c4t0d0`
Dev#	014				`c0t4d0`
Type	R2
ID	95	`c0t2d2`
Dev#	00A		`c0t4d2`
Type	R2
ID	50			`c3t0d2`
Dev#	012				`c4t3d2`
Type	R1
ID	95	`c0t15d0`
Dev#	040		`c0t15d0`
Type	GK
ID	50			`c3t15d1`
Dev#	041				`c5t15d1`
Type	GK
ID	95	`c4t3d2`
Dev#	028		`c4t3d2`
Type	BCV			n/a	n/a




	NOTE: The Symmetrix device number may be the same or different in each of the Symmetrix units for the same logical device. In other words, the device number for the logical device on the R1 side of the SRDF link may be different from the device number for the logical device on the R2 side of the SRDF link. The Symmetrix logical device numbers in these examples were configured to be the same number so that the cluster is easier to manage. If you are reconfiguring an existing cluster, the `Dev` and `RDev` devices will probably not be the same number. When you are determining the configuration for the Symmetrix devices for a new installation, it is recommended that you try to use the same Symmetrix device number for the R1 devices and the R2 devices. It is also recommended that the same target and LUN number be configured for all nodes that have access to the same Symmetrix logical device.

Preparing the Cluster for Data Replication

Technical documentation

» Table of Contents

» Glossary

» Index

Installing the Necessary Software

Building the Symmetrix CLI Database

Determining Symmetrix Device Names on Each Node