Data Storage Setup

By default, the SRDF link is configured with automatic link recovery after all the failure of all links. Work with EMC support engineers to disable this feature by setting the flag “Prevent Automatic Links Recovery after all Links Failure” in the Symmetrix bin file to “Yes.” This will keep the links from automatically trying to establish a new connection upon failure of all links. This is required so you would have time to split the BCV/R1 devices in the secondary Symmetrix from the mirror group before re-establishing the SRDF volume pairs between the primary Symmetrix and the secondary Symmetrix upon link recovery.

Ensure that the Symmetrix ICDAs are correctly cabled to each host system in the cluster that will run packages whose data reside on the Symmetrix.

Install the SymCLI software on each host system that has data residing on the Symmetrix. Using Figure 7-3 “Example of Symmetrix Device Setup”, the SymCLI software is installed on hosts clear, erie, newyork, seattle, atlanta and sanfran.

Create or rebuild the SymCLI database on each system with the most current information. Use the command:

# symcfg discover

Get a list of data for the Symmetrix devices visible to the host. Use the command:

# syminq -sym

to get a list that includes the following data for each Symmetrix logical device:

Gatekeeper devices are usually of size 2880. The Symmetrix device serial number is useful in matching the devices to the actual devices in the Symmetrix configuration (downloaded by EMC):

The BCV devices are configured to be visible on a different SA/FA than R1 or R2 devices. These devices will not show on the output of the syminq command unless there is a host connection to this SA/FA and the BCV’s are split from the mirror group. To get a list of these devices, use the command:

# symdev list

Get a list of the disk devices known to the host system. Use the following command, as appropriate:

# ioscan -kfnC disk

or

# ioscan -fnC disk

Match the device file names in this listing with the device file names in the output from the syminq command to see which devices are seen from this host to make sure that this host can see all necessary devices.

There are three Symmetrix device groups needed for each application package:

Use the symdg, symld, symbcv, and symgate commands on all nodes in both clusters to create the Symmetrix device groups. All devices belonging to Volume Groups that are owned by an application package must be defined in a single Symmetrix device group. A unique Symmetrix gatekeeper device should be created for each application package. The gatekeeper devices must be unique per Serviceguard package to prevent contention in the Symmetrix when commands are issued. Gatekeeper devices are unique to a Symmetrix unit. They are not replicated across the SRDF link. There should be at least two gatekeeper devices accessible from each host system. Each gatekeeper device is configured on different physical links. The SymCLI will switch to an alternate gatekeeper device if the path to the primary device failsSee the scripts in the /opt/cmcluster/toolkit/SGSRDF/ cascade/Samples directory for examples of how these commands are used to build the SymCLI database and setup the Symmetrix device groups. The sample scripts are used as follows:

After the device groups are created, the SymCLI database can be viewed with the commands:

Create a text file that contains a list of the standard device and BCV device pair in the secondary Symmetrix. This file is required when doing operations on the BCV devices in the secondary Symmetrix from a node that connects to the recovery Symmetrix.

This file has two columns. The first column contains the Symmetrix device number for the standard device. The second column contains the Symmetrix device number for the BCV device. Refer to the sample file midhop in the /opt/cmcluster/toolkit/SGSRDF/cascade/Samples directory.

Create a text file that contains a list of the standard device and BCV device pair in the recovery Symmetrix. This file is required when doing operations on the BCV devices in the recovery Symmetrix from a node that connects to the primary Symmetrix or the secondary Symmetrix. Refer to the sample file lasthop in the /opt/cmcluster/toolkit/SGSRDF/cascade/Samples directory.

Before creating the volume groups, make sure that the SRDF link is established between the R1 devices in primary Symmetrix and the R2 devices in the secondary Symmetrix, and the BCV/R1 devices in the secondary Symmetrix are established as mirrors of the local standard devices. On a node that connects to the primary Symmetrix, use the commands:

# symrdf -g <prisymdevgrpname> est -v

# symmir -g <prisymdevgrpname> -full est -rdf

Define the appropriate volume groups on each host system that might run the application package. Use the commands:

# mkdir /dev/vgxx

# mknod /dev/vgxx/group c 64 0xnn0000

where the name /dev/vgxx and the number nn are unique within the entire continental cluster.

Create the volume group only on one cluster node that connects to the primary Symmetrix. Use the vgcreate and perhaps the vgextend command, specifying the appropriate special device file names.

The LVM information needs to be replicated to the R2 devices in the recovery Symmetrix. See the script copytorec in the /opt/cmcluster/toolkit/SGSRDF/cascade/Samples directory for an example of how the replication is done. From a node that connects to the primary Symmetrix, run the script or do the following:

Generate a map file by exporting the VGs on the node that was used to create the VGs without removing the special device files. Uses the commands:

# vgchange -a n <vgname>

# vgexport -s -p -m <mapfilename <vgname>

Make sure to copy the mapfiles to all of the host systems.

Import the VGs on all of the other systems that might run the Serviceguard package and back up the LVM configuration. Do this on both the primary and recovery cluster nodes. Make sure the SRDF link between the primary Symmetrix and the secondary Symmetrix is split before importing the VGs.

Use the following commands:

# symrdf -g <prisymdevgrpname> split -v

# vgimport -s -m <mapfilename> <vgname>

# vgchange -a y <vgname>

# vgcfgbackup <vgname>

# vgchange -a n <vgname>

# symrdf -g <prisymdevgrpname> est -v

Confirm the volume groups have been imported using command:

# strings /etc/lvmtab

Split the SRDF links between the primary Symmetrix and the secondary Symmetrix (if linked). Issue the following command from a node that connects to the primary Symmetrix:

# symrdf -g <prisymdevgrpname> split -v

Activate the volume group on the primary node in the primary cluster, and mount the logical volumes.

# vgchange -a y <vgname>

# mkdir <tempdir>

# mount /dev/<vgname>/<lvolname> <tempdir>

Check the mount points:# bdf

Do they appear? If yes, then deactivate the volume groups:# umount <tempdir>

# vgchange -a n <vgname>

Perform step 3 on all adoptive nodes in the primary cluster and in the recovery cluster.