Setting up M by N Configurations

Metropolitan clusters using EMC/SRDF can be built in configurations that use more than two EMC Symmetrix disk arrays. In such configurations, M arrays located in Data Center A may be connected to N arrays located in Data Center B. This section describes how to set up this configuration using SymCLI and HP-UX commands. It is assumed that you have already set up the Symmetrix CLI database on each node as described in the previous section, "Preparing a Cluster for MetroCluster/SRDF." It is also assumed that Symmetrix PowerPath software is installed on all nodes.




	WARNING! M by N configurations cannot be used with R1/R2 swapping.

Figure 4-19 “Devices and Symmetrix Units in M by N Configurations” shows a 2 by 2 configuration. Data in this figure are used in the example commands given in the following sections. The example shows R1 devices at one data center and R2 devices with Business Continuity Volumes (BCVs) at the other, but a bidirectional configuration is also possible, with R1 devices on both sites.

Figure 4-19 Devices and Symmetrix Units in M by N Configurations

Creating Symmetrix Device Groups

For each node on the R1 side (node1 and node2), create the device groups as follows. You have to create two device groups because device groups do not span frames.




	NOTE: In a 1 by 2 or 2 by 1 configuration, on either the R1 or R2 side, whichever side has the one Symmetrix frame, a minimum of two device groups is required. The rule of thumb is that you need one device group per RDF group (RA group), even if you have only one frame on that side. The following examples are based on the 2 by 2 configuration shown in Figure 4-19 “Devices and Symmetrix Units in M by N Configurations”.

Create device groups using the following commands on each node on the R1 side:

# symdg -type RDF1 create dgoraA

# symdg -type RDF1 create dgoraB

For each node on the R2 side (node3 and node4), create the device groups as follows. Note that you have to create two device groups because device groups do not span frames. Do the following on each node on the R2 side:

# symdg -type RDF2 create dgoraA

# symdg -type RDF2 create dgoraB

For each node on the R1 side (node1 and node2), assign the R1 devices to the device groups as follows:

# symld -sid 638 -g dgoraA add dev 00C

# symld -sid 638 -g dgoraA add dev 00D

# symld -sid 130 -g dgoraB add dev 010

# symld -sid 130 -g dgoraB add dev 011

For each node on the R2 side (node3 and node4), assign the R2 devices to the device groups as follows:

# symld -sid 021 -g dgoraA add dev 018

# symld -sid 021 -g dgoraA add dev 019

# symld -sid 363 -g dgoraB add dev 050

# symld -sid 363 -g dgoraB add dev 051

On each node on the R2 side (node3 and node4), associate the local BCV devices to the R2 device group:

# symbcv -g dgoraA add dev 01A

# symbcv -g dgoraA add dev 01B

# symbcv -d dgoraB add dev 052

# symbcv -d dgoraB add dev 053

To manage the BCV devices from the R1 side, you need to associate the BCV devices with the device groups that are configured on the R1 side. Use the following commands on hosts directly connected to the R1 Symmetrix:

# symbcv -g dgoraA associate dev 01A -rdf

# symbcv -g dgoraA associate dev 01B -rdf

# symbcv -g dgoraB associate dev 052 -rdf

# symbcv -g dgoraB associate dev 053 -rdf

Now establish the BCV devices using the following commands from the R2 side:

# symmir -g dgoraA -full est

# symmir -g dgoraB -full est

Alternatively, you can establish the BCV devices with the following commands from the R1 side:

# symmir -g dgoraA -full est -rdf

# symmir -g dgoraB -full est -rdf

Configuring Gatekeeper Devices

You need a gatekeeper device for each device group in the consistency group that will be built in a later step. Use the following commands on all nodes on the R1 side to define gatekeepers and associate them with device groups:

# symgate -sid 638 define dev 010

# symgate -sid 130 define dev 009

# symgate -sid 638 -g dgoraA associate dev 010

# symgate -sid 130 -g dgoraB associate dev 009

Use the following commands on all nodes on the R2 side to define gatekeepers and associate them with device groups:

# symgate -sid 021 define dev 002

# symgate -sid 363 define dev 00B

# symgate -sid 021 -g dgoraA associate dev 002

# symgate -sid 363 -g dgoraB associate dev 00B

Defining the Consistency Groups

To configure consistency groups for use with MetroCluster/SRDF, you first create device groups and gatekeeper groups as described in previous sections. Then you create the consistency groups and add the devices to them as well.




	NOTE: Each package requires its own unique consistency group name.

Use the following steps for each package:

On each node in the cluster, create an empty consistency group using the symcg command:
# symcg create cgoradb
You can use the same name on all nodes.
Add each device that is going to be used in the consistency group. Use the appropriate SID numbers and device names for the data center that the node is a part of. For example, on node 1 and node 2 in Data Center A:
# symcg -cg cgoradb -sid 638 add dev 00C
# symcg -cg cgoradb -sid 638 add dev 00D
# symcg -cg cgoradb -sid 130 add dev 010
# symcg -cg cgoradb -sid 130 add dev 011
And on node 3 and node 4 in Data Center B:
# symcg -cg cgoradb -sid 021 add dev 018
# symcg -cg cgoradb -sid 021 add dev 019
# symcg -cg cgoradb -sid 363 add dev 050
# symcg -cg cgoradb -sid 363 add dev 051
Enable the consistency group:
NOTE: This important step must be carried out on every node.
# symcg -g cgoradb enable
Establish the BCV devices in the secondary Symmetrix as a mirror of the standard device. From either node3 or node4, issue the following commands:
# symmir -g dgoraA -full est
# symmir -g dgoraB -full est
Alternatively, from either node1 or node2, issue the following commands:
# symmir -g dgoraA -full est -rdf
# symmir -g dgoraB -full est -rdf

Creating Volume Groups

The following procedures assume you are creating volume groups for a cluster with a 2 by 2 Symmetrix configuration like that of Figure 4-19 “Devices and Symmetrix Units in M by N Configurations”. Use the following steps on node1:

Create the physical volumes:
# pvcreate -f /dev/rdsk/c6t0d0
# pvcreate -f /dev/rdsk/c6t0d1
# pvcreate -f /dev/rdsk/c5t0d2
# pvcreate -f /dev/rdsk/c5t0d3
Create the directories and special files for the volume groups.
# mkdir /dev/vgoraA
# mkdir /dev/vgoraB
# mknod /dev/vgoraA/group c 64 0x01000
# mknod /dev/vgoraB/group c 64 0x02000
Create the volume groups. Be careful not to span Symmetrix frames:
# vgcreate /dev/vgoraA /dev/rdsk/c6t0d0
# vgextend /dev/vgoraA /dev/rdsk/c6t0d1
# vgcreate /dev/vgoraB /dev/rdsk/c5t0d2
# vgextend /dev/vgoraB /dev/rdsk/c5t0d3
Create the logical volumes. (XXXX indicates size in MB)
# lvcreate -L XXXX /dev/vgoraA
# lvcreate -L XXXX /dev/vgoraB
Install a VxFS file system on the logical volumes:
# newfs -F vxfs /dev/vgoraA/rlvol1
# newfs -F vxfs /dev/vgoraB/rlvol1
Create map files to permit exporting the volume groups to other systems:
# vgchange -a n vgoraA
# vgchange -a n vgoraB
# vgexport -v -s -p -m /tmp/vgoraA.map vgoraA
# vgexport -v -s -p -m /tmp/vgoraB.map vgoraB
Copy the map files to the other nodes in the cluster:
# rcp /tmp/vgoraA.map node2:/tmp/vgoraA.map
# rcp /tmp/vgoraB.map node2:/tmp/vgoraB.map
Split the SRDF logical links:
# symrdf -g dgoraA split -v
# symrdf -g dgoraB split -v

On node2, node3, and node4, perform the following steps:

Create the volume group directories and special files:
# mkdir /dev/vgoraA
# mkdir /dev/vgoraB
Import the volume groups to each system:
# vgimport -v -s -m /tmp/vgoraA.map vgoraA
# vgimport -v -s -m /tmp/vgoraB.map vgoraB
After importing volume groups to all the other nodes, establish SRDF links:
# symrdf -gdgoraA establish -v
# symrdf -gdgoraB establish -v

Creating VxVM Disk Groups for Use with MetroCluster/SRDF

If you are using VERITAS storage, use the following procedure to create disk groups. It is assumed that you have already created a VERITAS root disk (rootdg) on the system where you are configuring the storage. The following section shows how to set up VERITAS disk groups. On one node do the following:

Check to make sure the devices are in a synchronized state:
# symrdf -g dgoraA query
# symrdf -g dgoraB query
Initialize disks to be used with VxVM by running the vxdisksetup command.
# vxdisksetup -i c5t0d0
Create the disk group to be used by using the vxdg command. On the primary system run the following command:
# vxdg init logdata /dev/dsk/c5t0d2 /dev/dsk/c5t0d3 /dev/dsk/c5t0d0 /dev/dsk/c5t0d1
Verify the configuration with the following command:
# vxdg list
Use the vxassist command to create the logical volume.
# vxassist -g logdata make logfile 2048m
Verify the configuration with the following command:
# vxprint -g logdata
Make the filesystem with the following command:
# newfs -F vxfs /dev/vx/rdsk/logdata/logfile
Create a directory to mount the volume group.
# mkdir /logs
Mount the volume group.
# mount /dev/vx/dsk/logdata/logfile /logs
Check if file system exits, then unmount the file system
# umount /logs

Validating VxVM Disk Groups for Use with MetroCluster/SRDF

The following section shows how to validate VERITAS diskgroups. On one node do the following:

Deport the disk group.
# vxdg deport logdata
Enable other cluster nodes to have access to the disk group.
# vxdctl enable
Split the SRDF link to enable R2 Read/Write permission.
# symrdf -g dgoraA split
# symrdf -g dgoraB split
Import the disk group.
# vxdg -tfC import logdata
Start the logical volume in the disk group.
# vxvol -g logdata startall
Create a directory to mount the volume.
# mkdir /logs
Mount the volume
# mount /dev/vx/dsk/logdata/logfile /logs
Check to make sure the file system is present, then unmount the file system.
# umount /logs
Establish the SRDF link:
# symrdf -g devgrpA establish

Setting up M by N Configurations

Technical documentation

» Table of Contents

» Glossary