When the following
procedures are completed, an adoptive node will be able to access
the data belonging to a package after it fails over. Use the convenience
scripts in the /opt/cmcluster/toolkits/SGSRDF/Samples to automate some of the tasks in the following sections:
mk3symgrps.nodename — to create EMC Symmetrix device groups.
mk4gatekpr.nodename — to create gatekeeper devices.
mk2imports — to import volume groups.
ftpit — to copy the configuration to other nodes in
the cluster.
pre.cmquery — to split SRDF links before applying the package configuration.
post.cmapply — to restore SRDF links after applying the package configuration.
These scripts should be copied from /opt/cmcluster/toolkits/SGSRDF to another directory, such as /etc/cmcluster/SRDF.
Installing the Necessary Software |
 |
Before any configuration can begin, you need to make sure
the following software is installed on all nodes:
Symmetrix SymCLI command line interface
that allows you to manage the Symmetrix disks from the node.
Symmetrix PowerPath software if you are building
an M by N configuration.
If using HP-PB F/W SCSI host adapters in HP-UX 10.20,
install and configure the HP-PB F/W SCSI Pass-Through driver. Otherwise,
the SymCLI commands will not work properly. In case of improper configuration,
you will see error messages in the package log file like:
Bad File Number.
MetroCluster with EMC SRDF should be installed on
all nodes according to the instructions in the MetroCluster
with EMC SRDF Release Notes.
Setting up Hardware for 1 by 1 Configurations |
|
Ensure
that the Symmetrix disk arrays are correctly cabled using PV links
to each node in the cluster that will run packages that access data on
the Symmetrix.
See the SymCLI manual for instructions on creating the appropriate pseudo
device files.
R1 and R2 devices must have been correctly
defined and assigned to the appropriate nodes in the internal configurations
that is downloaded by EMC support staff.
R1 devices are locally protected (RAID 1 or RAID
S).
R2 devices are locally protected (RAID 1, RAID S
or BCV).
Only Synchronous Mode is supported; Adaptive Copy
must be disabled.
Domino Mode is recommended to ensure data currency
on each EMC Symmetrix disk array.
To minimize contention, each package should be assigned
its own, unique gatekeeper device. See “Configuring Gatekeeper Devices”.
Setting up Hardware for M by N Configurations |
|
You can configure up to four Symmetrix disk arrays in the
following combinations:
An array in Data Center A connected
to one array in Data Center B.
An array in Data Center A connected to two arrays
in Data Center B.
Two arrays in Data Center A connected to an array
in Data Center B.
Two arrays in Data Center A connected to two arrays
in Data Center B.
Figure 4-8 “2 by 1 Node and Data Center
Configuration” shows a 2 by 1 configuration with
BCVs.
The figure indicates R1 volumes at Data Center A and R2 volumes
and BCVs at Data Center B for pkg A and pkg B.
Figure 4-9 “2 by 2 Node and Data Center
Configuration” shows a 2 by 2 configuration with
R1 volumes for pkg A and pkg B on the Symmetrix frames located in
Data Center A and R2 volumes and BCVs at Data Center B. Many of
the examples given later in this chapter are based on this configuration.
Figure 4-10 “Bidirectional 2 by 2 Configuration” below shows a bidirectional 2
by 2 configuration with additional packages on node3 and node4,
and R1 and R2 volumes at both data centers. In this configuration,
R1 volumes for pkg A and pkg B are at Data Center A, and R2 volumes
are at Data Center B. R1 volumes for pkg C and pkg D are at Data
Center B, and R2 volumes are at Data Center A.
Grouping the Symmetrix Devices at Each Data Center |
|
The use of R1/R2 devices in M by N configurations of multiple Symmetrix
frames is enabled by means of consistency groups.
A consistency group is a set of Symmetrix RDF devices that are configured to
act in unison to maintain the integrity of a database. Because MetroCluster/SRDF
works at the device group level, the consistency group is implemented
and managed as a single device group even though it spans multiple
Symmetrix frames. Consistency groups are managed by the EMC PowerPath
product. When PowerPath is installed, the Symmetrix tracks the I/Os
that are written to the devices in the consistency group. If an
I/O cannot be written to a remote Symmetrix due to a failure of
a remote device or an RDF link, the data flow to the other Symmetrix
will be halted in less than one second. Once mirroring is resumed,
any updates will propagate with normal SRDF operation.
Figure 4-11 “2 X 2 Node and Data Center
Configuration with Consistency Groups” shows that when there is a break
in the links between two of the Symmetrix frames, the use of consistency
groups (dashed oval lines) ensures that the other two links are
also suspended.
Building the Symmetrix CLI Database |
|
The Symmetrix CLI (Command Line
Interface) should be installed on all nodes running packages that
use data on the EMC Symmetrix disk arrays. Create the SymCLI database
on each system using the following steps. For complete information,
refer to the Symmetrix SymCLI manual.
Issue the following command on each node after the hardware
is installed:
# symcfg discover
This builds the CLI database on the node.
You can display what is in the SymCLI database with the commands:
symld -g symdevgrpname list
If you have not configured the SymCLI database, you will see
an error:
The Symmetrix configuration could not be loaded for a locally attached Symmetrix |
Determining Symmetrix Device Names on Each Node |
|
To correctly specify the device file names when creating Symmetrix device
groups, you need to know how the HP-UX device files map to the R1
and R2 Symmetrix devices. Use the following steps to gather the necessary
information.
Obtain a list of data for the Symmetrix
devices available, using the following command on each node without
any options:
# syminq
Sample output from both the R1 and R2 sides is shown in Figure 4-12 “Sample syminq Output from a
Node on the R1 Side” and Figure 4-13 “Sample syminq Output from a Node on the R2 Side”.
You need the following information from these listings
for each Symmetrix logical device:
HP-UX device file name (e.g. /dev/rdsk/c3t3d2).
device type (R1, R2, BCV, GK, or blank)
Symmetrix serial number
(e.g. 50006161), useful in matching the HP-UX device names to
the actual devices in the Symmetrix configuration downloaded by
EMC support staff. This number is further explained in Figure 4-14 “Parsing the Symmetrix Serial
Number”.
The Symmetrix ID is the same as the last two digits of the serial
number of the Symmetrix frame, in this example 50.
The next three hexadecimal digits are the unique
Symmetrix device number that is seen in the output of the status command:
symrdf -g symdevgrpname query
and is used by the MetroCluster with EMC SRDF control script and
saved in the file /etc/cmcluster/package_name/symrdf.out. The contents of this file may be useful for debugging
purposes.
The next three digits indicate the Symmetrix host
adapter (SA or FA) and port numbers; this is useful to see multiple
host links to the same Symmetrix device. For example, PV links will show
up as two HP-UX device file names with the same device number, but
with different host adapter and port numbers.
Use the symrdf command on each Symmetrix disk array (that is, from both
the R1 and the R2 side) to pair the logical device names for the
R1 and R2 sides of each SRDF link:
# symrdf list
Sample output is shown in Figure 4-15 “Sample symrdf list Output from
R1 Side” and Figure 4-16 “Sample symrdf list Output from
R2 Side”.
Match the logical device numbers in the symrdf listings with the HP-UX device file names in the output
from the syminq command to see which devices are seen from each node
to make sure that this node can see all necessary devices.
Use the Symmetrix ID to determine which Symmetrix array is connected
to the node. Then use the Symmetrix device number to determine which
devices are the same logical device seen by each node that is connected
to the same Symmetrix unit. Record the HP-UX device file names in
your table.
Table 4-6 “Symmetrix Device and HP-UX Device Correlation” show a partial mapping for a 4
node cluster connected to two Symmetrix arrays (95 and 50). Note
that you may have many R1 and R2 devices and many gatekeepers for
each package, so this table will be much larger for most clusters.
Also, with M by N configurations, the number of devices increases
according to the number of Symmetrix frames.
Table 4-6 Symmetrix Device and HP-UX Device Correlation
| Symmetrix ID, device #, and type | Node 1
/dev/rdsk
device file
name | Node 2
/dev/rdsk
device file
name | Node 3
/dev/rdsk
device
file name | Nodes 4
/dev/rdsk
device
file name |
|---|
| ID | 95 | c0t4d0 | | | |
| Dev# | 005 | | c6t0d0 | | |
| Type | R1 | | | | |
| ID | 50 | | | c4t0d0 | |
| Dev# | 014 | | | | c0t4d0 |
| Type | R2 | | | | |
| ID | 95 | c0t2d2 | | | |
| Dev# | 00A | | c0t4d2 | | |
| Type | R2 | | | | |
| ID | 50 | | | c3t0d2 | |
| Dev# | 012 | | | | c4t3d2 |
| Type | R1 | | | | |
| ID | 95 | c0t15d0 | | | |
| Dev# | 040 | | c0t15d0 | | |
| Type | GK | | | | |
| ID | 50 | | | c3t15d1 | |
| Dev# | 041 | | | | c5t15d1 |
| Type | GK | | | | |
| ID | 95 | c4t3d2 | | | |
| Dev# | 028 | | c4t3d2 | | |
| Type | BCV | | | n/a | n/a |
Printable version
