Redundant Disk Storage

Each node in a cluster has its own root disk, but each node is also physically connected to several other disks in such a way that more than one node can obtain access to the OPS database. This access is provided by a Storage Manager, such as Logical Volume Manager (LVM), VERITAS Volume Manager (VxVM), or VERITAS Cluster Volume Manager (CVM). For OPS data, a disk storage group can be activated by one or more nodes at a time using shared LVM or CVM. Disk storage for OPS data is made redundant by using RAID on disk arrays or, in two-node clusters only, by LVM mirroring. OPS 8.x and 9i use shared database disks for data, log files, and control files.

If you have configured your cluster to run packages that access non-OPS data, they will also require disks, separate from those that contain OPS database files. The disks are placed in volume groups whose logical volumes can be mirrored to a disk on a different bus. Storage groups used for non-OPS data can be activated only on one node at a time.

Supported Disk Interfaces

The following interfaces are supported by ServiceGuard OPS Edition for disks that are connected to two or more nodes (shared data disks):

Single-ended SCSI.
Fast/Wide SCSI.
FibreChannel.

Not all SCSI disks are supported. See the HP 9000 Servers Configuration Guide (available through your HP representative) for a list of currently supported disks.




	NOTE: In a cluster that contains systems with PCI SCSI adapters, you cannot attach both PCI and NIO SCSI adapters to the same shared SCSI bus.

External shared Fast/Wide SCSI buses must be equipped with in-line terminators for disks on a shared bus. Refer to the chapter "Chapter 8 “Troubleshooting Your Cluster”" for additional information.

When planning and assigning SCSI bus priority, remember that one node can dominate a bus shared by multiple nodes, depending on what SCSI addresses are assigned to the controller for each node on the shared bus. All SCSI addresses, including the addresses of all interface cards, must be unique for all devices on a shared bus. See the manual Configuring HP-UX for Peripherals for information on SCSI bus addressing and priority.

Data Protection

It is required that you provide data protection for your highly available system, using one of two methods:

Disk Mirroring
Disk Arrays using RAID Levels and Multiple Data Paths

Disk Mirroring

In disk mirroring (supported on two-node clusters only), the logical volumes used for OPS data are mirrored with HP's MirrorDisk/UX product. When you configure logical volumes using software mirroring, the members of each mirrored set contain exactly the same data. If one disk should fail, the storage manager will automatically keep the data available by accessing the other mirror. Three-way mirroring in LVM (or additional plexes with VxVM) may be used to allow for online backups or even to provide an additional level of high availability.

To protect against FibreChannel or SCSI bus failures, each copy of the data must be accessed by a separate bus; that is, you cannot have all copies of the data on disk drives connected to the same bus.

While it is valuable to mirror your root disks, it is critical for high availability that you mirror your data disks. In the event of damage to a root disk of one SPU, the other SPUs in the cluster will take over control of the applications. However, if a data disk is damaged, and not mirrored, any application which depends on that disk will not be available until the problem with the disk is repaired. Even then, it may not be possible to recover the data on that disk.

Disk Arrays using RAID Levels and Multiple Data Paths

An alternate method of achieving protection for your data is to employ a disk array with hardware RAID levels that provide data redundancy, such as RAID Level 1 or RAID Level 5. The array provides data redundancy for the disks. This protection needs to be combined with the use of redundant host bus interfaces (SCSI or FibreChannel) between each node and the array. The use of redundant interfaces, configured with LVM's PV Links feature or VxVM's dynamic multipathing (DMP), protects against single points of failure in the I/O channel, and RAID 1 or 5 configuration provides redundancy for the storage media. (PV links are also known as alternate links in LVM, or multiple paths in VxVM.)

DMP is available as a separate component of VxVM. DMP for active/active devices requires B9116AA, but DMP for active/passive devices is available for no charge with the base product, B7961AA.

Monitoring of Disks with Event Monitoring Service

You can configure disk monitoring to detect a failed mechanism by using the disk monitor capabilities of the EMS HA Monitors, available as a separate product (B5735AA). Monitoring can be set up to trigger a package failover or to report disk failure events to a target application such as ClusterView. Refer to the manual Using EMS HA Monitors (HP part number B5735-90001) for additional information.

Sample SCSI Disk Configurations

Figure 2-5 “Mirrored Disks Connected for High Availability ” shows a two node cluster. Each node has one root disk which is mirrored and one OPS instance for which it is the primary node. Resources have been allocated to each node so that each node may adopt the package from the other node. Each package has one disk volume group assigned to it and the logical volumes in that volume group are mirrored. Please note that Package A's disk and the mirror of Package B's disk are on one interface while Package B's disk and the mirror of Package A's disk are on a separate bus. This arrangement eliminates single points of failure and makes either the disk or its mirror available in the event one of the buses fails.

Figure 2-5 Mirrored Disks Connected for High Availability

Figure 2-6 “Cluster with High Availability Disk Array ” shows a similar cluster with a disk array connected to each node on two I/O channels. In this configuration, Logical Volume Manager's PV links are used to define the separate pathways to the data from one node.

Figure 2-6 Cluster with High Availability Disk Array

Sample FibreChannel Disk Configuration

In Figure 2-7 “Cluster with FibreChannel Switched Disk Array” below, the root disks are shown with simple mirroring, but the shared storage is now accessed via redundant FibreChannel switches attached to a disk array. The cabling is set up so that each node is attached to both switches, and both switches are attached to the disk array with redundant links.

Figure 2-7 Cluster with FibreChannel Switched Disk Array

This type of configuration also uses PV links or other multipath software such as VERITAS Dynamic Multipath (DMP) or EMC PowerPath.

Replacement of Failed Disk Mechanisms

With disk arrays, it is possible to replace a disk while the cluster stays up and the application remains online. Consult your disk array documentation for details.

Root Disk Limitations on Shared Buses

The IODC firmware does not support two or more nodes booting from the same SCSI bus at the same time. For this reason, it is important not to attach more than one root disk per cluster to a single SCSI bus. For example, Figure 2-8 “Root Disks on Different Shared Buses ” shows a supported configuration in which two nodes share an external SCSI bus and Node A has its primary root disk connected to the bus, but node B has its primary root disk connected to a different bus. (Numbers 0 to 3, 6 and 7 are SCSI addresses on the different buses.)

Figure 2-8 Root Disks on Different Shared Buses

Note that if both nodes had their primary root disks connected to the same bus, you would have an unsupported configuration.

You can put a mirror copy of Node B's root disk on the same SCSI bus as Node A's primary root disk, because three failures would have to occur for both systems to boot at the same time, which is an acceptable risk. In such a scenario, Node B would have to lose its primary root disk and be rebooted, and Node A would have to be rebooted at the same time Node B is, for the IODC firmware to run into a problem. This configuration is shown in Figure 2-9 “Primaries and Mirrors on Different Shared Buses ”.

Figure 2-9 Primaries and Mirrors on Different Shared Buses

Note that you cannot use a disk within a disk array as a root disk if the array is on a shared bus.