Volume Managers for Data Storage

A volume manager is a tool that lets you create units of disk storage known as storage groups. Storage groups contain logical volumes for use on single systems and in high availability clusters. In Serviceguard clusters, storage groups are activated by package control scripts.

Types of Redundant Storage

In Serviceguard, there are two types of supported shared data storage: mirrored individual disks (also known as JBODs, for “just a bunch of disks”), and external disk arrays which configure redundant storage in hardware. Two types of mirroring are RAID1 and RAID5. Here are some differences between the two storage methods:

If you are using JBODs, the basic element of storage is an individual disk. This disk must be paired with another disk to create a mirror (RAID1). (Serviceguard configurations usually have separate mirrors on different storage devices).
If you have a disk array, the basic element of storage is a LUN, which already provides storage redundancy via hardware RAID1 or RAID5.

Examples of Mirrored Storage

Figure 3-20 “Physical Disks Within Shared Storage Units” shows an illustration of mirrored storage using HA storage racks. In the example, node1 and node2 are cabled in a parallel configuration, each with redundant paths to two shared storage devices. Each of two nodes also has two (non-shared) internal disks which are used for the root file system, swap etc. Each shared storage unit has three disks, The device file names of the three disks on one of the two storage units are c0t0d0, c0t1d0, and c0t2d0. On the other, they are c1t0d0, c1t1d0, and c1t2d0.

Figure 3-20 Physical Disks Within Shared Storage Units

Figure 3-21 “Mirrored Physical Disks” shows the individual disks combined in a multiple disk mirrored configuration.

Figure 3-21 Mirrored Physical Disks

Figure 3-22 “Multiple Devices Configured in Volume Groups” shows the mirrors configured into LVM volume groups, shown in the figure as /dev/vgpkgA and /dev/vgpkgB. The volume groups are activated by Serviceguard packages for use by highly available applications.

Figure 3-22 Multiple Devices Configured in Volume Groups

Examples of Storage on Disk Arrays

Figure 3-23 “Physical Disks Combined into LUNs” shows an illustration of storage configured on a disk array. Physical disks are configured by an array utility program into logical units or LUNs which are then seen by the operating system.

Figure 3-23 Physical Disks Combined into LUNs




	NOTE: LUN definition is normally done using utility programs provided by the disk array manufacturer. Since arrays vary considerably, you should refer to the documentation that accompanies your storage unit.

Figure 3-24 “Multiple Paths to LUNs” shows LUNs configured with multiple paths (links) to provide redundant pathways to the data.

Figure 3-24 Multiple Paths to LUNs

Finally, the multiple paths are configured into volume groups as shown in Figure 3-25 “Multiple Paths in Volume Groups”.

Figure 3-25 Multiple Paths in Volume Groups

Types of Volume Manager

Serviceguard allows a choice of volume managers for data storage:

HP-UX Logical Volume Manager (LVM) and (optionally) Mirrordisk/UX
VERITAS Volume Manager for HP-UX (VxVM)—Base and add-on Products
VERITAS Cluster Volume Manager for HP-UX (CVM)

Separate sections in Chapters 5 and 6 explain how to configure cluster storage using all of these volume managers. The rest of the present section explains some of the differences among these available volume managers and offers suggestions about appropriate choices for your cluster environment.




	NOTE: The HP-UX Logical Volume Manager is described in Managing Systems and Workgroups. A complete description of VERITAS volume management products is available in the VERITAS Volume Manager for HP-UX Release Notes.

HP-UX Logical Volume Manager (LVM)

Logical Volume Manager (LVM) is the default storage management product on HP-UX. Included with the operating system, LVM is available on all cluster nodes. It supports the use of Mirrordisk/UX, which is an add-on product that allows disk mirroring with up to two mirrors (for a total of three copies of the data).

Currently, the HP-UX root disk can be configured as an LVM volume group. (Note that, in this case, the HP-UX root disk is not the same as the VERITAS root disk group, rootdg, which must be configured in addition to the HP-UX root disk on any node that uses VERITAS Volume Manager 3.5 products. The rootdg is no longer required with VERITAS Volume Manager 4.1 products.) The Serviceguard cluster lock disk also is configured using a disk configured in an LVM volume group.

LVM continues to be supported on HP-UX single systems and on Serviceguard clusters.

VERITAS Volume Manager (VxVM)

The Base VERITAS Volume Manager for HP-UX (Base-VXVM) is provided at no additional cost with HP-UX 11i. This includes basic volume manager features, including a Java-based GUI, known as VEA. It is possible to configure cluster storage for Serviceguard with only Base-VXVM. However, only a limited set of features is available.

The add-on product, VERITAS Volume Manager for HP-UX (B9116BA, B9117BA), provides a full set of enhanced volume manager capabilities in addition to basic volume management. This includes features such as mirroring, dynamic multipathing for active/active storage devices, and hot relocation.

VxVM can be used in clusters that:

are of any size, up to 16 nodes.
require a fast cluster startup time.
do not require shared storage group activation. (required with CFS)
do not have all nodes cabled to all disks. (required with CFS)
need to use software RAID mirroring or striped mirroring.
have multiple heartbeat subnets configured.

Propagation of Disk Groups in VxVM

With VxVM, a disk group can be created on any node, with the cluster up or not. The user then needs to go to each node and validate that disk group by trying to import it. Thus, although there are more steps required for propagating disk groups with VxVM than with CVM (see Cluster Volume Manager, CVM, below), you have the freedom to create the disk group from any node.

Package Startup Time with VxVM

With VxVM, each disk group is imported by the package control script that uses the disk group. This means that cluster startup time is not affected, but individual package startup time might be increased because VxVM imports the disk group at the time the package starts up.

VERITAS Cluster Volume Manager (CVM)

You may choose to configure cluster storage with the VERITAS Cluster Volume Manager (CVM) instead of the Volume Manager (VxVM). The Base-VXVM provides some basic cluster features when Serviceguard is installed, but there is no support for software mirroring, dynamic multipathing (for active/active storage devices), or numerous other features that require the additional licenses.

VxVM supports up to 16 nodes, CVM supports up to 8 nodes, and CFS supports up to 4 nodes.

VxVM 4.1 (B9116BA), CVM (VERITAS Cluster Volume Manager) 3.5 (B9117AA) and CVM 4.1 (B9117BA) are an enhanced versions of the VxVM volume manager that is specifically designed for cluster use. When installed with the VERITAS Volume Manager, the CVM add-on product provides most of the enhanced VxVM features in a clustered environment. CVM is truly cluster-aware, obtaining information about cluster membership from Serviceguard directly.

Cluster information is provided via a special system multi-node package, which runs on all nodes in the cluster. The cluster must be up and must be running this package in order to configure VxVM disk groups for use with CVM. The VERITAS CVM package for version 3.5 is named VxVM-CVM-pkg; the package for CVM version 4.1 is named SG-CFS-pkg.

CVM allows you to activate storage on more than one node at a time. CVM also allows you to activate storage on one node at a time, or you can perform write activation on one node and read activation on another node at the same time (for example, allowing backups). CVM provides full mirroring and dynamic multipathing (DMP) for clusters. If CVM is being used disk groups must be created from the CVM Master node and the cluster must be up.

CVM supports concurrent storage read/write access between multiple nodes by applications which can manage read/write access contention, such as Oracle Real Application Cluster.

CVM 4.1 can be used with VERITAS Cluster File System (CFS) in Serviceguard. Several of the HP Serviceguard Storage Management Suite bundles include features to enable both CVM and CFS.

CVM can be used in clusters that:

will run applications that require fast disk group activation after package failover.
require activation on more than one node at a time, for example to perform a backup from one node while a package using the volume is active on another node. In this case, the package using the disk group would have the disk group active in exclusive write mode while the node that is doing the backup would have the disk group active in shared read mode.
require activation on more than one node at the same time, for example Oracle RAC.

Heartbeat configurations are configured differently for CVM 3.5 and 4.1. See “Redundant Heartbeat Subnet Required ”.

CVM is supported on 8 nodes or fewer at this release. Shared storage devices must be connected to all nodes in the cluster, whether or not the node accesses data on the device.

Cluster Startup Time with CVM

With CVM, all shared disk groups (DGs) are imported when the system multi-node’s control script starts up CVM. Depending on the number of DGs, the number of nodes and the configuration of these (number of disks, volumes, etc.) this can take some time (current timeout value for this package is 3 minutes but for larger configurations this may have to be increased). Any failover package that uses a CVM DG will not start until the system multi-node package is up. Note that this delay does not affect package failover time; it is a one-time overhead cost at cluster startup.

Propagation of Disk Groups with CVM

With CVM, disk groups are created on one cluster node known as the CVM master node. The cluster has to be running to create shared disk groups. CVM will validate that each node can see each disk and will not allow invalid DGs to be created.

Redundant Heartbeat Subnet Required

Serviceguard recommends that you configure all subnets that interconnect cluster nodes as heartbeat networks, since this increases protection against multiple faults at no additional cost.

Heartbeat configurations are configured differently for CVM 3.5 and 4.1. Redundancy of the networks can be done in one of the following ways:

1) dual (multiple) heartbeat networks
2) single heartbeat network w/standby lan card(s)
3) single heartbeat network w/APACVM 3.5 only supports options 2 and 3. With CVM 4.1, options 1 and 2 are the minimum recommended configurations.

Comparison of Volume Managers

The following table summarizes some of the advantages and disadvantages of the volume managers that are currently available.

Table 3-5 Pros and Cons of Volume Managers with Serviceguard

Product	Advantages	Tradeoffs
Logical Volume Manager (LVM)	Software is provided with all versions of HP-UX. Provides up to 3-way mirroring using optional mirror disk UX software. Supports up to 16 nodes in a Serviceguard cluster Supports using PV links for multipl.e access paths to disk storage Supports exclusive activation as well as read-only activation from multiple nodes Can be used to configure a cluster lock disk Supports multiple heartbeat subnets; the one with the faster failover time is used to re-form the cluster.	Lacks flexibility and extended features of some other volume managers PV links are active/standby only, with one link active at a time
Mirrordisk/UX	Software mirroring Lower cost solution	Lacks extended features of other volume managers
Shared Logical Volume Manager (SLVM)	Provided free with SGeRAC for multi-node access to RAC data Supports up to 16 nodes in shared read/write mode for each cluster Supports exclusive activation Supports multiple heartbeat subnets. Online node configuration with activated shared volume groups (using specific SLVM kernel and Serviceguard revisions)	Lacks the flexibility and extended features of some other volume managers. Limited mirroring support.
Base-VxVM	Software is free with HP-UX 11i and later. Java-based administration through graphical user interface. Striping (RAID-0) support. Concatenation. Online resizing of volumes. Supports multiple heartbeat subnets. Supports up to 16 nodes	Cannot be used for a cluster lock. Using the disk as a root/boot disk is only supported for VxVM 3.5 or later, when installed on HP-UX 11.11. Supports only exclusive read or write activation. Package delays are possible, due to lengthy vxdg import at the time the package is started or failed over.
VERITAS Volume Manager— Full VxVM product B9116AA (VxVM 3.5) B9116BA (VxVM 4.1)	Disk group configuration from any node. DMP for active/active storage devices. Supports up to 16 nodes. Supports exclusive activation. Hot relocation and unrelocation of failed subdisks Supports up to 32 plexes per volume RAID 1+0 mirrored stripes RAID 1 mirroring RAID 5 RAID 0+1 striped mirrors Supports multiple heartbeat subnets, which could reduce cluster reformation time.	Requires purchase of additional license. Cannot be used for a cluster lock. Using the disk as a root/boot disk is only supported for VxVM 3.5 or later, when installed on HP-UX 11.11. Does not support activation on multiple nodes in either shared mode or read-only mode May cause delay at package startup time due to lengthy vxdg import
VERITAS Cluster Volume Manager— B9117AA (CVM 3.5) B9117BA (CVM 4.1)	Provides volume configuration propagation. Supports cluster shareable disk groups. Package startup time is faster than with VxVM. Supports shared activation. Supports exclusive activation. Supports activation in different modes on different nodes at the same time CVM version 4.1 supports the VERITAS Cluster File System (CFS)	Disk groups must be configured on a master node. CVM can only be used with up to 8 cluster nodes. CFS can be used with up to 4 nodes. Cluster startup may be slower than with VxVM Requires purchase of additional license No support for striped mirrors or RAID 5 Version 3.5 supports only a single heartbeat subnet (Version 4.1 supports more than one heartbeat) CVM requires all nodes to have connectivity to the shared disk groups.