Jump to content United States-English
HP.com Home Products and Services Support and Drivers Solutions How to Buy
» Contact HP
More options
HP.com home
Managing Serviceguard Version A.11.16, Eleventh EditionSecond Printing > Chapter 5  Building an HA Cluster Configuration

Creating a Storage Infrastructure with LVM

» 

Technical documentation

Complete book in PDF
» Feedback
Content starts here

 » Table of Contents

 » Index

In addition to configuring the cluster, you create the appropriate logical volume infrastructure to provide access to data from different nodes. This is done with Logical Volume Manager (LVM), VERITAS Cluster Volume Manager (CVM), or VERITAS Volume Manager (VxVM). You can also use a mixture of volume types, depending on your needs. LVM and VxVM configuration are done before cluster configuration, and CVM configuration is done after cluster configuration.

This section describes storage configuration with LVM. Separate procedures are given for the following:

  • Creating Volume Groups for Mirrored Individual Disks

  • Creating Volume Groups for Disk Arrays Using PV Links

  • Distributing Volume Groups to Other Nodes

The Event Monitoring Service HA Disk Monitor provides the capability to monitor the health of LVM disks. If you intend to use this monitor for your mirrored disks, you should configure them in physical volume groups. For more information, refer to the manual Using High Availability Monitors.

Creating Volume Groups for Mirrored Individual Data Disks

The procedure described in this section uses physical volume groups for mirroring of individual disks to ensure that each logical volume is mirrored to a disk on a different I/O bus. This kind of arrangement is known as PVG-strict mirroring. It is assumed that your disk hardware is already configured in such a way that a disk to be used as a mirror copy is connected to each node on a different bus than the bus that is used for the other (primary) copy.

For more information on using LVM, refer to the HP-UX Managing Systems and Workgroups manual.

Using SAM to Create Volume Groups and Logical Volumes

You can use SAM to prepare the volume group and logical volume structure needed for HA packages. In SAM, choose the “Disks and File Systems Area.” Then use the following procedure for each volume group and file system you are using with the package:

  1. Select the Volume Groups subarea.

  2. From the Actions menu, choose Create or Extend.

  3. Choose the first physical disk you wish to use in the volume group by selecting it from the list of available disks.

  4. Enter the volume group name, e.g., vgdatabase.

  5. Choose Create or Extend Volume Group.

  6. Choose Add New Logical Volumes.

  7. When adding logical volumes to the volume group, ensure that you are creating mirrored logical volumes with PVG strict allocation.

  8. Specify the file system that is to be mounted on the volume group, for example, /mnt1.

  9. Repeat the procedure for additional volume groups, logical volumes, and file systems.

Skip ahead to the section “Deactivating the Volume Group”.

Using LVM Commands to Create Volume Groups and Logical Volumes

If your volume groups have not been set up, use the procedure in the next sections. If you have already done LVM configuration, skip ahead to the section “Configuring the Cluster.”

Selecting Disks for the Volume Group

Obtain a list of the disks on both nodes and identify which device files are used for the same disk on both. Use the following command on each node to list available disks as they are known to each system:

lssf /dev/dsk/*  

In the following examples, we use /dev/rdsk/c1t2d0 and /dev/rdsk/c0t2d0, which happen to be the device names for the same disks on both ftsys9 and ftsys10. In the event that the device file names are different on the different nodes, make a careful note of the correspondences.

Creating Physical Volumes

On the configuration node (ftsys9), use the pvcreate command to define disks as physical volumes. This only needs to be done on the configuration node. Use the following commands to create two physical volumes for the sample configuration:

pvcreate -f /dev/rdsk/c1t2d0 
pvcreate -f /dev/rdsk/c0t2d0 
Creating a Volume Group with PVG-Strict Mirroring

Use the following steps to build a volume group on the configuration node (ftsys9). Later, the same volume group will be created on other nodes.

  1. First, set up the group directory for vgdatabase:

    mkdir /dev/vgdatabase 
  2. Next, create a control file named group in the directory /dev/vgdatabase, as follows:

    mknod /dev/vgdatabase/group c 64 0xhh0000 

    The major number is always 64, and the hexadecimal minor number has the form

    0xhh0000

    where hh must be unique to the volume group you are creating. Use a unique minor number that is available across all the nodes for the mknod command below. (This will avoid further reconfiguration later, when NFS-mounted logical volumes are created in the VG.)

    Use the following command to display a list of existing volume groups:

    ls -l /dev/*/group 
  3. Create the volume group and add physical volumes to it with the following commands:

    vgcreate -g bus0 /dev/vgdatabase /dev/dsk/c1t2d0 
    vgextend -g bus1 /dev/vgdatabase /dev/dsk/c0t2d0  

    The first command creates the volume group and adds a physical volume to it in a physical volume group called bus0. The second command adds the second drive to the volume group, locating it in a different physical volume group named bus1. The use of physical volume groups allows the use of PVG-strict mirroring of disks and PV links.

  4. Repeat this procedure for additional volume groups.

Creating Logical Volumes

Use the following command to create logical volumes (the example is for /dev/vgdatabase):

lvcreate  -L 120 -m 1 -s g /dev/vgdatabase 

This command creates a 120 MB mirrored volume named lvol1. The name is supplied by default, since no name is specified in the command. The -s g option means that mirroring is PVG-strict, that is, the mirror copies of data will be in different physical volume groups.

NOTE: If you are using disk arrays in RAID 1 or RAID 5 mode, omit the -m 1 and -s g options.

Creating File Systems

If your installation uses file systems, create them next. Use the following commands to create a file system for mounting on the logical volume just created:

  1. Create the file system on the newly created logical volume:

    newfs -F vxfs /dev/vgdatabase/rlvol1 

    Note the use of the raw device file for the logical volume.

  2. Create a directory to mount the disk:

    mkdir /mnt1 
  3. Mount the disk to verify your work:

    mount /dev/vgdatabase/lvol1 /mnt1 

    Note the mount command uses the block device file for the logical volume.

  4. Verify the configuration:

    vgdisplay -v /dev/vgdatabase 

Creating Volume Groups for Disk Arrays Using PV Links

If you are configuring volume groups that use mass storage on HP's HA disk arrays, you should use redundant I/O channels from each node, connecting them to separate ports on the array. Then you can define alternate links (also called PV links) to the LUNs or logical disks you have defined on the array. In SAM, choose the type of disk array you wish to configure, and follow the menus to define alternate links.

The following example shows how to configure alternate links using LVM commands. In the example, the following disk configuration is assumed:

8/0.15.0 /dev/dsk/c0t15d0  /* I/O Channel 0 (8/0) SCSI address 15 LUN 0 */
8/0.15.1 /dev/dsk/c0t15d1  /* I/O Channel 0 (8/0) SCSI address 15 LUN 1 */
8/0.15.2 /dev/dsk/c0t15d2  /* I/O Channel 0 (8/0) SCSI address 15 LUN 2 */
8/0.15.3 /dev/dsk/c0t15d3  /* I/O Channel 0 (8/0) SCSI address 15 LUN 3 */
8/0.15.4 /dev/dsk/c0t15d4  /* I/O Channel 0 (8/0) SCSI address 15 LUN 4 */
8/0.15.5 /dev/dsk/c0t15d5  /* I/O Channel 0 (8/0) SCSI address 15 LUN 5 */

10/0.3.0 /dev/dsk/c1t3d0   /* I/O Channel 1 (10/0) SCSI address 3 LUN 0 */
10/0.3.1 /dev/dsk/c1t3d1   /* I/O Channel 1 (10/0) SCSI address 3 LUN 1 */
10/0.3.2 /dev/dsk/c1t3d2   /* I/O Channel 1 (10/0) SCSI address 3 LUN 2 */
10/0.3.3 /dev/dsk/c1t3d3   /* I/O Channel 1 (10/0) SCSI address 3 LUN 3 */
10/0.3.4 /dev/dsk/c1t3d4   /* I/O Channel 1 (10/0) SCSI address 3 LUN 4 */
10/0.3.5 /dev/dsk/c1t3d5   /* I/O Channel 1 (10/0) SCSI address 3 LUN 5 */

Assume that the disk array has been configured, and that both the following device files appear for the same LUN (logical disk) when you run the ioscan command:

/dev/dsk/c0t15d0
/dev/dsk/c1t3d0

Use the following steps to configure a volume group for this logical disk:

  1. First, set up the group directory for vgdatabase:

    mkdir /dev/vgdatabase 
  2. Next, create a control file named group in the directory /dev/vgdatabase, as follows:

    mknod /dev/vgdatabase/group c 64 0xhh0000  

    The major number is always 64, and the hexadecimal minor number has the form

    0xhh0000

    where hh must be unique to the volume group you are creating. Use a unique number that is available across all the nodes. (This will avoid further reconfiguration later, when NFS-mounted logical volumes will be created in the VG.)

    Use the following command to display a list of existing group files:

    ls -l /dev/*/group 
  3. Use the pvcreate command on one of the device files associated with the LUN to define the LUN to LVM as a physical volume.

    pvcreate /dev/rdsk/c0t15d0 

    It is only necessary to do this with one of the device file names for the LUN.

  4. Use the following commands to create the volume group itself:

    vgcreate /dev/vgdatabase /dev/dsk/c0t15d0 
    # vgextend /dev/vgdatabase /dev/dsk/c1t3d0

You can now use the vgdisplay -v command to see the primary and alternate links. LVM will now recognize the I/O channel represented by /dev/dsk/c0t15d0 as the primary link to the disk; if the primary link fails, LVM will automatically switch to the alternate I/O channel represented by /dev/dsk/c1t3d0.

To create logical volumes, use the procedure described in the previous section, “Creating Logical Volumes.”

Distributing Volume Groups to Other Nodes

After creating volume groups for cluster data, you must make them available to any cluster node that will need to activate the volume group. The cluster lock volume group must be made available to all nodes.

Deactivating the Volume Group

At the time you create the volume group, it is active on the configuration node (ftsys9, for example). Before setting up the volume group for use on other nodes, you must first unmount any file systems that reside on the volume group, then deactivate it. At run time, volume group activation and file system mounting are done through the package control script.

Continuing with the example presented in earlier sections, do the following on ftsys9:

umount /mnt1 
vgchange -a n /dev/vgdatabase 

Distributing the Volume Group with LVM Commands

Use the following commands to set up the same volume group on another cluster node. In this example, the commands set up a new volume group on ftsys10 which will hold the same physical volume that was available on ftsys9. You must carry out the same procedure separately for each node on which the volume group's package can run.

To set up the volume group on ftsys10, use the following steps:

  1. On ftsys9, copy the mapping of the volume group to a specified file.

    vgexport -p -s -m /tmp/vgdatabase.map  /dev/vgdatabase 
  2. Still on ftsys9, copy the map file to ftsys10:

    rcp /tmp/vgdatabase.map ftsys10:/tmp/vgdatabase.map 
  3. On ftsys10, create the volume group directory:

    mkdir /dev/vgdatabase 
  4. Still on ftsys10, create a control file named group in the directory /dev/vgdatabase, as follows:

    mknod /dev/vgdatabase/group c 64 0xhh0000 

    Use the same minor number as on ftsys9. Use the following command to display a list of existing volume groups:

    ls -l /dev/*/group 
  5. Import the volume group data using the map file from node ftsys9. On node fsys10, enter:

    vgimport -s -m /tmp/vgdatabase.map /dev/vgdatabase  

    Note that the disk device names on ftsys10 may be different from their names on ftsys9. You should check to ensure that the physical volume names are correct throughout the cluster.

    When the VG can be activated on this node, perform a vgcfgbackup in the unlikely event that a vgcfgrestore must be performed on this node because of a disaster on the primary node and an LVM problem with the volume group. Do this as shown in the example below:

    # vgchange -a y /dev/vgdatabase
    # vgcfgbackup /dev/vgdatabase
    # vgchange -a n /dev/vgdatabase

  6. If you are using mirrored individual disks in physical volume groups, check the /etc/lvmpvg file to ensure that each physical volume group contains the correct physical volume names for ftsys10.

    NOTE: When you use PVG-strict mirroring, the physical volume group configuration is recorded in the /etc/lvmpvg file on the configuration node. This file defines the physical volume groups which are the basis of mirroring and indicate which physical volumes belong to each PVG. Note that on each cluster node, the /etc/lvmpvg file must contain the correct physical volume names for the PVG’s disks as they are known on that node. Physical volume names for the same disks may not be the same on different nodes. After distributing volume groups to other nodes, you must ensure that each node’s /etc/lvmpvg file correctly reflects the contents of all physical volume groups on that node. Refer to the following section, “Making Physical Volume Group Files Consistent.”
  7. Make sure that you have deactivated the volume group on ftsys9. Then enable the volume group on ftsys10:

    vgchange -a y /dev/vgdatabase 
  8. Create a directory to mount the disk:

    mkdir /mnt1 
  9. Mount and verify the volume group on ftsys10:

    mount /dev/vgdatabase/lvol1 /mnt1 
  10. Unmount the volume group on ftsys10:

    umount /mnt1
  11. Deactivate the volume group on ftsys10:

    vgchange -a n /dev/vgdatabase 

Making Physical Volume Group Files Consistent

Skip ahead to the next section if you do not use physical volume groups for mirrored individual disks in your disk configuration.

Different volume groups may be activated by different subsets of nodes within a Serviceguard cluster. In addition, the physical volume name for any given disk may be different on one node than it is on another. For these reasons, you must carefully merge the /etc/lvmpvg files on all nodes so that each node has a complete and consistent view of all cluster-aware disks as well as of its own private (non-cluster-aware) disks. To make merging the files easier, be sure to keep a careful record of the physical volume group names on the volume group planning worksheet (described in the “Planning” chapter).

Use the following procedure to merge files between the configuration node (ftsys9) and a new node (ftsys10) to which you are importing volume groups:

  1. Copy /etc/lvmpvg from ftsys9 to /etc/lvmpvg.new on ftsys10.

  2. If there are volume groups in /etc/lvmpvg.new that do not exist on ftsys10, remove all entries for that volume group from /etc/lvmpvg.new.

  3. If /etc/lvmpvg on ftsys10 contains entries for volume groups that do not appear in /etc/lvmpvg.new, then copy all PVG entries for that volume group to /etc/lvmpvg.new.

  4. Adjust any physical volume names in /etc/lvmpvg.new to reflect their correct names on ftsys10.

  5. On ftsys10, copy /etc/lvmpvg to /etc/lvmpvg.old to create a backup. Copy /etc/lvmvpg.new to /etc/lvmpvg on ftsys10.

Creating Additional Volume Groups

The foregoing sections show in general how to create volume groups and logical volumes for use with Serviceguard. Repeat the procedure for as many volume groups as you need to create, substituting other volume group names, logical volume names, and physical volume names. Pay close attention to the disk device names. For example, /dev/dsk/c0t2d0 on one node may not be /dev/dsk/c0t2d0 on another node.

Printable version
Privacy statement Using this site means you accept its terms Feedback to webmaster
© Hewlett-Packard Development Company, L.P.