Creating the Logical Volume Infrastructure for OPS

Creating Mirrors of Root Logical Volumes

It is highly recommended that you use mirrored root volumes on all cluster nodes. If you are using a disk array for your root logical volumes, skip ahead to the section, "Configuring Volume Groups on Arrays." If you are using a standalone root disk, it should be mirrored as shown here.

The following procedure assumes that you are using separate boot and root volumes; you create a mirror of the boot volume (/dev/vg00/lvol1), root volume (/dev/vg00/lvol3), and primary swap (/dev/vg00/lvol2). The procedure cannot be carried out with SAM. In this example and in the following commands, /dev/dsk/c4t5d0 is the primary disk and /dev/dsk/c4t6d0 is the mirror; be sure to use the correct device file names for the root disks on your system.

Creating OPS Volume Groups on Disk Arrays

On your disk arrays, you should use redundant I/O channels from each node, connecting them to separate controllers on the array. Then you can define alternate links to the LUNs or logical disks you have defined on the array. If you are using SAM, choose the type of disk array you wish to configure, and follow the menus to define alternate links. If you are using LVM commands, specify the links on the command line.

The following example shows how to configure alternate links using LVM commands. The following disk configuration is assumed:

8/0.15.0 /dev/dsk/c0t15d0  /* I/O Channel 0 (8/0) SCSI address 15 LUN 0 */
8/0.15.1 /dev/dsk/c0t15d1  /* I/O Channel 0 (8/0) SCSI address 15 LUN 1 */
8/0.15.2 /dev/dsk/c0t15d2  /* I/O Channel 0 (8/0) SCSI address 15 LUN 2 */
8/0.15.3 /dev/dsk/c0t15d3  /* I/O Channel 0 (8/0) SCSI address 15 LUN 3 */
8/0.15.4 /dev/dsk/c0t15d4  /* I/O Channel 0 (8/0) SCSI address 15 LUN 4 */
8/0.15.5 /dev/dsk/c0t15d5  /* I/O Channel 0 (8/0) SCSI address 15 LUN 5 */
 
10/0.3.0 /dev/dsk/c1t3d0   /* I/O Channel 1 (10/0) SCSI address 3 LUN 0 */
10/0.3.1 /dev/dsk/c1t3d1   /* I/O Channel 1 (10/0) SCSI address 3 LUN 1 */
10/0.3.2 /dev/dsk/c1t3d2   /* I/O Channel 1 (10/0) SCSI address 3 LUN 2 */
10/0.3.3 /dev/dsk/c1t3d3   /* I/O Channel 1 (10/0) SCSI address 3 LUN 3 */
10/0.3.4 /dev/dsk/c1t3d4   /* I/O Channel 1 (10/0) SCSI address 3 LUN 4 */
10/0.3.5 /dev/dsk/c1t3d5   /* I/O Channel 1 (10/0) SCSI address 3 LUN 5 */

Assume that the disk array has been configured, and that both the following device files appear for the same LUN (logical disk) when you run the ioscan command:

/dev/dsk/c0t15d0
/dev/dsk/c1t3d0

Use the following procedure to configure a volume group for this logical disk:

LVM will now recognize the I/O channel represented by /dev/dsk/c0t15d0 as the primary link to the disk; if the primary link fails, LVM will automatically switch to the alternate I/O channel represented by /dev/dsk/c1t3d0. Use the vgextend command to add additional disks to the volume group, specifying the appropriate physical volume name for each PV link.

Repeat the entire procedure for each distinct volume group you wish to create. For ease of system administration, you may wish to use different volume groups to separate logs from data and control files.

	NOTE: The default maximum number of volume groups in HP-UX is 10. If you intend to create enough new volume groups that the total exceeds ten, you must increase the maxvgs system parameter and then re-build the HP-UX kernel. In SAM, select the Kernel Configuration Area, then choose "Configurable Parameters." Maxvgs appears on the list.

Creating Logical Volumes for OPS

After you create volume groups and add PV links to them, you define logical volumes for data, logs, and control files. The following are some examples:

# lvcreate -n ops1log1.log -L 4 /dev/vg_ops  
# lvcreate -n opsctl1.ctl -L 4 /dev/vg_ops
# lvcreate -n system.dbf -L 28 /dev/vg_ops
# lvcreate -n opsdata1.dbf -L 1000 /dev/vg_ops

Oracle Demo Database Files

The following set of files is required for the Oracle demo database which you can create during the installation process.

^* The size of the logical volume is larger than the Oracle file size because Oracle needs extra space to allocate a header in addition to the file's actual data capacity.

Create these files if you wish to build the demo database. The three logical volumes at the bottom of the table are included as additional data files, which you can create as needed, supplying the appropriate sizes. If your naming conventions require, you can include the Oracle SID and/or the database name to distinguish files for different instances and different databases. If you are using the ORACLE_BASE directory structure, create symbolic links to the ORACLE_BASE files from the appropriate directory. Example:

# ln -s /dev/vg_ops/ropsctl1.ctl /u01/ORACLE/db001/ctrl01_1.ctl

(For more information about Oracle directories, refer to the Oracle Server for HP 9000 Installation and Configuration Guide.

After creating these files, set the owner to oracle and the group to dba with a file mode of 660. The logical volumes are now available on the primary node, and the raw logical volume names can now be used by the Oracle DBA.

Displaying the Logical Volume Infrastructure

To display the volume group, use the vgdisplay command:

# vgdisplay -v /dev/vg_ops 

Exporting the Logical Volume Infrastructure

Before the OPS volume groups can be shared, their configuration data must be exported to other nodes in the cluster. This is done either in SAM or by using HP-UX commands, as shown in the following sections.

	NOTE: A volume group using an HP High Availability Disk Array cannot be designated as a cluster lock volume group. You will need to use another volume group containing an independent disk for the cluster lock.

Exporting with SAM

In SAM, choose "Disks and File Systems," then choose "Volume Groups." Open the Actions menu and choose "Distribute." Follow the prompts in the dialog box to select the volume group that is to be distributed to one or more additional nodes. Enter the name of each node that is to receive the volume group and select "Add." When the list is complete, press "OK." SAM automatically configures the volume group for use on the other nodes.

	NOTE: In distributing the volume group, SAM does not import PV links on other nodes. Therefore, to complete the distribution, you use the vgextend command on all nodes to add the alternate PV links to the volume group you have distributed. Use the vgdisplay command to verify the existence of the correct groups of PV links on all nodes.

Creating Volume Groups for Packages

If you are creating packages, you must configure separate volume groups that can be activated each time the package starts up on a particular node. Packages activate volume groups in exclusive mode rather than shared mode, which is used for OPS volume groups. Another difference is that with package volume groups, you can define file systems that are mounted when the package starts up. By contrast, OPS data is stored on raw logical volumes, which do not use mounted file systems.

Creating Logical Volumes for Use with Packages

Use the lvcreate command to create logical volumes. Th following example is for a volume group named /dev/vgdatabase:

# lvcreate -L 120 -m 1 -s g /dev/vgdatabase

This command creates a 120 MB mirrored volume named lvol1. The name is supplied by default, since no name is specified in the command. The -s g option means that mirroring is PVG-strict, that is, the mirror copies of data will be in different physical volume groups.

	NOTE: If you are using disk arrays in RAID 1 or RAID 5 mode, omit the -m 1 and -s g options.

Creating Package Volume Groups with Mirrored Individual Data Disks

	NOTE: The procedure described in this section is for use only with application packages, not with OPS instances and not with OPS data. Software mirroring is not supported for OPS data.

Using SAM to Create Volume Groups and Logical Volumes

You can use SAM to prepare the volume group and logical volume structure needed for HA packages. In SAM, choose the "Disks and File Systems Area." Then use the following procedure for each volume group and file system you are using with the package:

1. Select the Volume Groups subarea.

2. From the Actions menu, choose Create or Extend.

3. Choose the first physical disk you wish to use in the volume group by selecting it from the list of available disks.

4. Enter the volume group name, e.g., vgdatabase.

5. Choose Create or Extend Volume Group.

6. Choose Add New Logical Volumes.

7. Specify the file system that is to be mounted on the volume group, for example, /mnt1.

8. Repeat the procedure for additional volume groups, logical volumes, and file systems.

Skip ahead to the section "Distributing the Package Volume Group".

Using LVM Commands to Create Volume Groups and Logical Volumes

You can use LVM commands to create volume groups for packages on mirrored disks. Use the procedures in this section.

Selecting Disks for the Volume Group

Obtain a list of the disks on both nodes and identify which device files are used for the same disk on both. Use the following command on each node to list available disks as they are known to each system:

# lssf /dev/dsk/*

In the following examples, we use /dev/rdsk/c1t2d0 and /dev/rdsk/c0t2d0, which happen to be the device names for the same disks on both ftsys9 and ftsys10. In the event that the device file names are different on the different nodes, make a careful note of the correspondences.

Creating Physical Volumes

On the configuration node (ftsys9), use the pvcreate command to define disks as physical volumes. This only needs to be done on the configuration node. Use the following commands to create two physical volumes for the sample configuration:

# pvcreate -f /dev/rdsk/c1t2d0 # pvcreate -f /dev/rdsk/c0t2d0

Creating a Volume Group.

Use the following steps to build a volume group on the configuration node (ftsys9). Later, the same volume group will be created on other nodes.

1. First, set up the group directory for vgdatabase:

   # mkdir /dev/vgdatabase

2. Next, create a control file named group in the directory /dev/vgdatabase, as follows:

   # mknod /dev/vgdatabase/group c 64 0xhh0000

   The major number is always 64, and the hexadecimal minor number has the form

   0xhh0000

   where hh must be unique to the volume group you are creating. Use the next hexadecimal number that is available on your system, after the volume groups that are already configured. Use the following command to display a list of existing volume groups:

   # ls -l /dev/*/group

3. Create the volume group and add physical volumes to it with the following commands:

   # vgcreate /dev/vgdatabase /dev/dsk/c1t2d0 # vgextend /dev/vgdatabase /dev/dsk/c0t2d0

   The first command creates the volume group and adds a physical volume to it. The second command adds the second physical volume to the volume group.

4. Repeat this procedure for additional volume groups.

Creating Logical Volumes

Use the following command to create logical volumes (the example is for a volume group named /dev/vgdatabase):

# lvcreate -L 120 -m 1 /dev/vgdatabase

This command creates a 120 MB mirrored volume named lvol1. The name is supplied by default, since no name is specified in the command.

	NOTE: If you are using disk arrays in RAID 1 or RAID 5 mode, omit the -m 1 option.

Creating File Systems

If your installation uses file systems, create them next. Use the following commands to create a file system for mounting on the logical volume just created:

1. Create the file system on the newly created logical volume:

   # newfs -F vxfs /dev/vgdatabase/rlvol1

   Note the use of the raw device file for the logical volume.

2. Create a directory to mount the disk:

   # mkdir /mnt1

3. Mount the disk to verify your work:

   # mount /dev/vgdatabase/lvol1 /mnt1

   Note the mount command uses the block device file for the logical volume.

4. Verify the configuration:

   # vgdisplay -v /dev/vgdatabase