Configuring OPS Clusters with MC/LockManager:

HP Part Number: B5158-90024

Published: May 1999

Legal Notices

Table of Contents

Printing History

Preface

1 MC/LockManager at a Glance

What is an OPS Cluster?

Using Packages in an OPS Cluster

Configuration Roadmap

2 Understanding MC/LockManager Hardware Configurations

Redundancy of Cluster Components

HP 9000 Systems

Redundant Network Components

Redundant Ethernet Configuration
Providing Redundant FDDI Connections
Using Dual Attach FDDI Stations
Fibre Channel Switched Configurations
Using a Serial (RS232) Heartbeat Line

Redundant Disk Storage

Supported Disk Interfaces
Data Protection using Disk Arrays
Monitoring of Disks with Event Monitoring Service
Replacement of Failed Disk Mechanisms
Root Disk Limitations on Shared Buses

Redundant Power Supplies

Larger Clusters

Up to Four Nodes with SCSI Storage
Up to Eight Nodes with EMC Disk Arrays

3 Understanding MC/LockManager Software Components

MC/LockManager Architecture

How the Cluster Manager Works

Configuration of the Cluster
Heartbeat Messages
Automatic Cluster Restart
Dynamic Cluster Re-formation
Cluster Quorum for Re-formation
Use of the Cluster Lock
Single Cluster Lock
Dual Cluster Lock
No Cluster Lock
Backing Up Cluster Lock Information

How MC/LockManager Supports OPS 7.3.x

DLM Configuration Files (OPS 7.3.x)

How MC/LockManager Supports OPS 8.0.5

How MC/LockManager Supports OPS 8.1.5 64 bit and OPS 8.1.6

How the Package Manager Works

Deciding When and Where to Run and Halt Packages
Startup Policies
Automatic Rotating Standby
Automatic Failback
Running Application Services
Service Monitor
Using the Event Monitoring Service
Using the EMS HA Monitors
Stopping the Package

How the Network Manager Works

Stationary and Relocatable IP Addresses
Adding and Deleting Relocatable IP Addresses
Monitoring LAN Interfaces and Detecting Failure

Responses to Failures

Transfer of Control (TOC) When a Node Fails
Responses to Hardware Failures
Responses to Package and Service Failures
Service Restarts
Network Communication Failure
Automatic Port Aggregation

4 Planning and Documenting and OPS Cluster

General Planning

Hardware Planning

SPU Information
LAN and RS232 Information
Setting SCSI Addresses for the Largest Expected Cluster Size
Disk I/O Information
Hardware Configuration Worksheet
Output of ioscan -fnC disk

Power Supply Planning

Power Supply Configuration Worksheet

Shared Logical Volume Planning

Planning Volume Groups
Planning Physical Volumes and PV Links
Planning Logical Volumes
Physical Volumes and PV Links Worksheet
Logical Volume Planning Worksheet

Cluster Configuration Planning

Choosing the Cluster Lock Volume Group
Cluster Manager Parameters
Cluster Configuration Worksheet

Planning for OPS 7.3.x

Cluster Interface Specific DLM Parameters
DLM Internal Parameters
Setting HP-UX Shared Memory Allocation
DLM Configuration Worksheet (OPS 7.3.x)

Planning for OPS 8.0.5

OPS 8.0.5 Cluster Parameters
OPS 8.0.5 Cluster Configuration Worksheet

Planning for OPS 8.1.5 64 bit and OPS 8.1.6

OPS 8.1.5 64 bit and OPS 8.1.6 Cluster Parameters
OPS 8.1.5 64 bit and OPS 8.1.6 Cluster Configuration Worksheet

Package Configuration Planning

Logical Volume and Filesystem Planning
Monitoring Registered Package Resources
Choosing Switching and Failover Behavior
Package Configuration File Parameters
Package Control Script Variables
Package Configuration Worksheet

5 Building an OPS Cluster Configuration

Preparing Your Systems

Editing Security Files
Allowing Non-Root Users to Run cmviewcl
Ensuring Consistency of Kernel Configuration
Enabling the Network Time Protocol

Installing MC/LockManager

Creating the Logical Volume Infrastructure for OPS

Creating Mirrors of Root Logical Volumes
Creating OPS Volume Groups on Disk Arrays
Creating Logical Volumes for OPS
Oracle Demo Database Files
Displaying the Logical Volume Infrastructure
Exporting the Logical Volume Infrastructure
Creating Volume Groups for Packages

Installing Oracle Parallel Server

Relinking

Configuring the Cluster

Using SAM to Configure the Cluster
Using MC/LockManager Commands to Configure the Cluster

Configuring the Distributed Lock Manager (OPS 7.3.x)

Using SAM to Configure the Distributed Lock Manager (OPS 7.3.x)
Using HP-UX Commands to Configure the Distributed Lock Manager (OPS 7.3.x)

Verifying the Cluster Configuration

Distributing the Binary Configuration File

Distributing the Configuration File with SAM
Distributing the Configuration File with HP-UX Commands
Storing Volume Group and Cluster Lock Configuration Data
Checking Cluster Operation

Preventing Automatic Activation of Volume Groups

Setting up Autostart Features

Changing the System Message

Deleting the Cluster Configuration

Creating OPS Startup and Shutdown Scripts (OPS 7.3.x)

Using Packages to Start and Stop OPS (OPS 8.0.5, OPS 8.1.5 64 bit or OPS 8.1.6)

Starting Up the Oracle Instance

Starting Up and Shutting Down Manually

6 Configuring Packages and Their Services

Creating the Package Configuration

Using SAM to Configure a Package
Using MC/LockManager Commands to Create a Package
Creating Packages to Launch OPS Instances (OPS 8.0.5, OPS 8.1.5 64 bit or OPS 8.1.6)
Configuring Packages that Access the OPS Database

Writing the Package Control Script

Using SAM to Write the Package Control Script
Using Commands to Write the Package Control Script
Customizing the Package Control Script
Adding Customer Defined Functions to the Package Control Script

Distributing and Verifying the Configuration

Distributing the Configuration File And Control Script with SAM
Copying Package Control Scripts with HP-UX commands
Distributing the Binary Cluster Configuration File with HP-UX Commands
Verifying Cluster and Package Configuration

7 Cluster and Package Maintenance

Viewing the Status of the Cluster

Using SAM to View Cluster Status
Using HP-UX Commands to View Cluster Status
Viewing the Status of Volume Groups

Starting and Stopping the Cluster

Using SAM to Stop the Cluster
Using HP-UX Commands to Stop the Cluster
Using SAM to Start the Cluster
Using HP-UX Commands to Start the Cluster

Starting and Stopping Individual Nodes

Using SAM to Remove a Node from the Cluster Temporarily
Using HP-UX Commands to Remove a Node from the Cluster Temporarily
Using SAM to Return a Node to the Cluster
Using HP-UX Commands to Return a Node to the Cluster

Managing Packages and Services

Starting a Package
Halting a Package
Moving a Package
Resetting the Service Restart Counter
Reconfiguring the Package
Responding to Cluster Events Affecting Packages

Changing the Permanent Cluster Configuration

Changing Lock Volume Group Configuration

Changing Oracle Parameters (OPS 7.3.x)

Making Volume Groups Shareable (HP-UX Commands Only)

Making a Volume Group Unshareable

Activating a Volume Group in Shared Mode (HP-UX Commands Only)

Deactivating a Shared Volume Group

Making Changes to Shared Volume Groups (HP-UX Commands Only)

Adding Additional Shared Volume Groups

Single-Node Operation

Removing MC/LockManager from a System

8 Troubleshooting Your Cluster

Testing Cluster Halt and Reconfiguration

Using SAM to Test Cluster Halt and Reconfiguration
Using HP-UX Commands to Test Cluster Halt and Reconfiguration

Monitoring Hardware

Using Event Monitoring Service
Using HP Predictive Monitoring

Adding Disk Hardware

Replacing Disks

Replacing a Faulty Array Mechanism
Replacing a Faulty Mechanism in an HA Enclosure
Replacing a Lock Disk
Online Hardware Maintenance with In-line SCSI Terminator

Replacing a LAN Card

Troubleshooting Approaches

Reviewing Cluster and Package States
Reviewing Package IP Addresses
Reviewing the System Log File
Reviewing Configuration Files
Reviewing the Package Control Script
Using cmquerycl and cmcheckconf
Using cmscancl
Using cmviewconf
Reviewing the LAN Configuration
Reviewing the Status of Shared Volume Groups
Using DLM Diagnostic Tools (OPS 7.3.x)

Solving Package Problems

System Administration Errors
Package Movement Errors
Node and Network Failures

A Configuration Parameter Limits

Minimum and Maximum Parameter Values for Cluster Configuration
Minimum and Maximum Parameter Values for Package Configuration
Maximum Parameter Values for OPS Configuration

B MC/LockManager Commands

C Designing Highly Available Cluster Applications

Automating Application Operation

Insulate Users from Outages
Define Application Startup and Shutdown

Controlling the Speed of Application Failover

Replicate Non-Data File Systems
Use Raw Volumes
Evaluate the Use of JFS
Minimize Data Loss
Use Restartable Transactions
Use Checkpoints
Design for Multiple Servers
Design for Replicated Data Sites

Designing Applications to Run on Multiple Systems

Avoid Node Specific Information
Avoid Using SPU IDs or MAC Addresses
Assign Unique Names to Applications
Use uname(2) With Care
Bind to a Fixed Port
Bind to Relocatable IP Addresses
Give Each Application its Own Volume Group
Use Multiple Destinations for SNA Applications
Avoid File Locking

Restoring Client Connections

Handling Application Failures

Create Applications to be Failure Tolerant
Be Able to Monitor Applications

Minimizing Planned Downtime

Providing Online Application Reconfiguration
Documenting Maintenance Operations

D MC/LockManager Error Messages

DLM Error Messages (OPS 7.3.x)

DLM Startup Errors
Normal Runtime Errors
Runtime Errors and Alerts
Reconfiguration Time Errors
DLM Internal Errors
Cluster Manager-DLM Interface Errors

GMS Error Messages (OPS 8.0.5)

GMS Configuration Messages
GMS Daemon Messages

E Blank Planning Worksheets

Worksheet for Hardware Planning
Power Supply Worksheet
Physical Volumes and PV Links Worksheet
Logical Volume Worksheet
Cluster Configuration Worksheet
DLM Configuration Worksheet (OPS 7.3.x)
Cluster Configuration Worksheet (OPS 8.0.5)
Cluster Configuration Worksheet (OPS 8.1.5 64 bit and OPS 8.1.6)
Package Configuration Worksheet

Index

List of Figures

1-1 Overview of Oracle Parallel Server Configuration on HP-UX
1-2 Tasks in Configuring an MC/LockManager Cluster
2-1 Redundant LANs
2-2 Redundant FDDI Configuration
2-3 Configuration with Dual Attach FDDI Stations
2-4 FibreChannel Interconnect in a Four-Node Cluster
2-5 Two-LAN Configuration with Redundant Serial (RS232) Line
2-6 Cluster with High Availability Disk Array
2-7 Root Disks on Different Shared Buses
2-8 Primaries and Mirrors on Different Shared Buses
2-9 Four-Node OPS Cluster
2-10 Eight Node Cluster
3-1 MC/LockManager Software Components
3-2 Package Moving During Failover
3-3 Rotating Standby Configuration before Failover
3-4 Rotating Standby Configuration after Failover
3-5 Rotating Standby Configuration before Failover
3-6 Cluster Before Local Network Switching
3-7 Cluster After Local Network Switching
3-8 Local Switching After Cable Failure
3-9 Before Package Switching
3-10 After Package Switching
3-11 Aggregated Networking Ports
4-1 Hardware Configuration Worksheet
4-2 Sample Power Supply Configuration Worksheet
4-3 OPS Physical Volume Worksheet
4-4 Logical Volume Worksheet
4-5 Cluster Configuration Worksheet
4-6 DLM Configuration Worksheet
4-7 OPS 8.0.5 Cluster Configuration Worksheet
4-8 OPS 8.1.5 64 bit and OPS 8.1.6 Cluster Configuration Worksheet
4-9 Package Configuration Worksheet
8-1 F/W SCSI Buses with In-line Terminators

List of Tables

1 Editions of Configuring OPS Clusters with MC/LockManager
3-1 Rotating Standby Node Naming
4-1 Fast/Wide SCSI Addressing in Cluster Configuration
4-2 Package Failover Behavior
5-1 Required Oracle File Names for Demo Database
8-1 Data Displayed by the cmscancl Command
A-1 Minimum, Maximum and Default Values of Cluster Configuration Parameters
A-2 Minimum, Maximum and Default Values of Package Configuration Parameters
B-1 MC/LockManager Commands

Configuring OPS Clusters with MC/LockManager:

Technical documentation

» Table of Contents

» Index