System Configuration and Imaging Overview

This section provides an overview of the initial system configuration and imaging process.

System Configuration Process

The system configuration process is the method by which the topology of the system is automatically discovered, applied, and stored in the configuration and management database after the installation of the head node has been completed.

The system topology is defined based on the manner in which the head and client nodes are connected to the administration network when the nodes are cabled into the appropriate switches.

The system configuration procedure described in this chapter consists of running the following HP XC commands and utilities:

Command or Utility Name	Description
cluster_prep	Prepares the system for discovery and configuration and configures the external Ethernet device on the head node, thus enabling network connectivity
discover	Discovers and enables all hardware components and populates the `/etc/hosts` and `/etc/dhcp.conf` files
cluster_config	Populates the configuration and management database with node role assignments, starts all services on the head node, and creates the golden system image
startsys	Turns on power to each node and downloads the SystemImager automatic installation environment to install and configure each node from the golden image

Internal Node Naming

It is important to understand how internal node names are assigned. The node naming scheme differs if HP server blades and enclosures are present in the hardware configuration.

When the hardware configuration does not contain enclosures and HP server blades, all nodes connected directly to the root switches are named with a user-defined node prefix and are numbered in descending order from the highest numbered node. All other nodes connected to a branch switch are named with the node prefix and are numbered in ascending order, starting at 1 on the first port of the first branch switch.
Internal node names are assigned in a dense fashion in which there are no missing numbers in the node numbering scheme except for a possible missing number between the branch nodes and those nodes that are connected to the root administration switch.
In an enclosure-based system with HP server blades, the discover command uses a sparse node numbering scheme. This means that internal node names are assigned based on the enclosure in which the node is located and the bay the node is plugged into.
For example, if a node is plugged into bay 10 of enclosure 1, the node is numbered {node_prefix}10. In a configuration with two enclosures in which there might be 16 nodes in each enclosure, the node in bay 10 in enclosure 2 is numbered {node_prefix}26.
In this release, the maximum number of server blade nodes in a real enclosure is 16. A real enclosure is defined as an enclosure with one of more Onboard Administrators. The maximum number of non-blade server nodes in a virtual enclosure is 48. A virtual enclosure is defined as a ProCurve switch that has at least one console port from a non-blade server node plugged into it.

Head Node Naming

The head node naming scheme differs if HP server blades and enclosures are present in the hardware configuration.

When the hardware configuration does not contain (real) enclosures and HP server blades, the cluster_prep command determines and assigns the head node name by using the number that represents the maximum number of nodes allowed in the hardware configuration, which you supply. For example, if the maximum number of nodes allowed in your hardware configuration is 128, the head node is node {node_prefix}128.
When the hardware configuration contains HP server blades, the head node is named for its location in the system just like every other node, as described in “Internal Node Naming”. The only exception is when the head node is a non-blade server node whose console port is not connected to the administration network ProCurve switch. In this case, the head node is named {node_prefix}0.

System Imaging Process

The SystemImager, an open source software component of the HP XC software stack, provides a simplified method for automating the installation of large Linux clusters. SystemImager provides the foundation for the image replication and distribution environment used to install client nodes in an HP XC system.

As part of the initial software installation, the head node is configured as the golden client, which is the node that represents the configuration from which all other nodes are replicated. Next, a golden image is created from the golden client, which is a replication of the local file system directories and files, starting from root (/). The golden image is stored on the image server, which is also resident on the head node in this release.

When nodes are instructed to PXE or network boot from the network adapter attached to the administration network, an installation process starts on the client nodes that does the following:

Partitions the local disk
By default, the first disk is partitioned because in this release, disks other than the first disk are not supported because of restrictions in the image replication environment.
Creates file systems
Downloads the golden image
Installs the appropriate boot loader

When the automatic installation process completes, each node is rebooted and continues its configuration process, eventually ending with the login prompt.

System Configuration and Imaging Log Files

A record of the configuration process is stored in the /var/log/nconfig.log file.

Logging of the system imaging process is stored in the following files:

/hptc_cluster/adm/logs/imaging.log
/var/log/systemimager/rsyncd
/hptc_cluster/adm/logs/startsys.log

The /hptc_cluster/adm/logs/imaging.log file contains log information produced during the imaging and node configuration processes.

Use the information in this file to diagnose imaging issues such as nodes not booting correctly or failing during the imaging operation. The node configuration portion of the log file provides useful information during the first boot after a node has been imaged. If a node configuration fails, the node is put into single-user mode with network access. The node is not allowed to join the HP XC system without user intervention. The node or nodes that are put into single-user mode are listed in the imaging.log file.

Two system services that are based on syslog-ng log the UDP based information sent by each client during installation and the node configuration phase (which is called the nconfigure phase). The hptc-ire-serverlog service runs persistently on the imaging server. It is responsible for accepting UDP messages from the imaging client and from the hptc-ire-clientlog service running when a node is running through its nconfigure phase. The hptc-ire-clientlog service is managed by autoinstall and the nconfig process. It is only started during these processes and turned off after the processes have been completed.

The rsync log daemon is responsible for logging start and completion time stamps for imaging and golden image updates in the /var/log/systemimager/rsyncd file. The information is sent through TCP for reliability. This is a required service because imaging completion timestamps are used to generate the per node entries in the /tftpboot/pxelinux.cfg file that denote a system has imaged successfully.

System Configuration and Imaging Overview

Technical documentation

» Table of Contents

» Glossary

» Index

System Configuration Process

Internal Node Naming

Head Node Naming

System Imaging Process

System Configuration and Imaging Log Files