 |
» |
|
|
 |
The next step in the configuration process is the discovery of all
system (hardware) components. In addition to configuring the system
topology by discovering all hardware components, the discover command also configures the administration network and interconnect
and populates the configuration and management database (CMDB). The discover process differs if HP server blades
and enclosures are present in the hardware configuration, how many
enclosures are present in the hardware configuration, and if you are
configuring compute nodes that are not yet present but you plan to
add in the future. Follow the discovery instructions in one of the
following sections: Discovering Non-Blade Hardware Configurations |  |
Follow this procedure if the hardware configuration does not contain HP server blades and enclosures: Begin this procedure as the
root user on the head node. Start a script to capture
command output into a file. This step is optional, but HP recommends
doing so. Change to the configuration
directory: # cd /opt/hptc/config/sbin |
Start the discovery of
all network components: # ./discover --system \
[--super] \
[--oldmp] \
[--outfile file_name] \
[--ic={AdminNet|Ethernet|Infiniband|Myrinet|Quadrics}] \
[--verbose] |
 |
Table 3-4 discover Command Options | Option | Description |
|---|
| --super | The --super option is required
for large-scale systems (systems with up to 1024 compute nodes) when
the hardware configuration consists of a Super Root Switch into which
multiple Root Administration Switches are connected. | | --oldmp | The --oldmp option is required
on HP Integrity servers (such as the rx8620) if their management processors
(MP) have statically set IP addresses and are
not configured to use DHCP. The --oldmp option
is no longer required if the HP Integrity server has been upgraded
with MP firmware Version 3.15 or newer. Do not include
the --oldmp option on the command line for HP ProLiant
nodes because that hardware family uses iLO and LO-100i as their console port management devices and uses DHCP
to obtain their IP addresses. | | --outfile file_name | The --outfile file_name option saves a copy of the HP XC system
information that is written to the configuration and management database.
The file name you specify must already exist on the head node. Saving
this information optional, but it enables you to use the --infile file_name option if
it becomes necessary to restore the HP XC system to the point
in the installation just after the discover command
was run but before the cluster_config utility is
run. | | --ic= | The --ic={AdminNet|Ethernet|Infiniband|Myrinet|Quadrics} option specifies the type of interconnect to configure. If you do
not specify this option, the discover utility configures
the fastest interface it finds available in the hardware configuration.
Your entries are case sensitive. When the interconnect
is configured on the administration network, a single LAN is used.
The administration network and the interconnect share the same ports
and switches. All other interconnect types (Myrinet,
InfiniBand, and Quadrics) are discovered automatically based on the
interconnect found on the head node. You are not required to specify
these interconnects on the command line. | | --verbose | The --verbose option provides
useful feedback during the discovery process that enables you to follow
along. HP recommends including it on the command line. |
Table 3-2 and discover(8) contain information about
additional keywords you can add to the command line to omit some of
the questions that will be asked during the discovery process. Use
of these keywords is optional. If you encounter problems during the discovery
process, see “Troubleshooting the Discovery Process” for troubleshooting guidelines.  |  |  |  |  | NOTE: The discover command does not properly discover
HP ProLiant DL140 and DL145 servers until the password is set. Running
the discover command now determines the IP address
of the console ports so you can use the telnet command
to access the console ports and set the password. This process is
described in “Modify the Default Password for HP ProLiant DL140 and DL145
Hardware Models”. |  |  |  |  |
Look at the screen and follow
along with the command output. Table 3-2 describes each prompt and provides information
to help you with your answers. The sample output shown
in Example 3-3 was created by using the --verbose option on
a small HP XC system with six HP ProLiant servers (Opteron
architecture). The sample output shown
in Example 3-4 was created on an large-scale HP XC system
with more than one region.
The output you see will be different. The information
you are prompted to supply depends on whether or not you entered additional
keywords on the command line. Example 3-3 discover Command Output On A Small Non-Blade Configuration  |
Enter the actual total number of nodes in this cluster: 6
Enter the number of nodes that are workstations: 0
Discovering Root switches...
Enter the MAC address of the admin root switch
in the format xx:xx:xx:xx:xx:xx : your_MAC_address 1
Please enter the ProCurve switch Administrator password: your_password 2
Please re-enter password: your_password
Restarting dhcpd
Enter the common user name for all console port management
devices: your_user_name 3
Please enter the password for your_user_name: your_password
Please re-enter password: your_password
Discovering 172.20.65.2 port 39 ... Console Port OK
Restarting dhcpd
discovered_cps is 4
Checking if all console ports are reachable ...
number of cps to check, 3
pinging 172.21.0.13 no response from 172.21.0.13
pinging 172.21.0.15 no response from 172.21.0.15
pinging 172.21.0.14 no response from 172.21.0.14
Following console ports did not respond:
172.21.0.13 172.21.0.15 172.21.0.14
number of cps to check, 3
pinging 172.21.0.13
pinging 172.21.0.15
pinging 172.21.0.14 .done
Opening /etc/hosts
Opening /etc/hosts.new.XC
Opening /etc/powerd.conf
Building /etc/powerd.conf ... Querying cp-n13
Querying cp-n14
Querying cp-n15 4
done
Attempting to start hpls power daemon ... done
Waiting for power daemon ... done
switchName necs1-1 switchIP 172.20.65.2 type 2650
switchName nems1-1 switchIP 172.20.65.1 type 2848
Attempting to power on nodes with nodestring 8n[13-15]
Powering on all known nodes ... done
Discovering Nodes...
running port_discover on 172.20.65.1
nodes Found = 1 nodes Expected = 4
running port_discover on 172.20.65.1
nodes Found = 1 nodes Expected = 4
running port_discover on 172.20.65.1
nodes Found = 1 nodes Expected = 4
running port_discover on 172.20.65.1
nodes Found = 1 nodes Expected = 4
running port_discover on 172.20.65.1
nodes Found = 3 nodes Expected = 4
running port_discover on 172.20.65.1
nodes Found = 3 nodes Expected = 4
running port_discover on 172.20.65.1
nodes Found = 3 nodes Expected = 4
running port_discover on 172.20.65.1
nodes Found = 3 nodes Expected = 4
running port_discover on 172.20.65.1
nodes Found = 3 nodes Expected = 4
running port_discover on 172.20.65.1
nodes Found = 4 nodes Expected = 4
All nodes initialized.
Powering off all nodes but head node ... done
Switch 172.20.65.1 port 42 ... Node Found
Switch 172.20.65.1 port 41 ... Node Found
Switch 172.20.65.1 port 40 ... Node Found
Switch 172.20.65.1 port 39 ... Node Found
Switch 172.20.65.1 port 38 ... NO Node Found 5
Restarting dhcpd
node n14 set to model ProLiant DL145 G1
node n13 set to model ProLiant DL145 G2
node n15 set to model ProLiant DL145 G1
node n16 set to model ProLiant DL585 G1
Opening /etc/hosts
Opening /etc/hosts.new.XC
Head Node CP device type set to iLO
Waiting for power daemon to restart... done |
 |
| 1 | Enter the MAC address of the switch that is connected
to the administration ports. Do not enter the MAC address of the switch
connected to the console ports. | | 2 | Enter the password for the Root Administration
Switch that you previously defined when you prepared the hardware.
If you did not preset a password, press the Enter key. | | 3 | Enter the common user name and password that you
previously defined for the console port management devices (the iLO,
LO-100i, or MP devices) when you prepared the hardware. If you did
not predefine these items, press the Enter key. | | 4 | If the discover process hangs while trying to
discover console ports, see Chapter 10. | | 5 | This is expected behavior. Because
this is a cluster discovery process, the command output No
Switch Found or No Node Found is expected.
This message does not indicate a failure unless a network component
is plugged into that port on the switch. If necessary, see Chapter 10 for information about troubleshooting
problems you might encounter during the discovery process. |
Example 3-4 shows the unique command output for a large-scale
system with two regions; all other command output is similar to the
previous example. Example 3-4 discover Command Output For Large-Scale Systems The discover process has detected 2 regions. Is this correct? [y/n] y
switchName nems0-1-0 switchIP 172.20.65.1 type 2848
Enter the actual total number of nodes in region 1: 526
Enter the number of nodes that are workstations: 0
Enter the actual total number of nodes in region 2: 504
Enter the number of nodes that are workstations: 0
discoverSystem called with switch nems1-1-1 and Region Number 1 |
If the hardware configuration
contains HP workstations and you installed SVA, turn on power
to the render and display workstations. Then, when prompted, turn
off power to the same nodes, leaving the head node power turned on. If you captured the output
of the discover command in a file (described in
step 2), press Ctrl-d to end the script session now. This file is for your information only. Store
this file in a directory where you keep administrative data. Do one of the following:
Discovering Hardware Configurations With Server Blades and
More Than One Enclosure |  |
The HP XC System Software has been designed to communicate
directly with the Onboard Administrator on each enclosure. Thus, the discover command obtains the required configuration information
directly from the Onboard Administrator to automatically discover
all components in the hardware configuration. Discovering hardware configurations with server
blades and more than one c3000 or c7000 enclosure is a multiple-step
process. First, because the hardware configuration contain at least
one external ProCurve switch, the switch or switches are discovered,
which activates the administration network. Next, you perform hardware
preparation tasks such as setting up passwords and making BIOS settings.
Lastly, the enclosures and nodes are discovered. Complete these tasks in the following order: Set the Onboard Administrator Password and Make BIOS SettingsDiscovering the switches activated the administration
network, so you can now log in to the Onboard Administrator on every
enclosure and set a common user name and password and make the appropriate
BIOS settings on all server blade nodes. These tasks are documented
in the HP XC Hardware Preparation Guide (BIOS settings depend upon the hardware model
type). Return here when you have finished with those
tasks and proceed to “Discover All Nodes and Enclosures”. Discover All Nodes and EnclosuresFollow this procedure to discover all enclosures
and all nodes (including server blades) in the hardware configuration.
This discovery process assigns IP addresses to all hardware components: Begin this procedure as the
root user on the head node. Start a script to
capture command output into a file. This step is optional, but HP
recommends doing so. Change to the following directory: # cd /opt/hptc/config/sbin |
Discover all enclosures: # ./discover --enclosurebased --enclosures
Discovery - XC Cluster version HP XC V32 20061022
Enter the common user name for all console port management devices:
your_username
Please enter the password for your_username: your_password
Please re-enter password: your_password
Discovering blade enclosures ...
Checking switch 172.31.32.2 for active ports ...done
Getting MAC addresses from switch 172.31.32.2 ... done
Checking switch 172.31.32.1 for active ports ...done
Getting MAC addresses from switch 172.31.32.1 ... done
Enclosure blc1n-enc09USE6391TF5 found
Discovering virtual enclosures ...
Checking switch 172.31.32.2 for active ports ... done
Getting MAC Addresses from switch 172.31.32.2 ... done
Checking switch 172.31.32.1 for active ports ... done
Getting MAC Addresses from switch 172.31.32.1 ...done |
Discover all nodes: # ./discover --enclosurebased --nodesonly --verbose |
Command output is similar to the following:  |
Discovery - XC Cluster version HP XC Vn.n <timestamp>
Enter the common user name for all console port management devices:
your_username
Please enter the password for your_username: your_password
Please re-enter password: your_password
Discovering Blades ...
Blade found in enclosure blc1n-enc09USE6391TF5 bay 1, name is blc1n1
Blade found in enclosure blc1n-enc09USE6391TF5 bay 2, name is blc1n2
Blade found in enclosure blc1n-enc09USE6391TF5 bay 3, name is blc1n3
Blade found in enclosure blc1n-enc09USE6391TF5 bay 4, name is blc1n4
Blade found in enclosure blc1n-enc09USE6391TF5 bay 5, name is blc1n5
Discovering Non-Blade systems ...
Setting system name to blc1n0
uploading database
Restarting dhcpd
Opening /etc/hosts
Opening /etc/hosts.new.XC
Opening /etc/powerd.conf
Building /etc/powerd.conf ... done
Attempting to start hpls power daemon ... done
Waiting for power daemon ... done
Checking if all console ports are reachable ...
number of cps to check, 5
checking 172.31.16.5
checking 172.31.16.1
checking 172.31.16.4
checking 172.31.16.3
checking 172.31.16.2 .done
Starting CMF for discover...
Stopping cmfd: [FAILED] 1
Starting cmfd: [ OK ]
Waiting for CMF to establish console connections .......... done
uploading database
Restarting dhcpd
Opening /etc/hosts
Opening /etc/hosts.new.XC
Opening /etc/powerd.conf
Building /etc/powerd.conf ... done
Attempting to start hpls power daemon ... done
Waiting for power daemon ... done
uploading database |
 |
| 1 | The discover command turns off the
console management facility (CMF) daemon. If the CMF daemon is not
running, a “FAILED” message is displayed. This message
is expected, and you can ignore it. |
If the hardware configuration
contains HP workstations and you installed SVA, turn on power
to the render and display workstations. Then, when prompted, turn
off power to the same nodes, leaving the head node power turned on. If you captured the output
of the discover command in a file (described in
step 2), press Ctrl-d to end the script session now. This file is for your information only. Store
this file in a directory where you keep administrative data. Do one of the following:
Discovering Hardware Configurations With Server Blades And
A Single Enclosure |  |
Follow this procedure to discover configurations
that have only one c3000 or c7000 enclosure present in the hardware
configuration. When only one enclosure is present, you must include
the --single option on the discover command line
and you are required to enter the MAC address of the Onboard Administrator.
Hardware configurations with one enclosure do not have any external
ProCurve switches.
Go to the enclosure, and obtain
the MAC address of the Onboard Administrator. The Onboard Administrator
is the small LED panel near the bottom of the enclosure. Write down
this MAC address because you will be asked to provide it. Log in as the root user on
the head node. Start a script to
capture command output into a file. This step is optional, but HP
recommends doing so. Change to the following directory: # cd /opt/hptc/config/sbin |
Start the discovery process: # ./discover --enclosurebased --verbose --single |
Command output is similar to the following:  |
Discovery - XC Cluster version HP XC V3.2.1 20071016
Enter the MAC address of the OA for the enclosure
in the format xx:xx:xx:xx:xx:xx : OA_MAC_address
Enter the common user name for all console port management devices: your_user_name
Please enter the password for your_user_name :
Please re-enter password:
Restarting dhcpd
waiting for OA to obtain IP address.
......................................
Enclosure blc4n-enc092UX740006L found
Restarting dhcpd
Opening /etc/hosts
Opening /etc/hosts.new.XC
uploading database
waiting for network component at 172.31.32.1 to become available.
...................
Discovering Blades ...
Blade found in enclosure blc4n-enc092UX740006L bay 1, name is blc4n1
Blade found in enclosure blc4n-enc092UX740006L bay 2, name is blc4n2
Blade found in enclosure blc4n-enc092UX740006L bay 3, name is blc4n3
Blade found in enclosure blc4n-enc092UX740006L bay 4, name is blc4n4
Blade found in enclosure blc4n-enc092UX740006L bay 5, name is blc4n5
Blade found in enclosure blc4n-enc092UX740006L bay 6, name is blc4n6
Headnode is blc4n1
Setting system name to blc4n1 ...
Discovering Non-Blade systems ...
No Virtual Enclosures found ... bypassing non-blade discovery
uploading database
Restarting dhcpd
Opening /etc/hosts
Opening /etc/hosts.new.XC
Opening /etc/powerd.conf
Building /etc/powerd.conf ... done
Attempting to start hpls power daemon ... done
Waiting for power daemon ... done
Discovery has found the following components:
0 Switches
0 Virtual Enclosures
1 Real Enclosures with 1 OnBoard Administrators
6 Nodes |
 |
If you captured the output
of the discover command in a file (described in
step 3), press Ctrl-d to end the script session now. This file is for your information only. Store
this file in a directory where you keep administrative data. Do one of the following:
Discovering Hardware Configurations To Plan For Future Compute
Node Expansion |  |
The discover command enables
you to plan for future expansion of the number of compute nodes in
the hardware configuration. During the discovery process, the discover command creates placeholder database entries for these planned additional
compute nodes and marks them as disabled to exclude them from command
output and job resource planning. Follow this procedure to discover all hardware
components and plan for future compute nodes: Begin this procedure as the
root user on the head node. Start a script to capture
command output into a file. This step is optional, but HP recommends
doing so. Change to the configuration
directory: # cd /opt/hptc/config/sbin |
Start the discovery process.
When you want to configure future planned compute nodes into the database,
the --enclosurebased --maxnodes= options are required
on the command line. The --maxnodes= option specifies the total number of nodes in the hardware configuration,
including the number of compute nodes you are planning for future
expansion. For example, if the current hardware configuration contains
100 nodes, and you plan to add 96 compute nodes in the future, specify --maxnodes=196. # ./discover --system --enclosurebased --maxnodes=n |
Table 3-4 lists and describes additional discover command-line options that might be appropriate
for your discovery process. Look at the screen and follow
along the discover process. The command output shown in Section is similar
to what you will see. The remainder of the discovery process is documented
there. If you need help answering the discover command prompts, Table 3-2 describes each prompt and provides information
to assist you with your answers.
The HP XC System Software Administration Guide describes how to add the planned
compute nodes into the database after the nodes are physically present
and cabled appropriately to the switches. Modify the Default Password for HP ProLiant DL140 and DL145
Hardware Models |  |
Perform this task only if the hardware configuration
contains HP ProLiant DL140 and DL145 hardware models (Generation 2
and 3) to modify the default console management device password. Changing
the default password is not required, but HP recommends changing the
factory default value for security purposes. Omit this step for all
other server models. Use the method of
your choice to view the /etc/dhcpd.conf file
and look for the characters cp- in host names to
determine console port names. Use the telnet command and the internal name of the console port to log in to each
node's console management device and change the default password.
Supply the factory default user name admin and
the default password admin to log in: # telnet cp-node_name
login: admin
password: admin |
The interface you use to change
the password differs depending upon the BMC firmware version. Thus,
do one of the following: For BMC Firmware Version
1.03: Press Esc and Shift-9 to enter the command-line mode. Use the C[hange
Password] option to change the console port password. The
factory default password is admin; change it to
the password of your choice. This password must be the same on every
node in your system. Lights-Out> C
Type the current password> admin
Type the new password (max 16 characters)> your_password
Retype the new password (max 16 characters)> your_password
New password confirmed.
Lights-Out> exit |
For BMC Firmware Version
1.24 or higher: Press Esc and Shift-9 to enter into the command-line mode. Change to the following
directory: /directory_name/-> cd map1/accounts |
List the pre-defined users
(by default, 16 users have been pre-defined): One user at a time, use
the show command until you find the first instance
of the user name admin . Start with user1 and continue executing the show command for each user in sequential
order: /directory_name/-> show user1
/directory_name/-> show user2 {...userN} |
For the purposes of this example, assume that user3 is the first instance of the user name admin . Change directory to the
user directory obtained in the previous step: /directory_name/-> cd user3 |
Set the password. Be very
careful with your key strokes; there is no password verification step: /directory_name/-> set password=your_password |
Exit from the interface:
Repeat this procedure to change
the console management device password on every HP ProLiant DL140
and DL145 node in the hardware configuration. Rerun the discover command with the same options.
|