 |
» |
|
|
|
To troubleshoot HP-UX IPSec, it is useful to understand a
few key points about its operation. This section contains high-level
descriptions of how IPSec establishes Security Associations (SAs)
and how IPSec processes packets. Establishing
Security Associations (SAs) | |
Before IPSec can authenticate or encrypt an IP packet using
an IPSec transformation—an Authentication Header (AH) or
Encapsulating Security Payload (ESP)—IPSec must establish
SAs with the remote system. You can think of the SAs as security
sessions, where the two systems agree on the type of authentication
and encryption, the encryption keys and other parameters. The procedure
for establishing SAs is described below: Authenticate
Identities Each system authenticates the other system's identity, using preshared
keys or security certificates (RSA signatures). Each system also
verifies ID types and ID values (HP-UX IPSec uses IP addresses as
ID values by default). This is part of the establishment of an ISAKMP
or Main Mode SA (ISAKMP/MM SA), as described in the next step. Establish ISAKMP/MM SA The two systems complete the establishment of the ISAKMP/MM SA.
The ISAKMP/MM SA is the “master” SA that the two
systems use as a secure channel to negotiate the SAs for AH and/or
ESP packets. Establish IPSec/QM SAs Once an ISAKMP/MM SA is established, the two systems have
a secure channel for negotiating IPSec or Quick Mode SAs (IPSec/QM SAs).
The IPSec/QM SAs determine the HP-UX IPSec transformation(s) used
(AH and/or ESP), the encryption keys for AH/ESP and other parameters.
Two IPSec/QM SAs are established: one for packets from the local
system to the remote system and one for packets from the remote
system to the local system.
Note that one ISAKMP/MM SA can be used to negotiate multiple
pairs of IPSec/QM SAs. Internal
Processing | |
This section provides an a high-level description of how HP-UX
IPSec processes packets. This information is useful to further troubleshoot HP-UX
IPSec and analyze the data reported by the HP-UX IPSec troubleshooting
tools. Query the Kernel
Policy Engine HP-UX IPSec first checks the kernel policy engine cache for
an existing decision on the action to take (secure, drop, or pass
in clear text) for the packet based on the IP addresses, protocol
and port numbers. If the action is secure (use an Authentication
Header, AH or use an Encapsulating Security Payload, ESP), there
may be a reference to an existing IPSec/QM SA that can be used. Query the Policy Manager
Daemon If no match is found in the policy engine cache, the Policy
Manager daemon is queried for the policy and action (secure, drop,
or pass in clear text) to take. On an end system (the local system is the source for the outbound packet),
the Policy Manager sequentially searches the host IPSec policies
in priority order for the first policy with an IP packet filter that
matches the packet. If no match is found, HP-UX IPSec uses the default
host IPSec policy. On a gateway system (the local system is forwarding the outbound packet),
the Policy Manager sequentially searches the gateway IPSec policies
in priority order. If no match is found, HP-UX IPSec uses the default
gateway IPSec policy. If the transform (action) specified in the matching IPSec
policy is to encrypt or authenticate the IP packets using AH or
ESP, IPSec SAs may already exist for the policy. The new packet
can use the existing IPSec/QM SAs if the IP addresses, ports and
protocols match. The new packet can also use the existing IPSec/QM
SAs if both IP addresses match and host-based keying is enabled
(the Exclusive bit is not set for the policy). Otherwise, new IPSec/QM
SAs are established. Establish an ISAKMP/MM SA Before establishing an IPSec/QM SA, the IKE daemon must establish
an ISAKMP/MM SA with the remote system, if none exists. To establish
an ISAKMP/MM SA, the Policy Manager daemon is queried for an IKE
policy, based on the remote system’s IP address. As part of the ISAKMP/MM SA negotiations, each system establishes
a Diffie-Hellman value using the Oakley group specified in the IKE
policy. Because the Diffie-Hellman algorithm is vulnerable to third-party
attacks, each system must also authenticate the identity of the
other system using the primary authentication method specified in
the IKE policy (preshared keys or RSA signatures with security certificates),
and verify the remote system’s ID type and ID value (HP-UX
IPSec uses IP addresses by default). The two systems also negotiate
the authentication and encryption algorithms for the ISAKMP/MM SA,
and SA lifetime according to values specified in the IKE policies. Establish IPSec/QM SAs Once the ISAKMP/MM SA is established, the IKE daemon uses
the secured channel to establish IPSec/QM SAs with its peers. Two IPSec/QM
SAs are established: one for packets from the local system to the
remote system, and one for packets from the remote system to the
local system. For the IPSec/QM SAs to be successfully established, both
systems must agree on the type of transform (AH, ESP), including
the authentication or encryption algorithm used. They must also negotiate
SA lifetimes. Add IPSec/QM SAs to the
Kernel SA Database The IPSec/QM SAs are added to the kernel SA database by the
IKE daemon. Each SA includes an SPI (Security Parameters Index)
a number assigned by the receiving system to reference the SA. The SPI
for outbound data is assigned by the remote system, while the SPI
for the inbound data is established by the local system. The SPI is
included in the AH or ESP header so that the destination system can
process inbound packets with the correct SA parameters including
encryption key(s).
AH
or ESP Packet If the inbound packet has an Authentication Header (AH) and/or
an Encapsulating Security Payload (ESP), HP-UX IPSec checks the kernel
SA database for inbound packets for an entry with the same SPI and
source IP address. If one exists, it uses the information in the
SA to properly decrypt or authenticate the packet. If the inbound packet
has multiple SPIs (a nested AH and ESP packet), HP-UX IPSec searches
the kernel SA database for each SPI. If no matching entry exists, HP-UX IPSec will check if there
is an IKE policy that applies to the remote system. If there is
not, this is an error and possible intrusion attempt. HP-UX IPSec
sends an audit message to the audit daemon. HP-UX IPSec discards
the packet. If the local system has an IKE policy that applies to the
remote system, HP-UX IPSec will assume that a valid IPSec/QM SA previously
existed, but the SPI entry no longer exists because the local system
has re-booted. The local system will attempt to establish a new
ISAKMP/MM SA with the remote system, and to send an ISAKMP INITIAL-CONTACT
notify message. The INITIAL-CONTACT notify message notifies the
remote system that the local system has re-started IPSec. The remote
system may delete its information for all SAs established with the
local node and attempt to re-establish a new SAs. If the remote
system does not delete the SAs, an administrator on the remote system
must manually delete the SAs. Clear Text Packet If the inbound packet has no AH or ESP (it is a normal IP
packet in clear text), HP-UX IPSec must still determine whether
the packet should be dropped or passed in clear text. HP-UX IPSec
checks the kernel policy engine cache for an existing decision on
the action to take (drop or pass in clear text) for the packet based
on the IP addresses, protocol, and port numbers. If the action is
to apply an AH or ESP transform, HP-UX IPSec sends an audit message
to the audit daemon. This is because the remote system should have
established IPSec/QM SAs before sending the packet. If no cache entry exists, HP-UX IPSec queries the policy manager daemon
for the appropriate action according to the host IPSec policy with
the filter that best matches the packet (or the default policy,
if no filters match). Again, if the action is to apply an AH or
ESP transform, HP-UX IPSec discards the packet and sends an audit message
to the audit daemon.
Establishing
Tunnel Security AssociationsIf HP-UX IPSec is processing an outbound packet and the selected
host or gateway IPSec policy specifies a tunnel IPSec policy, HP-UX
IPSec checks if it has an existing tunnel SA with the tunnel endpoint.
If not, it must establish a tunnel SA before it establishes the
end-to-end (transport) SA. The procedure for establishing a tunnel
SA is similar to establishing a transport SA (HP-UX IPSec uses/establishes
an ISAKMP/MM SA to establish the IPSec/QM SA), except the IKE entities also
include proxy address information during IPSec/QM SA negotiation. The
proxy address information identifies the end-to-end entities and allows
a tunnel endpoint to determine the identity of the end system or subnet
for which the other tunnel endpoint is establishing the tunnel. Processing
Inbound Tunnel PacketsIf HP-UX IPSec is processing an inbound packet, it searches
the kernel SA database for inbound packets for an entry with the
same SPI and source IP address. If one exists, it uses the information
in the SA to decrypt or authenticate the packet. If this is a tunnel
SA, HP-UX IPSec decapsulates the packet (removes the outer IP header)
and processes the IP header for the inner packet. If the destination
address in the inner packet is a local address, HP-UX IPSec searches
its host IPSec policies to determine the next action. If it is not
a local address, HP-UX IPSec searches its gateway IPSec policies
to determine the next action. If the SA uses manual keys, HP-UX
IPSec also verifies that the SA SPI for the tunnel policy referenced
in the host or gateway policy matches the SPI in the outer (tunnel)
packet. |
Printable version
