 |
» |
|
|
|
To troubleshoot HP-UX IPSec, it is useful to understand a
few key points about its operation. This section contains high-level
descriptions of how IPSec establishes Security Associations (SAs)
and how IPSec processes packets. Establishing
Security Associations (SAs) | |
Before IPSec can authenticate or encrypt an IP packet using
an IPSec transformation—an Authentication Header (AH)
or Encapsulating Security Payload (ESP)—IPSec must establish
SAs with the remote system. You can think of the SAs as security
sessions, where the two systems agree on the type of authentication
and encryption, the encryption keys and other parameters. The procedure
for establishing SAs is described below: Authenticate
Identities Each system authenticates the other system's identity, using preshared
keys or a certificate-based method: RSA signature. This is part
of the establishment of an ISAKMP or Main Mode SA (ISAKMP/MM SA),
as described in the next step. Establish ISAKMP/MM SA The two systems complete the establishment of the ISAKMP/MM SA.
The ISAKMP/MM SA is the “master” SA that the two
systems use as a secure channel to negotiate the SAs for AH and/or
ESP packets. Establish IPSec/QM SAs Once an ISAKMP/MM SA is established, the two systems have
a secure channel for negotiating IPSec or Quick Mode SAs (IPSec/QM SAs).
The IPSec/QM SAs determine the HP-UX IPSec transformation(s) used
(AH and/or ESP), the encryption keys for AH/ESP and other parameters.
Two IPSec/QM SAs are established: one for packets from the local
system to the remote system and one for packets from the remote
system to the local system.
Note that one ISAKMP/MM SA can be used to negotiate multiple
pairs of IPSec/QM SAs. Internal
Processing | |
This section provides an a high-level description of how HP-UX
IPSec processes packets. This information is useful to further troubleshoot HP-UX
IPSec and analyze the data reported by the HP-UX IPSec troubleshooting
tools. Query the Kernel
Policy Engine HP-UX IPSec first checks the kernel policy engine cache for
an existing decision on the action to take (secure, drop, or pass
in clear text) for the packet based on the IP addresses, protocol
and port numbers. If the action is secure (use an Authentication
Header, AH or use an Encapsulating Security Payload, ESP), there
may be a reference to an existing IPSec/QM SA that can be used. Query the Policy Manager
Daemon If no match is found in the policy engine cache, the Policy
Manager daemon is queried for the policy and action (secure, drop,
or pass in clear text) to take. The Policy Manager checks the hashed IPSec policies first
and tries to find the policy with the IP packet filter that best
matches the packet. The best match is the filter that matches the
packet and has the largest number of hashable fields that match
the packet. If multiple filters have the same number of hashable
fields that match the packet, HP-UX IPSec selects the first of these
policies according to the order in the configuration file. If HP-UX IPSec finds no match in the hashed IPSec policies,
it then sequentially searches the ordered IPSec policies for the
first policy with an IP packet filter that matches the packet. If
no match is found in the ordered policies, HP-UX IPSec uses the
default IPSec policy. If the transform (action) specified in the matching IPSec
policy is to encrypt or authenticate the IP packets using AH or
ESP, IPSec SAs may already exist for the policy. The new packet
can use the existing IPSec/QM SAs if the IP addresses, ports and
protocols match. The new packet can also use the existing IPSec/QM
SAs if both IP addresses match and host-based keying is enabled
(the Exclusive bit is not set for the policy). Otherwise, new IPSec/QM
SAs are established. Establish an ISAKMP/MM SA If the transform (action) specified in the matching IPSec
policy is to encrypt or authenticate the IP packets using AH or
ESP and no IPSec Security Associations already exist, the IKE daemon
(ikpmd) must establish an IPSec/QM SA. Before establishing an
IPSec/QM SA, the IKE daemon must establish an ISAKMP/MM SA with
the remote system if none exists. For the ISAKMP/MM SA to be successfully established, both
systems must authenticate the identity of the other system (primary authentication).
This is done using preshared keys, or a method using security certificates
and public-key encryption: RSA signatures. In addition, both systems must use the same Oakley group (Diffie-Hellman
initial values), and agree on the authentication algorithm and encryption
algorithm used for the ISAKMP/MM SA. They must also negotiate an
SA lifetime. Establish IPSec/QM SAs Once the ISAKMP/MM SA is established, the IKE daemon uses
the secured channel to establish IPSec/QM SAs with its peers. Two IPSec/QM
SAs are established: one for packets from the local system to the
remote system, and one for packets from the remote system to the
local system. For the IPSec/QM SAs to be successfully established, both
systems must agree on the type of transform (AH, ESP), including
the authentication or encryption algorithm used. They must also negotiate
SA lifetimes. Add IPSec/QM SAs to the
Kernel SA Database The IPSec/QM SAs are added to the kernel SA database by the
IKE daemon. Each SA includes an SPI (Security Parameters Index)
a number assigned by the receiving system to reference the SA. The SPI
for outbound data is assigned by the remote system, while the SPI
for the inbound data is established by the local system. The SPI is
included in the AH or ESP header so that the destination system can
process inbound packets with the correct SA parameters including
encryption key(s).
AH
or ESP Packet If the inbound packet has an Authentication Header (AH) and/or
an Encapsulating Security Payload (ESP), HP-UX IPSec checks the kernel
SA database for inbound packets for an entry with the same SPI and
source IP address. If one exists, it uses the information in the
SA to properly decrypt or authenticate the packet. If the inbound packet
has multiple SPIs (multiple transforms), HP-UX IPSec searches the
kernel SA database for each SPI. If no matching entry exists, this is an error and HP-UX IPSec
sends an audit message to the audit daemon. HP-UX IPSec discards
the packet. Clear Text Packet If the inbound packet has no AH or ESP (it is a normal IP
packet in clear text), HP-UX IPSec must still determine whether
the packet should be dropped or passed in clear text. HP-UX IPSec
checks the kernel policy engine cache for an existing decision on
the action to take (drop or pass in clear text) for the packet based
on the IP addresses, protocol, and port numbers. If the action is
to apply an AH or ESP transform, HP-UX IPSec sends an audit message
to the audit daemon. This is because the remote system should have
established IPSec/QM SAs before sending the packet. If no cache entry exists, HP-UX IPSec queries the policy manager daemon
for the appropriate action according to the IPSec policy with the
filter that best matches the packet (or the default policy, if no filters
match). Again, if the action is to apply an AH or ESP transform,
HP-UX IPSec discards the packet and sends an audit message to the
audit daemon.
|
Printable version
