Internet Protocol Security (RFC 4301) is vendor-agnostic. It is a collection of protocols and methods that encrypt traffic.

By design, IPSec supports IPv4 or IPv6. Any upper layer protocols that run on IP may be encrypted with IPSec.

IPSec provides data confidentiality, data integrity, origin authentication, and anti-replay services. This makes is a popular choice to use across an insecure network, such as the internet.

IPSec is independent of specific encryption algorithms. It can support several cryptographic methods, such as AES, DES, and so on.

The purpose of this article is to provide a very basic understanding of IPSec. For this reason, IKEv1 in main mode will be primarily discussed. The alternative to main mode is aggressive mode.

The Authentication Header (AH) provides data integrity and origin authentication. That is, AH provides security for the packet headers.

This verifies that the data is not altered in transit, and that the data is coming from the correct source.

The Encapsulating Security Payload (ESP) protocol provides data confidentiality. It prevents tampering with the packet payload.

This is the encryption of the main body of data (the payload) that is in transit.

A Security Association (SA) is a collection of procedures and parameters. Security devices agree on these parameters when creating a connection. The SA provides the framework for AH and ESP to work.

As part of the SA establishment, the devices are authenticated and keys are exchanged. Internet Security Association and Key Management Protocol (ISAKMP) does this. ISAKMP is only a framework., IKE does the actual key exchange.

Internet Key Exchange (IKE) is key management. This is a method of securely sharing keys between two security devices. There are currently two versions of IKE; IKEv1 and IKEv2.

An IPSec connection using IKEv1 has two main phases.

Phase 1 – IKE. Endpoints identify themselves, and mutually authenticate. They agree on security parameters, to create SA’s.

Phase 2 – Quick Mode. Using the Phase 1 tunnel, phase 2 creates the tunnel for data. Additional SA’s are created

In phase 1, IKEv1 uses a Diffe-Hellman (DH) key exchange to begin creating the SA between the devices. This creates a tunnel for the negotiation of further security parameters.

Phase 1 includes a policy containing:

• The authentication method; Such as certificates or preshared keys
• The encryption method; Such as DES or AES
• The hashing method, MD5 or SHA
• The DH Group; Usually 1, 2, or 5
• The lifetime in seconds

There are three exchanges between the two endpoints, containing six messages:

1. The initiator sends proposed policies to the responder
2. If the responder has a corresponding policy, it will accept one and reply
3. The initiator generates DH values and sends them
4. The responder generates DH values.
   • The responder sends te DH values
   • Both ends now used the DH values to generate a DH shared secret
5. The initiator sends its ID and a hash of its preshared key for authentication
6. The responder sends its ID and a hash of its preshared key
   • Both ends check that their hashes match

Three successful exchanges result in the creation of the initial tunnel.

Three messages are used in this phase:

1. The initiator sends some additional parameters to the responder
   • The encapsulation method; ESP or AH
   • The hashing method; MD5-HMAC or SHA-HMAC
   • The DH group; 1, 2, or 5
   • The SPI; The label for the end of the tunnel
2. The responder accepts the proposal if it can, and sends its own SPI and matching parameters
   • Both ends generate new DH keys for encryption
3. The initiator acknowledges that the message was received
   • The SA has now been established

Any traffic that passes over the tunnel is now encrypted, and will continue to be until the tunnel is terminated.

There are a few improvements in IKEv2. Notable changes are:

• Lower bandwidth usage
• Addition of EAP support
• Built-in NAT traversal
• Supports a tunnel ‘liveness’ check
• Adds support for the MOBIKE protocol (mobility and multihoming)