IPSec – Network Implementation – ANS-C01 Study Guide

IPSec

IPSec provides encapsulation of packets that are encrypted in transit between endpoint devices. IPSec is actually a family of protocols that work together for end-to-end encryption of data flows. IPSec will encrypt the packets and can authenticate the source of where the packets came from. It is the most common method used on networks to protect traffic from interception, snooping, and other malicious activity. Most VPN architectures use the IPSec protocol for transport.

There are two modes of IPSec encapsulation used: transport and tunnel modes. In transport mode, the transport layer and higher are encapsulated and encrypted. Transport mode protects only the original payload with no encryption for the IP header, as shown in Figure 6.8. The second method of IPSec encapsulation is tunnel mode, where the entire IP packet including the IP header is encrypted, as shown in Figure 6.9. The network layer IP payload is encapsulated with an application header (AH) or Encapsulation Security Protocol (ESP) header.

FIGURE 6.8 IPSec transport mode header

FIGURE 6.9 IPSec tunnel mode header

IPSec tunnel mode is used between two dedicated routers, with each router acting as one end of a virtual “tunnel” through a public or private corporate network. In tunnel mode, the original IP header that contains the final destination of the packet is encrypted, in addition to the packet payload. So, the intermediary routers know how to forward the packets, and IPSec adds a new IP header. At each end of the tunnel, the routers decrypt the IP headers and then deliver the packets to their destinations.

In transport mode, the payload of each packet is encrypted, but the original IP header is not. This approach allows the intermediary routers to view the destination address of each packet.

As noted earlier, IPSec is actually a family of protocols, and configurations vary widely with many different implementation and use cases. However, there are fundamental steps that we will cover here to give you a basic understanding of the protocol. IPSec can be very complex to understand and implement and is more part of the security realm. To gain a deeper knowledge, the AWS Security Specialty exam should be investigated.

The basic workflow of an IPSec session starts with the key exchange where a string of random characters is used to encrypt and decrypt the packets. IPSec standardizes the exchange of keys with a key exchange between the connected devices using many different protocols of your choice. The key exchange allows each device to decrypt the other device’s messages. Data that is sent over a network is broken down into smaller pieces called packets. Each packet has a payload, which is the user data that is being sent, and headers, which contain the information about that data so intermediate routers know where to forward them as with any IP packet. The protocol adds headers to the original packets that contain the authentication, encryption information, and IPSec trailers are also added at the end. At each receiving endpoint, the packets are decrypted, usually by a VPN client or a router running IPSec software; then the data is forwarded to the application.

IPSec authenticates each packet, with a stamp to ensure that each packet is from a trusted source and not received or tampered with by a malicious third party. Each packet is encrypted using an encryption protocol you define and that is mutually negotiated by each endpoint when the connection is established.

The AH protocol ensures that data packets are from a known, validated, and trusted source. AH ensures that the data has not been modified in transit, and AH is considered to be tamper proof. The AH headers do not provide any encryption so they can be read by intermediary devices. ESP encrypts the IP header and the payload of each packet. However, the transport mode variant encrypts the payload only. ESP stamps its own custom header and a trailer to the data packet. The Security Association (SA) is a suite of different protocols that negotiate the encryption keys and algorithms during the initial session creation and is used for the negotiation of authenticated key material. Each SA is a grouping of security parameters that are mutually agreed upon and are used for sharing information between peers across an IP network. The Internet Key Exchange (IKE) is the most common security association protocol. There is also the common Internet Security Association and Key Management Protocol (ISAKMP) used for the IKE. Security associations are unidirectional, so for end-to-end full duplex communications, there will always be two SAs per session, one in each direction. IPSec runs on top of the IP protocol, which is used for routing across both private and public networks. For IPSec to traverse a firewall, it is required to open port UDP 500 and protocol number 50.