An individual sa can implement both the ah and the esp protocol.

What is anti-replay protocol?

The anti-replay protocol provides Internet Protocol (IP) packet-level security by making it impossible for a hacker to intercept message packets and insert changed packets into the data stream between a source computer and a destination computer. By detecting packets that match the sequence numbers of those that have already arrived, the mechanism helps ensure that invalid packets are discarded and communication integrity is maintained.

This anti-replay protocol is a subprotocol of the Internet Engineering Task Force's IP Security (IPsec) standard suite of protocols. The two main protocols under IPsec -- Encapsulating Security Payload (ESP) and Authentication Header (AH) -- both use anti-replay protection.

An individual sa can implement both the ah and the esp protocol.
IPsec adds several components with security information and cryptographic algorithms to IP headers.

What is a replay attack?

A replay attack occurs when threat actors eavesdrop on a secure network to intercept and record its legitimate communications or data transmissions. They then either delay, resend or repeat this data in order to do the following:

  • impersonate a valid user;
  • misdirect the receiver into doing something they might not have done otherwise; or
  • disrupt legitimate connections.

The goal of a replay attack is to subvert security and cause some kind of negative impact on authorized users and connections.

Real-world example of a replay attack

An employee requests a financial transfer by sending an encrypted message to an authorized person within the company. This could be someone like the financial administrator or a senior accountant.

A hacker captures the message and resends it to the same target recipient. Because the message remains authentic -- it has not been altered -- is correctly encrypted and has simply been resent, it looks legitimate to the recipient.

As a result, receivers may honor the request for the money transfer, unaware that they -- and their organization -- have just been swindled out of a large sum of money.

An individual sa can implement both the ah and the esp protocol.
In a replay attack, a threat actor obtains a copy of a message and resends it to the recipient, fooling them into thinking it's coming from the original sender.

How does the anti-replay protocol work?

The answer to preventing replay attacks is encrypting messages and including a key. IPsec provides anti-replay protection against attackers who could potentially intercept, duplicate or resend encrypted packets.

The mechanism uses a unidirectional security association to establish a secure connection between the source and destination nodes in the network and check whether a received message is a replayed message. It works by assigning a monotonically increasing sequence number to each encrypted packet and then keeping track of the sequence numbers as packets arrive at the destination.

At the outset, the counters of both the source and destination are initialized at 0. The first sent packet has a sequence number of 1, the second packet is numbered 2 and so on. Each time a packet is sent, the receiver verifies that the number is not that of a previously sent packet. If the next packet has a lower number than the previous packet, the destination will drop the packet. If the number is larger, it keeps the packet.

Also, when a replayed packet is detected, the anti-replay protocol generates an error message. It also discards this packet and logs the event with entry identifiers, such as the date/time received, source address, destination address and sequence number.

Enhancing the anti-replay protocol

The anti-replay protocol is designed to overcome adversaries that can insert only replayed messages in the message stream. However, this protocol can be enhanced to overcome bad actors that can potentially insert any message into the message stream.

Here's how it works: A shared secret is provided to both the source and destination computer. The adversary does not know this shared secret. Each message sent by the source consists of three elements: the message sequence number, the message text and the message digest.

Before the destination receives the message, the protocol verifies if the message digest meets a specific value. If a bad actor inserts a message in the message stream between the source and destination, the protocol checks whether the message has been sent earlier by the source or if the digest does not meet the predetermined value.

In either case, the destination discards the message.

What is the AH protocol?

The IPsec AH protocol provides user authentication and integrity protection for packet headers and data. It may also provide replay protection and access protection but cannot encrypt any portion of packets. AH functions in two modes:

  1. Transport
    • does not alter the original IP header or create a new IP header
    • usually used in host-to-host architectures
  2. Tunnel
    • creates a new IP header for each packet

What is the ESP protocol?

ESP provides encryption for packet payload data and authentication to provide integrity protection -- but not for the outermost IP header and protection services. Like the AH protocol, ESP also functions in the transport and tunnel modes. In the latter scenario, it creates a new IP header -- for each packet -- which lists the endpoints of the ESP tunnel as the packet's source and destination.

This feature makes ESP ideal for virtual private network architecture models.

Can we use AH and ESP at the same time in IPsec?

The AH can be applied alone or together with the ESP when IPSec is in transport mode.

How security associations work when both AH and ESP are applied?

AH-style authentication authenticates the entire IP packet, including the outer IP header, while the ESP authentication mechanism authenticates only the IP datagram portion of the IP packet. Either protocol can be used alone to protect an IP packet, or both protocols can be applied together to the same IP packet.

Which two modes can IPsec use with AH or ESP?

IPsec can be used in tunnel mode or transport mode. Tunnel mode provides confidentiality (ESP) and/or authentication (AH) to the entire original packet, including the original IP headers.

How the IPsec do protocols ESP and AH provides replay protection?

IPSec uses two distinct protocols, Authentication Header (AH) and Encapsulating Security Payload (ESP), which are defined by the IETF. The AH protocol provides a mechanism for authentication only. AH provides data integrity, data origin authentication, and an optional replay protection service.