Internet Security Protocol

Dr. Dobb's Journal June 1997

Examining IPv6's IPSEC security layer

By Tom Markham

Tom is a senior engineer at Secure Computing Corp. He can be reached at markham@securecomputing.com.
Sidebar: "Protocol Selection Criteria"

The 1996 Computer Security Institute/FBI Computer Crime and Security Survey found that 42 percent of the respondents experienced unauthorized use of their computer systems in the last 12 months. Almost 40 percent of these respondents experienced frequent incidents through both remote dial-in and Internet connections. Clearly, information security is important to all of us. In this article, I'll examine the IPSEC protocol, a security layer that is part of the IPv6 protocol.

IP Security (IPSEC) operates in the IP layer and secures anything using UDP or TCP. IPSEC is attractive for several reasons, including:

IPSEC Architecture

IPSEC was designed by the Internet Engineering Task Force (IETF) to protect any system using IP. "Security Architecture for the Internet Protocol" (RFC 1825) describes the security mechanisms and the services it provides. Two headers are used within IPSEC: The Authentication Header (AH), used to provide source authentication and integrity for the IP datagram, and the Encapsulating Security Payload (ESP), used to provide confidentiality. A pair of communicating entities share a security association (cryptographic keys, algorithms, and related information) that is referenced by a security parameter's index (SPI).

The Authentication Header immediately follows the IPv4 header. As Figure 1 illustrates, the Authentication Header contains the next header, length, a reserved field, SPI, and authentication data. The authentication data is a value computed as a function of the IP datagram and the secret authentication keying material, which is part of the security association. The secret keying material is known only by the sender and receiver, so if the authentication data is valid, the receiver knows the data came from the other party in the security association. The receiver uses the SPI contained in the packet to determine which keying material to use when validating the authentication data.

Authentication data is calculated using a message digest (a secure hash algorithm). MD5 and SHA-1 are two algorithms commonly used with IPSEC. These algorithms have two important properties:

These properties make it computationally infeasible for an attacker to forge an IP datagram. The most secure technique for computing this authentication data is defined in the AH HMAC Internet Draft for MD5 and for SHA-1. MD5 is faster than SHA-1 but SHA-1 is regarded as more secure. The basic algorithm is authentication_data=MD (key1 XOR outer pad, MD (key2 XOR inner pad, data_to_be_authenticated)), where MD represents the message digest processing, key1 and key2 are shared secrets known only to the sender and receiver, and inner pad and outer pad are fixed values. MD5 and SHA are also discussed in the articles "One-Way Hash Functions," by Bruce Schneier (DDJ, September 1991) and "SHA: The Secure Hash Algorithm," by William Stallings (DDJ, April 1994), respectively.

An attacker does not know the secret authentication key shared between the sender and receiver, so the attacker cannot produce a pair (authentication_data, data_to_be_authenticated) that the receiver will accept as authentic.

Authenticating the source of IP datagrams is necessary to protect against attacks such as IP spoofing and hijacking. However, this is only half of the solution. The Encapsulating Security Payload (ESP) protocol is required to prevent an attacker from reading your sensitive information as it moves across the open network.

The typical ESP packet contains the security parameters index (SPI), followed by opaque data (typically an initialization vector and ciphertext); see Figure 2. The sender appends padding, pad length, and payload type to the plaintext payload data. The padding is necessary to make the size of the data a multiple of the width of the cryptographic mechanism. Most cryptographic algorithms operate on 64-bit blocks of data. The sender generates an initialization vector that both the sender and receiver use to set the initial state of the cryptographic mechanism. The sender then uses the secret key shared by sender and receiver to encrypt the payload data, padding, pad length, and payload type. The resulting ciphertext is placed in the IP datagram and sent to the receiver.

The receiver uses the SPI to determine which secret cryptographic key to use to decrypt the data. An attacker on the open network can observe the encrypted data but cannot recover the original plaintext without the secret key.

It is possible to use AH or ESP separately, but to protect yourself from subtle attacks, such as modification of the ESP header -- which must be sent in plaintext -- ESP should never be used without AH. For outbound packets, perform the ESP processing first, then treat the result as the upper protocol portion of the AH packet.

Processing

When it comes to cryptographic programming, the devil is in the details. Not only is performance sensitive to implementation decisions, but security vulnerabilities can be traced back to the coding stage of the development process. I'll provide an overview of the algorithms, identify performance factors, and point out a potential security vulnerability (and leave the "driving" to you).

Assume a sender has already performed ESP and AH processing to create an IPSEC packet. The steps required by a host receiving an IPSEC packet are:

1. Examine the protocol header on the packet. When the AH header is found, call AH processing.

2. Extract the SPI from the AH header. The SPI is used to extract the shared secret (keys for AH) from the security association database.

3. Using the shared secret, calculate the authentication data. Only an entity with the shared secret is capable of calculating the correct authentication data.

4. Test the received authentication data against that just calculated. If the values match, the integrity of the data has not been compromised.

5. Discard the AH header and extract the ESP header. The ESP header contains its own SPI, which may be different from the AH SPI.

6. Extract the ESP SPI from the ESP header. This SPI is used to access the security association database a second time to retrieve the ESP receive key. (Smart implementations will cache the entire security association record when it is retrieved for AH processing.)

7. Decrypt the payload. The ESP key is loaded into the cryptographic algorithm. The initialization vector is extracted from the header and used to initialize the state of the receiver's cryptography. The payload is then decrypted to recover the payload data and trailer information.

8. Remove the trailer and pass the payload on. The padding and pad length are stripped off. The payload can then be passed up the stack to the next protocol (UDP or TCP).

The process of encrypting and generating authentication data for outbound packets is essentially done in reverse order. On an outbound packet, however, the host looks at the destination address and searches the security association database for the corresponding keying material and SPI. Implementations designed to support many simultaneous connections (firewalls, for instance) should pay particular attention to optimizing the security association database lookup. The host must be able to quickly index into it using either the destination address, AH SPI, or ESP SPI.

Cryptography is a CPU hog (by design). Table 1 presents representative implementation speeds for the DES cryptographic algorithm. These numbers are for well-written implementations in C. Performance can be improved by as much as 40 percent by using carefully crafted assembly language. When analyzing timing and sizing analysis, keep in mind that these numbers are for the cryptographic processing only -- they do not include other protocol processing.

Don't fall into the trap of simply looking at the throughput in kilobytes per second. Most cryptographic algorithms have a set-up time that includes expanding the relatively short key (56 bits in the case of DES) to a longer key (768 bits for DES) for internal use. The switching and key expansion associated with processing small packets significantly degrades performance. The performance on small packets can be increased by caching the expanded DES key in the security association database.

The set-up time for the message-digest algorithms (MD5 and SHA-1) is also significant. Performance measurements on a Sparc 20/71 demonstrated an MD5 throughput of 60 MB per second when operating on a large file. However, the throughput when using 1-KB packets drops to only about 1 percent of this.

Buffer management can influence performance, although it is not as important as cryptographic processing. When designing parsing routines, keep track of variable-length fields and their location in the packet. Also keep in mind that the authentication data used in the AH header is a function of the data that follows it.

A common problem developers face when implementing cryptographic code is generating high-quality random numbers. A surprising number of systems have been successfully attacked because randomization was not performed properly. If you are planning on implementing your own random-number generator, you owe it to yourself to read "Truly Random Numbers," by Colin Plumb (DDJ, November 1994) and RFC 1750.

Potential Attacks

When developing security software, what you don't know can hurt you. I'm going to describe an attack, simplified for the sake of discussion and present a way to protect yourself. The point of this example is to force you to ask yourself, "What if this is used/abused in a manner not addressed by the design?"

Assume the configuration in Figure 3. "Your Host" has IPSEC security associations with two other hosts ("Good" and "Evil"), each connected via routers implementing IPSEC. Since all traffic coming into Your Host is protected by IPSEC, you might assume that Your Host could do access control based on the source address on the packet. This may not be the case. Assume that Evil Host wanted to send Your Host a packet claiming to be from Good. Evil could construct a packet that, when passed though its IPSEC router operating in tunnel mode, would have the format:

Outer IP address = Evil

Authentication Header IP address = Evil

IP address on tunneled packet = Good

When this packet arrives at your IPSEC router, it is processed like this:

1. The outer IP address is an address your router recognizes. The packet is sent to the IPSEC code for processing.

2. Evil has a valid security association (SA2) with your router. Your router extracts the SPI from the authentication header and validates the integrity of the packet. The integrity check confirms that the packet has not been corrupted (since it was sent from the remote router).

3. Your router extracts the SPI for ESP and decrypts the payload. This payload is a packet that was encrypted using tunnel mode.

Does your implementation verify that the IP address in the tunneled packet matches the IP address just authenticated with AH? If it does not, Your Host will make a flawed access-control decision.

This is just one example of the sorts of security bugs that are easily overlooked during development and testing. To produce good security code, you need to be familiar with the techniques that will be used to attack the system.

The Future of IPSEC

The IETF is continuing its IPSEC effort, developing new security transforms that will provide even greater security. The protocol is also being enhanced to better utilize bandwidth. IPSEC is a part of the emerging IPv6 protocol, so your investment in IPSEC will be protected as the Internet evolves.

The next major increase in functionality is the automated cryptographic key management in 1997. The Internet Security Association Key Management Protocol (ISAKMP) will allow public-key cryptography to be used to simplify security management. Reference implementations of ISAKMP are available at http://web.mit.edu/network/isakmp/.

The IETF, along with other members of the computer-security community, are working to field IPSEC implementations. Reference source code for IPSEC AH and ESP is available from ftp://ftp.ripe.net/ipv6/nrl/IPv6_domestic.tar.gz.

To learn more about the status of IPSEC and get the latest source code, check out the IPSEC web page at http://www.cs.arizona.edu/xkernel/www/ipsec/ipsec.html.

References

Bellare, M., R. Canetti, and H. Krawczyk. "HMAC-MD5: Keyed-MD5 for Message Authentication." Internet Draft (March 1996). http://ds.internic.net/internet-drafts/draft-ietf-ipsec-hmac-md5-00.txt.

"Breaking the RNG: The Cleverest Hack So Far." http://www.c2.org/hacknetscape/.

Chang, S. and R. Glenn. "HMAC-SHA IP Authentication with Replay Prevention." Internet Draft (May 1996). http://ds.internic.net/internet-drafts/draft-ietf-ipsec-ah-hmac-sha-00.txt.

Computer Incident Advisory Capability Information Bulletin C-18, Vulnerability in AT&T. /usr/etc/rexecd (February 1992).

Computer Incident Advisory Capability Information Bulletin C-21, AIX REXD Daemon Vulnerability (March 1991).

"Computer Security Issues and Trends." San Francisco, CA: Computer Security Institute (Spring 1996).

Department of Defense Trusted Computer System Evaluation Criteria, CSC-STD-001-83. Fort George G. Meade, MD: Department of Defense Computer Security Center (August 1983).

Eastlake, 3rd, D., S. Crocker, and J. Schiller. Request for Comment 1750: Randomness Recommendations for Security (December 1994).

Goldberg, Ian and David Wagner. "Randomness and the Netscape Browser." Dr. Dobb's Journal (January 1996).

Harkins, D. and D. Carrel. "Resolution of ISAKMP with Oakley." Internet Draft (June 1996). http://ds.internic.net/internet-drafts/draft-ietf-ipsec-isakmp-oakley-00.txt.

Hughes, Jim. ''Combined DES-CBC HMAC and Replay Prevention Security Transform." Internet Draft (June 1996). http://ds.internic.net/internet-drafts/draft-ietf-ipsec-esp-des-md5-02.txt.

Joncheray, Laurent. "A Simple Active Attack Against TCP." Paper presented at the Fifth USENIX UNIX Security Symposium. Salt Lake City, UT (June 1995).

Kent, Steve et al. IPSEC mailing list. http://www.cs.arizona.edu/xkernel/www/ipsec/ipsec-mail2.html#157.

Maughan, D., M. Schertler, M. Schneider, and J. Turner. "Internet Security Association and Key Management Protocol." Internet Draft (June 1996). http://ds .internic.net/internet-drafts/draft-ietf-ipsec-isakmp-05.txt.

Morris, Robert T. "A Weakness in the 4.2 BSD Unix TCP/IP Software." Murray Hill, NJ: AT&T Bell Laboratories.

Orman, H.K. "Oakley Key Determination Protocol." Internet Draft (May 1996). http://ds.internic.net/internet-drafts/draft-ietf-ipsec-oakley-01.txt.

Plumb, Colin. "Truly Random Numbers." Dr. Dobb's Journal (November 1994).

Request for Comment 1321: The MD5 Message-Digest Algorithm. ftp://ds.internic.net/rfc/rfc1321.txt.

Request for Comment 1825: Security Architecture for the Internet Protocol. ftp://ds.internic.net/rfc/rfc1825.txt.

Request for Comment 1826: Authentication Header. ftp://ds.internic.net/rfc/rfc1826.txt.

Request for Comment 1827: The Encapsulation Mode. ftp://ds.internic.net/rfc/rfc1827.txt.

Request for Comment 1829: ESP DES-CBC Transform. http://ds.internic.net/rfc/rfc1829.txt.

"Secure Hash Standard," FIPS PUB 180, Federal Information Processing Standards Publication. Washington, DC: U.S. Department Of Commerce, National Institute of Standards and Technology (May 1993). http://www.nist.gov/itl/lab/fips/fips180.txt.

DDJ


Copyright © 1997, Dr. Dobb's Journal