Trustwave SpiderLabs Security Advisory TWSL2017-012:
Remote un-authenticated DoS in IPsec-Tools Racoon

Published: 07/10/17
Version: 1.0

Vendor: IPsec-Tools (http://ipsec-tools.sourceforge.net/)
Product: Racoon
Version affected:  <= 0.8.2

Product description:
IPsec-Tools is a port of KAME's IPsec utilities to the Linux-2.6 IPsec 
implementation. It supports NetBSD and FreeBSD as well. The racoon deamon is a 
part of ipsec-tools and implements an Internet Key Exchange (IKE) daemon for 
automatically keying IPsec connections.

Finding 1: Remote un-authenticated denial of service
Credit: Neil Kettle of Trustwave
CVE: CVE-2016-10396

The ipsec-tools racoon daemon contains a remotely exploitable computational 
complexity attack when parsing and storing isakmp fragments. The implementation 
permits a remote attacker to exhaust computational resources on the remote 
endpoint by repeatedly sending isakmp fragment packets in a particular order 
such that the worst-case computational complexity is realized in the algorithm 
utilized to determine if reassembly of the fragments can take place.

The algorithm in question is a simple quadratic linked list walk and is in 
O(n(n+1)) hence O(n^2) for ’n’ fragments received. Since the implementation 
fails to identify repeated fragment indices, a remote attacker can repeatedly 
specify the same index. Worst-case complexity is realized if fragments are sent 
in reverse order, for instance:

253, 252 ... 3, 2, 1, 255 (last fragment)

The absence of fragment index 254 is not an error as this ensures fragment 
reassembly is not possible.

Remediation Steps:
The open-source project 'ipsec-tools' is unmaintained.  As a workaround, 
recompile recoon with ENABLE_FRAG set to false/0 (which completely compiles out 
fragmentation support). NetBSD 8 will include a fix when the branch is 
officially released.


Revision History:
10/18/16 - Vulnerability disclosed to IPsec-Tools
12/02/16 - Vulnerability disclosed to NetBSD
01/24/17 - Patch added in NetBSD HEAD
07/10/17 - Advisory published


References
1. http://cvsweb.netbsd.org/bsdweb.cgi/src/crypto/dist/ipsec-tools/src/racoon/isakmp_frag.c.diff?r1=1.5&r2=1.5.36.1

About Trustwave: Trustwave is the leading provider of on-demand and
subscription-based information security and payment card industry
compliance management solutions to businesses and government entities
throughout the world. For organizations faced with today's challenging
data security and compliance environment, Trustwave provides a unique
approach with comprehensive solutions that include its flagship
TrustKeeper compliance management software and other proprietary security
solutions. Trustwave has helped thousands of organizations--ranging from
Fortune 500 businesses and large financial institutions to small and
medium-sized retailers--manage compliance and secure their network
infrastructure, data communications and critical information assets.
Trustwave is headquartered in Chicago with offices throughout North
America, South America, Europe, Africa, China and Australia. For more
information, visit https://www.trustwave.com

About Trustwave's SpiderLabs: SpiderLabs is the advance security team at
Trustwave responsible for incident response and forensics, ethical hacking
and application security tests for Trustwave's clients. SpiderLabs has
responded to hundreds of security incidents, performed thousands of ethical
hacking exercises and tested the security of hundreds of business
applications for Fortune 500 organizations. For more information visit
https://www.trustwave.com/spiderlabs

Disclaimer: The information provided in this advisory is provided "as is"
without warranty of any kind. Trustwave disclaims all warranties, either
express or implied, including the warranties of merchantability and fitness
for a particular purpose. In no event shall Trustwave or its suppliers be
liable for any damages whatsoever including direct, indirect, incidental,
consequential, loss of business profits or special damages, even if
Trustwave or its suppliers have been advised of the possibility of such
damages. Some states do not allow the exclusion or limitation of liability
for consequential or incidental damages so the foregoing limitation may not
apply.