Are You Really Scanning What You Think

In a previous post we explored the importance of scanning hostnames instead of IP addresses in order to avoid missing certain content and we also briefly touched upon the behaviour of some common scanning tools. 

If you give your scanner a list of hosts you will receive a findings report back, but how can we be sure that the scanner done a thorough job of scanning everything you gave it? In some cases, it won't do a thorough job and often this information will be buried away in the small print of the report if it's present at all. Let's take a look at a few examples using commonly available tools:

Scenario 1: Your List Contains Mistyped Hostnames or Addresses

It is not uncommon to have a typo in scan list, especially with a larger list of targets. If you make a typo when entering your target list, you really need to know about it so you can ensure that the scan is conducted properly. But do common scanning tools detect and warn you about the mistakes like these?

NMap

So let's see how common port scanning tool NMap handles this scenario. We start by creating a simple file containing a list of targets for NMap:

$ cat targets
1.2.3.444
127.0.0.1
doesnotexist.example.org

In this example, we have an invalid IPv4 address (1.2.3.444), a valid IPv4 address (127.0.0.1), and a hostname that does not resolve (doesnotexist.example.org).

$ nmap -oA test.out -iL targets

Starting Nmap 7.60 ( https://nmap.org ) at 2020-07-14 23:05 CDT
Failed to resolve "1.2.3.444".
Failed to resolve "doesnotexist.example.org".
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00011s latency).
Not shown: 995 closed ports
PORT STATE SERVICE
22/tcp open ssh
111/tcp open rpcbind
1025/tcp open NFS-or-IIS
3128/tcp open squid-http
9003/tcp open unknown

Nmap done: 1 IP address (1 host up) scanned in 0.63 seconds

So what's happened? We've given NMap 3 targets to scan, but it has only scanned 1. It has however provided a warning that two of the targets failed to resolve. It has actually assumed that the invalid IPv4 address is a hostname rather than an IP and tried to perform a DNS lookup of it, which has subsequently failed as there are no fully numeric top level domains.

Unless you pay close attention to the output, it's quite easy to miss the fact that one of the addresses was found to be invalid and therefore not scanned. If you were scanning a larger number of targets then it's even easier to miss the warning in amongst the output. 

But how about the output files? NMap produces "greppable output" as well as an XML output, which is commonly ingested by a variety of tools. Let's start with the greppable output:

$ cat test.out.gnmap
# Nmap 7.60 scan initiated Tue Jul 14 23:05:44 2020 as: nmap -oA test.out -iL targets
Host: 127.0.0.1 (localhost) Status: Up
Host: 127.0.0.1 (localhost) Ports: 22/open/tcp//ssh///, 111/open/tcp//rpcbind///, 1025/open/tcp//NFS-or-IIS///, 3128/open/tcp//squid-http///, 9003/open/tcp//unknown/// Ignored State: closed (995)
# Nmap done at Tue Jul 14 23:05:44 2020 -- 1 IP address (1 host up) scanned in 0.63 seconds

So in this output, you have no warnings. You are presented with the results for the one valid target, and those invalid targets are totally gone. It's exactly the same with the XML output, the warnings are gone and you only have the output for the target that was valid. Many tools out there ingest the XML output from NMap, so if you rely on this workflow without reading the output from the command line you would have no idea what happened, and could easily miss one or two hosts in a large list.

Nessus

The vulnerability scanner Nessus behaves in a very similar way to NMap. If you specify a mix of targets where some are valid and some are not, you will get warnings in the "Notes" section of the Nessus web interface:

But when you export the report out of the web interface and view it as a PDF, the hosts that failed to resolve are now gone completely:

So if you receive a Nessus report like this, how do you know all your targets were scanned? In short, you don't, not unless you want to go through each host in the report manually and correlate it against your original target list, which would be very time consuming for a large scan.

Scenario 2: You specify a hostname that resolves to multiple addresses

This was briefly touched upon in the previous post. It is extremely common now for a single hostname to resolve to multiple addresses. The Internet is gradually transitioning from the legacy IPv4 protocol to IPv6 with many sites being simultaneously available via both protocols. Similarly, round-robin DNS has long been used as a crude form of load balancing. Many high profile sites resolve to multiple addresses:

$ host www.yahoo.com
www.yahoo.com is an alias for new-fp-shed.wg1.b.yahoo.com.
new-fp-shed.wg1.b.yahoo.com has address 72.30.35.10
new-fp-shed.wg1.b.yahoo.com has address 98.138.219.231
new-fp-shed.wg1.b.yahoo.com has address 72.30.35.9
new-fp-shed.wg1.b.yahoo.com has address 98.138.219.232
new-fp-shed.wg1.b.yahoo.com has IPv6 address 2001:4998:44:41d::3
new-fp-shed.wg1.b.yahoo.com has IPv6 address 2001:4998:44:41d::4
new-fp-shed.wg1.b.yahoo.com has IPv6 address 2001:4998:58:1836::11
new-fp-shed.wg1.b.yahoo.com has IPv6 address 2001:4998:58:1836::10

While in most cases all of these addresses will have the same services configured in the same way, this is not always the case. The SSL Labs SSL tester correctly scans all addresses and reports any anomalies, so a quick google search for "site:ssllabs.com "inconsistent server configuration" picks up a number of sites where the SSL configuration is not the same across all addresses, such as "filing.osha.gov.tw" where the IPv6 address had a weaker SSL configuration:

It is important to scan all addresses in order to identify such anomalies and any vulnerabilities that may arise as a result. Not doing so will leave gaps in your security. Note: no testing was performed against "filing.osha.gov.tw" by the author of this post. Only information publicly available via the SSLTest site and Google was used for this example.

NMap

As previously mentioned, the default behaviour of NMap is to only scan one address unless you manually specify the "--resolve-all" option, and even with "--resolve-all" it will only scan either IPv4 or IPv6, but never both at the same time. As with non-resolving hostnames, NMap will display a warning about the additional addresses on the command line output, but the machine parseable XML and Greppable formats contain no mention of the additional addresses whatsoever.

The behaviour is also misleading when you perform a scan against an IPv6 host from a system that does not have IPv6 connectivity. If you allow NMap to ping the host (default setting) it will declare the host as down, if you disable pings with the -Pn option then it will declare that all scanned ports are closed. The reverse is also true if the system does not have IPv4 connectivity.

Nessus

The Nessus behaviour is similar, but more dangerous. If a host resolves to multiple addresses, one of them will be scanned at random and the other addresses will be totally absent from the web interface or the exported report. No warning is ever given. If you scan the same target again, you could receive a report for a different address.

The workaround for Nessus is to specify each IP and hostname together as a pair in the list:

www.yahoo.com[2001:4998:58:1836::10]
www.yahoo.com[2001:4998:44:41d::3]
www.yahoo.com[2001:4998:58:1836::11]
www.yahoo.com[2001:4998:44:41d::4]
www.yahoo.com[72.30.35.9]
www.yahoo.com[72.30.35.10]
www.yahoo.com[98.138.219.232]
www.yahoo.com[98.138.219.231]

Summary

Common scanning tools Nessus, NMap and most likely others may not be thoroughly testing your systems unless explicitly configured to do so, and may provide no warning about incorrect configurations, while other tools such as SSLTest do perform as expected. If you are relying on a scanner that is not correctly configured, you may not be receiving thorough results and may not receive any warning that the results you have are incomplete.

Other scanning tools and/or services were not tested, and may require their own workarounds or in some instances will not be able to handle multiple addresses or dual-stack configurations at all. In particular, some outdated tools lack support for IPv6 and will ignore it completely.

When you perform a scan, it's very important to make sure that the scan was performed correctly:

• Make sure all the targets supplied are valid and free of typos, as the scanner may not warn you.
• Where targets resolve to multiple addresses, ensure that all the addresses get tested.
• If any of the targets support multiple protocols (IPv6/IPv4), ensure that they are scanned from systems which also support both.

If you are receiving a scan report from someone else, you need to make sure that they have performed the scan thoroughly.