PCAP Files Are Great Arn't They??

One of the most important skills in anyone's armory responsible for looking after the security of a corporation's networks should be how to analyze network capture files (PCAP files) obtained from sniffers. Putting a sniffer on the network can not only help you investigate network issues, but also give you a great insight into the "unseeable" security vulnerabilities that are occurring on a daily basis.

This is probably one of the cheapest security tools you can use on the network, as it's free, and can find a multitude of potential issues. This being the case, it is also one of the easiest attack vectors an attacker or disgruntled employee can use on your internal network to extract data, and not get noticed.

Now, on a switched network you can't just fire up your sniffer, and expect to capture all the traffic. You'll see traffic, but mostly only broadcast traffic. You could set up a SPAN or mirrored port on your switch, or set up a TAP, but that's assuming you manage your network. What an attacker would do would involves creating a man-in-the-middle attack using ARP poisoning techniques.

The majority of networks we test, there tends to be little or no ARP poisoning defenses in place, and a one-line command, using something like the tool ettercap soon allows you to redirect traffic on the local network via your host to capture all the traffic traversing the network. We can then sniff the data and write it to a PCAP file for analysis. Unless you have some Intrusion Detection System (IDS) or Network Access Control (NAC) system in place, you won't even notice this is going on. As a result, it is often used post exploitation on a PC or server by the bad guys, to just sit there and capture traffic, and retrieve the PCAP file for off-line analysis.

Now, if you are managing the security of the network and are not aware of what's travelling over your networks, then you should be aware! If on the other hand you are doing this, then great, good job :-)! Either way, you should be aware whether anyone can perform ARP poisoning on your network and capture this data gold mine. I'm not going to go over how to ARP poison, just know that it is easy and anyone can do it and you should know if you're vulnerable or not.

So, assuming you have captured network traffic into a PCAP file ready for analysis, I thought I'd collate some of my favourite methods of searching PCAP files for "hidden gems" you may find useful.

Wireshark is probably one of the best tools out there for this job (usually). It gives you a great number of options to analyze captured traffic on the network. One of its biggest failings however is it can't handle large PCAP files....computer says no! It just crashes and hangs. In this case I
always resort to the command line. Tools that can analyze PCAP files via the CLI have several advantages over the Wireshark GUI: firstly, you don't need a GUI so you can run them via a shell and can do it remotely. Secondly, they can handle large files, and thirdly they are generally scriptable. I'm sure there are other pros and cons, but that's enough for now :-)

So, below are my top 10 favorites I tend to use on a daily basis. These should be run to just get an idea of what the traffic is 'like' and to get you more information to delve in further. Note I used different CLI tools for different tasks, either for personal preference or just because it's easier!
There is no doubt different ways to achieve the same task! I would advise also exploring the tools used further to show how powerful they can be.

No 1) List the hosts in the pcap file using wireshark's CLI, tshark

This is useful to generate a quick target list of IP's.

tshark -r file.pcap -q -z hosts,ipv4

e.g.

# Host data gathered from file.pcap

192.168.100.11 so-dc.domain.co.uk

192.168.101.87 name

No 2) Show hierarchy stats using tshark (-z option is for listing stats)

This is useful to get an overview of all the protocols and services running on the network to see what has been captured.

tshark -r file.pcap -q -z io,phs

e.g.

Protocol Hierarchy Statistics

Filter:

eth frames:87837 bytes:34609121

ip frames:84316 bytes:34404858

tcp frames:81502 bytes:34009978

ssh frames:446 bytes:54908

tpkt frames:3083 bytes:361497

http frames:3936 bytes:2713999

data-text-lines frames:574 bytes:309642

tcp.segments frames:121 bytes:76011

json frames:29 bytes:22741

tcp.segments frames:2 bytes:820

ocsp frames:36 bytes:24755

tcp.segments frames:3 bytes:1775

image-gif frames:278 bytes:150491

tcp.segments frames:59 bytes:27149

malformed frames:3 bytes:4302

png frames:42 bytes:28379

tcp.segments frames:27 bytes:17360

media frames:69 bytes:49723

tcp.segments frames:33 bytes:13579

image-jfif frames:39 bytes:37590

tcp.segments frames:18 bytes:14910

tcp.segments frames:36 bytes:16771

malformed frames:21 bytes:1243

xml frames:8 bytes:3362

tcp.segments frames:2 bytes:1430

msnms frames:645 bytes:164923

ftp frames:153 bytes:15210

ftp-data frames:5788 bytes:8277176

ssl frames:2500 bytes:1413771

tcp.segments frames:126 bytes:115216

data frames:132 bytes:40583

nbss frames:130 bytes:24246

smb frames:50 bytes:9766

pipe frames:10 bytes:1768

lanman frames:10 bytes:1768

smb2 frames:66 bytes:13178

tcp.segments frames:2 bytes:570

icmp frames:1528 bytes:212944

udp frames:1286 bytes:181936

nbns frames:552 bytes:51136

db-lsp-disc frames:430 bytes:75588

data frames:85 bytes:12014

nbdgm frames:95 bytes:21185

smb frames:95 bytes:21185

mailslot frames:95 bytes:21185

browser frames:95 bytes:21185

http frames:52 bytes:15140

dns frames:64 bytes:5397

cldap frames:6 bytes:1296

ntp frames:2 bytes:180

arp frames:2052 bytes:109746

arp.duplicate-address-detected frames:650 bytes:29892

arp.duplicate-address-detected frames:452 bytes:18984

arp frames:452 bytes:18984

arp frames:198 bytes:10908

llc frames:736 bytes:46022

stp frames:729 bytes:45200

ipx frames:7 bytes:822

nbipx frames:6 bytes:588

nmpi frames:1 bytes:234

smb frames:1 bytes:234

mailslot frames:1 bytes:234

browser frames:1 bytes:234

ipv6 frames:58 bytes:17103

udp frames:58 bytes:17103

dhcpv6 frames:40 bytes:7200

http frames:18 bytes:9903

lldp frames:25 bytes:1500

3) Show conversations and destination ports using tshark

This is useful to see what ports are open on hosts without scanning, and
see who is talking to it.

tshark -o column.format:'"Source", "%s", "Destination",
"%d", "dstport", "%uD"' -r file.pcap |sort|uniq

e.g.

192.168.101.31 -> 192.168.101.255 138

192.168.101.3 -> 192.168.101.255 137

192.168.101.3 -> 192.168.101.255 138

192.168.101.3 -> 192.168.101.87 3389

192.168.101.3 -> 224.0.0.252 5355

192.168.101.4 -> 192.168.101.255 137

192.168.101.4 -> 192.168.101.255 138

192.168.101.4 -> 224.0.0.252 5355

4) Show top talkers for all packets using tcpdump

This is useful to see who are the noisiest hosts on the network. A noisy
host is a busy host and therefore requires further investigation.

tcpdump -tnr $FILE |awk -F '.' '{print $1"."$2"."$3"."$4}' | sort | uniq -c | sort -n | tail

e.g.

# of Pkts IP Addr

========= =======

333 IP 192.168.101.112

361 IP 192.168.101.7

520 ARP, Reply 192.168.101.200 is-at 00:11:22:33:44:55, length 28

521 IP 192.168.100.2

636 STP 802.1d, Config, Flags [none], bridge-id 8065.00:11:22:33:44:55.8002, length 42

637 STP 802.1d, Config, Flags [none], bridge-id 8000.66:77:88:99:11:22:db.8011, length 43

770 IP 192.168.101.88

2049 ARP, Request who-has 192.168.101.200 tell 192

2588 IP 192.168.101.3

4902 IP 192.168.101.87

5) Search for text strings using ngrep

This is useful to look for any specific string or regex you want e.g look for "password", "card","username" etc....This will normally find FTP, HTTP, or POP passwords as examples. Although this is a simple example, ngrep can be used for complex regex's.

ngrep -q -I file.pcap|grep -i user

e.g

..........< TRUNCATED>..........

en" id="secure_username" name="username" value="" />...<input type="hidden"
id="secure_password" name="password" value="" />...<input type="hidden"
id="secure_key" name="key" value="d8a7

..........< TRUNCATED>..........

6) Find emails using ngrep

As an example of ngrep's regular expression power, you can make up your regex, and then use grep to display it in a nice format. In this example, I'm searching anything that looks like an email address and output the result

ngrep -q -I file.pcap '[a-zA-Z0-9.]+\.?@[a-zA-Z0-9.]+\.[a-zA-Z0-9]+' |grep -Eo '[a-zA-Z0-9.]+\.?@[a-zA-Z0-9.]+\.[a-zA-Z0-9]+'|sort|uniq

NOTE - you will also get false positives using the above, so best to check manually.

e.g.

foo@hotmail.com

name@outlook.com

joe@blogs.co.uk....Rel

..e.E..1H.@.......e

E....z.1..j..F.WHP0@..U...90705.3.1

7) Finding Credit Card Numbers using ccsrch

This tool will look at all files in a directory, including PCAP files. You just need to give it the directory or file name. If you're doing PCI assessments this is a great tool for finding PAN data, credit card numbers and track data on windows and UNIX operating systems.

Again this can have a large number of false positives which will require further investigation. If it is a valid number then you will need to determine what is leaking the data. Its normally a database, when a client makes searches, to the server, but can also be found in emails, web application traffic traversing the network in clear text.

ccsrch /home

e.g.

/home/file.pcap VISA 469xxxxxxxxxx320

/home/file.pcap AMEX 3444xxxxxxxx444

......<TRUNCATED>...................

Files searched -> 15

Search time (seconds) -> 0

Credit card matches-> 12

8) Show all TCP streams using tshark

This is useful to show all the TCP sessions on the network

tshark -r file.pcap -q -z conv,tcp

e.g.

<......TRUNCATED.......>

192.168.101.3:23445 <-> 192.168.101.87:ms-wbt-server 4683 452766 2572 274418 7255
727184 0.070916000 1271.9423

192.168.101.88:42299 <-> 204.13.200.136:ssh 521 59058 769 66850 1290 125908 0.001257000 1271.0156

192.168.101.87:62467 <-> 192.168.100.11:netbios-ssn 12 1579 15 4707 27 6286 1135.718589000 13.7333

<......TRUNCATED.......>

9) Just show specific TCP data using tcpick

This is a great tool to easily look at TCP streams. It can be used as a sniffer or to analyze PCAP files. I tend to use it to look at specific port data to find ascii data from emails, ftp data,www traffic, telnet etc, but it can do much of the queries already discussed. Just another great command line tool.

e.g. to show all ftp payloads

tcpick -r file.pcap -yP "port 21"

(-yP means payload printable)

Timeout for connections is 600

tcpick: reading from file.pcap

setting filter: "port 21"

PORT 192,168,101,6,16,251

PORT 192,168,101,6,16,251

PORT 192,168,101,6,16,251

200 PORT command successful.

200 PORT command successful.

200 PORT command successful.

RETR data01.csv

RETR data01.csv

RETR data01.csv

10) See the connection status using tcpick

Again another simple command to quickly assess who is talking to who (brought to you in pretty Technicolor in the terminal!)

e.g.

tcpick -r file.pcap -C

1 SYN-SENT 192.168.101.87:62478 >192.168.100.11:loc-srv

1 SYN-RECEIVED 192.168.101.87:62478 >192.168.100.11:loc-srv

1 ESTABLISHED 192.168.101.87:62478 >192.168.100.11:loc-srv

2 SYN-SENT 192.168.101.87:62479 >192.168.100.11:49155

2 SYN-RECEIVED 192.168.101.87:62479 >192.168.100.11:49155

2 ESTABLISHED 192.168.101.87:62479 > 192.168.100.11:49155

3 SYN-SENT 192.168.101.87:62480 >192.168.100.11:ldap

3 SYN-RECEIVED 192.168.101.87:62480 >192.168.100.11:ldap

3 ESTABLISHED 192.168.101.87:62480 >192.168.100.11:ldap

4 SYN-SENT 192.168.101.87:62481 > 192.168.100.12:kerberos

4 SYN-RECEIVED 192.168.101.87:62481 >192.168.100.12:kerberos

Obviously the commands above should just be the start of your investigation and analysis of the PCAP file. It will give you an idea of potential issues, and will help direct you to delve deeper into the data as required....and yes there will be other ways of achieving the same results :-)

So, it time honoured fashion, here's a wee script I made earlier just pulling all of the above together....

#!/bin/sh
# Script to take a PCAP network capture file and try to extract useful
# information such as clear text, passwords, users, emails, CC data etc
# Ensure ngrep,tcpick are installed.
# written by D Kirkpatrick

ARGS=1
FILE="$1"
###################################################
if [ $# -ne "$ARGS" ]; then
printf "Usage: `basename $0` file.pcap \n "
echo "Searches a pcap file for useful information"
exit 0
fi
#########################################
# Search Criteria
########################################
# show hosts of IP traffic
echo "Show hosts"
echo "=========="
tshark -r $FILE -q -z hosts,ipv4
# Show Hierarcy stats using tshark (-z option is for listing stats)
echo "Show hierarchy"
echo "=============="
tshark -r $FILE -q -z io,phs
# Show all traffic and dest port info
echo "Show all sessions and destination ports"
echo "======================================="
tshark -o column.format:'"Source", "%s", "Destination", "%d", "dstport", "%uD"' -r $FILE |sort|uniq
# List the top talkers, and sort by number of bytes sent by each
echo "Top Talkers For All Traffic"
echo "============================="
echo
echo -e '# of Pkts \t IP Addr'
echo -e '========= \t ======='
tcpdump -tnr $FILE |awk -F '.' '{print $1"."$2"."$3"."$4}' | sort | uniq -c | sort -n | tail
# Show all conversations and port detail using tshark filters"
echo "Show all conversation"
echo "====================="
 tshark -o column.format:'"Source", "%s", "Destination", "%d", "dstport", "%uD"' -r $FILE |sort|uniq
# Show all TCP streams
echo "Show TCP Streams"
echo "================"
tshark -r $FILE -q -z conv,tcp
# Search for any word with user
echo "Show any packes with "User" in it"
echo "================================="
ngrep -q -I $FILE|grep -i user
# Search for any word with pass
echo "Show any words with "Pass" in it"
echo "================================"
ngrep -q -I $FILE|grep -i pass
# Search for emails
echo "Find anything looking like an email address"
echo "==========================================="
ngrep -q -I $FILE '[a-zA-Z0-9.]+\.?@[a-zA-Z0-9.]+\.[a-zA-Z0-9]+' |grep -Eo '[a-zA-Z0-9.]+\.?@[a-zA-Z0-9.]+\.[a-zA-Z0-9]+'|sort|uniq
# Look at TCP data in printable form
echo "Show Port 25 data"
echo "================="
tcpick -r $FILE -yP "port 25"
echo "Show Port 21 data"
echo "================="
tcpick -r $FILE -yP "port 21"
echo "Show Port 23 data"
echo "================="
tcpick -r $FILE -yP "port 23"
echo "Show Port 80 data"
echo "================="
tcpick -r $FILE -yP "port 80"
echo "Show all TCP flows"
echo "=================="
tcpick -r $FILE -C