Pentesting like an Eastern European

Through SpiderLabs' Incident Response and Penetration Testing services we get a chance to both examine 'bad actor' techniques in the field and help our clients see how their security controls will stand up to them.

One trend we've seen is a move away from malicious parties stealing 'data at rest' to targeting it as flows through IT infrastructure. Increasingly sensitive data will be well protected when stored; so network sniffers and memory dumpers are being used to pilfer information en route.

On the other side of the fence, pentesting is designed to give a 'real world' security test of these same sensitive assets. So in light of the trend I thought I'd elaborate on some of the tools and techniques we use to target dynamic data during pentests. I'll first cover the general background and then dive into the specific tools and techniques.

Network Sniffing

Network sniffing is probably the best understood but most poorly addressed attack scenario facing corporate networks. From a pentest point of view this makes network sniffing 'less interesting' but our experience is that the vast majority of corporate networks are vulnerable to some form of it.

ARP and NetBIOS spoofing are perennial favourites with pentesters and attackers alike. NetBIOS is limited to Windows-based environments, even one's running Windows 7 and 2008 exclusively, but almost every network will suffer from ARP Spoofing 'out of the box'.

The best way to address network sniffing is through a combination of strong network controls and appropriate application-layer encryption. Systems of differing security levels should be segregated and protected with firewalls. NAC and other network-layer controls should be used to control who can access your network and what they can do once they're on it. And if you're running Windows it wouldn't hurt to disable NetBIOS altogether.

Memory Dumping

Unlike network sniffing, the ability to steal sensitive data from RAM is an often overlooked weakness, both by administrators and pentesters. The terabytes of corporate data sitting on file shares and in databases presents a tempting target for attackers, but the comparatively small size and 'working memory' role of RAM can make it a quick win.

Some examples of information we find in memory during a pentest include passwords and password hashes for users, private keys for remote access and encryption software, and even sensitive corporate information that has been recently processed.

The value of searching memory was illustrated during a recent penetration test we conducted for a payment card processor. In this case cards were being encrypted end-to-end from customer's browser to the merchant bank using a combination of SSL encryption and a Hardware Security Module (HSM). This protected against any type of network sniffing attack.

However payment data on its journey from the customer to the bank passed through a 'middleware' server that packaged it up in format supported by the HSM. During the pentest we were able to gain access to this server which was found to be running Centos Linux.

Despite credit cards never been written to disk, or otherwise stored, we were able to quickly steal credit cards directly from the server's live memory (PANs censored to protect the innocent):

modprobe crash
dd if=/dev/crash of=/opt/logs/crypto1.mem.dump

strings crypto1.mem.dump | grep -Ei '[3-6][0-9]{14,15}|creditcard|pan|cvv' | sort | uniq
...
<PAN>376012xxxxxxxxxx</PAN><CVV>907</CVV><ExpiryDate><Month>09</Month><Year>12</Year></ExpiryDate>
<PAN>521729xxxxxxxxxx</PAN><CVV>422</CVV><ExpiryDate><Month>04</Month><Year>15</Year></ExpiryDate>
<PAN>531358xxxxxxxxxx</PAN><CVV>953</CVV><ExpiryDate><Month>05</Month><Year>13</Year></ExpiryDate>
<PAN>546827xxxxxxxxxx</PAN><CVV>816</CVV><ExpiryDate><Month>02</Month><Year>14</Year></ExpiryDate>
...

This was possible despite the client having gone to great lengths to become PCI Compliant.

Unfortunately, there are no silver-bullets that will protect your infrastructure against memory dumping. It comes down to having strong host-level security; making sure that an attacker has no chance to compromise sensitive servers. This is an uphill battle unless you already have in place a programme of strong network segregation, configuration hardening and a comprehensive patching.

Memory dumping techniques

So I've covered the background on how 'bad actors' can target dynamic data when stealing sensitive information. I'd now like to elaborate on some of the tools and techniques we use to when targeting this information during penetration tests.

Some tools will do examination of memory for you, but grabbing a full copy of running memory is generally the preferred option. The 4GB to 16GB size of a typical memory dump is much easier to search than the hundreds of gigabytes found on disk drives. In almost all cases you will require high-level administrator privileges to dump memory. How that is obtained is left as an exercise for the reader.

How you go about grabbing running memory will greatly depend on the target operating system. Even then there are numerous different ways for each OS. These are the techniques that have worked best for me.

Windows

Windows is probably the most straight forward of all the OSs when it comes to dumping a full memory image, largely thanks to Matthieu Suiche and his DumpIt tool. DumpIt will give you a full memory dump with a simple double click. It even checks to make sure you have enough free space for the dump.

Linux

If you want to grab a memory dump from a server running a Linux 2.4 kernel (or earlier) then you're in luck! The kernel handily provides access via the /dev/mem (physical) and /dev/kmem (virtual memory) device drivers. Obtaining an image is as easy as:

dd if=/dev/mem of=/tmp/memdump.raw bs=1024

Be-careful to make sure you have enough space for your dump! (Use cat /proc/meminfo and df -a commands.)

However since the Linux 2.6 kernels, they have clamped down on direct access to memory (for the obvious security reasons). The best way to grab memory is now through the 'crash' kernel modules that replicate the old /dev/mem device functionality.

Redhat (and hence Centos as well) is nice enough to prepackage the crash module for you. So once again it is as simple as:

modprobe crash
dd if=/dev/crash of=/tmp/memdump.raw bs=1024

On other distributions things are little more difficult. You will probably need to compile your own crash module. 'JL' has been kind enough to port the Redhat crash driver to Ubuntu if that is the system you're targeting. Compiling it will require having the appropriate kernel header sources however.

OSX

There are a number of options on a Mac, but the MacMemoryReader is the easiest in my experience. Running it is as simple as:

sudo ./MacMemoryReader -p memdump.raw

Targeted memory dump tools and utilities

So once you've got a full memory dump, pulling out passwords and other sensitive information might be as easy as running a few grep commands. However there are a numbered of specialised tools that target specific types of data in memory.

Windows password hashes are probably the piece of data we most often pilfer from memory. There are numerous tools that will do this but most involve code injection into Windows Kernel. On the other hand the Windows Credentials Editor tool uses a memory reading approach. No need for a full dump, it will read memory for you.

More interesting than WCE is Nicolas Collignon and Jean-Baptiste Aviat's Passe-partout tool ("Master Key" in English). Written for Unix-type OSs (including OSX) it uses a generic technique for identifying RSA and DSA encryption keys. It is particularly handy for pulling 'cached' SSH keys from memory if ssh-agent is running, but can also be used against Apache (to snarf private SSL certificates) and OpenVPN (to steal VPN authentication keys).

The AESKeyfinder tool developed as part of the Princeton research into Cold Boot attacks is similar to Pass-partout but slightly less user friendly. Using that AESKeyfinder, it is possible to recover the raw AES keys for any TrueCrypt or FileVault data that may be 'open'. Attacking FileVault is even easier than that however, as the user password string can be found in your full memory dump.

Obtaining the FileVault password is often of limited use if you already have memory dumping ability on a machine, however we see password reuse enough to make it worth snatching and trying against other systems or accounts.

At this point I should mention the great Volatility Memory Forensics Framework that my colleagues in Incident Response use regularly. It is suite of analysis tools for examining a memory dump,, allowing you to pull out the processes that were running, 'netstat' like information on network connections, open files, etc. Its support for Windows is the most comprehensive, but there are Linux and OS X branches. In a pentest scenario you would expect to have access to most of this information 'live' so it is more handy for forensic investigations.

So they are some of the tools and techniques we use to target memory during a pentest. The amount of sensitive data that can be found in RAM and its comparatively small size lends itself to 'smash and grabs'. This is not something that has escaped the notice of bad guys either, so it is worth considering (or investigating yourself!) exactly what is sitting in the memory of your critical systems, and how it is currently being protected.