Building my own personal password cracking box
Since 2003, I've spent a majority of my workdays hacking systems. I've collected tons of penetration testing tips and tricks and have shared some of them on this blog. As a part of my work as a penetration tester, cracking password hashes is something I need to do regularly.
So, I decided I needed to build a personal cracking box to speed things up. Some co-workers have them, and it's a pain to ask to use their rigs every time. I've now constructed my own and share it with my LAC (Latin America and Caribbean) colleagues. Below I'll explain how I built my cracking box in case it might help you in some of your projects. In a future post, I will elaborate on customizations and improvements, keep watching the SpiderLabs blog for more details! :)
First thing, I love John the Ripper, but Hashcat is a monster when breaking passwords with GPU cards. I really enjoy the good ol' John the Ripper, however, it is much slower. Last time I checked, it used CPU to generate candidate passwords for GPU.
Something else I needed to consider was my living in a hot--really hot--climate. I live in Ribeirão Preto - Brazil. Currently, the temperature hovers around 36 degrees Celsius (~97 degrees Fahrenheit). While using my cracking box, I really need to keep my air conditioning system turned on. I had two options, go with liquid cooling or good ol' airflow systems. It's a personal decision, but I prefer airflow in order to save a few bucks. I also have to admit that I worried that a liquid leak that could destroy my system - yes, I know that the liquid cooling lovers will hate me for that. :)
My system is for sure not the best configuration, but it was what I could make work with my budget. The way I crack passwords is using some scripts that mix CPU and GPU. I generate a custom password file that is really huge (500GB+) per session, so I need a fast storage. Depending on how you crack passwords, you may not need it.
I want to thank friends that helped me buy the parts in the USA to build my box in Brazil. Thanks to Garret Picchioni (co-worker), Luiz Eduardo (former boss), Rodrigo Rubira Branco (friend) and Gabriel Negreira Barbosa (friend). You rock, guys!
Now I'll briefly explain my configuration and why I selected it.
But first, don't forget your anti-static wrist strap!
- CoolerMaster CM690 III
This chassis has a fair price, decent air flow and simple cable management, curved metal mesh, HDD/SSD combo cage that can be switched between 3.5-inch or 2.5-inch drives, and dust filters at the bottom, top, and front. It was a good choice. - EVGA x79 SLI
This motherboard is one of the most simple. It is an average motherboard and is limited in its features and slots. I chose it because a friend had used it for only two months and sold it to me for a very good price. :) - i7 4930K (CPU)
As I said, I also use CPU cracking. This processor is an Ivy Bridge with LGA 2011 socket type with 6 cores and total of 12 threads running from 3.4 GHz up to 3.9 GHz (max turbo frequency). This is a bit expensive, but a good processor, the overclocking is not incredible, but acceptable. - Cooler Noctua NH-D14 (CPU)
The processor above gets really hot, so I wanted a great CPU cooler. I chose Noctua. The price is high, but it's really good and their support is awesome. As a bonus, they sent me an additional mounting-kit for free (including international shipping)!
- SeaSonic X-SERIES X-1050 1050W
Most people really don't care about their Power Supply Unit (PSU), but it's really important and benefits the whole build. This PSU from SeaSonic provides great performance, contains quality components, is very stable and hybrid and 80 Plus Gold certified. It also doesn't get hot while working on heavy load. A great PSU is essential since I'm working with GPUs that require a lot of power.
- Samsung SSD 840 EVO-Series 1TB 2.5-Inch SATA III
As I mentioned before, I generate really big custom wordlists for every cracking process, so I wanted a really fast storage device to read and write all the data from my wordlist. This 840 EVO is really good, faster than I expected. It's expensive, but it's worth it.
- 2x Radeon R9 290X (GPU)
The R9 290X is a great GPU that cracks passwords quickly like a beast. However, they get very hot and require a lot of energy. I acquired two of them, and they are great. But they also affected my energy bill. And a decent cooling system is required to keep them operating at peak performance.
- Memory 32 GB (4 x 8GB - speed 1600 MHz)
Memory is always useful and hopefully decreases in price with time. Again, my motherboard is nothing special, but it did allow me to hook up 32GB supporting dual channel DDR3 running at 1600 MHz at the moment (the specifications says it supports 2133 MHz).
With my cracking box built, it was time to check whether it worked and test its memory.
Next, I needed to configure Linux. I chose Ubuntu 14.04.1, and in sequence I configured the GPUs and basic tuning. All was good and and operation. But what about performance, right?
First a brief test to see how fast it could calculate the MD5 of a 52GB wordlist.
wendel@WS-CrackingBox:~$ ls -lah WS-CWG/dict-INSANE2.txt
-rw-rw-r-- 1 wendel wendel 52G May 16 00:09 WS-CWG/dict-INSANE2.txt
wendel@WS-CrackingBox:~/WS-CWG$ time wc -l dict-INSANE2.txt
4520197856 dict-INSANE2.txt
real 1m35.257s
user 0m31.782s
sys 0m28.394s
wendel@WS-CrackingBox:~/WS-CWG$ time md5sum dict-INSANE2.txt
fd6bc6c2d74976629d42fae72eba0042 dict-INSANE2.txt
real 1m57.313s
user 1m25.245s
sys 0m19.373s
This is really fast, but in general we need a parameter for comparison. So, here's a comparison.
I have a Mac OS X with Intel i7 2.6GHz, 8GB of Ram and a 750 GB HD - it's the default configuration for this model (15-inch, Mid 2012). Here's the results of this Mac calculating the MD5 of another file.
MD5 (WS-Cracking.tgz) = 030d5f3cc805814b3bef26d2816dfac3
real 5m21.272s
user 0m50.030s
sys 0m7.245s
And now on the cracking box.
030d5f3cc805814b3bef26d2816dfac3 WS-Cracking.tgz
real 0m36.097s
user 0m33.062s
sys 0m3.021s
As you can see, the processor is much faster and helped a lot, but the SSD performance is fantastic. CPU performance is not bad as demonstrated below with the John the Ripper benchmark feature - I selected a few common hash formats to make it smaller.
Benchmarking: descrypt, traditional crypt(3) [DES 128/128 AVX-16]... (12xOMP) DONE
Many salts: 34357K c/s real, 2865K c/s virtual
Only one salt: 26001K c/s real, 2166K c/s virtual
Benchmarking: md5crypt, crypt(3) $1$ [MD5 128/128 AVX 12x]... (12xOMP) DONE
Raw: 293760 c/s real, 24480 c/s virtual
Benchmarking: LM [DES 128/128 AVX-16]... (12xOMP) DONE
Raw: 101597K c/s real, 8466K c/s virtual
Benchmarking: dominosec, Lotus Notes/Domino 6 More Secure Internet Password [8/64]... (12xOMP) DONE
Many salts: 3280K c/s real, 273408 c/s virtual
Only one salt: 1686K c/s real, 140310 c/s virtual
Benchmarking: mysql, MySQL pre-4.1 [32/64]... (12xOMP) DONE
Raw: 109226K c/s real, 26040K c/s virtual
Benchmarking: netlm, LM C/R [DES 32/64]... (12xOMP) DONE
Many salts: 23003K c/s real, 1966K c/s virtual
Only one salt: 2011K c/s real, 853620 c/s virtual
Benchmarking: netlmv2, LMv2 C/R [MD4 HMAC-MD5 32/64]... (12xOMP) DONE
Many salts: 9400K c/s real, 783360 c/s virtual
Only one salt: 7004K c/s real, 583194 c/s virtual
Benchmarking: netntlm, NTLMv1 C/R [MD4 DES (ESS MD5) 128/128 AVX 12x]... DONE
Many salts: 10870M c/s real, 10979M c/s virtual
Only one salt: 47536K c/s real, 47536K c/s virtual
Benchmarking: netntlmv2, NTLMv2 C/R [MD4 HMAC-MD5 32/64]... (12xOMP) DONE
Many salts: 8810K c/s real, 731768 c/s virtual
Only one salt: 6679K c/s real, 560774 c/s virtual
The temperature of CPU was good during my tests, showing that Noctua cooler works well.
Physical id 0: +59.0°C (high = +85.0°C, crit = +95.0°C)
Core 0: +50.0°C (high = +85.0°C, crit = +95.0°C)
Core 1: +59.0°C (high = +85.0°C, crit = +95.0°C)
Core 2: +53.0°C (high = +85.0°C, crit = +95.0°C)
Core 3: +54.0°C (high = +85.0°C, crit = +95.0°C)
Core 4: +54.0°C (high = +85.0°C, crit = +95.0°C)
Core 5: +54.0°C (high = +85.0°C, crit = +95.0°C)
And here are the benchmarks with my GPU:
wendel@WS-CrackingBox:~$ ./oclHashcat64.bin -b
Hashtype: MD4
Workload: 1024 loops, 256 accel
Speed.GPU.#1.: 22880.1 MH/s
Speed.GPU.#2.: 22880.5 MH/s
Speed.GPU.#*.: 45760.6 MH/s
Hashtype: MD5
Workload: 1024 loops, 256 accel
Speed.GPU.#1.: 11717.0 MH/s
Speed.GPU.#2.: 11718.5 MH/s
Speed.GPU.#*.: 23435.4 MH/s
Hashtype: SHA1
Workload: 1024 loops, 256 accel
Speed.GPU.#1.: 3691.0 MH/s
Speed.GPU.#2.: 3691.0 MH/s
Speed.GPU.#*.: 7382.0 MH/s
Hashtype: SHA256
Workload: 512 loops, 256 accel
Speed.GPU.#1.: 1540.8 MH/s
Speed.GPU.#2.: 1540.9 MH/s
Speed.GPU.#*.: 3081.7 MH/s
Hashtype: SHA512
Workload: 256 loops, 256 accel
Speed.GPU.#1.: 244.6 MH/s
Speed.GPU.#2.: 244.6 MH/s
Speed.GPU.#*.: 489.2 MH/s
Below are benchmarks as a result of me overclocking the GPUs. Basically I increased the core speed to 1050 and allowed it to use more power (electricity), if required. Let's see the difference:
Hashtype: MD4
Workload: 1024 loops, 256 accel
Speed.GPU.#1.: 23993.0 MH/s
Speed.GPU.#2.: 23994.0 MH/s
Speed.GPU.#*.: 47987.0 MH/s
Hashtype: MD5
Workload: 1024 loops, 256 accel
Speed.GPU.#1.: 12314.8 MH/s
Speed.GPU.#2.: 12314.7 MH/s
Speed.GPU.#*.: 24629.5 MH/s
Hashtype: SHA1
Workload: 1024 loops, 256 accel
Speed.GPU.#1.: 3874.9 MH/s
Speed.GPU.#2.: 3874.9 MH/s
Speed.GPU.#*.: 7749.8 MH/s
Hashtype: SHA256
Workload: 512 loops, 256 accel
Speed.GPU.#1.: 1617.6 MH/s
Speed.GPU.#2.: 1617.6 MH/s
Speed.GPU.#*.: 3235.1 MH/s
Hashtype: SHA512
Workload: 256 loops, 256 accel
Speed.GPU.#1.: 256.9 MH/s
Speed.GPU.#2.: 256.8 MH/s
Speed.GPU.#*.: 513.7 MH/s
Fan coolers were configured to 95% on both cases. The difference is not huge, but you can see it. The temperature also was fine. Even during days working non-stop, I never reached the maximum temperature. The closest was 3 degrees Celsius, but it was infrequent. Pay attention to temperature--too hot and your performance will drastically decrease. You need to keep the environment temperature cold. Remember that the fans are there to bring in cold air from outside the chassis and pussh out hot air from inside of chassis.
I did a simple test, I used a file with a few MD5 hashes and I tested all of them against the dictionary file mentioned above with 52GB of size. I didn't use any rules or permutations.
With John the Ripper (CPU) it took 15.5 hours to finish.
With John the Ripper (GPU) it took 2.10 hours to finish.
With Hashcat (GPU) it took 1.26 hours to finish.
Thanks for sticking with me. I hope my experience might have taught you something you can apply to your own cracking box projects. In a future post I will share more details about tuning, overclocking and more. Stay tuned.