6. State of the Art Malware
1. The Rootkit Technology
Rootkits are among the biggest challenges faced by the security researchers and providers today. They can be difficult to detect, especially when they operate in the kernel. This is because a kernel rootkit can alter functions used by all software, including those needed by security software. A rootkit can prevent detection or anti-malware software from running or working properly by hiding files and concealing running processes from the computer’s operating system. The most recent development in the field of rootkits is Virtual Machine Based Rootkit (VMBR).This type of rootkit forces the user to use an operating system that executes within a virtual machine. The advantages to the potential attacker are obvious; the user would be oblivious to any malicious programs executing outside the virtual machine. Thus any security software running on the target virtual OS would be totally incapable of detecting any suspicious activity, being fully under the control of the VMBR.
2. Dormant Malware
This class of malware plunges into action only when a specific environmental or temporal condition is met. Until then, the malware behaves as any other legitimate program. Common examples are bot programs that wait for external input from their command and control servers, or malware programs that execute their malicious payload only before (or after) a certain date.
Dynamic analysis techniques suffer from the problem of limited coverage which means that it is unlikely for them to reveal the entire range of capabilities of a given binary. The reason is that the analysis can only observe behaviours for which the corresponding code is actually executed.
Hence if the dynamic analysis techniques explore multiple execution paths to be able to detect dormant malware, they have to deal with the path explosion problem. Path explosion occurs because, for each interesting branch in the program, the analysis has to follow two successor paths. This leads to an exponential growth in the overall number of paths that need to be explored. Various heuristics are used to first select more promising continuations. However, these heuristics rarely achieve full code coverage. Thus, even though multi-path analysis can increase the number of behaviours that are observed during a dynamic analysis run, it is unlikely that the entire code is executed. Moreover, multipath analysis is costly, which is a significant limitation when considering the tens of thousands of samples that the anti-malware vendors need to analyze daily.
3. Split Personality / Analysis Aware Malware
To thwart automated screening, malware authors have developed a number of ways to check for the presence of malware analysis tools and popular sandbox environments. When the malware detects presence of a malware analysis system, it typically suppresses the execution of malicious functionality or simply terminates.
The anti-malware products employ one or a combination of methods such as signature-based detection, static analysis, dynamic detection, behavioural monitoring and sandbox technology. State of the art malware is characterised by a broad spectrum of obfuscation techniques that result in the evasion of most of these anti-malware methods. The advanced malware retains the same core malicious functionality and remote control mechanisms. The following are some of the obfuscation mechanisms used in the wild:
Crypters (or “Cryptors”) encrypt malware so that signature detection systems and static analysis processes are ineffectual. Crypters typically encrypt the contents of the malware executable, and then only decrypt sections of code that are in the process of being executed on the victim’s computer.
As a result, the host-based detection technologies cannot inspect the executable content prior to it being loaded from disk and into memory. Also, the popular reverse engineering tools such as IDA Pro will struggle with their analysis without full knowledge of the decryption algorithm in order to work properly. Similarly, signature-based antivirus products work by sifting the executable code of the malware for atypical coding markers, and then search for pre-defined regular expressions. If antivirus detects known malicious strings, it will delete or quarantine the suspicious file. Crypters prevent antivirus scanners from seeing these key coding markers.
This technology was originally designed for commercial use to protect, for example, Windows applications against modern cracking tools by putting those applications into a strong protection “shell”, online games from abuse (e.g. reverse engineering that led to game stats manipulation, etc.) and as a DRM protection technology. Ironically it is now more commonly employed by cyber-criminals and is a relatively new class of AV-evasion technique. Protectors automatically add specific anti-debugging features to malware that prevent security researchers and automated sandbox analysis technologies from dissecting samples.
Packers are software programs that compress and encrypt other executable files in a disk and restore the original executable images when the packed files are loaded into memories so that they take less space to store and less time to transfer over slow channels. However, the malware authors also found them useful to conceal their parasites.
Packers offer a safe haven for malware authors by disguising their malicious code and data. In some cases, packers provide them with different looking binaries each time they repack their code. A technique commonly used against checksumming.
A more recent advancement has been that of polymorphic packers, in which the malware binary is structurally different every time the packed version is executed. The most popular packer technology in operation today is UPX, both among legitimate application vendors and malware authors.
Binders are an old technology typically used by malware authors to “embed” and Trojan other software packages. These tools are a method for aiding propagation of the malware component, tricking victims into executing a popular file or something that looks legitimate.
Binders have proved popular with botnet masters that use Torrent networks and newsgroups to spread their malware. The malicious code is embedded inside files that are frequently searched for and downloaded. Binders are also used to create the packages that malware downloaders automatically install on victims of drive-by attacks – thereby increasing the breadth of potential victims and the probability of successful compromise.
PolyPack is a web-based service that employs an array of packers and antivirus engines to pack an input binary with the packer that results in the maximal evasion of detection by the antivirus engines. A typical usage of the PolyPack service is as follows:
1) A user submits an unpacked binary via PolyPack’s web interface,
2) The binary is packed using an array of packers and each packed version is analyzed by an array of antivirus engines, and
3) The detection results from the antivirus engines are analyzed to select the optimally packed version to provide the most AV evasion and the results are returned to the user.
Although, PolyPack is targeted at providing a useful service for penetration testers who require the ability to create payloads that will evade the signature detection of a number of anti-virus engines, it is an interesting concept with respect to the crimeware industry for the obvious obfuscatory gain.
5. Increase in Botnet Control Sophistication
Increasingly, attackers are using the HTTP and HTTPS web protocols as the communication method between bots and the C&C server. This means that it is more difficult for network operators, firewalls and intrusion prevention/detection systems to detect and block bot communications to or from their network as it is hidden among the vast volume of normal web traffic.
Innovative C&C models are designed to make it more difficult for security practitioners to stop botnet hosted attacks using the increasing use of the peer to peer (P2P) model. The peer to peer model lacks a central hierarchy of communication which makes the botnet more resilient to dismantling. It is therefore extremely difficult to stop attacks launched from botnets that communicate using P2P as there is no single point of failure. When one C&C is taken down, another appears on the network within minutes.
In addition to the models above, botnets are increasingly using what is known as “fast flux” networks to evade detection. Fast flux networks are networks of compromised computer systems with public DNS records that change constantly thus making it more difficult to track and shut down malicious activity. Furthermore, this model abandons the traditional centralised C&C server and uses proxies to hide the servers controlling the fast flux network. 
 Gaith Taha, Counterattacking Packers, McAfee Avert Labs, Aylesbury, UK, 2007.
 Gunter Ollmann, Serial Variant Evasion Tactics, Damballa,2009
 Ralf Benzmüller & Sabrina Berkenkopf, GData Malware Report, Half year report Jan-June 2010.
 OECD, Malicious Software, A Security Threat to Internet Economy, 2008.