Dynamic Malware Analysis
Dynamic analysis means to observe one or more bevaviors of a software artefact to analyse its properties by executing the software itself. We have already argued above that dynamic analysis is preferable to static (code) analysis when it comes to malware. There exist two different approaches to dynamic malware analysis with different result granularity and quality:
- taking an image of the complete system state before and comparing this to the complete system state right after the malware execution
- monitoring all actions of the malware application during its execution, e.g., with the help of a debugger or a specialized tool
It is evident that the first option is easier to implement, but delivers more coarse-grained results, which sometimes are sufficient, though. This approach can only analyze the cumulative effects and does not take dynamic changes into account. If for example a file is generated during the malware's execution and this file is deleted before the malware terminates, the first approach will not be able to observe this behavior. The second approach is harder to implement, but delivers much more detailed results, so we chose to use this approach in the CWSandbox.
API Hooking
The Windows API is a programmer's interface which can be used to access the Windows resources, e.g., files, processes, network, registry and all other major parts of Windows. User applications use the API instead of making direct system calls and thus this offers a possibility for behavior analysis: we get a dynamic analysis if we monitor all relevant API calls and their parameters. The API itself consists of several DLL files that are contained in the Windows System Directory. Some of the most important files are kernel32.dll, advapi32.dll, ws2_32.dll, and user32.dll. Nearly all API functions do not call the system directly, but are only wrappers to the so called Native API which is implemented in the file ntdll.dll. With the Native API, Microsoft introduces an additional API layer. By that Microsoft increases the portability of Windows applications: the implementation of native API functions often changes from one Windows version to another, but the implementation and the interface of the regular Windows API functions nearly never changes.
The Native API is not the end of the execution chain which is performed when an API function is executed. Like in other operating systems, the running process has to switch from usermode (Ring 3) to kernelmode (Ring 0) in order to perform operations on the system resources. This is mostly done in the ntdll.dll, although some Windows API functions switch to kernelmode by themselves. The transfer to kernelmode is performed by initiating a software interrupt, Windows uses int 0x2e for that purpose, or by using processor specific commands, i.e., sysenter for Intel processors or syscall for AMD processors. Control is then transfered to ntoskrnl.exe which is the core of the Windows operating system.
In order to observe the control flow from a given malware sample, we need to somehow get access to these different API function. A possible way to achieve this is hooking. Hooking of a function means the interception of any call to it. When a hooked function should be executed, control is delegated to a different location, where customized code resides: the hook or hook function. The hook can then perform its own operations and later transfer control back to the original API function or prevent its execution completely. If hooking is done properly, it is hard for the calling application to detect that the API function was hooked and that the hook function was called instead of the original one. However, the malware application could try to detect the hooking function and thus we need to carefully implement it and try to hide as good as possible the analysis environment from the malware process.