Recently, security researchers demonstrated how the use of the trap bit in x86 processors could inform running malware if it is running in a virtual environment or not. What is the purpose of the trap bit, how can it benefit malware, and what does this mean for future CPU hardware and virtualisation?

What is the trap bit?

In the x86 CPU architecture, the trap bit is a special flag in the EFLAG register that raises an interrupt after completing a single instruction once the flag is set. For example, a piece of machine code would first set the trap flag, execute an instruction, and this would then trigger the CPU to execute a special interrupt that runs a subroutine.

While there is no specific purpose for the trap flag, it is convenient for debugging as it allows for code to be executed step-by-step. Furthermore, the interrupt allows for viewing the CPU contents, including registers, program counter, and stack pointer.

How the trap bit can be abused by malware

Recently, researchers from Palo Alto Networks demonstrated how the trap bit in x86 processors could be abused by malware to determine if the malware is being executed on a real computer or in a virtual machine. The cause of the exploit lies in how virtual machines emulate the behaviour of the trap flag. If a piece of malware sets the trap flag after executing certain special instructions such as RDTSC and CPUID, the CPU should return to the malware code with the trap bit cleared. While this is the case in real hardware, virtual machines may not catch this and return to the code with the trap bit still set.

The calling of a special instruction sees the CPU handle the interrupt, but if the trap bit is enabled on a basic instruction such as NOP, the malware can use its interrupt handler to detect this. Thus, if the malware interrupt handler is fired with the trap bit set, it knows it is running on a virtual machine. If no exception is thrown, then the malware knows that it is running on a real system.

Why is virtualisation detection an issue for fighting against malware?

One practical use of virtualisation is to test unknown code and applications to see how they behave. For example, an individual could find a USB flash drive lying around with no idea what is inside. While one could risk infection by simply plugging it into a computer and viewing the contents, it is better to use a virtual machine to view the contents. If malware infects the virtual machine, no damage is done, and the system can be reverted to its original state.

The use of the trap bit to detect virtualisation allows for malware to determine if it should execute or not. If virtualisation is detected, malware can disguise itself to look harmless and/or inactive (for example, provide false outputs to the user to appear safe). Once the user determines that the malware is safe, they would then insert the USB drive into a real system, whereby the malware could cause havoc.

While this vulnerability may not affect all virtualisation systems, as pointed out by Palo Alto Networks, it does raise concern in the security sector, where a virtualisation is a crucial tool for malware detection. A crude method for fighting against trap bit detection is using a real computer system isolated from networks and external drives. As the malware would be running on a real system, its true nature would show, and it is tough to determine if a computer is just a test bench or a computer integrated into a more comprehensive network.