Windows Vista Kernel Patch Protection (a.k.a. PatchGuard)

As the sand sifted through the hourglass on the countdown to the release of Windows Vista, a couple of enterprise security vendors, namely Symantec and McAfee, became quite vocal about their opposition to PatchGuard, the Microsoft security measure designed to protect the Windows kernel. In a nutshell, by locking down the kernel to protect it from bad guys, Microsoft also locked down the kernel so that security vendors that relied on some form of kernel-patching to get the job done are now forced to explore alternative methods of providing that same security.

PatchGuard is only present on the 64-bit version of Vista. In terms of market share, 64-bit products are a growing segment, but still pale in comparison with good old 32-bit software. As small a target as this segment represents, the fact that Microsoft intended to prevent security vendors from accessing or modifying the kernel remained a source of contention. After some chest-thumping in the headlines and some pressure from the EU, Microsoft did agree to work with the third-party security vendors to develop an API or otherwise help them to work with or get through the PatchGuard protection.

What is the big deal, anyway? Why does Microsoft feel that kernel patch protection is so important, and why do some software vendors consider it such a violation of their free-market ability to produce and sell security applications? How, exactly, is this PatchGuard helping to improve security in Windows Vista? We’ll get to that, but first let’s cover a brief overview of what the kernel is.

The Heart and Soul of the Operating System

To understand how PatchGuard helps to secure Windows Vista or why application vendors are frustrated by it, you first need to understand what the kernel is. The kernel is the fundamental core of the operating system. Just as the motherboard BIOS provides the base instruction set that allows the computer to function at all, the operating system kernel provides the base instruction set that allows Windows to boot up in the first place and on top of which all other applications, including the Windows operating system itself, run.

By hooking into, or modifying the code of the kernel, viruses and other malware such as rootkits are able to attack and operate in a stealth mode. By compromising the kernel, a malware attack functions at the core level of the operating system and cannot be seen or detected by any normal means.

In order to monitor and protect the kernel, many anti-malware products actually modify, or patch, the kernel themselves. These products are obviously not attacking the kernel, but Microsoft does not support any product modifying the kernel. These attempts at kernel-patching are often the source of the dreaded BSoD (Blue Screen of Death).

How Does PatchGuard Help?

Microsoft set out to make Windows Vista the most secure version of the operating system to date. In so doing, it looked at the existing threat landscape and determined that many of the more insidious threats to Windows security involved some form of kernel-patching to compromise the core functionality of Windows and fly ”under the radar,” so to speak.

To prevent such activity, Microsoft developed Kernel Patch Protection, commonly referred to as PatchGuard. PatchGuard works in much the same way that anti-malware products described earlier do. It monitors kernel activity and identifies attempts to patch or alter the kernel code. Specifically, PatchGuard monitors for attempts to patch the kernel, modify the interrupt descriptor table (IDT), use kernel stacks not allocated by the kernel, modify the system services tables or modify the global descriptor table (GDT).

If an attempt to patch the kernel is detected, PatchGuard will shut down the computer system. Unfortunately for software developers, Microsoft does not make any distinction between benign applications or malware attacks. All attempts to modify the kernel are detected and blocked.

Kernel-Patching Alternatives for Applications With Issues

If you install a 64-bit application which relies on kernel-patching to function on a 64-bit Windows Vista system, the result will be repeated shutdowns each time the application starts up and attempts to modify the system kernel. This creates a sort of denial-of-service loop on the system and renders it fairly useless.

What are software developers to do, then? How can they continue to perform the tasks they have in the past without being able to access the kernel directly? Microsoft has not simply left the application developer community out in the cold.

Rather than relying on kernel patching, an application can use the Windows Filtering Platform (WFP). The WFP allows an application to analyze and change TCP/IP packets before allowing them to be processed any further.

To monitor file system activity and detect attempted malware attacks or suspicious actions, antivirus or other security software can rely on Vista’s File System Mini Filter model. Additionally, Microsoft has implemented a method to allow developers to access and monitor activity in the Registry by using Registry notification hooks.

More Secure or Just More Frustrating?

Enterprises and consumers alike have continued to demand that Microsoft do more to secure the Windows operating system, as well as other common applications such as Microsoft Office and Internet Explorer. Many resent having to purchase or install third-party add-on products to provide the security and protection they feel should have been built in to begin with.

With Vista, and with PatchGuard in particular, Microsoft is taking those criticisms to heart and making the changes it feels are necessary to provide the best protection possible. Unfortunately, many of the third-party add-on security products relied on the inherent insecurity of previous versions of Windows to provide their protection, so PatchGuard breaks their applications as well.

One argument from the security vendors has been that these new measures place too much faith in Microsoft. If Microsoft could be trusted to be 100 percent secure and truly protect the kernel, that would be great. However, if Microsoft’s kernel protection is flawed, an attacker may be able to exploit it and the consumer will not even have alternative protection to turn to because Microsoft locked everyone from accessing the kernel.

Microsoft has provided a framework to allow application developers to work around kernel-patching, though, and it is making efforts to cooperate with security vendors to ensure that their products are able to function harmoniously with PatchGuard. PatchGuard is only available for the 64-bit version of Windows Vista, but it appears to be a significant step toward a more secure Windows operating system.