Staying One Step Ahead of Modern Hackers

Security threats continue to evolve and find even more creative ways to work around traditional IT barriers. In response, new malware mitigation tools have hit the marketplace. While some of these tools and techniques are destined to die a quick death, others such as next-­generation firewalls, sandboxing and reputation services are slowly becoming more widely accepted by IT managers.

The Next Generation of Firewalls

Traditional firewall rule bases focus on the “five-tuple” standard for determining data flow. A security manager combines these elements — source and destination Layer 3 IP addresses, and Layer 4 port numbers and protocols — to decide whether to block or allow traffic.

This focus on Layer 3 and Layer 4 information leaves some holes in coverage open, however, requiring security managers to make dangerous assumptions when writing access control rules.

Next-generation firewalls are all about making the tuple wider by moving up the stack to include Layer 7 information. That way, applications (not just protocols) and users or user groups (not just IP addresses or zone information) are included. As a result, security managers are provided with a broader vocabulary to work with, allowing for a more precise binding between security policy and the actual, precise instructions of the firewall rule base.

For example, network managers who want to write firewall policies based on departmental information are usually tied to physical attributes such as IP addresses and subnets. With next-generation firewall technology, however, rules can be tied to the user rather than the network segment being used. Application policies can also be more accurate. A policy to disallow outbound Simple Mail Transfer Protocol doesn’t just block port 25, as does a traditional firewall, but actively identifies SMTP running on any TCP/IP port number.

Next-generation firewalls also partially replace the functionality of web security gateways. For instance, traditional techniques depend on a content filter to identify virtual private network (VPN) servers in order to block out-of-policy tunneling. Rather than depend on an out-of-date URL database to identify anonymization and VPN proxies, a good next-generation firewall will peek into the application layer stream and identify traffic that is being tunneled out from the corporate network by its application fingerprint.

So, depending on exactly what features are being used in a web security gateway (proxy), a next-generation firewall may be able to completely replace that gateway function.

Using Reputation Services to Combat Spam

Anti-spam vendors have known for years that IP reputation information is a great tool to block spam from entering the network. Firewall and intrusion prevention system (IPS) vendors are now moving to incorporate this type of awareness into their products as well.

Reputation services work by collecting information from customers and security operations centers about ongoing attacks and infected hosts. For example, if some remote site is seen by an IPS as attacking one customer, this information can be fed back to a central database, with the IP address performing the attack being assigned a bad reputation. The results can be used in a variety of ways, from outright blocking of connections from that IP address to simply throttling them.

A number of security vendors are working to fit reputation services, which can protect end users browsing the Internet as well as servers under attack from hackers, into existing products such as firewalls and IPS devices. Savvy network managers will jump on these new features as soon as they are available.

Reputation service databases can be fed from many sources, including honeypots, IPSs, anti-malware tool reports and manual feedback from users.

Sentence Suspected Malware to the Sandbox

Signature-based anti-malware systems are useful for some attacks, but zero-day threats and polymorphic malware have reduced the overall effectiveness of traditional tools. The most promising of a new crop of tools to complement these existing systems use a combination of techniques based on sandboxing to identify malware before it bites.

While sandboxing techniques vary, the general idea is to identify suspicious attachments and web objects that are worth investigating further. Sandbox devices are usually placed in-line between end users and the Internet to monitor all traffic, with a focus on inbound data streams such as instant messaging, web, email and file transfers.

When something is downloaded that matches some criteria (such as a PDF file with embedded objects), the sandbox device fires up a convincing virtual sandbox environment to prevent the object from communicating with the outside world. The sandbox then tries to open the object while monitoring for inappropriate activity, environmental changes or application compromises.

Because of the high performance requirements for the virtual execution of sandboxes, many sandboxing vendors combine on-premises devices with cloud-based services or are moving entirely to the cloud as a proxy to protect web traffic.

Although sandbox technologies have been around for nearly a decade, it is only in the past few years that they have become mature and fast enough for mainstream deployment. Network managers will still need their traditional endpoint security anti-malware tools, but sandboxing technology does provide another opportunity for IT staff to block malware before it goes too far.