Although the argument suggests that some protection for your IT networks is better than none, signature-based intrusion detection systems really only scratch the surface of what most blue chip organisations need to protect themselves against. Arnt Brox examines why anomaly-based testing is a better bet for corporate network safety.
Intrusion detection has become big business on the Internet and, to be honest, it's not surprising. With the profusion of e-commerce web sites, on-line banking and other high profile applications, it's fully understandable that organisations should want to avail themselves of the best possible protection against unauthorised system entry.

In truth, the threat of network intrusion is much wider than those heavily publicised incidents of web site defacement would have us believe. It would also be a misnomer to imply that this is merely an Internet-only problem.

The threat of network intrusion hangs heavy over any organisation that possesses a network open to the outside world. And, in taking that particular statement on board, we open up the true dangers inherent in virtually every Internet protocol network. That, by definition, involves virtually every computer network in existence today – networks used for everything from process industry control through to Internet banking and office automation.

Connectivity is the byword
Since the byword of almost every modern organisation is connectivity, even those companies that have no direct Internet presence remain vulnerable to hacker attack and intrusions.

Just because you don't have a web site or, equally, because your site doesn't feature any e-commerce capabilities, that doesn't make you immune to the possibility of someone gaining unauthorised access to your network.

Just think about it for a moment. Most organisations running a network have the capability to allow members of staff and even outside contractors to connect to their systems remotely. This makes it easier for workers to connect from home, or while on the move. Importantly, it also renders the network susceptible to unauthorised entry by unwelcome third parties.

And therein, as they say, lies the rub. Our modern work practices make it essential that we provide a reasonable degree of external connectivity to our networks, regardless of whether we are a bank or a financial institution (or even an e-commerce site with sensitive customer data just waiting to be exposed).

The fact is, then, that virtually every organisation running anything other than a perfectly closed loop network is leaving itself open to possible intruder attack.

So what's the answer? Well, one of the most prevalent solutions is the installation of a sophisticated firewall system. Undoubtedly, this can help in 'hiding' major parts of your system from unwanted attention. However, the problem remains that we still need to provide external connectivity, data communications, Internet access and maybe even Voice-over-IP for the organisation as a whole. Inevitably, this means that the firewall cannot be used simply to pull the proverbial shutters down.

The need to communicate brings us the necessity to search for a second line of defence. That's where intrusion detection comes into the equation. As there's no universally adopted definition of what this actually is, it's probably easier to describe the whole concept by reference to more familiar analogies.

Access to the network
Think of intrusion detection as a well-trained guard dog and you'll derive the general idea. Now imagine that the rooms in your home represent your network, and the perimeter fence around the garden is standing in for the firewall. You need to gain access to the outside world and, equally, authorised guests, visitors and invitees must be able to enter your property. After all, how else would you be able to receive the morning post and newspapers, and have the electricity and gas meters read?

Being the prudent householder, you'll realise that there's a distinct possibility some visitors to your premises will not be welcome. Now, because you have a gate to allow you to mingle with the outside world, and vice versa, this leaves you vulnerable to the attentions of such undesirable elements – the networking equivalent of double glazing salesmen or, worse still, burglars. This is the time when your trusty guard dog makes its presence felt.

Due to the fact that he or she has been trained to sniff out unwanted guests, it duly sounds a warning whenever it detects the presence of any unauthorised third party coming through the gate. This is the basis of intrusion detection.

That said, as you can imagine not all guard dogs are perfectly trained. Some will happily bark at anything or anyone that approaches your gate, while others will sit down and wait to be patted as the burglar walks past and strolls into your lounge. Thus we have a problem. While there are many intrusion detection solutions on the market, some are more efficient than others in the elimination of what we call 'false positives', as well as in the correct identification of unauthorised traffic.

Most intrusion detection systems are what is known as signature-based, meaning that they operate in much the same way as a virus scanner by searching for a known identity (or signature) for each specific intrusion event. And, while signature-based intrusion detection is very efficient at sniffing out known styles of attack, it does – much like anti-virus software – depend on receiving regular signature updates such that it can stay in touch with variations to hackers' techniques.

In other words, signature-based intrusion detection systems are only as good as the database of stored signatures. It's a bit like training our proverbial guard dog to watch the front door, but then forgetting to tell it to watch the back of the house as well.

As signature-based intrusion detection can only ever be as good as the extent of the database, further problems immediately arise. For one, it becomes all-too-easy to fool signature-based solutions by changing and obfuscating the ways in which an attack is made. This technique simply skirts around the signature database stored in the detection system, offering the hacker an ideal opportunity to gain access to the network.

Hackers like a challenge
Make no mistake that hackers enjoy a challenge, and like nothing better than to be able to test their software and pit their skills against many of the commercially-available intrusion detection systems. As the attacker knows that the intrusion system will trigger an alarm when it detects certain attack signatures, that hacker will tend to evade detection by disguising the attack.

In network traffic terms, anomaly-based intrusion detection captures all the headers of the IP packets running towards the network. From here, it filters out all known and legal traffic (including web traffic) to the organisation’s web server, mail traf

For example, hackers are aware that signature-based intrusion detection systems traditionally have a problem with the complexities of application interactions. This is compounded by the fact that application protocols have become increasingly complex as they expand to provide support for features like Unicode.

Briefly, Unicode allows uniform computer representation of every character in every language by providing a unique code point or identifier for each character. It's a standard requirement of well-known computer languages such as Java and XML, making it a feature of many modern operating systems. As signature-based intrusion detection can miss characters written in Unicode Transformation Format, it then becomes relatively easy for an attacker to submit a URL containing an exploit that would allow other programmes to be run and files to be accessed.

Due to these known problems, signature-based intrusion detection is really only suited to very basic levels of protection. For any organisation wanting to implement a more thorough – and hence safer – solution, it's better to use anomaly-based intrusion detection. By its very nature, this is a rather more complex animal.

In fact, to borrow once more from our earlier analogy, it's like our guard dog personally interviewing everyone at the gate before they're allowed to walk down the driveway.

Anomaly-based detection
In network traffic terms, anomaly-based intrusion detection captures all the headers of the IP packets running towards the network. From here, it filters out all known and legal traffic (including web traffic) to the organisation's web server, mail traffic to and from its mail server, outgoing web traffic from company employees and DNS traffic to and from its DNS server.

Even though this level of filtering significantly narrows down the amount of data to be analysed, anomaly-based intrusion detection can still create large amounts of log data. This is then analysed using database functionality. Since anomaly-based intrusion detection sees all of the traffic running into the network, there are far fewer places in which to conceal malicious hacker codes.

There are other equally obvious advantages to using anomaly-based detection systems. For example, because it detects any traffic that's either new or unusual, this method is particularly good at identifying sweeps and probes towards network hardware. Therefore, it can give early warnings of potential intrusions (probes and scans are the predecessors of all attacks).

The more targeted the probes and scans, the more likely that the hacker is serious about attacking your network. Equally, the technique is ideal for detecting every new piece of hardware installed on the network. This applies to any new service installed on any item of hardware and forgotten about when the maintenance was finished.

As such, anomaly-based intrusion detection is perfect for detecting anything from port and web anomalies through to misinformed attacks where the URL is deliberately typed incorrectly.

Some Internet security commentators argue that effective anomaly testing isn't possible. They claim that, because the technique requires trained human resources, not to mention sophisticated hardware and software, the procedures involved simply aren't viable.

Admittedly, anomaly testing requires more hardware spread further across the network than is required with signature-based solutions. This is particularly true for larger networks and, with the bandwidth connections, it's therefore necessary to install the anomaly sensors closer to the servers and network being monitored.

The rationale here is that the amount of data is lessened the closer the sensors are to the application than if they were located close to or at the network backbone. Placing them too close to the main backbone simply results in too much data being detected.

However, none of this detracts from the inescapable fact that anomaly testing is a more effective way of pinpointing possible attacks. In fact, most of the operational criticisms levelled against anomaly-based intrusion detection systems are equally applicable to signature-based testing.

Both methods need tuning to reduce the number of false alerts. Very important, as the temptation to tune a system too tightly can cause the loss of some of the 'events of interest' that you're hoping to detect.

A 'straight-from-the-box' answer?
It's worth remembering that anomaly-based detection certainly isn't the 'straight-from-the-box' solution that signature testing purports to be. Ideally, anomaly testing parameters and criteria need to be configured in conjunction with the organisation's own engineers – that is, with the input of IP services and the corresponding network addresses (ie computers with the IP services exposed to the network to which they're connected). Once installed, any anomalies detected need to be analysed by trained operatives.

Some may argue that this makes an anomaly-based solution much more of a hands-on service than signature intrusion detection systems. Looking at the amount of labour involved in nursing a normal signature-based system, I'd say this isn't the case.

All of this makes anomaly testing much more capable of correctly identifying the basis of a hacker attack than straightforward signature-based techniques. What it doesn't do, however, is explain what can be done once hacker activity has been detected.