Tip

Network security monitoring using transaction data

Welcome back to Traffic Talk, a regular SearchNetworkingChannel.com series for network solution providers and consultants who troubleshoot business networks. We took a break, but we're back with more articles on using network traffic to make your business more productive and secure.

In this article, I discuss network security monitoring (NSM) and introduce one specific form of NSM data -- transaction data.

In my books

    Requires Free Membership to View

"The Tao of Network Security Monitoring" and "Extrusion Detection," I explained how four forms of NSM data could be used to better detect and respond to intrusions. Briefly, these forms are the following:

- Alert data: Judgments made by tools that inspect network traffic.
- Statistical data: Overall summaries or profiles of network traffic.
- Session data: Conversations or flows generated from network traffic.
- Full content data: Actual packets collected by storing network traffic.

These four forms of NSM data are extremely useful, but I'd like to use this article to introduce a fifth form that I have begun to apply to my daily defensive operations: transaction data. I define transaction data as application-specific records generated from network traffic. Let me demonstrate an example before I try to explain the reasons to generate this form of NSM data.

More about network security monitoring
Network security monitoring: Know your network  

Tactics for attacking network security monitoring  

HTTP records as transaction data

I'll demonstrate the creation of NSM transaction data for HTTP using Jason Bittel's Httpry program. Here I run Httpry on a live sensor interface and tell it to generate records using the HTTP fields you see on the command line.

# httpry -i bge0 -o /tmp/httpry_31mar09.txt -q -u richard -s timestamp,source-ip,x-forwarded-for,direction,dest-ip,method,host,request-uri,user-agent,referer,status-code,http-version,reason-phrase

A selection from the log file appears below.

# httpry version 0.1.3
# Fields: timestamp,source-ip,x-forwarded-for,direction,dest-ip,method,host,request-uri,user-agent,referer,status-code,http-version,reason-phrase
03/31/2009 20:52:41 74.125.91.99 - < 24.126.62.67 - - - - - 200 HTTP/1.0 OK
03/31/2009 20:52:45 24.126.62.67 192.168.2.106 > 212.77.1.150 GET mv.vatican.va /3_EN/pages/x-Schede/SDRs/SDRs_03_02_020.html Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.8) Gecko/2009032711 Ubuntu/8.04 (hardy) Firefox/3.0.8 http://www.bejtlich.net/lab.html - HTTP/1.0 -
03/31/2009 20:52:45 212.77.1.150 - < 24.126.62.67 - - - - - 200 HTTP/1.1 OK

For comparison's sake, here is the same record one might retrieve from a Squid proxy log:

1238547165.722 1357 192.168.2.106 TCP_MISS/200 6623 GET http://mv.vatican.va/3_EN/pages/x-Schede/SDRs/SDRs_03_02_020.html - DIRECT/212.77.1.150 text/html "http://www.bejtlich.net/lab.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.8) Gecko/2009032711 Ubuntu/8.04 (hardy) Firefox/3.0.8"

So, why generate the HTTP record using Httpry when you have access to Squid proxy logs? Well, imagine that you do not have access to Squid proxy logs. Maybe you don't have proxies in place. Maybe you do have proxies but don't have logging enabled, for performance reasons. Maybe you do have logging enabled, but you do not have easy access to the logs. Thanks to an application-specific traffic inspection tool like Httpry, you can collect NSM transaction records for HTTP.

Why not generate these records from the full content data when you need them? You could do that, but you will probably have better luck generating, compressing and storing text records like those created by Httpry. You can also specify exactly which fields you need.

Conclusion

You could easily extend this sort of transaction data approach to a variety of protocols. Another powerful example involves DNS requests and replies. Records of hosts that try to resolve various names could be invaluable for incident detection and response. This month's Conficker malware shows the prominence of DNS records, if you can collect them. Using something as simple as Tcpdump with a filter for port 53 can do the trick.

If you're wondering whether there are other programs to generate NSM transaction data, you're correct. In future articles we will examine a few, including Bro, a tool that can also generate the HTTP records seen in this article.

Richard Bejtlich is director of incident response at General Electric and author of the TaoSecurity Blog.


 

This was first published in April 2009

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.