Using Snort 2.8.3 to inspect HTTP traffic

Since the last Snort Report, Snort 2.8.3 has been released. In this issue, expert Richard Bejtlich discusses the best methods to learn about the new features of Snort 2.8.3 and how to use them when inspecting HTTP traffic.

Solution provider takeaway: Solution providers will learn new features in Snort 2.8.3 to improve the granularity...

of inspecting HTTP traffic.

Welcome to the 20th edition of the Snort Report! In July, we described new features in Snort 2.8.2 and how to identify them when compared to Snort 2.8.0 and intervening releases. Since then, Snort, and 2.8.3 have arrived. In this issue of the Snort Report, we'll use the previously explained techniques to learn what's new in Snort 2.8.3, and then try those techniques ourselves.

The first place to look for news on recent Snort features is still the release notes. A look at the release notes for and show mainly "improvements" that involve a "fix" action, like "* Fix issue with dsize range check."

More on Snort
For more on the open source network intrusion detection software, check out our Snort Reports.

That's interesting -- I wonder what issue was fixed? The dsize keyword is used to test packet payload size. For example, you could tell Snort to accept a match only if a packet's payload (above layer 4) is less than a certain number of bytes, or greater than a certain number of bytes, or between a certain number of bytes.

For example, the following Emerging Threats rule uses the dsize keyword to match only on packets with data payloads of less than 30 bytes.

alert tcp $HOME_NET any -> $EXTERNAL_NET 20000 (msg:"ET MALWARE Realtimegaming.com Online Casino Spyware Gaming Checkin"; flow:established,to_server; dsize:<30; content:"|43 01 00|"; depth:4; content:"Casino"; nocase; classtype:trojan-activity; sid:2008402; rev:2;)

The following Emerging Threats rule looks for packets with data payloads between 9 and 18 bytes, (probably) inclusive.

alert tcp $EXTERNAL_NET 1024:5000 -> $HOME_NET 1024:65535 (msg:"ET MALWARE SOCKSv4 Port 25 Inbound Request (Windows Source)"; dsize:9<>18; flow:established,to_server; content:"|04 01 00 19|"; offset:0; depth:4; threshold:type both, track by_src, count 2, seconds 900; classtype:protocol-command-decode; sid:2003256; rev:3;)

The beauty of an open source program like Snort is that one can peruse the source code or, better yet, CVS tree for changes like the one involving the dsize keyword. In the old days, the Snort project supported a mailing list (https://lists.sourceforge.net/lists/listinfo/snort-cvsinfo) that published individual comments as email messages. If that method was still supported, one could search the list archives for recent mentions of "dsize".

With that mechanism no longer supported, I decided to grep the src/ directory of the 2.8.3 archive for "dsize" to see what might appear. I noticed decode.c, decode.h and multiple .c files in the detection-plugins directory contained the string dsize. I turned to the online Snort CVS website and browsed the contents of the src/ directory, with contents sorted by age:


I didn't see anything interesting in the CVS logs for decode.c and decode.h, so I moved into the detection-plugins directory. Right away, I found the last log entry for sp_dsize_check.c with the following: "Fix issue with rule option 'dsize' range check. Thanks to Bhadresh Patel." That makes sense -- even the .c file is named for the issue I'm researching here. By visiting the CVS log for sp_dsize_check.c I could compare the latest version (1.23) with the previous (1.22) by clicking the link for "Diff to previous 1.22."

Finally, I see that these lines:

if ((ds_ptr->dsize <= p->dsize) &&
(ds_ptr->dsize <= p->dsize))

were changed to these:

if ((ds_ptr->dsize <= p->dsize) &&
(ds_ptr->dsize2 <= p->dsize))

I'm not a C programmer, so I can't tell if there are security implications of this change. I doubt it, because Sourcefire tends to tell the world when a security advisory is required. Still, if you have the will and skill, you could determine the impact of this change for yourself.

Before turning to the real gem in the 2.8.3 release notes, I'd like to briefly mention the second way I like to learn of new Snort features: examine changes to the snort.conf file. In the following example, I use diff to search for differences between the snort.conf shipped with 2.8.2 and the snort.conf in 2.8.3:

freebsd70:/usr/local/src# diff snort-2.8.2/etc/snort.conf snort-2.8.3/etc/snort.conf
< # http://www.snort.org Snort 2.8.2 Ruleset
> # http://www.snort.org Snort 2.8.3 Ruleset
> #

As you can see, the only change is a renumber from 2.8.2 to 2.8.3.

Turning back to the release notes, we see this interesting item:

* New Feature for HTTP Inspect to split requests into 5 components -
Method, URI, Header (non-cookie), Cookies, Body. Added HTTP server
specific configurations to normalize HTTP header and/or cookie buffers.
Provided content and PCRE modifiers to allow searches within one or
more of those individual buffers. Added content modifier to allow rule
writer to specify content to be used for fast pattern matcher.
Updated dynamic rule API to allow searches within the new buffers.

This reminds me of the third place I like to review to understand new Snort features: the doc/ directory. A fast way to find new features is to view the CVS doc/ directory and identify the newest files. (README.dcerpc has been heavily modified, but that's a topic for a future Snort Report.) A look at the differences between README.http_inspect 1.23 and 1.24 explains the new capabilities of that preprocessor, basically changing "flow_depth" to "server_flow_depth" and "client_flow_depth", and adding "normalize_headers" and "normalize_cookies".

This discovery brings us to the final place one can look for more detail on Snort features: the manual. A look at the entry for HTTP Inspect example provides details on the four changes just listed. However, the HTTP request split into "five components" doesn't appear there. Instead, reviewing the Payload Detection Rule Options part of the manual shows five new rule options: http_client_body, http_cookie, http_header, http_method and http_uri.

Let's try those options, using the methodology outlined in the issue of the Snort Report on Snort 2.8.0 new features -- IPv6 and port lists. Start by compiling Snort 2.8.3 and moving critical files into place.

freebsd70:/usr/local/src/snort-2.8.3# mkdir /usr/local/snort-2.8.3

freebsd70:/usr/local/src/snort-2.8.3# ./configure --enable-dynamic-plugin --prefix=/usr/local/snort-2.8.3
freebsd70:/usr/local/src/snort-2.8.3# make
freebsd70:/usr/local/src/snort-2.8.3# make install

freebsd70:/usr/local/src/snort-2.8.3# cp etc/snort.conf /usr/local/snort-2.8.3/
freebsd70:/usr/local/src/snort-2.8.3# cp etc/snort.conf /usr/local/snort-2.8.3/snort.conf.2.8.3
freebsd70:/usr/local/src/snort-2.8.3# cp etc/unicode.map /usr/local/snort-2.8.3/
freebsd70:/usr/local/src/snort-2.8.3# cp etc/classification.config /usr/local/snort-2.8.3/
freebsd70:/usr/local/src/snort-2.8.3# cp etc/reference.config /usr/local/snort-2.8.3/
freebsd70:/usr/local/src/snort-2.8.3# cd /usr/local/snort-2.8.3/

Next test Snort 2.8.3.

freebsd70:/usr/local/snort-2.8.3# bin/snort -V

,,_ -*> Snort! <*-
o" )~ Version 2.8.3 (Build 16) FreeBSD
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
(C) Copyright 1998-2008 Sourcefire Inc., et al.
Using PCRE version: 7.4 2007-09-21

Create a simple rule:

# cat snortconf.test
alert icmp any any -> any any (msg:"LOCAL ICMP echo test"; itype:8; sid:2000000;)

Run Snort with the rule using the -T switch:

freebsd70:/usr/local/snort-2.8.3# bin/snort -T -c snortconf.test
Running in Test mode with config file: snortconf.test
Running in IDS mode
Snort successfully loaded all rules and checked all rule chains!

Finally, try running the simple rule against a live interface. Note that we have to specify a log directory with the -l switch. In a separate terminal, I generate a single ICMP packet using "ping -c 1" to trigger the simple test alert.

freebsd70:/usr/local/snort-2.8.3# bin/snort -i le0 -c snortconf.test -A console -l /tmp/

Running in IDS mode
Initializing Network Interface le0
Decoding Ethernet on interface le0

[ Port Based Pattern Matching Memory ]

--== Initialization Complete ==--

,,_ -*> Snort! <*-
o" )~ Version 2.8.3 (Build 16) FreeBSD
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
(C) Copyright 1998-2008 Sourcefire Inc., et al.
Using PCRE version: 7.4 2007-09-21

09/27-22:29:37.455614 [**] [1:2000000:0] LOCAL ICMP echo test [**] [Priority: 0] {ICMP} ->

So far, so good. Let's create rules testing all but the http_client_body features. We have to enable frag3, stream5 and http_inspect before we can use any rules that rely on http_inspect.

freebsd70:/usr/local/snort-2.8.3# cat snortconf.test

preprocessor frag3_global: max_frags 65536
preprocessor frag3_engine: policy first detect_anomalies

preprocessor stream5_global: max_tcp 8192, track_tcp yes, \
track_udp no
preprocessor stream5_tcp: policy first, use_static_footprint_sizes

preprocessor http_inspect: global \
iis_unicode_map unicode.map 1252

preprocessor http_inspect_server: server default \
profile all ports { 80 8080 8180 } oversize_dir_length 500

alert icmp any any -> any any (msg:"LOCAL ICMP echo test"; itype:8; sid:2000000;)
alert tcp any any -> any 80 (msg:"LOCAL http_header test for gzip"; content: "gzip"; http_header; sid:2000001;)
alert tcp any any -> any 80 (msg:"LOCAL http_method test for GET"; content: "GET"; http_method; sid:2000002;)
alert tcp any any -> any 80 (msg:"LOCAL http_uri test for books"; content: "books"; http_uri; sid:2000003;)
alert tcp any any -> any 80 (msg:"LOCAL http_cookie test for PHPSES"; content: "PHPSES"; http_cookie; sid:2000004;)

Sid 2000001 looks for the string gzip in any client-sent HTTP header. Sid 2000002 looks for a GET method. Sid 2000003 looks for the string books in the URI. Sid 2000004 looks for PHPSES in any cookies sent by the client.

While browsing bestbookbuys.com via Lynx to trigger alerts, I saw the following:

09/27-22:45:55.530086 [**] [1:2000004:0] LOCAL http_cookie test for PHPSES [**] [Priority: 0] {TCP} ->
09/27-22:46:03.596011 [**] [1:2000003:0] LOCAL http_uri test for books [**] [Priority: 0] {TCP} ->
09/27-22:46:03.596011 [**] [1:2000001:0] LOCAL http_header test for gzip [**] [Priority: 0] {TCP} ->
09/27-22:46:03.596011 [**] [1:2000002:0] LOCAL http_method test for GET [**] [Priority: 0] {TCP} ->

When I get a decent example of a http_client_body rule that works, I'll post it on my blog.

Hopefully, this article shared methods you can use to learn about new Snort features and also showed how recent enhancements to HTTP Inspect can be brought to life in your customer's environment.

About the author
Richard Bejtlich is founder of TaoSecurity, author of several books on network security monitoring, including Extrusion Detection: Security Monitoring for Internal Intrusions, and operator of the TaoSecurity blog.

Dig Deeper on Managed network security services