DNS troubleshooting and analysis

This first edition of Traffic Talk takes a look at the Domain Name System (DNS), the mechanism that translates IP addresses to hostnames and back, plus a slew of other functions. Learn how to identify problems with DNS records and learn more about DNS servers themselves to ensure the protection and secure operation of your clients' websites.

Welcome to the first edition of Traffic Talk, a regular SearchNetworkingChannel.com series for junior to intermediate networkers who troubleshoot business networks. In these articles we examine a variety of open source tools that expose and analyze different types of network traffic. In this edition we explore the Domain Name System (DNS), the mechanism that translates IP addresses to hostnames and back, plus a slew of other functions....

As has been demonstrated by the discussions on Dan Kaminsky's DNS vulnerability disclosure, DNS is key to the proper and secure operation of every site connected to the Internet.

 

This series uses FreeBSD as the demonstration operating system. Where possible, tools from FreeBSD's ports tree will be used. In almost all cases, tools from FreeBSD's ports tree are readily available on other Unix-like operating systems.

We'll try, where possible, to examine real-world conditions on live networks, but only when the actions taken in no way adversely affect those networks. For example, you'll never see a Traffic Talk article that shows how to conduct a denial-of-service attack on a production Internet host!

Where useful, network traffic snippets are displayed using the command line Tshark tool shipped with Wireshark.

Let's start our discussion of DNS troubleshooting by looking at the native host command, the modern equivalent of the familiar nslookup tool.

$ host

Usage: host [-aCdlriTwv] [-c class] [-N ndots] [-t type] [-W time]

[-R number] [-m flag] hostname [server] -a is equivalent to -v -t ANY -c specifies query class for non-IN data -C compares SOA records on authoritative nameservers -d is equivalent to -v -i IP6.INT reverse lookups -N changes the number of dots allowed before root lookup is done -r disables recursive processing -R specifies number of retries for UDP packets -s a SERVFAIL response should stop query -t specifies the query type -T enables TCP/IP mode -v enables verbose output -w specifies to wait forever for a reply -W specifies how long to wait for a reply -4 use IPv4 query transport only -6 use IPv6 query transport only -m set memory debugging flag (trace|record|usage)

To demonstrate use of the host command, let's identify the nameserver for bpc.bt domain.

$ host -t ns bpc.bt.

bpc.bt name server ns1.bpc.bt.

The traffic is a simple UDP DNS query and response.

14:10:48.733610 68.48.240.186 -> 68.87.73.242 DNS Standard query NS bpc.bt 14:10:48.745794 68.87.73.242 -> 68.48.240.186 DNS Standard query response NS ns1.bpc.bt

DNS queries and responses can be conducted over a single TCP socket, if we pass the -T switch to host.

$ host -T -t ns bpc.bt.

bpc.bt name server ns1.bpc.bt.

The traffic shows a standard TCP three-way handshake, followed by the query, an ACK, a response and a three-packet graceful close.

14:11:03.876558 68.48.240.186 -> 68.87.73.242 TCP 61087 > 53 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=3 TSV=3806834396 TSER=0 14:11:03.884374 68.87.73.242 -> 68.48.240.186 TCP 53 > 61087 [SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1460 14:11:03.884406 68.48.240.186 -> 68.87.73.242 TCP 61087 > 53 [ACK] Seq=1 Ack=1 Win=65535 Len=0 14:11:03.884461 68.48.240.186 -> 68.87.73.242 DNS Standard query NS bpc.bt 14:11:03.903977 68.87.73.242 -> 68.48.240.186 TCP 53 > 61087 [ACK] Seq=1 Ack=27 Win=5840 Len=0 14:11:03.903989 68.87.73.242 -> 68.48.240.186 DNS Standard query response NS ns1.bpc.bt 14:11:03.904213 68.48.240.186 -> 68.87.73.242 TCP 61087 > 53 [FIN, ACK] Seq=27 Ack=61 Win=65535 Len=0 14:11:03.913437 68.87.73.242 -> 68.48.240.186 TCP 53 > 61087 [FIN, ACK] Seq=61 Ack=28 Win=5840 Len=0 14:11:03.913466 68.48.240.186 -> 68.87.73.242 TCP 61087 > 53 [ACK] Seq=28 Ack=62 Win=103 Len=0

TCP-based DNS is used whenever a DNS reply exceeds 512 bytes. For example, consider asking the amazon.com domain for ANY records-using host.

$ host -t any amazon.com ;; Truncated, retrying in TCP mode. amazon.com descriptive text "v=spf1 ip4:207.171.160.0/19 ip4:87.238.80.0/21 ip4:72.21.193.0/24 ip4:72.21.197.0/24 ip4:72.21.196.0/24 ip4:72.21.208.0/24 ip4:72.21.209.0/24 ip4:194.154.193.200/28 ip4:194.7.41.152/28 ~all" amazon.com descriptive text "spf2.0/pra ip4:207.171.160.0/19 ip4:87.238.80.0/21 ip4:72.21.193.0/24 ip4:72.21.197.0/24 ip4:72.21.196.0/24 ip4:72.21.208.0/24 ip4:72.21.209.0/24 ip4:194.154.193.200/28 ip4:194.7.41.152/28 ~all" amazon.com has address 72.21.210.11 amazon.com has address 72.21.206.5 amazon.com has address 72.21.203.1 amazon.com mail is handled by 10 smtp-fw-4101.amazon.com. amazon.com mail is handled by 10 smtp-fw-2101.amazon.com. amazon.com mail is handled by 10 smtp-fw-9101.amazon.com.

The ";; Truncated, retrying in TCP mode." line indicates we'll see a TCP DNS query follow the standard UDP query:

16:03:29.940081 68.48.240.186 -> 68.87.73.242 DNS Standard query ANY amazon.com

16:03:29.955542 68.87.73.242 -> 68.48.240.186 DNS Standard query response TXT TXT A 72.21.210.11 A 72.21.206.5 A 72.21.203.1

16:03:29.955832 68.48.240.186 -> 68.87.73.242 TCP 56849 > 53 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=3 TSV=3813443145 TSER=0 16:03:29.976128 68.87.73.242 -> 68.48.240.186 TCP 53 > 56849 [SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1460 16:03:29.976154 68.48.240.186 -> 68.87.73.242 TCP 56849 > 53 [ACK] Seq=1 Ack=1 Win=65535 Len=0 16:03:29.976192 68.48.240.186 -> 68.87.73.242 DNS Standard query ANY amazon.com 16:03:30.001324 68.87.73.242 -> 68.48.240.186 TCP 53 > 56849 [ACK] Seq=1 Ack=31 Win=5840 Len=0 16:03:30.001337 68.87.73.242 -> 68.48.240.186 DNS Standard query response TXT TXT A 72.21.210.11 A 72.21.206.5 A 72.21.203.1 MX 10 smtp-fw-4101.amazon.com MX 10 smtp-fw-2101.amazon.com MX 10 smtp-fw-9101.amazon.com 16:03:30.001646 68.48.240.186 -> 68.87.73.242 TCP 56849 > 53 [FIN, ACK] Seq=31 Ack=624 Win=65535 Len=0 16:03:30.009778 68.87.73.242 -> 68.48.240.186 TCP 53 > 56849 [FIN, ACK] Seq=624 Ack=32 Win=5840 Len=0 16:03:30.009808 68.48.240.186 -> 68.87.73.242 TCP 56849 > 53 [ACK] Seq=32 Ack=625 Win=65534 Len=0

TCP-based DNS is different from a zone transfer, which also uses TCP. A zone transfer requests a list of hosts known by a DNS server, as demonstrated using the host command and the bpc.pt domain.

$ host -l bpc.bt. ns1.bpc.bt. Using domain server: Name: ns1.bpc.bt. Address: 202.144.155.53#53 Aliases:

bpc.bt name server ns1.bpc.bt. ftp.bpc.bt has address 202.144.155.53 mail.bpc.bt has address 202.144.155.54 ns1.bpc.bt has address 202.144.155.53 www.bpc.bt has address 202.144.155.53

We learned of four hosts in the bpc.pt domain, although we already knew of the Web server via a simple Google search. We also knew about the name server from our first host query. This traffic snippet is TCP, as expected.

14:12:18.265473 68.48.240.186 -> 202.144.155.53 TCP 53736 > 53 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=3 TSV=3806907269 TSER=0 14:12:18.579381 202.144.155.53 -> 68.48.240.186 TCP 53 > 53736 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=312352364 TSER=3806907269 WS=7 14:12:18.579413 68.48.240.186 -> 202.144.155.53 TCP 53736 > 53 [ACK] Seq=1 Ack=1 Win=66608 Len=0 TSV=3806907577 TSER=312352364 14:12:18.579480 68.48.240.186 -> 202.144.155.53 DNS Standard query AXFR bpc.bt 14:12:18.891653 202.144.155.53 -> 68.48.240.186 TCP 53 > 53736 [ACK] Seq=1 Ack=27 Win=5888 Len=0 TSV=312352442 TSER=3806907577 14:12:18.891666 202.144.155.53 -> 68.48.240.186 DNS Standard query response SOA ns1.bpc.bt NS ns1.bpc.bt MX 10 mail.bpc.bt A 202.144.155.53 A 202.144.155.54 A 202.144.155.53 A 202.144.155.53 SOA ns1.bpc.bt 14:12:18.891921 68.48.240.186 -> 202.144.155.53 TCP 53736 > 53 [FIN, ACK] Seq=27 Ack=221 Win=66608 Len=0 TSV=3806907883 TSER=312352442 14:12:19.207218 202.144.155.53 -> 68.48.240.186 TCP 53 > 53736 [FIN, ACK] Seq=221 Ack=28 Win=5888 Len=0 TSV=312352521 TSER=3806907883 14:12:19.207250 68.48.240.186 -> 202.144.155.53 TCP 53736 > 53 [ACK] Seq=28 Ack=222 Win=66600 Len=0 TSV=3806908192 TSER=312352521

If we use the host -t any command against the bpc.pt domain, we can confirm the mail information.

$ host -t any bpc.bt. bpc.bt has SOA record ns1.bpc.bt. postmaster.bpc.bt. 2007120400 10800 900 604800 86400 bpc.bt name server ns1.bpc.bt. bpc.bt mail is handled by 10 mail.bpc.bt.

Looking beyond host, we turn to the DNSwalk tool by David Barr. DNSwalk examines zone transfer results for errors:

$ dnswalk bpc.bt. Checking bpc.bt. BAD: bpc.bt. has only one authoritative nameserver Getting zone transfer of bpc.bt. from ns1.bpc.bt...done. SOA=ns1.bpc.bt contact=postmaster.bpc.bt 0 failures, 0 warnings, 1 errors.

The results complain that the bpc.bt zone only has one authoritative name server, so if something happens to it the entire domain is unreachable.

If we want to learn what version of BIND (the Berkeley Internet Name Daemon) ns1.bpc.pt is running, we can ask it using the old dig command:

$ dig @ns1.bpc.bt txt chaos version.bind

; <<>> DiG 9.4.2 <<>> @ns1.bpc.bt txt chaos version.bind ; (1 server found) ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5803 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0 ;; WARNING: recursion requested but not available

;; QUESTION SECTION:

;version.bind. CH TXT

;; ANSWER SECTION:

version.bind. 0 CH TXT "9.3.2"

;; AUTHORITY SECTION:

version.bind. 0 CH NS version.bind.

;; Query time: 317 msec ;; SERVER: 202.144.155.53#53(202.144.155.53) ;; WHEN: Sat Jul 5 14:17:21 2008 ;; MSG SIZE rcvd: 62

That question is asked via a UDP DNS query and response:

14:17:21.276766 68.48.240.186 -> 202.144.155.53 DNS Standard query TXT version.bind 14:17:21.594404 202.144.155.53 -> 68.48.240.186 DNS Standard query response TXT

We can try to learn the type of name server using Fpdns by Roy Arends and Jakob Schlyter:

$ fpdns ns1.bpc.bt fingerprint (ns1.bpc.bt, 202.144.155.53): ISC BIND 9.2.3rc1 -- 9.4.0a0 [recursion enabled]

As you can see, Fpdns relies on odd DNS queries to fingerprint its target:

14:17:29.133096 68.48.240.186 -> 68.87.73.242 DNS Standard query A ns1.bpc.bt 14:17:29.143410 68.87.73.242 -> 68.48.240.186 DNS Standard query response A 202.144.155.53 14:17:29.146689 68.48.240.186 -> 202.144.155.53 DNS Inverse query A  14:17:29.457935 202.144.155.53 -> 68.48.240.186 DNS Inverse query response, Not implemented 14:17:29.459390 68.48.240.186 -> 202.144.155.53 DNS Zone change notification A  14:17:29.774529 202.144.155.53 -> 68.48.240.186 DNS Zone change notification response, Format error

We can even use Fyodor's Nmap to identify the DNS server:

# nmap -n -sV -sU -p 53 ns1.bpc.bt

Starting Nmap 4.20 ( http://insecure.org ) at 2008-07-05 14:27 EDT Interesting ports on 202.144.155.53: PORT STATE SERVICE VERSION 53/udp open domain ISC Bind (Fake version: 9.3.2)

Service detection performed. Please report any incorrect results at http://insecure.org/nmap/submit/ . Nmap finished: 1 IP address (1 host up) scanned in 5.107 seconds

Let's look at one other domain to see what these tools can do. Cameroon's government provides an example. First find the name servers for the gov.cm domain:

# host -t ns gov.cm. gov.cm name server mbapit.gov.cm. gov.cm name server kim.camnet.cm.

Next try a zone transfer:

# host -l gov.cm. mbapit.gov.cm Using domain server: Name: mbapit.gov.cm Address: 195.24.203.1#53 Aliases:

gov.cm name server mbapit.gov.cm. gov.cm name server kim.camnet.cm. ...edited... tchadtalent.gov.cm name server mbapit.pm.gov.

tchadtalents.gov.cm name server mbapit.pm.gov.

I omitted several dozen records. This is a good candidate for inspection with DNSwalk.

# dnswalk gov.cm. Checking gov.cm. Getting zone transfer of gov.cm. from mbapit.gov.cm...done. SOA=mbapit.pm.gov contact=administrateur.gov.cm BAD: colloque-es.gov.cm NS mbapit.pm.gov: unknown host ...edited... WARN: kim.gov.cm A 195.24.92.35: no PTR record ...edited... WARN: spm_dns.spm.gov.cm: invalid character(s) in name ...edited... BAD: tchadtalents.gov.cm NS mbapit.pm.gov: unknown host 0 failures, 7 warnings, 17 errors.

As you can see from the edited results, the gov.cm domain has many problems -- domain names with no IP addresses ("unknown host" BAD errors), invalid characters and so on.

If we use the host command to check for mail records, we find another problem:

# host -t mx gov.cm.

gov.cm has no MX record

This means no one can email anyone in the gov.cm domain. Finally, let's see if we can identify their name servers using dig, Fpdns and Nmap.

$ dig @kim.camnet.cm. txt chaos version.bind ; <<>> DiG 9.4.2 <<>> @kim.camnet.cm. txt chaos version.bind ; (1 server found) ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 386 ;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; WARNING: recursion requested but not available

More tips from Richard Bejtlich
Learn how to use Snort to analyze network traffic and protect your clients' networks against intrusions.

;; QUESTION SECTION:

;version.bind. CH TXT

;; Query time: 234 msec ;; SERVER: 195.24.192.35#53(195.24.192.35) ;; WHEN: Sat Jul 5 14:40:32 2008 ;; MSG SIZE rcvd: 30

$ fpdns kim.camnet.cm fingerprint (kim.camnet.cm, 195.24.192.35): ISC BIND 9.2.3rc1 -- 9.4.0a0

# nmap -n -sV -sU -p 53 kim.camnet.cm

Starting Nmap 4.20 ( http://insecure.org ) at 2008-07-05 16:17 EDT Interesting ports on 195.24.192.35: PORT STATE SERVICE VERSION 53/udp open domain

Service detection performed. Please report any incorrect results at http://insecure.org/nmap/submit/ . Nmap finished: 1 IP address (1 host up) scanned in 8.960 seconds

Dig and Nmap produced no results, but Fpdns believes BIND is being used.

$ dig @mbapit.gov.cm. txt chaos version.bind

; <<>> DiG 9.4.2 <<>> @mbapit.gov.cm. txt chaos version.bind ; (1 server found) ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOTIMP, id: 63065 ;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; WARNING: recursion requested but not available

;; QUESTION SECTION: ;version.bind. CH TXT

;; Query time: 246 msec ;; SERVER: 195.24.203.1#53(195.24.203.1) ;; WHEN: Sat Jul 5 14:40:45 2008 ;; MSG SIZE rcvd: 30

$ fpdns mbapit.gov.cm fingerprint (mbapit.gov.cm, 195.24.203.1): No match found

# nmap -n -sV -sU -p 53 mbapit.gov.cm

Starting Nmap 4.20 ( http://insecure.org ) at 2008-07-05 16:18 EDT Interesting ports on 195.24.203.1: PORT STATE SERVICE VERSION 53/udp open domain Microsoft DNS Service Info: OS: Windows

Service detection performed. Please report any incorrect results at http://insecure.org/nmap/submit/ . Nmap finished: 1 IP address (1 host up) scanned in 4.308 seconds

Here, dig and Fpdns produce no results, but Nmap discovers a Microsoft DNS server.

Using simple open source tools like host, dig, Fpdns, DNSwalk and Nmap, you can identify problems in your client's DNS architecture fairly easily.

About the author Richard Bejtlich is the founder of TaoSecurity, author of several books on network security monitoring, including Extrusion Detection: Security Monitoring for Internal Intrusions, and operator of the TaoSecurity blog.


This was first published in July 2008

Dig deeper on Network Management Services

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

MicroscopeUK

SearchCloudProvider

SearchSecurity

SearchStorage

SearchNetworking

SearchCloudComputing

SearchConsumerization

SearchDataManagement

SearchBusinessAnalytics

Close