Monitoring DNS Traffic for Threats

Here are several methods to monitor DNS traffic for security threats.
Firewalls
Let's begin at the most prevalent security system: your firewall. All firewalls should let you define rules to prevent IP spoofing. Include a rule to deny DNS queries from IP addresses outside your allocated numbers space to prevent your name resolver from being exploited as an open reflector in DDoS attacks.
Next, enable inspection of DNS traffic for suspicious byte patterns or anomalous DNS traffic to block name server software exploit attacks.

Documentation describing how popular firewalls provide this feature is readily available (e.g., Palo Alto Networks, Cisco Systems, WatchGuard). Sonicwall and Palo Alto can detect and block certain DNS tunneling traffic, as well.

Intrusion detection systems
Whether you use Snort, Suricata, or OSSEC, you can compose rules to report DNS requests from unauthorized clients.

You can also compose rules to count or report NXDOMAIN responses, responses containing resource records with short TTLs, DNS queries made using TCP, DNS queries to nonstandard ports, suspiciously large DNS responses, etc. Any value in any field of the DNS query or response message is basically "in play." You're essentially limited only by your imagination and mastery of DNS. Intrusion prevention services in firewalls provide permit/deny rules for many of the most common of these checks.

Traffic analyzers
Use cases for both Wireshark and Bro show that passive traffic analysis can be useful in identifying malware traffic. Capture and filter DNS traffic between your clients and your resolver, and save to a PCAP file.

Create scripts to search the PCAP for the specific suspicious activities you are investigating, or use PacketQ (originally DNS2DB) to SQL query the PCAP file directly.

(Remember to block your clients from using any resolver or nonstandard port other than your local resolvers).
Passive DNS replication
This involves using sensors at resolvers to create a database that contains every DNS transaction (query/response) through a given resolver or set of resolvers.

Including passive DNS data in your analysis can be instrumental in identifying malware domains, especially in cases where the malware uses algorithmically generated domain names (DGAs). Palo Alto Networks firewalls and security management systems that use Suricata as an IDS engine (like AlienVault USM or OSSIM) are examples of security systems that pair passive DNS with IPS to block known malicious domains.

Logging at your resolver
The logs of your local resolvers are a last and perhaps most obvious data source for investigating DNS traffic. With logging enabled, you can use tools like Splunk plus getwatchlist or OSSEC to collect DNS server logs and explore for known malicious domains.
Passive DNS replication
This involves using sensors at resolvers to create a database that contains every DNS transaction (query/response) through a given resolver or set of resolvers. Including passive DNS data in your analysis can be instrumental in identifying malware domains, especially in cases where the malware uses algorithmically generated domain names (DGAs). Palo Alto Networks firewalls and security management systems that use Suricata as an IDS engine (like AlienVault USM or OSSIM) are examples of security systems that pair passive DNS with IPS to block known malicious domains.
Logging at your resolver
The logs of your local resolvers are a last and perhaps most obvious data source for investigating DNS traffic. With logging enabled, you can use tools like Splunk plus getwatchlist or OSSEC to collect DNS server logs and explore for known malicious domains