Sep 12, 2015 - A DNS Analysis Tale

Since our last blog entry the architectural design of RaTA DNS has changed. We needed to reduce the network traffic generated by our Packet Parser -currently named Speedy, after “Speedy Gonzalez”. To do this, we designed and implemented Fievel, a new module which’s main purpose is to perform preliminar packet filtering-transformation.

<img src=/ratadns/images/monitor.png width=80%>

So, lets summarize what we are doing here. Think we have a DNS server and we want to have a realtime monitoring tool. A way that doesn’t add overhead to the server is to do port mirroring to an interface connected to an analyzer machine, just by configuring the switch. In this machine we have our RaTA DNS pipeline working: we are sniffing DNS packets with tcpdump, then we serialize the packets into a slightly-modified-JSON formatted message with the help of Speedy, following with the preliminar packet filtering-transformation given by Fievel. Fievel has several preliminar filtering-transformers (PFT) inside, each of these knows how to serialize a processed packet window. For example, a PFT may be configured to send the processed window directly to a visualization module or to a data-aggregation module on another machine.

What does Fievel exactly do?

To start running Fievel we have to configure L, a list of PFTs, and W, its windows size parameter: the number of packets that the module is going to analyze before it reset the complete state of the program. Once a serialized DNS packet is received from Speedy, Fievel’s main loop send this packet to every preliminar filtering-transformer in L. When Fievel receive the W-th packet, it tells each PFT to serialize the processed window and to reset it state.

And what is this PFT thing?

A PFT is an object that stores some state, knows how to process one packet at a time, knows what to do when Fievel received W packets, and knows how to reset its own state. A simple example of a PFT is our Packets Per Seconds showed below:

from prer import PreR
import time

class PacketsPerSecond(PreR):
    def __init__(self, f):
        PreR.__init__(self, f)
        self.counter = 0
        self.start = time.time()

    def __call__(self, d):
        self.counter += 1

    def get_data(self):
        data = { 'pps' : self.counter / (time.time() - self.start) }
        return data

    def reset(self):
        self.counter = 0
        self.start = time.time()

So that’s it! All our code is publicly available under a MIT license. If you have any suggested PFT, or any suggestion in general don’t hesitate to contact us!

Jul 24, 2015 - RaTA-DNS Project Architecture

RaTA-DNS is composed by three modules: a packet analyzer, a processing logic framework, and a data visualization module.

Packet analyzer is our equivalent to the DSC collector tool. It takes a DNS packet stream and converts them into JSON objects (a lightweight data-interchange format), which are compressed and sent via a secure connection to a hub. How to capture packets depends on the needs of the DNS admins, but in the first instance tcpdump or even the same DSC collector can be used for this purpose. The default pipeline defined in this module is as follows:

  • Capture packets with tcpdump.
  • Scan them with the packet stream analyzer.
  • Compress them with LZ4.
  • Send to a hub machine.

Same pipeline has to be used in each of the servers wanted to be analyzed.

When the data is received by the hub machine, it is decompressed and sent into a Kafka server.

Then, the Apache Storm framework read the data from the hub and distribute the work to multiple nodes, in order to obtain different types of statistics and aggregations defined by TLD administrators. Finally, aggregations and statistics are sent to a distributed database, such as Redis, in order to be displayed.

Visualization is an on-going work that will work in a distributed manner, with an HTML5 frontend. It is planned to use the R programming language with the Shiny web framework.

Apr 14, 2015 - Packet Analyzer first draft published


  • ldns: DNS C library, used to parse DNS packets
  • json-c: JSON C library, used to generate the output
  • pkg-config (Optional): Helper tool used to locate and json-c library folders to link in Makefile. If you don’t have pkg-config installed, you have to modify the Makefile according to this site.



  • Read from libpcap stream
  • Filter DNS packets (by UDP protocol & port 53)
  • Parse IPv4 Header information
  • Parse IPv6 Header information (partial)
  • Parse DNS Queries section