Replay Mode

Case of use

Most of the attacks or interesting events happening in a DNS server are impossible to anticipate. So, sooner than later you will miss an interesting event in your servers.

Solution

Usually to solve this type of problems you will use tcpreplay, but in this case as the module might be in the same network, using tcpreplay will impact the network if the packets are injected with the same addresses. Or, if the addresses are appropriately changed, it will cause a loss of information. That is why we just added to fievel the capability to retransmit an event stored in a pcap file.

How to use it.

We tried to keep the configuration file as simple as possible, there are three fields you have to modify in order to have this new feature working:

{
  ...
  "Capture" : {
    "InputMethod" : "capture_file",
    "InputSource" : "/path/to/the/pcap_file",
    "Speed" : n,
    ...
  },
  ...
}

Sample of a configuration file.

• InputMethod: You tell the system you want to read the data from a capture file.
• InputSource: The path to the pcap file you want to read from.
• Speed: This option allows you to set the speed the system will read the input. There are three different behaviors for this value:
• n = 0: The system will read the data as fast as possible, no delay.
• n = 1: The system will read the data with the same delay specified by the timestamps in the capture file.
• n > 0 && n != 1: The system will read the data n times faster that the time specified in the capture file. This will speed up the reading if n > 1 or speed it down if n < 1. For instance, if you set n = 2 the data will be read 2 times faster, but it you set n = 0.25 it will take 4 times the original time to retransmit the data.

Implementation details.

The delay was introduced by replacing the use of pcap_loop by our own loop. This loop includes calls to nanosleep in order to provide the pauses in the execution. As we are using nanosleep do not expect a better resolution than nanoseconds in the speed of the replay.

Real Time Analysis over DNS Packet Streams

RaTA DNS is a real time monitoring framework for multiple DNS servers, which is currently in development phase. The system’s architecture is modular, to allow for easy extension and optimization of each component.

The system consists of a chain of packet processors, which starts in the network stream capture, goes through a serializer, then by a set of preliminary reducers, each of which sends its result through a different channel to an aggregator. The latter is responsible for generating the information needed to finally be consumed by the web visualization.

Monitor

A RaTA DNS monitor consists of a series of tools that define a pipeline. This pipeline is used to preprocess the data captured from a single server. At the end of this preprocessing, the results are sent to the aggregator. The aggregator receives preprocessed data from multiple monitors. Generally, we describe a RaTA DNS monitor as a machine whose sole function is to provide this service. However, RaTA DNS monitor can run on the same machine that runs the DNS server. The decision on which option to choose depends on network’s needs.

The tools that comprise the monitor are TCPDUMP, Speedy and Fievel. Also, we have another tool that interacts with data from multiple monitors, Jerry, the aggregator/visualization module. All of these tools will be explained later.

Network Stream Capture

To capture the network stream, 3 cases are considered: Capture directly on the same computer that works as server, capture via switch level port mirroring or replay a previous packet capture. The first two cases correspond to real-time capture of the packet, and the third case is a repetition of events in the past, this with forensic purposes.

Capturing the DNS packet stream, either directly on the server machine or through port mirroring, it’s performed with the help of tcpdump. It has a stable and widespread format, commonly called PCAP. In addition to dump the captured packets into the disk, TCPDUMP may generate a stream into stdout. It’s this mechanism that is used in Speedy -our packet serializer- to do its job.

Speedy: Packet Serializer

TCPDUMP output is redirected through a pipe to Speedy, our packet serializer. This module takes a network packet in the PCAP format and interprets it as a DNS packet, including IP and UDP headers. Then analyzes this packet and writes it as a JSON message with a well-defined format. Which parts of the packet are serialized can be chosen when launching the tool. What parts may be chosen?. You may choose the source and destination IP addresses, the DNS’s queries section, the DNS’s answers section and the authoritative name servers DNS’s section.

The module was developed in the C programming language, with the help of several libraries: libpcap was used to read the packets, ldns to read them as DNS packets and json-c packages to serialize the data in the JSON format. Furthermore, it was developed with the use of multiple cores in mind, by the use of worker threads. The amount of worker threads can be passed as a parameter when running the program.

The serialization format is a length-prefixed JSON message, so message blocks can be quickly read. An example of Speedy output, both for a query to and an answer from the server is showed below. IP addresses are anonymized with a hash function due to privacy concerns:

Query

128{"source":"0E402AE7","dest":"6DFA5FD7","id":"2e13","flags":"10","queries":[{"qname":"www.ejemplo.cl.","qtype":"1","qclass":"1"}]}

Answer

584{"source":"6DFA5FD7","dest":"0E402AE7","id":"2e13","flags":"8010","queries":[{"qname":"www.ejemplo.cl.","qtype":"1","qclass":"1"}],"authorities":[{"name":"ejemplo.cl.","type":"2","class":"1","ns":["ejemplo.ns.cl."]},{"name":"ejemplo.cl.","type":"2","class":"1","ns":["ejemplo.ns.cl."]},{"name":"ejemplo.cl.","type":"2","class":"1","ns":["ejemplo.ns.cl."]},{"name":"ejemplo.cl.","type":"2","class":"1","ns":["ejemplo.ns.cl."]},{"name":"ejemplo.cl.","type":"2","class":"1","ns":["ejemplo.ns.cl."]},{"name":"ejemplo.cl.","type":"2","class":"1","ns":["ejemplo.ns.cl."]}]}

Speedy writes its data to the standard output, ready to be redirected to our next module: Fievel, the data preprocessing module.

Fievel: Data Preprocessor

Fievel is RaTA DNS’s data preprocessing module. Its objectives are to reduce the bandwidth occupied by the system and distribute the processing work across multiple monitors. It is a programmable module, where you can write different preliminary reducers, which are the real responsible for data processing.

To work, a window size and a preliminar reducers list is configured. Also, each preliminar reducer has its own configuration section, where it is defined which channel will be used to send the already preprocessed data. These channels may be the standard output, a file or a Redis channel.

In Fievel’s mainloop, each JSON message sent by Speedy is received, and then it is transformed into a data structure known by preliminary reducers. They take the data structure and do their work. When all the packets that make a window are received, the mainloop tells the preliminar reducers to restart their state.

Multiple preliminar reducers are already coded. For example, a queries per second calculator, a malformed queries aggregator, a domain names counter, a queries summary grouped by host aggregator, etc. Some of them are showed below.

Queries Name Counter:

{
  "type": "QueriesNameCounter",
  "serverId": "server1",
  "data": {
    "www.ejemplo1.cl.": 5,
    "www.ejemplo2.cl.": 3,    
    "ns.ejemplo3.cl.": 4,
    "ejemplo4.cl.": 1,
  }
}

Queries Per Second:

{
  "type": "QueriesPerSecond",
  "serverId": "server1",
  "data": {
    "qps": 426.08993504359
  }
}

Queries With Underscored Name:

{
  "type": "QueriesWithUnderscoredName",
  "serverId": "server1",
  "data": {
    "malformed_query.domainname.cl.": [
      {
        "query": {
          "qclass": "1",
          "qtype": "1",
          "qname": "MALFORMED_QUERY.domainname.cl."
        },
        "sender": "be347435",
        "server": "c8107010"
      }
    ]
  }
}

Jerry: Aggregator and Information Visualization

Jerry keeps listening several Redis channels, one for each PreR. Multiple monitors send their data through the same channels. Jerry extracts the data, and start to process it. With this, aggregated data may be queried by the visualization. More details will came in the future, with its own blog entry.

RaTA DNS as a Forensic Tool

One of the RaTA DNS objectives is to study the behaviour that attackers had with the monitored services, somewhere in the past. With this, attacks patterns may be studied to know how to react in the future.

To do this, it is required to permanently store DNS packets. Once this files are obtained, the tcpreplay tool is used to repeat the attack. With this tool, you can replay the captured network traffic, configuring even how many times faster do you want to repeat the simulation.

More details on this will be discussed later, too.

Since our last blog entry the architectural design of RaTA DNS has changed. We needed to reduce the network traffic generated by our Packet Parser -currently named Speedy, after “Speedy Gonzalez”. To do this, we designed and implemented Fievel, a new module which’s main purpose is to perform preliminar packet filtering-transformation.

So, lets summarize what we are doing here. Think we have a DNS server and we want to have a realtime monitoring tool. A way that doesn’t add overhead to the server is to do port mirroring to an interface connected to an analyzer machine, just by configuring the switch. In this machine we have our RaTA DNS pipeline working: we are sniffing DNS packets with tcpdump, then we serialize the packets into a slightly-modified-JSON formatted message with the help of Speedy, following with the preliminar packet filtering-transformation given by Fievel. Fievel has several preliminar filtering-transformers (PFT) inside, each of these knows how to serialize a processed packet window. For example, a PFT may be configured to send the processed window directly to a visualization module or to a data-aggregation module on another machine.

What does Fievel exactly do?

To start running Fievel we have to configure L, a list of PFTs, and W, its windows size parameter: the number of packets that the module is going to analyze before it reset the complete state of the program. Once a serialized DNS packet is received from Speedy, Fievel’s main loop send this packet to every preliminar filtering-transformer in L. When Fievel receive the W-th packet, it tells each PFT to serialize the processed window and to reset it state.

And what is this PFT thing?

A PFT is an object that stores some state, knows how to process one packet at a time, knows what to do when Fievel received W packets, and knows how to reset its own state. A simple example of a PFT is our Packets Per Seconds showed below:

from prer import PreR
import time

class PacketsPerSecond(PreR):
    def __init__(self, f):
        PreR.__init__(self, f)
        self.counter = 0
        self.start = time.time()

    def __call__(self, d):
        self.counter += 1

    def get_data(self):
        data = { 'pps' : self.counter / (time.time() - self.start) }
        return data

    def reset(self):
        self.counter = 0
        self.start = time.time()

So that’s it! All our code is publicly available under a MIT license. If you have any suggested PFT, or any suggestion in general don’t hesitate to contact us!