Network Traffic Analysis
Network Traffic Analysis (NTA) can be described as the act
of examining network traffic to characterize common ports and protocols
utilized, establish a baseline for our environment, monitor and respond
to threats, and ensure the greatest possible insight into our
organization's network. This process helps security specialists determine anomalies, including
security threats in the network, early and effectively pinpoint threats.
Network Traffic Analysis can also facilitate the process of meeting
security guidelines. Attackers update their tactics frequently to avoid
detection and leverage legitimate credentials with tools that most
companies allow in their networks, making detection and, subsequently,
response challenging for defenders. Everyday use cases of NTA include: Collecting information, Identifying and Detecting malicious activity. The list below contains many different tools and equipment types that can be utilized to perform network traffic analysis. Each will provide a different way to capture or dissect the traffic. Some offer ways to copy and capture, while others read and ingest. This module will explore just a few of these (Wireshark and tcpdump mostly). Keep in mind these tools are not strictly geared for admins. Many of these can be used for malicious reasons as well.
Common Traffic Analysis Tools
| Tool | Description |
|---|---|
tcpdump |
tcpdump is a command-line utility that, with the aid of LibPcap, captures and interprets network traffic from a network interface or capture file. |
Tshark |
TShark is a network packet analyzer much like TCPDump. It will capture packets from a live network or read and decode from a file. It is the command-line variant of Wireshark. |
Wireshark |
Wireshark is a graphical network traffic analyzer. It captures and decodes frames off the wire and allows for an in-depth look into the environment. It can run many different dissectors against the traffic to characterize the protocols and applications and provide insight into what is happening. |
NGrep |
NGrep is a pattern-matching tool built to serve a similar function as grep for Linux distributions. The big difference is that it works with network traffic packets. NGrep understands how to read live traffic or traffic from a PCAP file and utilize regex expressions and BPF syntax. This tool shines best when used to debug traffic from protocols like HTTP and FTP. |
tcpick |
tcpick is a command-line packet sniffer that specializes in tracking and reassembling TCP streams. The functionality to read a stream and reassemble it back to a file with tcpick is excellent. |
Network Taps |
Taps (Gigamon, Niagra-taps) are devices capable of taking copies of network traffic and sending them to another place for analysis. These can be in-line or out of band. They can actively capture and analyze the traffic directly or passively by putting the original packet back on the wire as if nothing had changed. |
Networking Span Ports |
Span Ports are a way to copy frames from layer two or three networking devices during egress or ingress processing and send them to a collection point. Often a port is mirrored to send those copies to a log server. |
Elastic Stack |
The Elastic Stack is a culmination of tools that can take data from many sources, ingest the data, and visualize it, to enable searching and analysis of it. |
SIEMS |
SIEMS (such as Splunk)
are a central point in which data is analyzed and visualized. Alerting,
forensic analysis, and day-to-day checks against the traffic are all
use cases for a SIEM. |
| and others. |
Addressing Mechanisms
Now that we have gone over the basic concepts driving networking behavior let us take some time to discuss the addressing mechanisms that enable the delivery of our packets to the correct hosts. We will begin with Media Access Control addresses first.
MAC-Addressing
Each logical or physical interface attached to a host has a Media Access Control (MAC) address. This address is a 48-bit six octet address represented in hexadecimal format. If we look at the image below, we can see an example of one .
IP Addressing
IPv4
The most common addressing mechanism most are familiar with is the Internet Protocol address version 4 (IPv4).
IPv4 addressing is the core method of routing packets across networks
to hosts located outside our immediate vicinity. The image below shows
us an example of an IPv4 address.
IPv6
After a little over a decade of utilizing IPv4, it was determined
that we had quickly exhausted the pool of usable IP addresses. IPv6 provides us a much larger address space that can be utilized for any networked purpose. IPv6 is a 128-bit address 16 octets represented in Hexadecimal format. We can see an example of a shortened IPv6 address in the image below.
TCP / UDP, Transport Mechanisms
TCP vs. UDP
Let us take a second to examine these two protocols side by side.
TCP VS. UDP
| Characteristic | TCP | UDP |
|---|---|---|
Transmission |
Connection-oriented | Connectionless. Fire and forget. |
Connection Establishment |
TCP uses a three-way handshake to ensure that a connection is established. | UDP does not ensure the destination is listening. |
Data Delivery |
Stream-based conversations | packet by packet, the source does not care if the destination is active |
Receipt of data |
Sequence and Acknowledgement numbers are utilized to account for data. | UDP does not care. |
Speed |
TCP has more overhead and is slower because of its built-in functions. | UDP is fast but unreliable. |
HTTP Methods
To perform operations such as fetching webpages, requesting items for download, or posting your most recent tweet all require the use of specific methods. These methods define the actions taken when requesting a URI. Methods:
| Method | Description |
|---|---|
HEAD |
required is a safe method that requests a response from
the server similar to a Get request except that the message body is not
included. It is a great way to acquire more information about the
server and its operational status. |
GET |
required Get is the most common method used. It requests information and content from the server. For example, GET http://10.1.1.1/Webserver/index.html requests the index.html page from the server based on our supplied URI. |
POST |
optional Post is a way to submit information to a
server based on the fields in the request. For example, submitting a
message to a Facebook post or website forum is a POST action. The actual
action taken can vary based on the server, and we should pay attention
to the response codes sent back to validate the action. |
PUT |
optional Put will take the data appended to the message
and place it under the requested URI. If an item does not exist there
already, it will create one with the supplied data. If an object already
exists, the new PUT will be considered the most up-to-date, and the
object will be modified to match. The easiest way to visualize the
differences between PUT and POST is to think of it like this; PUT will
create or update an object at the URI supplied, while POST will create
child entities at the provided URI. The action taken can be compared
with the difference between creating a new file vs. writing comments
about that file on the same page. |
DELETE |
optional Delete does as the name implies. It will remove the object at the given URI. |
TRACE |
optional Allows for remote server diagnosis. The
remote server will echo the same request that was sent in its response
if the TRACE method is enabled. |
OPTIONS |
optional The Options method can gather information on
the supported HTTP methods the server recognizes. This way, we can
determine the requirements for interacting with a specific resource or
server without actually requesting data or objects from it. |
CONNECT |
optional Connect is reserved for use with Proxies or other security devices like firewalls. Connect allows for tunneling over HTTP. (SSL tunnels) |
FTP Commands
| Command | Description |
|---|---|
USER |
specifies the user to log in as. |
PASS |
sends the password for the user attempting to log in. |
PORT |
when in active mode, this will change the data port used. |
PASV |
switches the connection to the server from active mode to passive. |
LIST |
displays a list of the files in the current directory. |
CWD |
will change the current working directory to one specified. |
PWD |
prints out the directory you are currently working in. |
SIZE |
will return the size of a file specified. |
RETR |
retrieves the file from the FTP server. |
QUIT |
ends the session. |
This is not an exhaustive list of the possible FTP control commands that could be seen. These can vary based on the FTP application or shell in use.
SMB) is a protocol most widely seen
in Windows enterprise environments that enables sharing resources
between hosts over common networking architectures. SMB is a
connection-oriented protocol that requires user authentication from the
host to the resource to ensure the user has correct permissions to use
that resource or perform actions. In the past, SMB utilized NetBIOS as
its transport mechanism over UDP ports 137 and 138. Since modern
changes, SMB now supports direct TCP transport over port 445, NetBIOS
over TCP port 139, and even the QUIC protocol.The Analysis Process
detailed examination of an event or process,
determining its origin and impact, which can be used to trigger
specific precautions and/or actions to support or prevent future
occurrences. With network traffic, this means breaking down the data
into understandable chunks, examining it for anything that deviates from
regular network traffic, for potentially malicious traffic such as
unauthorized remote communications from the internet over RDP, SSH, or
Telnet, or unique instances preceding network issues. While performing
our analysis, we are also looking to see what the trends look like
within the traffic and determine if it matches a baseline of typical
operational traffic. Traffic analysis is a highly versatile and essential tool to have in our defensive toolbox. Analysis Dependencies
Traffic capturing and analysis can be performed in two different ways, active or passive.
Each has its dependencies. With passive, we are just copying data that
we can see without directly interacting with the packets. For active
traffic capture and analysis, the needs are a bit different. Active
capture requires us to take a more hands-on approach. This process can
also be referred to as in-line traffic captures. With both,
how we analyze the data is up to us. We can perform the capture and
analysis once done, or we can perform analysis in real-time while the
traffic is live. The table below lays out the dependencies for each.
Traffic Capture Dependencies
| Dependencies | Passive | Active | Description |
|---|---|---|---|
Permission |
☑ |
☑ |
Depending on the organization we are working in, capturing data can be against policy or even against the law in some sensitive areas like healthcare or banking. Be sure always to obtain permission in writing from someone with the proper authority to grant it to you. We may style ourselves as hackers, but we want to stay in the light legally and ethically. |
Mirrored Port |
☑ |
☐ | A switch or router network interface configured to copy data from other sources to that specific interface, along with the capability to place your NIC into promiscuous mode. Having packets copied to our port allows us to inspect any traffic destined to the other links we could normally not have visibility over. Since VLANs and switch ports will not forward traffic outside of their broadcast domain, we have to be connected to the segment or have that traffic copied to our specific port. When dealing with wireless, passive can be a bit more complicated. We must be connected to the SSID we wish to capture traffic off of. Just passively listening to the airwaves around us will present us with many SSID broadcast advertisements, but not much else. |
Capture Tool |
☑ |
☑ |
A way to ingest the traffic. A computer with access to tools like TCPDump, Wireshark, Netminer, or others is sufficient. Keep in mind that when dealing with PCAP data, these files can get pretty large quickly. Each time we apply a filter to it in tools like Wireshark, it causes the application to parse that data again. This can be a resource-intensive process, so make sure the host has abundant resources. |
In-line Placement |
☐ | ☑ |
Placing a Tap in-line requires a topology change for the network you are working in. The source and destination hosts will not notice a difference in the traffic, but for the sake of routing and switching, it will be an invisible next hop the traffic passes through on its way to the destination. |
Network Tap or Host With Multiple NIC's |
☐ | ☑ |
A computer with two NIC's, or a device such as a Network Tap is required to allow the data we are inspecting to flow still. Think of it as adding another router in the middle of a link. To actively capture the traffic, we will be duplicating data directly from the sources. The best placement for a tap is in a layer three link between switched segments. It allows for the capture of any traffic routing outside of the local network. A switched port or VLAN segmentation does not filter our view here. |
Storage and Processing Power |
☑ |
☑ |
You will need plenty of storage space and processing power for traffic capture off a tap. Much more traffic is traversing a layer three link than just inside a switched LAN. Think of it like this; When we passively capture traffic inside a LAN, it's like pouring water into a cup from a water fountain. It's a steady stream but manageable. Actively grabbing traffic from a routed link is more like using a water hose to fill up a teacup. There is a lot more pressure behind the flow, and it can be a lot for the host to process and store. |
Descriptive Analysis
Descriptive analysis is an essential step in any data analysis. It serves to describe a data set based on individual characteristics. It helps to detect possible errors in data collection and/or outliers in the data set.
-
What is the issue?- Suspected breach? Networking issue?
-
Define our scope and the goal. (what are we looking for? which time period?)- Target: multiple hosts potentially downloading a malicious file from bad.example.com
- When: within the last 48 hours + 2 hours from now.
- Supporting info: filenames/types 'superbad.exe' 'new-crypto-miner.exe'
-
Define our target(s) (net / host(s) / protocol)- Scope: 192.168.100.0/24 network, protocols used were HTTP and FTP.
Diagnostic Analysis
Diagnostic analysis clarifies the causes, effects, and interactions of conditions. In doing so, it provides insights that are obtained through correlations and interpretation. Characteristic here is a backward-looking view, as in the closely related descriptive analytics, with the subtle difference that it tries to find reasons for events and developments.
-
Capture network traffic- Plug into a link with access to the 192.168.100.0/24 network to capture live traffic to try and grab one of the executables in transfer. See if an admin can pull PCAP and/or netflow data from our SIEM for the historical data.
-
Identification of required network traffic components (filtering)- Once we have traffic, filter out any packets not needed for this investigation to include; any traffic that matches our common baseline and keep anything relevant to the scope of the investigation. For example, HTTP and FTP from the subnet, anything transferring or containing a GET request for the suspected executable files.
-
An understanding of captured network traffic- Once we have filtered out the noise, it is time to dig for our targets—filter on things like
ftp-datato find any files transferred and reconstruct them. For HTTP, we can filter onhttp.request.method == "GET"to see any GET requests that match the filenames we are searching for. This can show us who has acquired the files and potentially other transfers internal to the network on the same protocols.
- Once we have filtered out the noise, it is time to dig for our targets—filter on things like
Predictive Analysis
By evaluating historical and current data, predictive analysis creates a predictive model for future probabilities. Based on the results of descriptive and diagnostic analyses, this method of data analysis makes it possible to identify trends, detect deviations from expected values at an early stage, and predict future occurrences as accurately as possible.
-
Note-taking and mind mapping of the found results- Annotating everything we do, see, or find throughout the investigation is crucial. Ensure we are taking ample notes, including:
- Timeframes we captured traffic during.
- Suspicious hosts within the network.
- Conversations containing the files in question. ( to include timestamps and packet numbers)
-
Summary of the analysis (what did we find?)- Finally, summarize what we have found explaining the relevant details so that superiors can decide to quarantine the affected hosts or perform more significant incident response.
- Our analysis will affect decisions made, so it is essential to be as clear and concise as possible.
By performing an evaluation of the data we have found, comparing it to our baseline traffic, and known bad data such as markers of infiltration or exploitation (like signatures for viruses and other hacking tools), we are performing Predictive Analysis. In this process, we paint a clear picture so that appropriate actions can be taken in response.
Predictive Analysis
By evaluating historical and current data, predictive analysis creates a predictive model for future probabilities. Based on the results of descriptive and diagnostic analyses, this method of data analysis makes it possible to identify trends, detect deviations from expected values at an early stage, and predict future occurrences as accurately as possible.
-
Note-taking and mind mapping of the found results- Annotating everything we do, see, or find throughout the investigation is crucial. Ensure we are taking ample notes, including:
- Timeframes we captured traffic during.
- Suspicious hosts within the network.
- Conversations containing the files in question. ( to include timestamps and packet numbers)
-
Summary of the analysis (what did we find?)- Finally, summarize what we have found explaining the relevant details so that superiors can decide to quarantine the affected hosts or perform more significant incident response.
- Our analysis will affect decisions made, so it is essential to be as clear and concise as possible.
-
What is the issue? - Define our scope and the goal (what are we looking for? which time period?)
-
Define our target(s) (net / host(s) / protocol) - Capture network traffic
-
Identification of required network traffic components (filtering) -
An understanding of captured network traffic -
Note-taking and mind mapping of the found results. - Summary of the analysis (what did we find?)
Often this process is not a once-and-done kind of thing. It is usually cyclic, and we will need to rerun steps based on our analysis of the original capture to build a bigger picture. This could have been a much larger attack than what is in the examples. Suppose a full-scale incident response is deemed necessary.
Tcpdump Fundamentals
Tcpdump is a command-line packet sniffer that can directly
capture and interpret data frames from a file or network interface. It
was built for use on any Unix-like operating system and had a Windows
twin called WinDump.TCPDump is available for most Unix systems and Unix derivatives, such as
AIX, BSD, Linux, Solaris, and is supplied by many manufacturers already
in the system. Due to the direct access to the hardware, we need the root or the administrator's privileges to run this tool. we should first familiarize ourselves with the tool's essential
features. Let us discuss some basic TCPDump options, demo some commands,
and show how to save traffic to PCAP files and read from these. Basic Capture Options
Below is a table of basic Tcpdump switches we can use to modify how our captures run. These switches can be chained together to craft how the tool output is shown to us in STDOUT and what is saved to the capture file. This is not an exhaustive list, and there are many more we can use, but these are the most common and valuable.
| Switch Command | Result |
|---|---|
| D | Will display any interfaces available to capture from. |
| i | Selects an interface to capture from. ex. -i eth0 |
| n | Do not convert addresses (i.e., host addresses, port numbers, etc.) to names. |
| e | Will grab the ethernet header along with upper-layer data. |
| X | Show Contents of packets in hex and ASCII. |
| XX | Same as X, but will also specify ethernet headers. (like using Xe) |
| v, vv, vvv | Increase the verbosity of output shown and saved. |
| c | Grab a specific number of packets, then quit the program. |
| s | Defines how much of a packet to grab. |
| S | change relative sequence numbers in the capture display to absolute sequence numbers. (13248765839 instead of 101) |
| q | Print less protocol information. |
| r file.pcap | Read from a file. |
| w file.pcap | Write into a file |
-w will write our capture to a file. Keep in mind
that as we capture traffic off the wire, we can quickly use up open disk
space and run into storage issues if we are not careful. The larger our
network segment, the quicker we will use up storage. EX: sudo tcpdump -i eth0 -w ~/output.pcapTcpdump Packet Filtering
Tcpdump provides a robust and efficient way to parse the data included in our captures via packet filters. This section will examine those filters and get a glimpse at how it modifies the output from our capture. Utilizing more advanced filtering options like those listed below will enable us to trim down what traffic is printed to output or sent to file. It is highly recommended to explore the more advanced filters and find different combinations.
Helpful TCPDump Filters
| Filter | Result |
|---|---|
| host | host will filter visible traffic to show anything involving the designated host. Bi-directional |
| src / dest | src and dest are modifiers. We can use them to designate a source or destination host or port. |
| net | net will show us any traffic sourcing from or destined to the network designated. It uses / notation. |
| proto | will filter for a specific protocol type. (ether, TCP, UDP, and ICMP as examples) |
| port | port is bi-directional. It will show any traffic with the specified port as the source or destination. |
| portrange | portrange allows us to specify a range of ports. (0-1024) |
| less / greater "< >" | less and greater can be used to look for a packet or protocol option of a specific size. |
| and / && | and && can be used to concatenate two different filters together. for example, src host AND port. |
| or | or allows for a match on either of two conditions. It does not have to meet both. It can be tricky. |
| not | not is a modifier saying anything but x. For example, not UDP. |
Interpreting Tips and Tricks
Using the -S switch will display absolute sequence
numbers, which can be extremely long. Typically, tcpdump displays
relative sequence numbers, which are easier to track and read. However,
if we look for these values in another tool or log, we will only find
the packet based on absolute sequence numbers. For example,
13245768092588 to 100.
The -v, -X, and -e switches can help you increase the amount of data captured, while the -c, -n, -s, -S, and -q switches can help reduce and modify the amount of data written and seen.
Many handy options that can be used but are not always directly valuable for everyone are the -A and -l switches. A will show only the ASCII text after the packet line, instead of both ASCII and Hex. L will tell tcpdump to output packets in a different mode. L will line buffer instead of pooling and pushing in chunks. It allows us to send the output directly to another tool such as grep using a pipe |.
Tips For Analysis
Below is a list of questions we can ask ourselves during the analysis process to keep on track.
| what type of traffic do you see? (protocol, port, etc.) |
| Is there more than one conversation? (how many?) |
| How many unique hosts? |
| What is the timestamp of the first conversation in the pcap (tcp traffic) |
| What traffic can I filter out to clean up my view? |
| Who are the servers in the PCAP? (answering on well-known ports, 53, 80, etc.) |
| What records were requested or methods used? (GET, POST, DNS A records, etc.) |
0 Comments
Thanks For your comment