Network packet analysis – Part I: Theory
A brief intro to TCP/IP layers, protocols, packets, and internet communication overall
Network packet analysis can help you understand the performance and security implications of a network or application. Packets are important because the internet is a packet-switched network — as opposed to circuit-switched networks such as voice telephone networks (see a comparison table of packet vs circuit switching by GeeksforGeeks).
Information sent over the web is split into small data units (packets) where each packet includes all the information necessary for identification and routing, such as source and destination addresses and protocol identifiers.
Thus, the internet is inherently a connectionless network, which means that:
- there's no established connection between the server and the client
- packets are independent of each other
- every packet chooses the best route available for transmission (i.e. different packets can use different routes for delivery)
The composition of a network packet
Network packets have a layered structure where each layer:
- is associated with a distinct stage of network communication
- exchanges data between different hardware and software platforms belonging to that stage (e.g. routers, device drivers, network interface controllers, ports, web servers, browsers, etc.)
- is implemented by different communication protocols
What are communication protocol layers?
The TCP/IP model (also known as the Internet Protocol Suite) specifies the standards of network communication over the internet. It consists of four layers — each communication protocol belongs to one layer. The layers are as follows:
- LAYER 1 – Link (or network access) layer (protocols: Ethernet, PPP, etc.)
- LAYER 2 – Internet layer (protocols: IP, IPsec, ARP, etc.)
- LAYER 3 – Transport layer (protocols: UDP, TCP, QUIC, etc.)
- LAYER 4 – Application layer (protocols: HTTP, DNS, TLS, etc.)
The data encapsulated in a packet belongs to three or four layers — therefore, it's regulated by a set of protocols. The layers together represent a part or the whole of the TCP/IP model (e.g. an HTTP/1.1 packet also includes the TCP, IP, and Ethernet layers).
In the diagram below, you can see the structure of a theoretical packet carrying application data — in real life, the top layer would belong to an application-layer protocol, such as HTTP, DNS, TLS, etc. It uses TCP as transport protocol and also includes the IP and physical (a.k.a. MAC or Ethernet) addresses of the destination machine:
Explanation of the above diagram from the cited book:
"When an application sends data using TCP, the data is sent down the protocol stack, through each layer, until it is sent as a stream of bits across the network. Each layer adds information to the data by prepending headers (and sometimes adding trailer information) to the data that it receives. Figure 1.7 shows this process. The unit of data that TCP sends to IP is called a TCP segment. The unit of data that IP sends to the network interface is called an IP datagram. The stream of bits that flows across the Ethernet is called a frame."
How do packets flow over the network?
To see how this all works in practice, let's see a simplified example of what happens on the wire when a web page loads in the browser (for the sake of simplicity, I didn't include the TLS protocol used for encryption in HTTPS requests — the below is the description of an insecure HTTP/1.1 request that doesn't encrypt the data but transmits it as plain text):
- The user enters a URL into their browser's address bar.
- The browser (client) sends a DNS lookup request to the user's ISP (Internet Service Provider) to resolve the domain name to a numeric IP address — see the process of a DNS lookup in detail (DNS uses UDP for transport).
- The ISP sends back the IP address of the requested domain to the user's browser.
- The browser establishes a TCP connection with the web server using the IP address — sends a TCP SYN (synchronize) packet for requesting authentication (first message in the three-way TCP handshake).
- The server responds with a SYN-ACK (synchronize-acknowledgement) packet — it authenticates itself and also requests authentication from the browser (second message in the TCP handshake).
- The browser sends a TCP ACK packet (third message in the TCP handshake) together with the first HTTP request.
- The TCP socket connection between the client and server is established (further messages can be sent without authentication).
Analyzing network packets
The best way to understand how network packets move around the network is to install a packet analyzer application on your own computer that monitors the incoming and outgoing network traffic, captures packets, and visualizes them.
In the second part of this post, you'll find a brief intro to Wireshark, a popular network sniffer, and examples of packet captures of the most frequently used protocols — i.e. UDP, TCP, QUIC, TLS, HTTP, and DNS.
To understand the theoretical background of network packet analysis and communication protocols better, check out one or all of the resources below.
- W. Richard Stevens: TCP/IP Illustrated, Vol. 1: The Protocols (Addison-Wesley Professional Computing Series) – see the related Wikipedia article or have a look at the first edition at Flylib or the second edition at O'Reilly Learning (the first edition is available on Amazon, too)
- Ilya Grigorik: High Performance Browser Networking – read the ebook at hpbn.co
On this blog
- Network packet analysis – Part II: Practice – Capturing network traffic with Wireshark (with examples of HTTP, QUIC, TCP, UDP, and other packet captures)
- The most important RFCs to understand HTTP/3 and QUIC – A non-comprehensive collection of specifications related to HTTP and the underlying security and transport-layer protocols