TCP IP
TCP/IP
TCP/IP is the foundational protocol suite of the internet. IP (Internet Protocol) handles addressing and routing — getting packets from source to destination across networks. TCP (Transmission Control Protocol) runs on top of IP and provides reliable, ordered, connection-oriented byte-stream delivery. Together they form the transport layer that HTTP, gRPC, databases, and virtually every networked application depend on.
Understanding TCP/IP matters for debugging latency, connection issues, and designing systems that handle network failures correctly.
The TCP/IP Stack
Application Layer HTTP, gRPC, WebSocket, SMTP, DNS
Transport Layer TCP (reliable) / UDP (unreliable)
Internet Layer IP (addressing + routing)
Link Layer Ethernet, Wi-Fi (physical transmission)
Each layer adds a header and passes the packet down. On the receiving end, each layer strips its header and passes the payload up.
TCP Connection: Three-Way Handshake
Before data flows, TCP establishes a connection with a three-way handshake:
Client → Server: SYN (seq=100)
Server → Client: SYN-ACK (seq=200, ack=101)
Client → Server: ACK (ack=201)
── Connection established ──
Client → Server: DATA
- SYN: client proposes a starting sequence number.
- SYN-ACK: server acknowledges and proposes its own sequence number.
- ACK: client acknowledges the server's sequence number.
This handshake adds one round-trip of latency before the first byte of data can be sent. HTTP/2 and QUIC (HTTP/3) reduce this overhead.
Reliability Mechanisms
TCP guarantees delivery through:
- Sequence numbers: every byte is numbered. The receiver reorders out-of-order segments.
- Acknowledgments (ACK): the receiver acknowledges received bytes. Unacknowledged segments are retransmitted.
- Retransmission timeout (RTO): if no ACK arrives within the timeout, the segment is retransmitted.
- Duplicate ACKs / Fast Retransmit: three duplicate ACKs signal a lost segment; TCP retransmits without waiting for the timeout.
Flow Control and Congestion Control
Flow control prevents the sender from overwhelming the receiver. The receiver advertises a receive window (how many bytes it can buffer). The sender cannot have more unacknowledged bytes in flight than the window size.
Congestion control prevents the sender from overwhelming the network. TCP starts with a small congestion window and grows it exponentially (slow start) until it detects packet loss, then backs off. Algorithms: CUBIC (Linux default), BBR (Google, latency-optimized).
Connection Teardown: Four-Way Handshake
Client → Server: FIN
Server → Client: ACK
Server → Client: FIN
Client → Server: ACK
── Connection closed ──
The TIME_WAIT state keeps the connection in memory for 2×MSL (Maximum Segment Lifetime, typically 60s) to handle delayed packets. High-throughput servers can exhaust ephemeral ports if connections are closed too frequently — use connection pooling and SO_REUSEADDR.
Pitfalls
Nagle's Algorithm Causing Latency
What goes wrong: small writes (e.g., sending a 10-byte command) are buffered by Nagle's algorithm until the buffer fills or an ACK arrives. This adds 40–200ms latency for interactive protocols.
Why it happens: Nagle's algorithm coalesces small packets to reduce network overhead. It's enabled by default.
Mitigation: disable Nagle's algorithm with TCP_NODELAY for latency-sensitive connections (database clients, real-time APIs). Most database drivers and HTTP clients do this automatically.
var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socket.NoDelay = true; // disables Nagle's algorithm
TIME_WAIT Port Exhaustion
What goes wrong: a high-throughput service opens and closes many short-lived connections. The OS runs out of ephemeral ports because they're all in TIME_WAIT.
Why it happens: each closed connection holds its port for 60s in TIME_WAIT.
Mitigation: use connection pooling (HTTP clients, database connection pools) to reuse connections instead of closing them. On Linux, tune net.ipv4.tcp_tw_reuse = 1 to allow reuse of TIME_WAIT sockets for new connections.
TCP vs UDP
| TCP | UDP | |
|---|---|---|
| Connection | Connection-oriented (handshake) | Connectionless |
| Reliability | Guaranteed delivery, ordered | Best-effort, no ordering |
| Overhead | Higher (headers, ACKs, retransmits) | Lower |
| Latency | Higher (handshake, retransmits) | Lower |
| Use cases | HTTP, databases, file transfer | DNS, video streaming, gaming, QUIC |
Decision rule: use TCP for anything where data loss is unacceptable (web APIs, databases, file transfer). Use UDP when you control reliability at the application layer or when low latency matters more than guaranteed delivery (real-time video, DNS, QUIC/HTTP/3).
Questions
- The handshake synchronizes sequence numbers and confirms both sides are reachable before sending data.
- It adds one full round-trip of latency before the first byte of application data can be sent.
- For short-lived connections (single HTTP request, DNS-over-TCP), handshake latency dominates total request time.
- HTTP/2 multiplexing amortizes the handshake across many requests on one connection. QUIC (HTTP/3) eliminates it entirely by combining transport and TLS handshakes.
- Tradeoff: the handshake guarantees reliable connection setup but costs latency — use connection pooling or QUIC when handshake overhead is unacceptable.
- Flow control protects the receiver: the receive window limits how much unacknowledged data the sender can push.
- Congestion control protects the network: the congestion window limits send rate based on detected packet loss.
- The effective send rate is the minimum of both windows.
- Tuning only one side causes problems: a large receive window with aggressive congestion control still causes network drops; a small receive window with conservative congestion control wastes available bandwidth.
- Tradeoff: flow control is endpoint-local and deterministic; congestion control is network-wide and heuristic — both must be tuned together for optimal throughput.
References
- Transmission Control Protocol (RFC 793) — the original TCP specification; defines the three-way handshake, sequence numbers, flow control, and state machine.
- TCP/IP Illustrated, Volume 1 (W. Richard Stevens) — the definitive practitioner reference for TCP/IP internals; covers every mechanism with packet traces.
- High Performance Browser Networking (Ilya Grigorik) — free online book chapter on TCP building blocks: handshake latency, slow start, congestion control, and how HTTP/2 and QUIC address TCP's limitations.
- Beej's Guide to Network Programming — practical socket programming guide covering TCP/UDP sockets, connection setup, and common pitfalls.