TCP vs UDP

The 30-second version

When a Go program calls net.Dial("tcp", …) or a browser loads a web page, the data is chopped into small packetsPacketOne small chunk of data sent across a network on its own, with a header saying where it's from and where it's going. Big messages are split into many packets. that travel across the internet on their own. The internet makes no promises about them: packets get lost, arrive out of order, or arrive twice. TCP and UDP are two different answers to "what should we do about that?"

What you'll learn

How a TCP connection is opened (§03) and closed (§09); how it recovers lost data (§04); why one Write doesn't equal one Read and how to fix it (§05); how TCP decides how fast to send (§06); why one lost packet can stall a whole web page (§07); how to use UDP from Go and Node (§08); and the production traps: a mysterious 40 ms delay (§10), connection costs, timeouts and port exhaustion (§11).

What the network really gives you

Packets, not files

When your program sends 1 MB, it doesn't travel as one piece. It's cut into packetsPacketOne small chunk of data sent across a network on its own, with a header saying where it's from and where it's going. Big messages are split into many packets. of about 1,500 bytesByte8 bits — enough to store a number from 0 to 255, or one ASCII character. Network speeds are usually quoted in bits per second, file sizes in bytes. each (the MTUMTUMaximum transmission unit: the largest packet a link will carry in one piece, typically 1500 bytes on Ethernet. Bigger data gets split up. of a normal network). Each packet carries a small header with the sender's and receiver's IP addressesIP addressThe number that identifies a machine on a network, like a postal address: 93.184.216.34 (IPv4) or 2606:2800:220:1::248 (IPv6)., and travels on its own through a chain of routersRouterA device that forwards packets between networks, choosing the next hop toward each packet's destination address. Your home Wi-Fi box is one; the internet is thousands of them., each one passing it one hop closer to the destination.

That design is cheap and robust, but it has consequences. Press the button and watch what can happen to five packets:

Here's why each of those things happens:

• Lost. A router's queue is full (too much traffic at once), so it simply throws the packet away. This is normal, not an error: on a busy link, a small percentage of packets is dropped all the time.
• Reordered. Two packets can take different routes, or sit in different queues, so a later one can overtake an earlier one.
• Duplicated. Rare, but it happens: a piece of network equipment resends a packet it wrongly thought was lost, and both copies arrive.
• Damaged. Bits can flip on the way. Each packet carries a ChecksumChecksumA small number computed from a block of data and sent along with it. The receiver recomputes it; if the two don't match, the data was damaged in transit. Also used to check that two copies of data are identical., and a damaged packet is dropped, which turns "damaged" into "lost".

The network doesn't tell anyone about any of this. This "do your best, promise nothing" service is called best effort.

Ports: which program gets the packet?

An IP address gets a packet to the right machine. But one machine runs many programs: a web server, a database, your browser. A PortPortA number from 0 to 65535 that picks which program on a machine should get the traffic — like an apartment number after the street address. HTTPS servers usually listen on port 443. number (0–65535) picks the program, like an apartment number after the street address. A web server usually listens on port 443; Postgres on 5432.

Both TCP and UDP put the source port and destination port at the start of each packet's PayloadPayloadThe actual data a message carries, as opposed to the headers wrapped around it for delivery.. That's the one thing they have in common. Everything after that is different.

Sockets: your program's end of it

Your code never touches packets directly. It asks the operating system (the Operating system (OS) / kernelOperating system (OS) / kernelThe core program (Linux, macOS, Windows) that manages hardware, memory, threads, files and network connections for every other program. The kernel is its innermost part.) for a SocketSocketThe OS's handle for one open network connection. Your program reads and writes bytes through it.: a handle you write bytes into and read bytes out of. The OS kernel then does all the TCP or UDP work (numbering, resending, waiting) behind that handle.

tcpConn, err := net.Dial("tcp", "example.com:80")   // a TCP connection
udpConn, err := net.Dial("udp", "8.8.8.8:53")       // a UDP "connection" (just a remembered address)

Opening a connection: the three-way handshake

Segments and sequence numbers, first

TCP calls each chunk it sends a segmentSegment (TCP)One chunk of a TCP byte stream, sent inside one IP packet: a small TCP header (ports, sequence number, ACK number, flags, window) followed by up to ~1460 bytes of your data.: a small header plus some of your data, sent inside one packet. Two header fields matter most:

• The sequence numberSequence numberA number in every TCP segment that says where its first byte sits in the stream. Each side starts from a random number (its ISN, initial sequence number) and counts up by one per byte sent. The receiver uses it to put bytes back in order and spot gaps. says where this segment's first byte sits in the stream. If I start at 1000 and send 100 bytes, they're bytes 1000–1099, and my next segment starts at 1100.
• The ACK numberACK (acknowledgement)A TCP receiver's “got it” message. ack=N means: “I have every byte before N; the next byte I expect is N.” One ACK covers everything before N, not just the last segment. is the receiver's reply: "ack=1100" means "I have everything before 1100; send 1100 next."

There are also a few flagsTCP flags (SYN, ACK, FIN, RST)On/off bits in the TCP header that say what a segment is for. SYN = “let's start, here's my starting sequence number”; ACK = the ack number field is valid; FIN = “I've finished sending”; RST = “abort this connection right now”., on/off bits that say what a segment is for. The handshake uses two: SYN ("let's synchronize our numbers") and ACK ("my ack number is meaningful").

The handshake, step by step

Step through it. Watch the numbers: each side picks its own starting number, and the other side acknowledges it by replying with that number plus one.

Why three messages, not two? Each side needs to hear "I got your starting number." The client's SYN gets acknowledged in step 2, the server's SYN in step 3. Step 2 does two jobs at once (the server's ACK of the client and the server's own SYN), which is why it's three messages and not four.

The cost: one round trip

The client can't send data until the SYN-ACK comes back. That's one round-trip timeRound-trip time (RTT)How long it takes for a message to go from one machine to another and for the reply to come back. Roughly 1 ms inside a data center, 70–150 ms across an ocean., the time for a message to go there and the reply to come back. Some real numbers (approximate):

Add TLSTLSTransport Layer Security: the encryption layer that makes HTTPS secure. It proves the server is who it claims to be and scrambles everything so nobody in between can read or change it. for HTTPS and you pay another round trip or more on top (see TLS). This is the main reason production systems reuse connections instead of opening a new one per request (§11).

In code

The OS keeps track of each connection's stateConnection state (TCP)The stage the OS records for each TCP connection: LISTEN, SYN_SENT, ESTABLISHED, FIN_WAIT, TIME_WAIT, CLOSED and so on. The ss command shows it for every socket. (LISTEN, ESTABLISHED, …), like the labels in the widget above; ss shows them (end of this section).

From your program, the whole handshake is one call. On the server, the OS completes handshakes on its own and queues the finished connections; Accept just takes the next one.

These are two separate programs: run the server in one terminal, the client in another.

ln, err := net.Listen("tcp", ":8080") // state: LISTEN
if err != nil {
    log.Fatal(err)
}
for {
    conn, err := ln.Accept() // returns a connection whose handshake is already done
    if err != nil {
        continue
    }
    go handle(conn) // one goroutine per connection
}

If nobody listens on that port, the server's OS answers the SYN with an RSTRST (reset)A TCP segment that kills a connection immediately, with no polite goodbye. Sent when a packet arrives for a port nobody listens on, or a program aborts a connection. Your code sees “connection refused” or “connection reset by peer”. segment ("no such connection") and your Dial fails immediately with connection refused. If the machine doesn't answer at all (it's down, or a FirewallFirewallA filter that allows or blocks network traffic by rules — e.g. “only port 443 from the internet”. silently drops the SYN), the client resends its SYN with growing gaps, and on Linux gives up only after about 2 minutes. Always set a connect timeout (§11).

See it for real

tcpdumptcpdump / WiresharkTools that capture and display every packet going through a network interface. tcpdump prints them in the terminal; Wireshark shows them in a GUI with each header field decoded. prints every packet. Run it in one terminal, then curl http://example.com in another:

sudo tcpdump -n -i any 'tcp port 80'
# client > server: Flags [S],  seq 1983547913                       ← SYN
# server > client: Flags [S.], seq 2477113491, ack 1983547914       ← SYN-ACK ("." = ACK)
# client > server: Flags [.],  ack 1                                ← ACK (tcpdump now shows relative numbers)
# client > server: Flags [P.], seq 1:76, ack 1, length 75           ← the HTTP request (P = push: hand it to the app now)

And ssss / netstatCommand-line tools that list a machine's sockets and their TCP states (LISTEN, ESTAB, TIME-WAIT, …). ss is the modern Linux one; netstat is older and on most systems. lists your machine's connections with their state:

ss -tan            # -t TCP, -a all (incl. listening), -n numeric ports
# State    Local Address:Port    Peer Address:Port
# LISTEN   0.0.0.0:8080          0.0.0.0:*
# ESTAB    10.0.0.5:51234        93.184.216.34:80

Reliability: ACKs, timers and retransmission

The two pieces

To keep things readable, from here on we number whole segments 1, 2, 3… instead of bytes. Real TCP numbers bytes (§03), but the logic is identical.

Cumulative ACKs. An ACKACK (acknowledgement)A TCP receiver's “got it” message. ack=N means: “I have every byte before N; the next byte I expect is N.” One ACK covers everything before N, not just the last segment. doesn't say "I got segment 5". It says "I have every byte before N; the next one I expect is N." So if segments 1, 2 and 4 arrived, the receiver keeps answering "next I expect 3", even after 4 arrives. Those repeats are duplicate ACKsDuplicate ACKAn ACK that repeats the same number as the previous one. The receiver sends it when a segment arrives after a gap: “thanks, but I'm still waiting for byte N”..

A receive buffer. Data that arrives early (segment 4 while 3 is missing) waits in the OS's socket bufferSocket bufferMemory inside the OS kernel for each connection: a send buffer (bytes you wrote that aren't acknowledged yet) and a receive buffer (bytes that arrived but your program hasn't read yet).. Your program's Read never sees it until the gap is filled. This is how TCP turns "1, 2, 4, 5, 3" back into "1, 2, 3, 4, 5".

Two ways to notice a loss

1. The timer. When the sender sends a segment, it starts a retransmission timeoutRetransmission timeout (RTO)How long TCP waits for an ACK before it assumes a segment was lost and sends it again. It's computed from measured round-trip times (Linux never goes below 200 ms) and doubles after each failed retry. (RTO). If no ACK covers it before the timer fires, it resends. The RTO is based on measured round trips, so it adapts: about 200 ms is the Linux minimum, and it doubles on every failed retry.
2. Three duplicate ACKs. If the sender keeps getting "next I expect 3" while it has already sent 4, 5 and 6, then 3 is almost certainly lost. It resends 3 right away, without waiting for the timer. That's fast retransmitFast retransmitTCP's shortcut for loss: after 3 duplicate ACKs for the same byte, the sender resends that segment at once instead of waiting for the retransmission timeout.. Why three and not one? Packets are sometimes just reordered; one or two duplicates might only mean 3 is a little late.

Try it. Start with the defaults (lose #3, fast retransmit on), then switch to timeout only, then lose #6:

What you should have seen, with numbers (one-way trip 50 ms, RTO 300 ms in this sim):

That last row is a real production effect: losing the last packet of a response (a "tail loss") is much slower to repair than losing one in the middle. Modern Linux sends a small probe early to cut this down, but the shape stays the same.

Sending without waiting

Notice the sender didn't wait for ACK 1 before sending 2. If it did ("stop and wait"), a 100 ms RTT would limit it to one segment per 100 ms: about 15 KB/s, whatever the link speed. Instead, TCP keeps many segments in flightIn flight (bytes)Data that has been sent but not yet acknowledged. It might be on the wire, sitting in a router's queue, or lost; the sender can't tell yet. at once: each returning ACK lets it send another. Compare:

How many segments to keep in flight is the subject of §06.

TCP is a byte stream: the framing problem

The surprise

TCP is a byte streamByte streamA connection that delivers one long sequence of bytes with no built-in message boundaries. TCP is a byte stream: three writes of 10 bytes may arrive as one read of 30, or as reads of 7 and 23.: think of a water pipe, not a conveyor belt of boxes. You pour bytes in at one end; they come out at the other in the same order, but the pipe doesn't remember how you poured them. Here is a client that writes "hello" and then "world", and a server that reads once a moment later. Both programs were run on Go 1.24 and Node 22:

ln, err := net.Listen("tcp", "127.0.0.1:0") // port 0 = "pick any free port"
if err != nil {
    log.Fatal(err)
}

go func() { // client, in its own goroutine
    conn, _ := net.Dial("tcp", ln.Addr().String())
    conn.Write([]byte("hello"))
    conn.Write([]byte("world"))
    conn.Close()
}()

conn, _ := ln.Accept()             // server
time.Sleep(100 * time.Millisecond) // let both writes arrive first
buf := make([]byte, 1024)
n, _ := conn.Read(buf)             // ← one read…
fmt.Printf("read %d bytes: %q\n", n, buf[:n])
// read 10 bytes: "helloworld"     ← …gets both writes, glued together

It also goes the other way: a 100 KB write arrives as many reads of whatever size the network and the OS felt like (often 1–64 KB). The splits depend on timing, packet sizes and buffer sizes, so code that assumes one write = one read passes every test on your laptop (loopbackLoopback (127.0.0.1, localhost)A fake network interface that only reaches the same machine — packets sent to it never leave. 127.0.0.1 (or ::1 in IPv6) means “this computer”, and “localhost” is its name. is fast and tidy) and then breaks in production, randomly.

The fix: framing

FramingFramingMarking where each message starts and ends inside a byte stream, so the receiver can cut it back into messages: e.g. put each message's length in front of it, or end each one with a newline. means putting markers in the stream so the receiver can cut it back into messages. The two common ways:

• Length prefix. Before each message, write its length as a fixed-size number (say 4 bytes). The receiver reads 4 bytes, learns "the next message is 5 bytes", reads exactly 5 more. Used by gRPC, Postgres, Kafka, HTTP/2.
• Delimiter. End each message with a special byte sequence that never appears inside a message, usually a newline (\n, or \r\n in many older protocols). Simple and human-readable. Used by SMTP, Redis (partly), line-delimited JSON, and HTTP/1.1's headers.

Try all three readers against different chunkings:

Length prefix in Go and Node

// writeMsg sends one message: a 4-byte length, then the bytes.
func writeMsg(w io.Writer, msg []byte) error {
    var hdr [4]byte
    binary.BigEndian.PutUint32(hdr[:], uint32(len(msg))) // length → 4 bytes
    if _, err := w.Write(hdr[:]); err != nil {
        return err
    }
    _, err := w.Write(msg)
    return err
}

// readMsg reads exactly one message, however the bytes were split up.
func readMsg(r io.Reader) ([]byte, error) {
    var hdr [4]byte
    if _, err := io.ReadFull(r, hdr[:]); err != nil { // ← waits for all 4 bytes
        return nil, err
    }
    n := binary.BigEndian.Uint32(hdr[:])
    if n > 1<<20 { // refuse anything over 1 MiB: never trust a length from the network
        return nil, fmt.Errorf("message too big: %d bytes", n)
    }
    msg := make([]byte, n)
    _, err := io.ReadFull(r, msg) // ← waits for all n bytes
    return msg, err
}

// usage: wrap the connection in a buffered reader, then loop
r := bufio.NewReader(conn)
for {
    msg, err := readMsg(r)
    if err != nil {
        return // io.EOF = the other side closed cleanly
    }
    fmt.Printf("message: %q\n", msg)
}

Delimiter framing in Go

For newline-separated messages, Go's bufio.Scanner does all of it for you:

sc := bufio.NewScanner(conn) // splits on '\n' by default
for sc.Scan() {
    fmt.Printf("line: %q\n", sc.Text()) // one complete line, without the '\n'
}
if err := sc.Err(); err != nil { // nil at a clean end of stream
    log.Println("read error:", err)
}
// even if the client sent "hello\nwor" then "ld\n", this prints "hello" then "world"

In Node, readline.createInterface({ input: sock }) gives you a "line" event per line. A Scanner refuses lines over 64 KB by default (bufio.Scanner: token too long); call sc.Buffer(…) to raise it.

UDP doesn't have this problem at all: every read returns exactly one whole message (§08). It has other problems instead.

How fast to send: flow control and congestion control

§04 showed that TCP keeps several segments in flightIn flight (bytes)Data that has been sent but not yet acknowledged. It might be on the wire, sitting in a router's queue, or lost; the sender can't tell yet. at once. Too few, and a fast link sits idle. Too many, and something overflows. There are two different things that can overflow, so there are two limits.

Flow control: don't drown the receiver

The receiving program may be slow: busy, or just reading slowly. Unread bytes pile up in its socket bufferSocket bufferMemory inside the OS kernel for each connection: a send buffer (bytes you wrote that aren't acknowledged yet) and a receive buffer (bytes that arrived but your program hasn't read yet).. So every ACK also carries the receive windowReceive window (rwnd)How many more bytes the receiver has room for in its buffer. It's sent in every ACK, and the sender may never have more unacknowledged bytes in flight than this. That's flow control: don't overwhelm the receiver.: "I have room for 48 KB more." The sender never has more unacknowledged data out than that.

If the receiver stops reading entirely, the window shrinks to zero and the sender stops. In Go, the sender's Write then blocks once its own send buffer is full; in Node, write() starts returning false. That's TCP's built-in BackpressureBackpressurePushing back on whoever sends you work when you're full — making them wait or rejecting them — instead of piling it up in memory., and it's why a slow consumer slows the producer down instead of crashing it. Worker Pools §06 shows how to pass that signal through your own code.

Congestion control: don't drown the network

Between the two machines are routers shared with everyone else. If every sender blasts at full speed, router queues overflow and packets get dropped, which causes retransmissions, which add more traffic. In 1986 this actually happened: parts of the early internet slowed to about 1/1000th of their capacity.

The fix (by Van Jacobson, 1988) is that each sender keeps a congestion windowCongestion window (cwnd)The sender's own guess at how much data the network path can carry at once without dropping it. The sender keeps at most min(cwnd, rwnd) unacknowledged bytes in flight. That's congestion control: don't overwhelm the network. (cwnd): its own guess at how much the path can take. Nobody tells it the right number; it probes:

1. Slow startSlow startHow a new TCP connection finds its speed: start with a small congestion window (10 segments on Linux) and double it every round trip until something is lost. “Slow” is a historical joke; the growth is exponential.: start small (10 segments, about 14 KB, on Linux) and double every round trip: 10, 20, 40, 80…
2. When a loss shows up, the guess was too high. Remember half of it as the slow-start thresholdssthresh (slow-start threshold)The congestion-window size where TCP stops doubling (slow start) and switches to growing by one segment per round trip. After a loss it's set to about half the window at the time of the loss..
3. From then on, grow by just +1 segment per round trip, and halve on each loss. This phase is called congestion avoidanceCongestion avoidanceThe phase after slow start: the congestion window grows by only one segment per round trip, carefully probing for spare room instead of doubling.; the grow-slowly, cut-sharply rule is called AIMDAIMDAdditive increase, multiplicative decrease: grow a limit slowly (+1) while things go well, cut it sharply (e.g. halve it) when you see trouble. TCP uses it to find a network's speed., and it draws a sawtooth. This classic version of the algorithm is known as RenoReno (TCP)The classic textbook congestion-control algorithm: slow start, then +1 segment per round trip, and halve the window on each loss. It draws the well-known sawtooth; modern stacks use refinements like CUBIC..

Run it with the real path (it drops packets above 60 segments), then try the buttons:

Two kinds of "loss"

• 3 duplicate ACKs (§04): one segment went missing, but later ones are still arriving, so the network is mostly working. Halve the window and carry on.
• A retransmission timeout: nothing came back at all. That could be heavy congestion, so start over from 1 segment and slow-start again. One timeout can cost a connection seconds of throughput.

Put both together: the sender may have at most min(cwnd, rwnd) bytes in flight. Usually rwnd is large and cwnd is the real limit.

Why this matters for your code

A brand-new connection is slow. It starts at 10 segments and needs several round trips to ramp up. A 1 MB download (about 700 segments) over a fresh connection with 100 ms RTT sends 10, 20, 40, 80, 160, 320… segments per round trip, so it needs about 7 round trips, ≈ 0.7 s. A warmed-up connection whose window is already large could do it in one or two. A warmed-up, reused connection skips all of that. This, plus the handshake, is why connection poolsConnection poolA set of open database (or HTTP) connections that your program keeps and reuses instead of opening a new one per request. Pools usually have a size limit; when every connection is busy, the next caller waits for one to be returned. matter (§11).

Head-of-line blocking: one loss stalls everything

That's Head-of-line blockingHead-of-line blockingOne slow item at the front of a queue holds up everything behind it, even items that would be quick.: the item at the front of the line is stuck, so everyone behind it waits. With TCP it doesn't matter much when the connection carries one thing (one file download): you need all the bytes anyway. It matters a lot when one connection carries several independent things.

The HTTP/2 problem

HTTP/2 does exactly that. To avoid opening many connections (each costing a handshake and slow start), it sends all of a page's requests over one TCP connection at the same time, chopping each response into small pieces and interleaving them. That's multiplexingMultiplexingSending several independent conversations over one shared channel at the same time, by cutting them into small labelled pieces and interleaving them. HTTP/2 multiplexes many requests over one TCP connection., and it's a great idea, until a packet is lost.

TCP doesn't know about HTTP/2's streams. It sees one long byte stream. Play it, then switch to QUIC:

On a clean network this rarely matters. On a lossy one (mobile, crowded Wi-Fi, 1–2% loss), HTTP/2 can actually do worse than HTTP/1.1, which used ~6 separate TCP connections per site: a loss on one of them stalled only that one.

The fix: move the streams below the ordering

You can't fix this inside TCP: in-order delivery of one byte stream is the whole definition. So the fix was to build a new transport, QUICQUICA transport protocol built on UDP that does TCP's job (reliable, ordered delivery, congestion control) plus encryption, with independent streams so a lost packet only delays the stream it belongs to. HTTP/3 runs on it., that knows about streams. Each stream has its own ordering: a lost packet for stream A only holds up stream A.

Why build it on UDPUDPUser Datagram Protocol: sends individual packets with no connection, no ordering and no resending. Fast and simple; used by DNS, video calls, games and QUIC (HTTP/3).? Because UDP is the thinnest thing that gets packets through today's internet. Every IP packet carries a small label saying what's inside it (TCP, UDP, …), and a brand-new label would be blocked by countless firewallsFirewallA filter that allows or blocks network traffic by rules — e.g. “only port 443 from the internet”., NAT boxesNATNetwork Address Translation: a router rewrites private addresses (like 192.168.1.20) to its one public address on the way out, and back on the way in. Why your laptop isn't directly reachable from the internet. and routers that only understand TCP and UDP. So QUIC sends UDP datagramsDatagramOne self-contained message sent with UDP: it arrives whole or not at all, never merged with or split across other datagrams. Think of a postcard. and does everything TCP does (handshake, ACKs, retransmission, congestion control) itself, in the application instead of the kernel, plus TLSTLSTransport Layer Security: the encryption layer that makes HTTPS secure. It proves the server is who it claims to be and scrambles everything so nobody in between can read or change it. encryption built in. HTTP/3 is HTTP on top of QUIC.

The details of HTTP/2 and HTTP/3 (frames, header compression, connection migration) are in the HTTP topic.

UDP: datagrams, and when postcards win

A UDP datagramDatagramOne self-contained message sent with UDP: it arrives whole or not at all, never merged with or split across other datagrams. Think of a postcard. is a postcard: one message, sent on its own, delivered whole or not at all. UDP's whole header is 8 bytes: source port, destination port, length, ChecksumChecksumA small number computed from a block of data and sent along with it. The receiver recomputes it; if the two don't match, the data was damaged in transit. Also used to check that two copies of data are identical.. That's all it adds to a raw IP packet.

An echo server in Go and Node

There's no Listen + Accept and no per-client connection. The server has one socket and every datagram arrives on it, tagged with who sent it:

conn, err := net.ListenUDP("udp", &net.UDPAddr{Port: 9000}) // no Accept: one socket for everyone
if err != nil {
    log.Fatal(err)
}
defer conn.Close()

buf := make([]byte, 1500)
for {
    n, from, err := conn.ReadFromUDP(buf) // one call = one whole datagram
    if err != nil {
        return
    }
    conn.WriteToUDP(buf[:n], from) // reply to whoever sent it
}

And a client. A Go "UDP dial" sends nothing; it just remembers the address, so later Writes know where to go:

c, err := net.Dial("udp", "127.0.0.1:9000") // no handshake: nothing is sent yet
if err != nil {
    log.Fatal(err)
}
defer c.Close()

c.Write([]byte("hello"))
c.Write([]byte("world"))

c.SetReadDeadline(time.Now().Add(time.Second)) // a reply may never come: always set a deadline
buf := make([]byte, 1500)
for i := 0; i < 2; i++ {
    n, err := c.Read(buf)
    if err != nil {
        log.Println("gave up:", err) // read udp …: i/o timeout
        return
    }
    fmt.Printf("got %q\n", buf[:n]) // got "hello", then got "world": never "helloworld"
}

Notice the difference from §05: two sends arrive as two reads. UDP keeps message boundaries, so you don't need framing. The trade is that any of those messages can silently vanish, arrive twice, or arrive out of order (§02), and there's no error when that happens.

Things that surprise people

• A small buffer silently cuts the message. If a datagram is bigger than your read buffer, you get the first part and the rest is thrown away, with no error in Go. Use a buffer at least as big as your largest datagram.
• Big datagrams are a bad idea. The protocol allows up to about 64 KB (Node: send EMSGSIZE above that), but anything over the path's MTUMTUMaximum transmission unit: the largest packet a link will carry in one piece, typically 1500 bytes on Ethernet. Bigger data gets split up. (~1,500 bytes) gets split by IP into several packets called fragments, and if one fragment is lost the whole datagram is gone. Keep datagrams under ~1,200 bytes; that's what QUIC does.
• "Connection refused" can show up on a read. If nobody listens on the port, the other machine may send back an error packet, and Go reports it on your next Read as read: connection refused. Often nothing comes back at all.
• No speed control. UDP will happily send faster than the network or receiver can take; the excess is just dropped. If you send a lot, you need your own pacing.

When to choose UDP

If you need every byte, in order, use TCP. Rebuilding reliability on UDP well (ACKs, timers, congestion control, §04 and §06) is a lot of hard work; use a library like QUIC rather than inventing your own.

Closing a connection: FIN, RST and TIME_WAIT

The polite goodbye

A TCP connection is really two one-way streams: client → server and server → client. Close() sends a FIN on your direction, meaning "I'm done sending." The other side acknowledges it, and later sends its own FIN when it's done. Four segments in all. Step through it:

On the receiving side, a FIN shows up as end of stream: Go's Read returns io.EOF, Node emits "end". That's how a reader knows "there will be no more data" as opposed to "no data yet".

The rude goodbye: RST

An RSTRST (reset)A TCP segment that kills a connection immediately, with no polite goodbye. Sent when a packet arrives for a port nobody listens on, or a program aborts a connection. Your code sees “connection refused” or “connection reset by peer”. (reset) ends a connection instantly, with no FIN dance. You'll see it when:

• a packet arrives for a connection the other machine doesn't know about (e.g. it rebooted) → connection reset by peer (ECONNRESET in Node);
• a program closes a socket while unread data is still sitting in its receive buffer: the kernel sends RST instead of FIN to signal "data was thrown away";
• a Load balancerLoad balancerA component that spreads incoming requests or connections across several servers, so no single one is overloaded and a dead one can be skipped. or FirewallFirewallA filter that allows or blocks network traffic by rules — e.g. “only port 443 from the internet”. forgets an idle connection and rejects the next packet.

ECONNRESET in your logs usually means "the other side, or something in between, gave up on this connection". It's common with idle pooled connections (§11).

TIME_WAIT: why it exists

After the final ACK, the side that closed first doesn't forget the connection straight away. It stays in TIME_WAITTIME_WAITThe state the side that closes a TCP connection first stays in for a while afterwards (60 seconds on Linux). It remembers the old connection so a stray late packet can't be mistaken for part of a new one. for twice the maximum segment lifetimeMSL (maximum segment lifetime)The longest a TCP segment is assumed to be able to survive wandering in the network. TIME_WAIT lasts 2 × MSL so any stray segment has died out; Linux fixes the total at 60 seconds.; on Linux that's fixed at 60 seconds. Two reasons:

1. Late packets. A segment from this connection might still be wandering around the network, delayed in some queue. If a brand-new connection with the exact same 4-tuple4-tuple (connection identity)The four values that identify one TCP connection: source IP, source port, destination IP, destination port. No two open connections on a machine can have the same four. (both IPs, both ports) started right away, that old segment could land in it as if it were new data. TIME_WAIT keeps that 4-tuple reserved until any stray packets have died out.
2. A lost final ACK. If the last ACK is lost, the other side resends its FIN. The side in TIME_WAIT is still around to answer it, instead of replying with an RST that would turn a clean close into an error.

Why servers see thousands of them

TIME_WAIT is held by whichever side closes first. Each one costs only a little kernel memory, so thousands are harmless in themselves. You'll see them in ssss / netstatCommand-line tools that list a machine's sockets and their TCP states (LISTEN, ESTAB, TIME-WAIT, …). ss is the modern Linux one; netstat is older and on most systems.:

ss -tan state time-wait | wc -l       # how many sockets are in TIME_WAIT right now
ss -tan state time-wait | head -3
# Recv-Q Send-Q  Local Address:Port   Peer Address:Port
# 0      0       10.0.0.5:44312       10.0.0.9:5432
# 0      0       10.0.0.5:44318       10.0.0.9:5432

The real trouble is on the client side of busy services, for example an API server that opens a new connection to the same database or upstream service for every request. Each of those connections uses a fresh ephemeral portEphemeral portA temporary source port the OS picks for an outgoing connection, from a fixed range (Linux default: 32768–60999). It's freed again shortly after the connection closes. for its own end, and every closed one keeps its port tied up for 60 s. Linux has about 28,000 of those ports by default, so:

28,000 ports ÷ 60 s ≈ 470 new connections per second to the same destination, and then connect starts failing with cannot assign requested address (EADDRNOTAVAIL).

Watch it happen, then turn on connection reuse:

That's ephemeral port exhaustion, and §11 covers the real fix: reuse connections instead of closing them.

The 40 ms mystery: Nagle and delayed ACKs

Two good ideas

• Nagle's algorithmNagle's algorithmA TCP sender rule from 1984: while any sent data is still unacknowledged, hold back new small writes and glue them into one bigger segment. It saves packets but can add delay. (1984), on the sender: if you write tiny pieces (one keystroke over a remote terminal = 1 byte of data + 40 bytes of headers), the network fills with mostly-header packets. So: while any sent data is still unacknowledged, hold new small writes and glue them together until the ACK arrives.
• Delayed ACKDelayed ACKA receiver rule: don't ACK every segment right away; wait a little (40 ms minimum on Linux, up to 200 ms or more elsewhere) in case you can ACK two at once or piggy-back the ACK on a reply., on the receiver: don't send an ACK for every segment right away. Wait a little, in case you can ACK two segments at once or put the ACK on your reply for free. Linux waits at least 40 ms; other systems up to 200 ms.

The trap

Now a client writes one request in two small pieces (a header, then a body) and waits for the reply:

1. Piece 1 goes out immediately (nothing is unacknowledged yet).
2. Piece 2 is small and piece 1 isn't acknowledged, so Nagle holds it.
3. The server got piece 1, but can't reply until it has the whole request. With no reply to piggy-back on, delayed ACK waits for its timer.
4. Each side waits for the other until the delayed-ACK timer fires (≈40 ms). Then the ACK goes, piece 2 goes, the reply comes.

This isn't theory. Here's a measurement from a Linux machine (Node 22, client and server on the same host, the client writes 5 bytes, then another 5 bytes, then waits for a 2-byte reply; median of 20 rounds):

A thousand times slower, on a machine where the network round trip is ~0.03 ms. On a real network the stall is the same 40 ms (or 200 ms with a Windows peer) on top of the normal round trip.

The fix

conn, err := net.Dial("tcp", "db.internal:5432")
if err != nil {
    log.Fatal(err)
}
// Go sets TCP_NODELAY on every TCP connection it creates (dialed or accepted),
// so Nagle is already off. You'd only call this to turn Nagle back ON:
conn.(*net.TCPConn).SetNoDelay(false)

We checked these defaults with strace, which shows every System callSystem call (syscall)A request from your program to the operating system — “read this file”, “open a connection”. While a blocking syscall runs, the calling thread waits inside the OS. a program makes: Go 1.24 calls setsockopt(…, TCP_NODELAY, [1]) on both dialed and accepted connections; a plain Node 22 net.connect never does, while Node's http server and fetch do.

In production: pools, timeouts, floods and ports

Reuse connections

Everything so far adds up: a new connection costs a handshake round trip (§03), a TLSTLSTransport Layer Security: the encryption layer that makes HTTPS secure. It proves the server is who it claims to be and scrambles everything so nobody in between can read or change it. handshake on top for HTTPS, then slow start (§06), and finally a TIME_WAIT slot and an ephemeral portEphemeral portA temporary source port the OS picks for an outgoing connection, from a fixed range (Linux default: 32768–60999). It's freed again shortly after the connection closes. for a minute after it closes (§09).

So keep connections open and reuse them: HTTP Keep-aliveKeep-alive (connection reuse)Leaving a connection open after a request finishes so the next request can reuse it, skipping the cost of a new handshake., database connection poolsConnection poolA set of open database (or HTTP) connections that your program keeps and reuses instead of opening a new one per request. Pools usually have a size limit; when every connection is busy, the next caller waits for one to be returned., one long-lived gRPC connection. Both standard HTTP clients do this already, if you let them:

// One client (and so one Transport, which holds the pool) for the whole program.
// Clone the default Transport rather than writing &http.Transport{}: a blank one
// loses the defaults (HTTPS_PROXY support, 30 s dial timeout, TLS timeout, HTTP/2).
func newClient() *http.Client {
    t := http.DefaultTransport.(*http.Transport).Clone()
    t.MaxIdleConnsPerHost = 32             // default is only 2!
    t.IdleConnTimeout = 90 * time.Second   // close idle connections after this
    return &http.Client{
        Timeout:   10 * time.Second, // whole request: connect + send + read body
        Transport: t,
    }
}

var client = newClient()

resp, err := client.Get(url)
if err != nil {
    return err
}
defer resp.Body.Close()        // ← without this, the connection is never returned
io.Copy(io.Discard, resp.Body) // ← read to the end, or it can't be reused

Set a timeout on everything

Without a TimeoutTimeoutA limit on how long to wait for something (a connection, a response) before giving up with an error, so a slow or dead peer can't hang your program forever., a connect to a dead machine takes ~2 minutes to fail on Linux (§03), and a read from a peer that vanished without a FIN can wait forever: TCP has nothing to send, so it never notices. Every network call needs a limit.

d := net.Dialer{
    Timeout:   3 * time.Second,  // give up on the handshake after 3 s
    KeepAlive: 30 * time.Second, // TCP keepalive probes (see below; default 15 s)
}
conn, err := d.DialContext(ctx, "tcp", "db.internal:5432") // ctx can cancel it too
if err != nil {
    return err
}
defer conn.Close()

conn.SetDeadline(time.Now().Add(5 * time.Second)) // an absolute time, not a duration!
_, err = conn.Read(buf)
if errors.Is(err, os.ErrDeadlineExceeded) {
    // err.Error() == "read tcp 10.0.0.5:42134->10.0.0.9:5432: i/o timeout"
}

How long? Base it on the real p99p99 (99th percentile)The value that 99% of measurements are below — i.e. how bad the slowest 1% is. Averages hide slow outliers; p99 shows them. latency of the thing you're calling, plus a margin, and keep each inner timeout shorter than the caller's. Retry rules and budgets are in Worker Pools.

Notice dead peers: keepalive vs heartbeats

If the other machine loses power or a NATNATNetwork Address Translation: a router rewrites private addresses (like 192.168.1.20) to its one public address on the way out, and back on the way in. Why your laptop isn't directly reachable from the internet. box in between forgets your connection, no FIN or RST ever arrives. An idle connection looks perfectly healthy until you write to it.

• TCP keepaliveTCP keepalive (probes)An OS feature: after a connection has been idle for a while, the kernel sends tiny empty probes and closes the connection if the other side never answers. Not the same as HTTP keep-alive (reusing a connection). asks the kernel to send empty probes after some idle time and close the connection if nobody answers. Go enables it on every TCP connection (we measured 15 s idle, 15 s between probes, 9 probes on Go 1.24); a plain Node socket doesn't: call sock.setKeepAlive(true, 30_000). Its main real-world value is keeping NAT and load-balancer mappings alive and eventually cleaning up dead connections.
• An application heartbeatHeartbeat (application)A small “are you still there?” message your own protocol sends every few seconds. If a few heartbeats go unanswered, the program treats the other side as gone. (a ping message in your own protocol, like WebSocket ping/pong or gRPC keepalive pings) proves more: that the program on the other side is alive and responding, not just its kernel. If the peer process is stuck, TCP keepalive still succeeds; a heartbeat fails.

Rule of thumb: keep TCP keepalive on, and for long-lived connections where you need to detect failure in seconds, add a heartbeat with a deadline.

SYN floods and the accept queue

During a handshake, the server stores a little state for each half-open connection (it got a SYN, sent SYN-ACK, is waiting for the ACK). A SYN floodSYN floodAn attack that sends huge numbers of SYNs (often from fake addresses) and never finishes the handshake, trying to fill the server's table of half-open connections so real clients can't connect. sends millions of SYNs from fake addresses and never completes them, trying to fill that table.

Linux defends with SYN cookiesSYN cookiesA defence against SYN floods: instead of storing anything for a half-open connection, the server encodes what it needs into its own starting sequence number, and checks it when the final ACK arrives. (on by default): when the table is full, the server stores nothing and instead encodes the connection's details into its own starting sequence number. A real client's ACK echoes that number back, and the server can rebuild the state from it. Large floods are still usually absorbed by the cloud provider's or CDNCDNContent Delivery Network: servers spread around the world that keep copies of your files close to users, so they're served from a nearby city instead of your origin server.'s edge servers before they reach you.

The related everyday limit is the listen backlogListen backlog (accept queue)The kernel's queue of connections that have finished the handshake but haven't been picked up by your program's Accept() yet. If it's full, new connections are dropped or refused.: connections that finished the handshake but your program hasn't called Accept on yet. Go asks for the system maximum (4096 on modern Linux, net.core.somaxconn); Node uses 511 by default. If your process stalls (a long garbage-collectionGarbage collector (GC)Part of the language runtime that automatically frees memory nobody can reach anymore. Anything still referenced — like a stuck goroutine — is never freed. pause, or a blocked Event loopEvent loopThe loop inside every JS runtime that repeatedly picks the next piece of waiting work (a callback) and runs it to completion on the single main thread.), the queue fills and new clients see timeouts even though "the server is up".

Ephemeral port exhaustion

§09's arithmetic: closing ~470 connections per second to one destination uses up Linux's ~28,000 ephemeral ports, and connect fails with cannot assign requested address. Fixes, best first:

1. Reuse connections (the Reuse connections part above). This fixes it outright.
2. Spread load across more destination IPs or ports (each destination has its own set of 4-tuples).
3. Widen the range: sysctl net.ipv4.ip_local_port_range="1024 65000".
4. Let the server close first (so TIME_WAIT lands on the server), or enable net.ipv4.tcp_tw_reuse=1, which lets new outgoing connections reuse TIME_WAIT ports safely using timestamps.

Seeing what's happening

• ssss / netstatCommand-line tools that list a machine's sockets and their TCP states (LISTEN, ESTAB, TIME-WAIT, …). ss is the modern Linux one; netstat is older and on most systems. -tan state established '( dport = :5432 )' | wc -l: how many connections to Postgres right now?
• ss -ti: per connection cwnd, RTT and retransmit counters (§04, §06).
• tcpdumptcpdump / WiresharkTools that capture and display every packet going through a network interface. tcpdump prints them in the terminal; Wireshark shows them in a GUI with each header field decoded. -n host 10.0.0.9 and port 5432: the actual packets, including SYNs that get no answer and RSTs.
• Go's net/http/httptrace reports, per request, whether a connection was reused and how long DNS, connect, TLS and the first byte took:

trace := &httptrace.ClientTrace{
    GotConn: func(info httptrace.GotConnInfo) {
        log.Printf("reused connection: %v", info.Reused) // false every time = no pooling
    },
    ConnectDone: func(network, addr string, err error) {
        log.Printf("new TCP connection to %s (err=%v)", addr, err)
    },
}
req, _ := http.NewRequest("GET", url, nil)
req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
resp, err := client.Do(req)

Pitfalls gallery

Eight mistakes that show up again and again in code reviews and incident reports, each with the fix.

n, _ := conn.Read(buf)
handleMessage(buf[:n]) // might be half a message, or two

r := bufio.NewReader(conn)
msg, err := readMsg(r) // length prefix + io.ReadFull (§05)

resp, err := http.Get(url) // can hang forever

var client = &http.Client{Timeout: 10 * time.Second}
resp, err := client.Get(url)

sock.setTimeout(5000); // and then… nothing

sock.setTimeout(5000, () => sock.destroy(new Error("idle")));

func call() {
    c := &http.Client{Transport: &http.Transport{}} // fresh pool every call
    resp, _ := c.Get(url)
    data, _ := io.ReadAll(resp.Body) // Body never closed
    use(data)
}

var client = &http.Client{Timeout: 10 * time.Second} // shared

func call() error {
    resp, err := client.Get(url)
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    data, err := io.ReadAll(resp.Body)
    if err != nil {
        return err
    }
    use(data)
    return nil
}

sock.write(header);
sock.write(body); // held back until header is ACKed

sock.setNoDelay(true);
sock.write(Buffer.concat([header, body])); // one write, sent now

buf := make([]byte, 64)
n, _ := conn.Read(buf) // longer datagrams cut off; may block forever

buf := make([]byte, 1500)
conn.SetReadDeadline(time.Now().Add(time.Second))
n, err := conn.Read(buf) // then retry or give up on timeout

Check yourself

Predict the answer before you click. Each explanation points back to the section that covers it.

// client
conn.Write([]byte("ping"))
conn.Write([]byte("ping"))

// server, a moment later
n, _ := conn.Read(buf)
fmt.Printf("%q\n", buf[:n])

// client sends two datagrams: "hello", then "world"
buf := make([]byte, 3)
for i := 0; i < 2; i++ {
    n, _, err := srv.ReadFromUDP(buf)
    fmt.Printf("%q %v\n", buf[:n], err)
}

// RTT between client and server: 80 ms. No TLS.
// The server answers instantly. How long from Dial()
// until the first byte of the response arrives?
conn, _ := net.Dial("tcp", "api.internal:8080")
conn.Write(request)
conn.Read(buf)

// A Reno-style TCP sender has cwnd = 40 segments.
// Instead of 3 duplicate ACKs, the retransmission
// timer fires. What is cwnd right afterwards?

const sock = net.connect(5432, "db.internal");
sock.setTimeout(5000);
// the server goes silent for 10 seconds…

// An HTTP/2 connection is downloading streams A, B and C.
// One TCP packet carrying part of stream A is lost.
// Until it's retransmitted, which data is stuck in the
// receiver's buffer, unable to reach the browser?

Cheat sheet

TCP or UDP?

Defaults worth knowing (Linux, Go 1.24, Node 22)

Commands