File Descriptors & Sockets

Foundations§01

The 30-second version

In one sentenceA file descriptor is a small number, like 3 or 7, that the operating system gives your program to stand for something it opened for you: a file, a network connection, a pipe, a terminal. Your program only ever holds the number.

Every time your code opens a file, accepts a network connection or starts a child process, the operating system's kernel creates a real object and hands you back one of these numbers — an fd. To use the object, you pass the number back: "read 100 bytes from fd 7", "close fd 7".

"Everything is a file"

Here's what makes fds powerful. Unix (and so Linux and macOS) gives very different things the same handful of operations: open (or socket, pipe, accept) to get a number, read and write to move bytes, and close to give the number back. These are system calls: requests from your program to the kernel. A file on disk, a Socket to another computer, a Pipe to another program, and your terminal all look alike from your code's point of view.

One process, one table of small numbers

fd

what the kernel holds

kind

0

──▶

terminal (your keyboard) · device

device

1

──▶

terminal (your screen) · device

device

2

──▶

terminal (your screen) · device

device

3

──▶

/var/log/app.log · regular file

regular file

4

──▶

epoll instance (the “doorbell”) · kernel object

kernel object

5

──▶

listening socket, port 8080 · socket

socket

6

──▶

connection to 203.0.113.7:51234 · socket

socket

7

──▶

pipe to a child process · pipe

pipe

Your program only ever holds the numbers on the left. The things on the right live inside the kernel. Very different things, one interface: read, write, close.

That's the famous Unix motto "everything is a file". It's not literally true: a socket isn't stored on a disk. What's true is that everything is read, written and closed through an fd. So the same code can copy bytes from a file, a socket or a pipe without caring which one it has.

Here's that idea in code: one program reads a file and a network connection with the same function. (Port 9000 is a little test server that sends one line and hangs up. To try it yourself, start one in another terminal with echo "hello from a socket" | nc -l 9000.)

copy.go

f, err := os.Open("notes.txt") // the kernel opens the file → an fd
if err != nil {
    log.Fatal(err)
}
defer f.Close() // ← give the ticket back when done

conn, err := net.Dial("tcp", "localhost:9000") // a socket → another fd
if err != nil {
    log.Fatal(err)
}
defer conn.Close()

io.Copy(os.Stdout, f)    // file   → fd 1 (your screen)
io.Copy(os.Stdout, conn) // socket → fd 1: the very same function

$ go run copy.go
hello from a file
hello from a socket

🔤

Reading the snippets

Go defer f.Close() runs f.Close() when the surrounding function returns (see defer). io.Copy(dst, src) loops "read a chunk from src, write it to dst" until src has nothing left. It accepts a file and a connection alike because both have a Read method — Go's version of "everything is a file".
JS Node wraps fds in streams. pipeline(a, b) moves every chunk from a into b and resolves when a runs out. { end: false } means "don't close b afterwards" — we don't want to close our own stdout.

What you'll learn

Basics (§01–§02): fds 0, 1 and 2, and how the shell's >, 2>&1 and | work.
How it works (§03–§07): what an fd really points at, what's shared after fork, how to see a program's fds, why every connection is an fd, and the limits.
Keeping it healthy (§08–§09): fd leaks and how to close things correctly.
Staff-level (§10–§11): fd number reuse bugs, accept loops when you're out of fds, handing a socket to another process, and SIGPIPE.
Practice (§12–§14): a pitfalls gallery, a quiz and a one-screen cheat sheet.

This topic builds on Threads & Processes (what a process is, fork and exec) and TCP/IP (ports and sockets). How a program waits on many fds at once (epoll) and what a system call costs are covered in System calls.

Foundations§02

stdin, stdout, stderr and the shell

In one sentenceEvery program starts life with three fds already open: 0 = stdin (input), 1 = stdout (normal output), 2 = stderr (errors). Your program doesn't choose where they go — whoever started it does.

These three are called stdin / stdout / stderr. When you run a program in a terminal, all three point at that terminal: you type into fd 0, and fds 1 and 2 both show up on screen. That's why normal output and error messages look mixed together.

Why two output streams?

Because output is often data for the next program, and errors are for a human. If you run ./export-users > users.csv, you want the CSV in the file and the "warning: 3 users skipped" on your screen, not in the middle of the CSV. Keeping errors on fd 2 makes that possible.

Both languages give you the three fds as ready-made objects:

both.go

package main

import (
    "fmt"
    "log"
    "os"
)

func main() {
    fmt.Println("to stdout")                // fmt.Println writes to os.Stdout = fd 1
    fmt.Fprintln(os.Stderr, "to stderr")    // explicitly fd 2
    log.Println("log also goes to stderr")  // the log package defaults to fd 2

    var name string
    fmt.Fscanln(os.Stdin, &name)            // read a line from fd 0
}

The shell rewires fds before your program starts

The shell (bash, zsh…) is the program that starts your program. Before it does, it can point fds 0, 1 and 2 somewhere else. That's Redirection. Your program never notices: it still writes to fd 1, but fd 1 now leads to a file.

You type	What the shell does to the new process's fds
`cmd > out.txt`	Open `out.txt` (emptying it), point fd 1 at it. Same as `1> out.txt`.
`cmd >> out.txt`	Same, but append instead of emptying the file first.
`cmd 2> err.txt`	Point fd 2 at `err.txt`. The number before `>` picks the fd.
`cmd < in.txt`	Point fd 0 at `in.txt`, so the program reads the file as its input.
`cmd 2>&1`	Point fd 2 at wherever fd 1 points right now. `&1` means "fd 1", not a file named `1`.
`cmd > /dev/null`	/dev/null is a special file that throws away everything written to it.
`a \| b`	Connect `a`'s fd 1 to `b`'s fd 0 through a pipe (wired up in §04).

The trap is 2>&1. It doesn't mean "send errors to stdout forever". It means "copy fd 1's current destination into fd 2" — at the moment the shell reads it, going left to right. So the order of redirections matters. Step through it with a two-line script, both.sh, that prints one line to each stream:

Shell redirection, step by step

Pick a command. Watch the shell rewire fds 0, 1, 2 from left to right before the script runs.

Code

1# both.sh
2echo "to stdout"        # writes to fd 1
3echo "to stderr" >&2    # writes to fd 2
4 
5$ ./both.sh > out.txt 2>&1

fd table of ./both.sh

0 → terminal

1 → terminal

2 → terminal

your terminal shows

nothing yet

files / pipe receive

no files opened

The shell starts a new process for ./both.sh. It inherits the shell's fds, so 0, 1 and 2 all point at your terminal. Now, before the script runs, the shell applies the redirections from left to right.

1 / 6

Here's the same thing on a real machine. Notice that with 2>&1 > out2.txt, the error lands on the screen:

$ ./both.sh > out1.txt 2>&1     # first move fd 1, then copy it into fd 2
$ cat out1.txt
to stdout
to stderr

$ ./both.sh 2>&1 > out2.txt     # copy first (fd 1 is still the terminal!), then move fd 1
to stderr
$ cat out2.txt
to stdout

Practitioner§03

Three tables: fd → open file → inode

In one sentenceAn fd number doesn't point straight at a file. It points at an open file description, the kernel's record of "this file, opened this way, currently at byte 4". That middle record can be shared — and that's where the surprises come from.

💡

A book, a bookmark and a sticky note

The book is the file itself. There's one copy on the shelf, however many people read it. The kernel calls its record of the file an inode.
A bookmark says where one reader is up to (and whether they may write in the margins). That's the Open file description. Each open() makes a new bookmark.
A numbered sticky note in your own notebook says "#3 = that bookmark". That's the fd, kept in your process's File descriptor table.

Two people who open the same book get two bookmarks: each reads at their own pace. But if you write "#3" and "#5" on two sticky notes pointing at the same bookmark, reading through either one moves that one bookmark.

The position stored in the bookmark is the file offset: every read starts there and moves it forward. So whether two fds share an open file description decides whether they share an offset. Three things create fds, and they behave differently:

Action	New fd number?	New open file description (own offset)?
`open()` the same file again	yes	yes — independent offset
dup / dup2 an existing fd	yes	no — shares the offset
fork the process	same numbers, in the child	no — parent and child share every offset

Try all three. The 16-byte file contains ABCDEFGHIJKLMNOP; the colored marks under the letters show where each open file description will read next:

The three tables, live

Open letters.txt (16 bytes: A…P), then try dup, fork and read. Click an fd to select it. Watch which offsets move together.

① fd tablesper process

parent (pid 100)

0 1 2 → terminal

② open file descriptionskernel

none yet: press open()

③ the file (inode)16 bytes

letters.txt · inode 697958

A

B

C

D

E

F

G

H

I

J

K

L

M

N

O

P

The bytes live here once. Offsets don't: they live in ②.

Try this: open() twice and read from each — then reset, open() once, dup() it, and read from both fds. Then try fork().

The same thing in real code

Go and Node both pass these calls straight to the kernel. Here, a and b are two separate open calls; c is a dup of a:

dup.go

package main

import (
    "fmt"
    "os"
    "syscall"
)

func read4(f *os.File) string {
    buf := make([]byte, 4)
    n, _ := f.Read(buf)
    return string(buf[:n])
}

func main() {
    a, _ := os.Open("letters.txt") // open #1
    b, _ := os.Open("letters.txt") // open #2: its own open file description
    defer a.Close()
    defer b.Close()

    newFd, _ := syscall.Dup(int(a.Fd())) // a second number for open #1
    c := os.NewFile(uintptr(newFd), "dup-of-a")
    defer c.Close()

    fmt.Println("fds:", a.Fd(), b.Fd(), c.Fd())
    fmt.Println("a:", read4(a)) // ABCD
    fmt.Println("b:", read4(b)) // ABCD — b has its own offset
    fmt.Println("c:", read4(c)) // EFGH — c shares a's offset
    fmt.Println("a:", read4(a)) // IJKL
}

$ go run dup.go
fds: 3 6 7
a: ABCD
b: ABCD
c: EFGH
a: IJKL

$ node dup.mjs
fds: 20 21
a: ABCD
b: ABCD
a: EFGH

Two details in that output are worth a second look. Go's fds jump from 3 to 6: when Go opened the first file, its runtime also created two fds of its own (an epoll and an eventfd — §05 shows them). Node starts at 20, because before your code runs Node already holds fds 0–19: stdio plus 17 of its own (§05 lists them).

The fork case, in your shell

You can see fork sharing without writing any code. exec 3< letters.txt opens the file once, as fd 3 of your shell. Each head command is a child process that inherits fd 3 — and with it, the shared offset:

$ exec 3< letters.txt     # the shell opens the file once, as fd 3
$ head -c 4 <&3; echo     # child 1 reads 4 bytes through fd 3
ABCD
$ head -c 4 <&3; echo     # child 2 continues where child 1 stopped
EFGH
$ head -c 4 < letters.txt; echo   # a fresh open() starts at the beginning
ABCD

That's not just trivia. It's why (date; whoami) > log.txt produces two lines instead of the second overwriting the first: date and whoami are two separate child processes that inherited the same fd 1, so they share one open file description, and the second write continues at the first one's offset.

Why this matters in practice

1

close() frees the number, not the thing

The open file description (and the socket or pipe behind it) lives until the last fd pointing at it is closed, in any process. A copy left open in a child keeps a connection alive (§04, §11).

2

Shared offsets race

If a parent and child both read the same inherited fd, they steal bytes from each other. Give each its own open(), or have only one side read.

3

Lowest free number

A new fd always gets the lowest unused number. Close 3, and the next open is 3 again. That makes "use after close" bugs nasty (§10).

Practitioner§04

Inheritance: fork, exec, pipes and close-on-exec

In one sentenceWhen a process starts a child, the child inherits copies of its fds. That one rule is how the shell connects ls | grep — and the reason a forgotten fd can make a program hang forever.

Quick recap from Threads & Processes §06: starting a program on Unix is two steps. fork clones the current process, and exec replaces the clone's program with a new one. Between those two steps, the child is still running the parent's code, so it can rearrange its fds. And exec keeps the fd table: the new program starts with whatever fds were open.

How `ls | grep go` is wired

A pipe is a small buffer inside the kernel with two fds: a write end and a read end. Here is, step by step, what the shell does with one. The code is simplified pseudocode, not real Go. The key call is dup2(old, new): "make fd new point at the same thing as fd old" (it's the dup / dup2 from §03, but you choose the number):

How the shell wires ls | grep go

Pseudocode for what bash does. Watch the three fd tables.

Code

1pipe(p)            // p[0] read, p[1] write
2if fork() == 0 {   // child 1 → ls
3    dup2(p[1], 1)  // stdout = write end
4    close(p[0]); close(p[1])
5    exec("ls")
6}
7if fork() == 0 {   // child 2 → grep
8    dup2(p[0], 0)  // stdin = read end
9    close(p[0]); close(p[1])
10    exec("grep", "go")
11}
12close(p[0]); close(p[1])
13wait()             // for both children

shell

0 → terminal

1 → terminal

2 → terminal

child 1 (→ ls)

not started

child 2 (→ grep)

not started

You type ls | grep go. The shell (bash, zsh…) is a normal process with fds 0, 1, 2 on your terminal. Here's what it does, simplified (real shells add error handling).

1 / 9

Notice that neither ls nor grep knows anything about pipes. ls writes to fd 1 and grep reads from fd 0, exactly as they would with a terminal. The shell did all the wiring by rearranging fds between fork and exec.

The rule that causes hangs: EOF needs every write end closed

A reader of a pipe gets EOF (a read returns 0 bytes) only when every fd pointing at the write end, in every process, is closed. One stray write end anywhere — even one nobody will ever write to — and the reader waits forever. Flip the stepper's toggle to see the shell forget one.

You hit this in your own code whenever you feed a child program through a pipe:

sort.go

r, w, err := os.Pipe() // r = read end, w = write end
if err != nil {
    log.Fatal(err)
}
cmd := exec.Command("sort")
cmd.Stdin = r          // the child's fd 0 = the read end
cmd.Stdout = os.Stdout
if err := cmd.Start(); err != nil {
    log.Fatal(err)
}
r.Close()              // the parent doesn't read: drop its copy

w.Write([]byte("pear\napple\n"))
w.Close()              // ← without this, sort never sees EOF and Wait hangs forever
cmd.Wait()             // prints apple, pear

The kernel's pipe buffer is 64 KiB by default on Linux. While it has room, writes return immediately; when it's full, the writer waits until the reader catches up. That's free Backpressure. It's also a deadlock if the parent writes everything before reading anything while the child writes a lot back: both sides fill a buffer and wait for each other. For big inputs, write and read at the same time (a goroutine in Go; Node's streams already do this).

Close-on-exec: don't leak fds into strangers

Inheritance is great for fds 0, 1 and 2. For everything else, it's usually a bug. Imagine a web server holding 500 client sockets and a database connection, and it runs convert to resize an image. Without protection, convert would inherit all 501 sockets. Those copies keep the connections open even after the server closes its own fds (the socket lives until the last fd is closed — §03), and a buggy child could write into them.

The fix is a per-fd flag, Close-on-exec: "close this fd automatically when exec runs". The good news: Go and Node set it on every fd they open — files, sockets and pipes alike. You can see the flag in /proc (more on that in §05). The flags field shows the fd's open flags as one number in octal (base 8): each flag is one digit position, and the parts simply add up. The 02000000 part is close-on-exec. (0100000, O_LARGEFILE, is always set on 64-bit Linux — ignore it.)

$ cat /proc/11570/fdinfo/3       # a Go program's log file
pos:	16
flags:	02102001                  # 02000000 O_CLOEXEC + 0100000 O_LARGEFILE + 02000 append + 1 write-only
$ node -e 'const fs = require("fs"); const fd = fs.openSync("/etc/hostname");
           console.log(fs.readFileSync(`/proc/self/fdinfo/${fd}`, "utf8"))'
pos:	0
flags:	02100000                  # 02000000 O_CLOEXEC + 0100000 O_LARGEFILE (+ 0 = read-only)

So a child started with Go's os/exec or Node's child_process gets only fds 0, 1 and 2, plus anything you pass on purpose:

extrafiles.go

f, err := os.Open("/etc/hostname") // Go opens it with O_CLOEXEC
if err != nil {
    log.Fatal(err)
}
cmd := exec.Command("ls", "-l", "/proc/self/fd")
cmd.Stdout = os.Stdout
cmd.ExtraFiles = []*os.File{f} // ← explicitly pass it: it becomes the child's fd 3
cmd.Run()

$ go run extrafiles.go     # trimmed; fd 1 is wherever your own stdout points
lr-x------ 1 root root 64 Oct 30 13:40 0 -> /dev/null
l-wx------ 1 root root 64 Oct 30 13:40 1 -> /dev/pts/0
l-wx------ 1 root root 64 Oct 30 13:40 2 -> /dev/null
lr-x------ 1 root root 64 Oct 30 13:40 3 -> /etc/hostname
lr-x------ 1 root root 64 Oct 30 13:40 4 -> /proc/16368/fd

The child sees exactly what we gave it: stdin and stderr went to /dev/null (Go's default when you don't set cmd.Stdin / cmd.Stderr), stdout to our terminal, and fd 3 is the file we passed. Fd 4 is ls itself reading the /proc/self/fd folder. None of the parent's other fds leaked in.

Practitioner§05

Seeing them: /proc, lsof and ss

In one sentenceOn Linux, every process's fd table is visible as a folder: /proc/<pid>/fd. When you suspect a leak or a stuck connection, look before you guess.

Linux has a special folder, /proc, that the kernel fills in live. /proc/<pid>/fd has one entry per open fd of the process with that PID, and each entry is a Symlink showing what the fd points at. Nothing is stored on disk; you're reading the kernel's tables.

The specimen: a tiny Go and a tiny Node server

Both servers open a log file, listen for HTTP, and write each request's path to the log. A client then opened two Keep-alive connections to each.

server.go

package main

import (
    "fmt"
    "log"
    "net/http"
    "os"
)

func main() {
    logf, err := os.OpenFile("app.log", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644)
    if err != nil {
        log.Fatal(err)
    }
    defer logf.Close()

    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(logf, r.URL.Path)
        fmt.Fprintln(w, "hi")
    })
    log.Println("pid", os.Getpid())
    log.Fatal(http.ListenAndServe(":18480", nil))
}

`/proc/<pid>/fd`: the raw table

$ ls -l /proc/11570/fd          # the Go server (output trimmed to the name and target)
0 -> /dev/null
1 -> /tmp/fdx/srv/go.out
2 -> /tmp/fdx/srv/go.out
3 -> /tmp/fdx/srv/app.log
4 -> anon_inode:[eventpoll]
5 -> anon_inode:[eventfd]
6 -> socket:[1290583]
7 -> socket:[1290704]
8 -> socket:[1290705]

Reading it top to bottom:

0, 1, 2 are stdio. This server was started as ./server > go.out 2>&1 with no input, so 0 is /dev/null and 1 and 2 both point at go.out (that's §02's 2>&1).
3 is app.log, the first file the program opened.
4 and 5 belong to Go's runtime, not your code. eventpoll is an epoll / kqueue / IOCP instance: the "doorbell" the Netpoller uses to wait for network activity on many sockets at once (see System calls). eventfd is an eventfd, which the runtime uses to ring that doorbell itself when it needs to wake a sleeping thread.
6, 7, 8 are sockets. The number in brackets is the socket's inode number. Sockets get one too, even though nothing is on disk: treat it as a unique ID. Which is which? /proc doesn't say. Two tools do.

`lsof`: the same table, explained

lsof ("list open files") decodes each fd. -p picks the process; the FD column is the fd number plus r/w/u for read, write or both:

$ lsof -p 11570                 # rows for memory-mapped files and the program itself removed
COMMAND   PID USER   FD      TYPE  DEVICE SIZE/OFF   NODE NAME
server  11570 root    0r      CHR     1,3      0t0      3 /dev/null
server  11570 root    1w      REG   254,0       30 697949 /tmp/fdx/srv/go.out
server  11570 root    2w      REG   254,0       30 697949 /tmp/fdx/srv/go.out
server  11570 root    3w      REG   254,0       24 697958 /tmp/fdx/srv/app.log
server  11570 root    4u  a_inode    0,16        0   1030 [eventpoll:5,6,7,8]
server  11570 root    5u  a_inode    0,16        0   1030 [eventfd:97]
server  11570 root    6u     IPv4 1290583      0t0    TCP *:18480 (LISTEN)
server  11570 root    7u     IPv4 1290704      0t0    TCP localhost:18480->localhost:41708 (ESTABLISHED)
server  11570 root    8u     IPv4 1290705      0t0    TCP localhost:18480->localhost:41720 (ESTABLISHED)

(On a machine with IPv6 enabled, Go's ":18480" listens on IPv4 and IPv6 at once, so you'll see IPv6 and *:18480 / [::]:18480 instead of IPv4 and 0.0.0.0.)

Now it's clear: fd 6 is the listening socket on port 18480, and 7 and 8 are the two client connections (§06 explains why each client gets its own fd). The epoll line even lists what it watches: [eventpoll:5,6,7,8] — the eventfd, the listener, and both connections. TYPE tells you the kind: REG regular file, CHR a device, IPv4 a socket, FIFO a pipe.

The Node server with the same two connections has 25 fds:

$ lsof -a -d 0-100 -p 11571     # -a -d 0-100: only real fds, not memory maps
COMMAND   PID USER   FD      TYPE  DEVICE SIZE/OFF    NODE NAME
node    11571 root    0r      CHR     1,3      0t0       3 /dev/null
node    11571 root    1w      REG   254,0       10  697957 /tmp/fdx/srv/node.out
node    11571 root    2w      REG   254,0       10  697957 /tmp/fdx/srv/node.out
node    11571 root    3u  a_inode    0,16        0    1030 [eventpoll]
node    11571 root    4u  a_inode    0,16        0 1291665 [io_uring]
node    11571 root    5r     FIFO    0,15      0t0 1291666 pipe
node    11571 root    6w     FIFO    0,15      0t0 1291666 pipe
  … fds 7–19: more pipes, eventfds, epoll and io_uring instances (V8 and libuv internals)
node    11571 root   20r      CHR     1,3      0t0       3 /dev/null
node    11571 root   21u     IPv4 1291678      0t0     TCP *:18481 (LISTEN)
node    11571 root   22w      REG   254,0       24  697958 /tmp/fdx/srv/app.log
node    11571 root   23u     IPv4 1290707      0t0     TCP localhost:18481->localhost:33592 (ESTABLISHED)
node    11571 root   24u     IPv4 1290708      0t0     TCP localhost:18481->localhost:33598 (ESTABLISHED)

Fds 3–19 are Node's own machinery: several epoll instances, io_uring rings, eventfds and pipes that libuv and V8's helper threads use to wake each other up. Fd 20, a second /dev/null, is a spare that libuv opens when a server starts listening — §10 shows what it's for. You don't need to know each one. What matters: a Node process uses about 20 fds before your code opens anything, and your app's files and sockets come after them.

`ss -tanp`: sockets, from the network side

ss lists TCP connections (-t), all states (-a), numeric (-n), with the owning process and fd (-p). It's how you find which fd in which process holds a given connection:

$ ss -tanp | grep 18480        # spacing condensed
LISTEN  0  4096    0.0.0.0:18480      0.0.0.0:*     users:(("server",pid=11570,fd=6))
ESTAB   0  0     127.0.0.1:18480    127.0.0.1:41708 users:(("server",pid=11570,fd=7))
ESTAB   0  0     127.0.0.1:18480    127.0.0.1:41720 users:(("server",pid=11570,fd=8))
ESTAB   0  0     127.0.0.1:41708    127.0.0.1:18480 users:(("node",pid=12000,fd=21))
ESTAB   0  0     127.0.0.1:41720    127.0.0.1:18480 users:(("node",pid=12000,fd=22))

Each connection shows up twice because both ends are on this machine: the server's side (fds 7 and 8 in pid 11570) and the client's side (fds 21 and 22 in the client, pid 12000). Each end is an fd in its own process.

More places to look

Question	Command
How many fds does it have right now?	`ls /proc/<pid>/fd \| wc -l`
What's its limit?	`grep 'open files' /proc/<pid>/limits` (§07)
Where is fd 3's offset, and which flags?	`cat /proc/<pid>/fdinfo/3` (`pos:` and `flags:`; §04)
Who has port 5432 open?	`lsof -i :5432` or `ss -tanp \| grep 5432`
Who holds this (deleted) file open?	`lsof +L1` lists open files that were deleted but still take disk space
What is it opening and closing, live?	`strace -f -e trace=openat,close,accept4 -p <pid>` (see strace)

Practitioner§06

Sockets are fds: listeners, connections, pollers

In one sentenceA server has one fd for its listening socket, and gets one new fd for every connection it accepts. 10,000 connected clients means 10,000 fds, plus a few.

TCP/IP §07 covered the network side: a server listens on one Port, and each connection is identified by its 4-tuple. Here's the fd side of the same story. These are the system calls a server makes:

socket() → a new, unconnected socket. One fd.
bind() to a port and listen() → it's now the listening socket. Still that one fd.
accept() → the kernel takes the next client that finished the handshake (waiting in the backlog) and returns a brand-new fd for that one connection. The listening fd stays as it was, ready for the next client.

accept.go

ln, err := net.Listen("tcp", ":8080") // socket + bind + listen → one fd
if err != nil {
    log.Fatal(err)
}
for {
    conn, err := ln.Accept() // ← each call returns a NEW fd for one client
    if err != nil {
        if errors.Is(err, net.ErrClosed) {
            return // the listener was closed (shutdown): stop the loop
        }
        log.Println("accept:", err)
        time.Sleep(50 * time.Millisecond) // simplified backoff; §10 explains why
        continue
    }
    go func() {
        defer conn.Close() // ← this connection's fd is freed here
        io.Copy(conn, conn) // echo back whatever the client sends
    }()
}

Fill one up

The server below has fds 0–2 (stdio), an epoll fd (3) and its listener (4) before the first client arrives. With a limit of 12 fds, that leaves room for 7 clients. Connect a few, then a burst:

A server's fd table

A tiny server with ulimit -n 12. Every client that connects costs one fd. Fill it up.

fd table5 / 12 used

0stdin

1stdout

2stderr

3epoll (the doorbell)

4listening socket :8080

·free

epoll is watching

fd 4 (new connections)

kernel backlognot accepted yet

empty

Five fds are gone before the first client arrives. Only 7 clients fit.

Three things to notice:

A full fd table doesn't refuse the TCP connection. The kernel finishes the handshake and parks the client in the backlog. It's only your program's accept() that fails, with "too many open files" (§07). The client sees a slow server, not an error — until the backlog overflows too.
Freed numbers are reused at once. When a client hangs up, the very next accept gets the same fd number. Remember this for §10.
One poller watches them all. Go's Netpoller and Node's libuv each register every connection's fd with one epoll / kqueue / IOCP instance, then ask it "which of these have data?" in a single call. That's how one process handles thousands of connections without thousands of threads; Async/await §03 and System calls cover the waiting part.

Count both directions

It's easy to count only the clients connecting to you. Every connection you open is an fd too:

In a typical API server	fds
stdio, runtime internals (epoll, eventfd; Node ≈ 20)	5–20
Listening sockets (HTTP, metrics, admin)	1–3
Client connections (keep-alive, WebSockets)	one each, e.g. 10,000
Database Connection pool	e.g. 20
Outbound HTTP pools to other services	up to hundreds
Open files: logs, uploads being written, temp files	a few to hundreds

A server holding 10,000 WebSocket clients needs a limit comfortably above 10,000. A worker that fires 2,000 outbound requests at once needs 2,000 fds for that alone. §07 is about the limit.

ℹ️

Sockets that aren't TCP connections

A UDP socket is one fd for all peers: there are no connections, so no per-client fds.
A Unix domain socket (e.g. /var/run/docker.sock) works exactly like TCP here: one listening fd, one fd per accepted connection.
After you close a TCP socket, it may linger in the TIME_WAIT state (TCP & UDP §09). That costs the kernel some memory and blocks the port combination for a while, but it no longer uses one of your fds — ss shows those rows with no process attached.

Practitioner§07

Limits: ulimit -n and “too many open files”

In one sentenceEvery process has a maximum number of open fds. At the limit, anything that would create an fd fails — open, accept, socket, pipe — with the error EMFILE: "too many open files".

In the coat-check picture from §01, this is the number of hooks. The kernel enforces it per process, and the limit applies to fd numbers: with a limit of 1,024, a process can use fds 0 to 1,023.

Soft and hard

The limit is called RLIMIT_NOFILE, and it comes as a pair:

The soft limit is the one actually enforced.
The hard limit is a ceiling. A process may raise its own soft limit up to the hard limit at any time, no permission needed. Raising the hard limit requires root.

The shell command ulimit shows and changes them for the shell and every program it starts afterwards (children inherit limits):

$ ulimit -Sn                 # soft
1024
$ ulimit -Hn                 # hard
524288
$ ulimit -Sn 4096            # raise the soft limit for this shell and its future children
$ grep 'open files' /proc/12345/limits     # any running process's limits
Max open files            1024                 524288               files
$ prlimit --nofile --pid 12345             # same, and prlimit can also change it

Why is the soft limit so often a stingy 1,024 when the hard limit is huge? History. An old system call, select, can only handle fds below 1,024, and a program using it crashes in strange ways above that. So systems keep the default low for old software and let modern programs raise it themselves.

Hitting it, for real

Each program below opens the same file over and over without closing it, until the kernel says no. It's run with ulimit -n 64:

emfile.go

package main

import (
    "fmt"
    "os"
)

func main() {
    var files []*os.File // keep every file open on purpose
    for i := 0; ; i++ {
        f, err := os.Open("/etc/hostname")
        if err != nil {
            fmt.Printf("opened %d files, then: %v\n", i, err)
            return
        }
        files = append(files, f)
    }
}

$ ulimit -n 64
$ go run emfile.go
opened 59 files, then: open /etc/hostname: too many open files
$ node emfile.js
opened 44 files, then: EMFILE: too many open files, open '/etc/hostname'
EMFILE -24 open

The numbers tell the story of the earlier sections. Go got 59 = 64 − 3 (stdio) − 2 (its epoll and eventfd). Node got only 44 because its runtime already held about 20 fds (§05). EMFILE is the kernel's error code; "too many open files" is its human-readable text.

Who raises the limit for you

Here's something surprising: both Go and Node quietly raise their own soft limit at startup.

Runtime	What it does with the soft limit	What child processes get
Go ≥ 1.19 (Linux, macOS…)	Raises it to the hard limit when the program starts. (Current Go sets it to hard − 1, to detect outside changes.)	The original soft limit, restored for programs started with `os/exec`
Node (checked on Node 22)	Raises it to the hard limit at startup	The raised limit
Python, most C programs	Leaves it alone	Whatever they had

Who raises the soft limit — and what children get

Go ≥ 1.19

your shell

soft

1,024

hard

524,288

↓ starts

Go program

soft

524,287

hard

524,288

raises soft to hard − 1

↓ exec

its os/exec child

soft

1,024

hard

524,288

original soft limit restored

Node (22)

your shell

soft

1,024

hard

524,288

↓ starts

Node program

soft

524,288

hard

524,288

raises soft to hard

↓ exec

its child_process child

soft

524,288

hard

524,288

inherits the raised limit

Python / most C

your shell

soft

1,024

hard

524,288

↓ starts

Python program

soft

1,024

hard

524,288

leaves it alone

↓ exec

its child

soft

1,024

hard

524,288

Each box shows two bars: soft (enforced) and hard (the ceiling), on a log scale. The shell starts with soft 1,024 / hard 524,288. The hard limit never moves — it's set by whoever starts the first process.

We checked by starting each with ulimit -Sn 1024 and a hard limit of 20,000: Go reported soft=19999 hard=20000, Node 20000 20000, a Python script 1024. So for Go and Node services, the hard limit is the one that matters. Whoever starts your process sets it.

Where the limit comes from in production

1

systemd services

systemd sets each service's limit from LimitNOFILE= in its unit file. Recent systemd versions default to soft 1,024 / hard 524,288 — fine for Go and Node, which raise the soft one. Set it explicitly anyway: LimitNOFILE=65536.

2

Docker and Kubernetes

A Container gets its limit from the container runtime, and the default has changed between versions. Set it with docker run --ulimit nofile=65536:65536; Kubernetes has no per-pod setting, so it's configured on the node's container runtime. Check with ulimit -n inside the container.

3

Your terminal

Your shell got its limit from your login session. That's why "it works on my laptop" (often 1,024 on Linux, 256 on macOS) can differ from production in both directions.

Reading the limit from inside your program is handy in a startup log line:

limit.go

var lim syscall.Rlimit
if err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &lim); err == nil {
    log.Printf("fd limit: soft=%d hard=%d", lim.Cur, lim.Max)
}

Practitioner§08

Leaks: how fds escape, and how to catch them

In one sentenceAn fd leak is an open without a matching close on some path. Nothing fails at first: the count just creeps up, until one day accept or open fails with "too many open files", usually somewhere far from the bug.

That's what makes leaks nasty. The code that leaks works fine. The code that crashes is whatever next tries to get an fd: a healthy request handler, the logger rotating its file, the database driver opening a connection. Play with the three closing strategies, then try the larger limit:

fd leak simulator

A server handles 6 new requests per tick; each one holds an fd for 4 ticks. Choose how carefully it closes them.

open fds over timetick 0 / 500

open fds

8

leaked (never closed)

0

requests served

0

failed: EMFILE

0

A slow, steady climb. Raising the limit only buys time — try ulimit 1024 and see the line still climb.

Where leaks come from

1

Early returns and error paths

The happy path closes; the if err != nil { return } or continue three lines later doesn't. This is the most common leak, and the reason for defer right after a successful open (§09).

2

HTTP response bodies

A response you never read to the end and close keeps its connection — and its socket fd — tied up. See HTTP pitfall #2.

3

Stuck goroutines and callbacks

A Goroutine leak that holds a connection never reaches its Close. In Node, a stream whose 'error' you don't handle may never be destroyed.

Also common: a new HTTP client, database pool or Redis client per request (each brings its own connections); defer f.Close() inside a long loop (nothing closes until the function returns); child processes whose pipes you never close; and file watchers (fs.watch) nobody stops.

Reproduced: the unclosed response body

Each program runs a local test server and calls it 100 times while ignoring the response, then 100 more times the right way. It counts its own fds by listing /proc/self/fd:

bodyleak.go

func openFDs() int {
    entries, _ := os.ReadDir("/proc/self/fd")
    return len(entries)
}

func main() {
    srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(w, "hello")
    }))
    defer srv.Close()

    fmt.Println("start:", openFDs())
    for i := 1; i <= 100; i++ {
        resp, err := http.Get(srv.URL)
        if err != nil {
            panic(err)
        }
        _ = resp // ← BUG: body never read or closed
        if i%25 == 0 {
            fmt.Printf("after %3d requests (leaky): %d fds\n", i, openFDs())
        }
    }
    for i := 1; i <= 100; i++ {
        resp, err := http.Get(srv.URL)
        if err != nil {
            panic(err)
        }
        io.Copy(io.Discard, resp.Body) // read to the end…
        resp.Body.Close()              // ← …and close: the connection is reused
    }
    fmt.Printf("after 100 more (closed):   %d fds\n", openFDs())
}

$ go run bodyleak.go                    $ node bodyleak.mjs
start: 7                                start: 23
after  25 requests (leaky): 57 fds      after 25 requests (leaky): 73 fds
after  50 requests (leaky): 107 fds     after 50 requests (leaky): 123 fds
after  75 requests (leaky): 157 fds     after 75 requests (leaky): 173 fds
after 100 requests (leaky): 207 fds     after 100 requests (leaky): 223 fds
after 100 more (closed):   209 fds      after 100 more (drained):   225 fds

The starting counts are the baseline: Go's 7 are 3 stdio + epoll + eventfd + the test server's listener + the fd that ReadDir itself has open on /proc/self/fd; Node's 23 are its usual fds 0–19, libuv's spare /dev/null (§10), the listener and the readdirSync fd. Then: exactly 2 fds per ignored request: both the client end and the server end of each new connection live in this one process. The response was never finished, so the connection couldn't go back into the pool, so the next request opened a fresh one. With the fix, 100 requests cost just 2 more fds: one connection, reused 100 times.

Catching a leak

Graph the count. Export the number of open fds as a gauge and alert when it passes ~80% of the limit, or keeps rising for hours. The standard Prometheus client libraries for Go and Node already export process_open_fds and process_max_fds with their default metrics. A healthy service's line is flat with bumps; a leak is a ramp.
Find what's growing. Group the process's connections by the address at the other end, and its files by path. Run it twice, a few minutes apart, and see which line grows:

$ ss -tnp | grep 'pid=4120,' | awk '{print $5}' | sort | uniq -c | sort -rn | head -3
    389 10.0.9.12:443
     20 10.0.4.2:5432
      3 10.0.7.30:6379
$ ls -l /proc/4120/fd | awk '{print $NF}' | sort | uniq -c | sort -rn | head -3

awk '{print $5}' keeps the 5th column (the peer address), uniq -c counts repeats, sort -rn puts the biggest first. (Illustrative numbers.) If the 389 connections to 10.0.9.12:443 are 566 five minutes later while the database stays at 20, look at the code that calls that service. The second command does the same for files: the same path open 300 times is a file leak.

Name the code. In Go, a goroutine dump (/debug/pprof/goroutine?debug=2) often shows hundreds of goroutines parked inside the same HTTP or database call. In Node, run with --trace-warnings to get a stack trace for "Closing file descriptor N on garbage collection" warnings, and log every stream 'error' — a stream that errored and was never destroyed is a likely leak.

🧠

The garbage collector might hide a leak — don't let it

Both runtimes have a safety net, and it makes leaks intermittent rather than absent:

Go attaches a cleanup to every *os.File and network connection: when the garbage collector frees one you forgot, it closes the fd. In our test, 100 unclosed files went from 106 fds to 6 after runtime.GC(). But the GC runs when memory is short, not when fds are. A program with lots of RAM can hit EMFILE long before the next collection.
Node closes a forgotten fs/promises FileHandle on GC, with a loud warning: Warning: Closing file descriptor 20 on garbage collection — plus a deprecation notice saying a future version will throw instead. Plain fd numbers (fs.openSync) are just numbers to the GC: they are never closed for you.

Practitioner§09

Closing correctly

In one sentenceClose every fd exactly once, on every path, as soon as you're done with it. For files you wrote to, the result of Close is part of the result of the write.

The habit: close right after a successful open

The rule that prevents most leaks from §08: the line right after the error check schedules the close. Then no later return, continue or thrown error can skip it.

read.go

f, err := os.Open(path)
if err != nil {
    return err // nothing was opened, nothing to close
}
defer f.Close() // ← runs however the function exits

// … read from f; any return below still closes it

For writes, check what Close says

When you write to a file, the kernel usually just copies your bytes into memory and says "done". The actual disk write happens later. On network file systems (NFS, some FUSE mounts), a write error may only show up at Close; on local disks it shows up at Write or Sync. If you ignore it, you can report success for data that was never saved. And if the data must survive a power cut, call fsync (Go's f.Sync()) first — Transactions explains why databases do this on every commit.

write.go

func writeFile(path string, data []byte) (err error) {
    f, err := os.Create(path)
    if err != nil {
        return err
    }
    defer func() {
        if cerr := f.Close(); cerr != nil && err == nil {
            err = cerr // ← report a failed Close, unless we already have an error
        }
    }()

    if _, err = f.Write(data); err != nil {
        return err
    }
    return f.Sync() // ← fsync: wait until the bytes are really on disk
}

🔤

Reading the Go version

(err error) in the signature is a named result: err is a variable the function will return. A deferred function can change it after the return statement has set it — that's how the Close error gets reported.
defer func() { … }() defers a call to an anonymous function, so several lines can run at exit.
if cerr := f.Close(); cerr != nil && err == nil runs f.Close(), stores its error in cerr, then checks it. We only report it if nothing failed earlier; the first error is usually the more useful one.
For a whole file at once, os.WriteFile(path, data, 0o644) already returns the Close error for you (it doesn't Sync, though).
The JS version isn't quite equivalent: if writeFile throws and close throws, the close error replaces the first one. Usually acceptable; wrap close in its own try if you need the original error.

What about closing twice? Go's *os.File remembers: the second Close just returns close out.txt: file already closed. Node's FileHandle is similar. A raw fd number (fs.closeSync(fd) twice) is dangerous: by the second call, that number may belong to a different file someone else just opened (§10).

Streams: use `pipeline`, not `.pipe`

In Node, files and sockets are usually streams. The old a.pipe(b) does not close a if b fails, or b if a fails — a classic leak on the error path. stream.pipeline destroys (and so closes) every stream in the chain when any of them errors:

gzip.mjs

import { createReadStream, createWriteStream } from "node:fs";
import { pipeline } from "node:stream/promises";
import { createGzip } from "node:zlib";

// If any step fails, pipeline destroys all three streams: both fds get closed.
await pipeline(createReadStream("big.log"), createGzip(), createWriteStream("big.log.gz"));

A write stream emits 'finish' when all data has been handed to the kernel and 'close' once its fd is actually closed. If the next step depends on the file being complete and closed (renaming it, uploading it), wait for pipeline to resolve or for 'close'.

Sockets: close, half-close, and closing on shutdown

Closing a TCP socket's fd sends the other side a FIN: "no more data from me" (TCP & UDP §09). Sometimes you want to say "I'm done sending" but keep reading the answer. That's a Half-close: Go's conn.(*net.TCPConn).CloseWrite(), or Node's socket.end(), which sends FIN but keeps reading until the other side finishes. (allowHalfOpen: true is the reverse case: it lets you keep writing after the peer has ended. With the default false, Node ends your side automatically when the peer's FIN arrives.) The fd stays open until the connection is fully closed.

Half-close: the client is done sending, the answer keeps flowing

client →

request bytes

FIN

sending side closed

← server

response bytes keep flowing

FIN

fds

both fds still open

closed

The client's FIN closes only its sending direction. The server keeps reading until it sees that FIN (end of file), then sends the rest of its answer and its own FIN. Only then do both sides close their fds.

Servers have one more thing to close: the listening socket. A Graceful shutdown closes it first, so no new clients are accepted, then lets in-flight requests finish, then closes the remaining connections:

shutdown.go

ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
// Closes the listening socket(s), closes idle connections,
// waits for active requests to finish (or for ctx to expire).
if err := srv.Shutdown(ctx); err != nil {
    srv.Close() // deadline passed: close everything that's left
}

The details — draining queues, what to do with long requests, Kubernetes timing — are in Worker pools: graceful shutdown. §11 shows how to hand the listening socket to a new process instead of closing it, so a restart refuses nobody.

Staff-level§10

Staff-level: fd reuse bugs and accept loops under EMFILE

In one sentenceThe kernel recycles fd numbers immediately, so a stale number silently points at someone else's file. And when fds run out, the way your accept loop reacts decides whether you get a slowdown or an outage.

Use-after-close: the wrong file, with no error

§03 said a new fd always takes the lowest free number. Combine that with a stale copy of a number and you get a Use-after-close bug. Here it is in Node, where fds are plain numbers:

stale.js

const fs = require("node:fs");
const a = fs.openSync("/etc/hostname"); // a = 20
fs.closeSync(a);
const b = fs.openSync("/etc/passwd");   // b = 20 as well: the lowest free number
const buf = Buffer.alloc(8);
fs.readSync(a, buf);                     // "a" is stale…
console.log(JSON.stringify(buf.toString()));

$ node stale.js
"root:x:0"

No error: the read through a returned the first bytes of /etc/passwd. Go's *os.File protects you here: after Close, its methods return file already closed instead of touching the number. The danger in Go is code that passes raw numbers around (f.Fd(), syscall calls, cgo).

In a busy concurrent program, the worst version is a double close. Goroutine A closes fd 9. Goroutine B opens a database connection and gets 9. Then A's error path closes 9 again — and B's database connection dies with a confusing "bad file descriptor" or "connection reset", in code that did nothing wrong. Writes are worse: a log line sent to a stale number can land inside someone else's TLS connection.

When accept runs out of fds

Every connection needs an fd (§06). When the process is at its limit, accept() fails with EMFILE, but the waiting client stays in the backlog, so the listening socket still looks "ready". The loop's reaction matters enormously:

Accept loop vs EMFILE

The process is out of fds and clients keep arriving. Three ways the accept loop can react.

Code

1var delay time.Duration
2for {
3    conn, err := ln.Accept()
4    if err != nil {
5        if delay == 0 {
6            delay = 5 * time.Millisecond
7        } else {
8            delay = min(2*delay, time.Second)
9        }
10        log.Printf("accept: %v; retrying in %v", err, delay)
11        time.Sleep(delay) // ← give in-flight work time to free fds
12        continue
13    }
14    delay = 0
15    go handle(conn)
16}

fd table

1024 / 1024 fds in use

backlog (waiting clients)

client A

client B

log / effect

—

Same situation: every fd in use, two clients waiting in the backlog. (var delay time.Duration declares a length of time; it starts at 0.)

1 / 5

We ran both real runtimes into this. A Go net/http server limited to 20 fds, hit by 40 slow clients at once, backed off exactly like the middle tab, and every client was eventually served:

2026/10/30 13:39:20 http: Accept error: accept tcp 127.0.0.1:18482: accept4: too many open files; retrying in 5ms
2026/10/30 13:39:20 http: Accept error: accept tcp 127.0.0.1:18482: accept4: too many open files; retrying in 10ms
2026/10/30 13:39:20 http: Accept error: accept tcp 127.0.0.1:18482: accept4: too many open files; retrying in 20ms
2026/10/30 13:39:20 http: Accept error: accept tcp 127.0.0.1:18482: accept4: too many open files; retrying in 40ms

A Node http server limited to 30 fds served 8 of the 40 clients; the other 32 got ECONNRESET, and the server's 'error' event never fired. That's the spare-fd trick from the third tab: libuv keeps a spare /dev/null open (the fd 20 you saw in §05) for exactly this moment.

1

Go: write your own loop carefully

net/http backs off for you. A hand-written ln.Accept() loop must do the same, and must exit when errors.Is(err, net.ErrClosed) — that's the "listener was closed on shutdown" error, not a reason to retry.

2

Node: watch for resets

You won't see an error in your server logs. The symptom is clients reporting ECONNRESET while your process sits at its fd limit. Monitor process_open_fds (§08).

3

Cap connections below the limit

Better than both: refuse before the fd limit. netutil.LimitListener (in golang.org/x/net) and Node's server.maxConnections cap concurrent connections, leaving fds free for log files, DNS and the database.

Staff-level§11

Staff-level: handing fds to other processes, and SIGPIPE

In one sentenceAn open fd can be handed to another process. The receiver gets a new number for the same open socket — which is how a server can be replaced without refusing a single connection.

Remember §03: an fd number is only a pointer to an open file description in the kernel. Nothing stops the kernel from giving a second process another pointer to the same one. Then both processes hold the same listening socket, and clients can't tell which process accepts them.

Handing over a socket

A new number in the receiver, the same open socket in the kernel.

old process

9points at the socket

kernel

listening socket :8080

open

new process

not received yet

The old process holds the listening socket on fd 9. The socket itself (its port, its backlog of waiting clients) lives in the kernel.

1 / 4

Zero-downtime restarts

Restarting the usual way — stop the old server, start the new one — leaves a gap. For a moment, nothing listens on the port and new clients get "connection refused". A Zero-downtime deploy avoids that gap by never closing the listening socket:

Handing over the listening socket

old process

accepting

draining

new process

start

accepting

listen fd

one socket, open the whole time — passed as fd 3

The listening socket stays open for the whole swap, so the kernel keeps accepting handshakes into its backlog. The old process drains its in-flight requests while the new one is already accepting.

The simplest way to pass it is inheritance: start the new version as a child with the socket in ExtraFiles (§04). Here's the whole handover in one Go program that plays both roles:

handoff.go

func main() {
    if os.Getenv("IS_CHILD") == "1" {
        // The new process: fd 3 is the listening socket we were handed.
        f := os.NewFile(3, "listener")
        ln, err := net.FileListener(f) // makes its own dup of fd 3…
        if err != nil {
            log.Fatal(err)
        }
        f.Close() // …so close the original, or fd 3 leaks
        log.Printf("child pid %d serving on %s", os.Getpid(), ln.Addr())
        http.Serve(ln, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            fmt.Fprintf(w, "served by pid %d\n", os.Getpid())
        }))
        return
    }

    // The old process: owns the listening socket.
    ln, err := net.Listen("tcp", "127.0.0.1:18484")
    if err != nil {
        log.Fatal(err)
    }
    f, err := ln.(*net.TCPListener).File() // a dup of the socket's fd
    if err != nil {
        log.Fatal(err)
    }
    cmd := exec.Command(os.Args[0]) // start this same program again
    cmd.Env = append(os.Environ(), "IS_CHILD=1")
    cmd.Stderr = os.Stderr
    cmd.ExtraFiles = []*os.File{f} // ← becomes fd 3 in the child
    if err := cmd.Start(); err != nil {
        log.Fatal(err)
    }
    ln.Close() // the parent stops accepting; the socket stays open in the child
    f.Close()
    log.Printf("parent pid %d handed off the listener and exits", os.Getpid())
}

$ ./handoff
2026/10/30 13:41:08 parent pid 16628 handed off the listener and exits
2026/10/30 13:41:08 child pid 16635 serving on 127.0.0.1:18484
$ curl 127.0.0.1:18484
served by pid 16635

🔤

Reading the snippet

ln.(*net.TCPListener) is a type assertion: "I know this net.Listener is really a *net.TCPListener; give me that". Its File() method returns a duplicate of the socket's fd as an *os.File (a dup / dup2, so closing ln afterwards doesn't close the copy).
net.FileListener(f) does the reverse: wraps an fd that already is a listening socket back into a net.Listener.
os.Args[0] is the path of the running program; os.Environ() is a copy of its environment variables, to which we add IS_CHILD=1.

Node can accept an inherited listening socket too: server.listen({ fd: 3 }). We checked by starting a Node child from the Go parent above; it served requests on the inherited socket.

Real deployments add a signal ("upgrade now", often SIGHUP or SIGUSR2) and let the old process drain before exiting. Nginx and HAProxy reload this way, and Go libraries such as cloudflare/tableflip package it up. On Kubernetes you usually get the same effect differently: a Rolling deploy starts new pods before stopping old ones, and the Load balancer stops sending traffic to a pod before it shuts down.

systemd socket activation

Socket activation moves the listening socket out of your program entirely. systemd opens port 8080 itself, then starts your service with the socket already open as fd 3 and the Environment variable LISTEN_FDS=1. Restart or upgrade the service as often as you like: systemd holds the socket, so the port never closes, and connections that arrive while you're restarting simply wait in the backlog.

myapp.socket

[Socket]
ListenStream=8080

Your program uses the same calls as the handover child above: net.FileListener(os.NewFile(3, "listener")) in Go (the coreos/go-systemd library wraps this), server.listen({ fd: 3 }) in Node.

Passing an fd to a process that's already running: SCM_RIGHTS

Inheritance only works from parent to child. To give an fd to an unrelated, already running process, send it over a Unix domain socket with SCM_RIGHTS. The fd travels next to a normal message, and the kernel installs a new number in the receiver pointing at the same open file description:

scm.go

func send(conn *net.UnixConn, f *os.File) error {
    rights := syscall.UnixRights(int(f.Fd())) // "attach this fd"
    _, _, err := conn.WriteMsgUnix([]byte("x"), rights, nil)
    return err
}

func receive(conn *net.UnixConn) (*os.File, error) {
    buf := make([]byte, 1)
    oob := make([]byte, syscall.CmsgSpace(4)) // room for one fd (4 bytes)
    _, oobn, _, _, err := conn.ReadMsgUnix(buf, oob)
    if err != nil {
        return nil, err
    }
    msgs, err := syscall.ParseSocketControlMessage(oob[:oobn])
    if err != nil {
        return nil, err
    }
    fds, err := syscall.ParseUnixRights(&msgs[0])
    if err != nil {
        return nil, err
    }
    return os.NewFile(uintptr(fds[0]), "received"), nil // a NEW number, same open file
}

In our test, the sender's fd 9 arrived as fd 10, and reading it returned the file's contents. In Node you rarely do this by hand: child.send(message, server) from child_process passes a net.Server or net.Socket to a child over the IPC channel, and that's SCM_RIGHTS underneath. Node's cluster module uses it to share listening sockets between workers.

When the reader goes away: SIGPIPE and EPIPE

The opposite problem: you write to a pipe or socket whose reader has gone away. The kernel can't deliver the bytes, so it sends your process SIGPIPE / EPIPE, a Signal whose default action is to kill it. If the signal is ignored, the write fails with the error EPIPE, "broken pipe". The everyday case is piping a program into head, which exits after reading the lines it wanted:

$ ./printforever | head -2           # Go: prints "line N" in an endless loop
line 0
line 1
$ echo "${PIPESTATUS[@]}"            # exit codes of both sides of the pipe
141 0
$ node printforever.js | head -2     # the same in Node
line 0
line 1
node:events:497
      throw er; // Unhandled 'error' event
Error: write EPIPE

Exit code 141 is 128 + 13, and 13 is SIGPIPE. Go deliberately lets SIGPIPE kill the program when the broken pipe is fd 1 or 2, which is what Unix tools are expected to do. For any other fd, such as a network socket, Go ignores the signal and your Write returns an error (write: broken pipe) instead. Node always ignores the signal: the write fails, and a stream emits an 'error' event. On process.stdout nobody listens for it, so the process crashes with that stack trace.

Practitioner§12

Pitfalls gallery

#1

Closing on the happy path only

The open succeeds, then an early return or a thrown error skips the Close. Each failure leaks one fd, so the leak rate follows your error rate — invisible in tests, deadly during an incident (§08).

✕ Don't

f, err := os.Open(path)
if err != nil {
    return err
}
cfg, err := parse(f)
if err != nil {
    return err // ← f is never closed
}
f.Close()
return apply(cfg)

✓ Do

f, err := os.Open(path)
if err != nil {
    return err
}
defer f.Close() // ← every path below closes it

cfg, err := parse(f)
if err != nil {
    return err
}
return apply(cfg)

#2

defer inside a long loop

defer waits for the function to return, not the iteration. Process 50,000 files in one function and all 50,000 stay open until the end (§09).

✕ Don't

for _, p := range paths {
    f, err := os.Open(p)
    if err != nil {
        return err
    }
    defer f.Close() // ← piles up
    count(f)
}

✓ Do

for _, p := range paths {
    if err := countFile(p); err != nil {
        return err
    }
}

func countFile(p string) error {
    f, err := os.Open(p)
    if err != nil {
        return err
    }
    defer f.Close() // ← closes when countFile returns
    return count(f)
}

#3

Leaving HTTP response bodies open

A response you don't read to the end and close keeps its connection — and its socket fd — out of the pool. Every request then opens a new connection: in §08, 100 ignored responses cost 200 fds. Close the body right after the error check, on every status code; in Node, consume or res.resume() it. Details in HTTP pitfalls #2–#3.

#4

A new client or pool per request

Each &http.Client{Transport: &http.Transport{}}, new http.Agent(), database pool or Redis client keeps its own connections open. Create them per request and every request leaves idle sockets behind, which only time out minutes later. Create one at startup and share it. (A bare &http.Client{} is fine: with no Transport it shares http.DefaultTransport and its pool.)

#5

2>&1 in the wrong place

cmd 2>&1 > out.log sends errors to the terminal, not the file: 2>&1 copies where fd 1 points at that moment (§02). Write cmd > out.log 2>&1. In cron jobs and CI scripts this is how error messages vanish.

#6

Keeping a pipe's write end open

A reader only sees end-of-file when every write end is closed, in every process (§04). A parent that keeps its copy of the write end makes the child wait forever for more input.

✕ Don't

r, w, _ := os.Pipe()
cmd := exec.Command("sort")
cmd.Stdin = r
cmd.Start()
w.Write(data)
cmd.Wait() // ← hangs: w is still open

✓ Do

r, w, _ := os.Pipe()
cmd := exec.Command("sort")
cmd.Stdin = r
cmd.Start()
r.Close()  // parent doesn't read
w.Write(data)
w.Close()  // ← now sort sees EOF
cmd.Wait()

Easier still: cmd.StdinPipe() in Go, or child.stdin.end(data) in Node, which manage the ends for you.

#7

Raising ulimit to make a leak go away

The service dies with too many open files every few days, so someone raises the limit from 65,536 to 1,048,576. Now it dies every few months instead — or runs out of memory first. If process_open_fds keeps rising, find the leak (§08). Raise the limit only when the steady count is legitimately close to it.

#8

Retrying accept() in a tight loop

When accept fails with EMFILE, retrying immediately just fails again — a million times a second — burning CPU the in-flight requests need to finish and free fds. Sleep with backoff, like Go's net/http, and exit the loop on net.ErrClosed (§10).

#9

Holding on to raw fd numbers

The kernel reuses a closed number for the very next open (§03). A stale number reads or writes someone else's file or socket, and a double close kills another component's connection. Pass *os.File / net.Conn / FileHandle around, not integers, and give every fd exactly one owner (§10).

#10

Ignoring the error from Close after writing

For a file you wrote, some errors (mainly on network file systems such as NFS) only appear at Close. defer f.Close() throws that error away and your code reports success for lost data. Check it, and Sync first if the data must survive a crash (§09).

#11

Testing with a different limit than production

Your laptop, your CI runner and your container each start your process with a different hard limit (§07). A load test that passes locally can hit EMFILE in production — or the other way round. Log the limit at startup and run ulimit -n inside the real container.

Practitioner§13

Check yourself

Predict the answer before you click. Q1 is covered in §03 and §10, Q2 in §02, Q3 in §03, Q4 in §05–§06, Q5 in §04, Q6 in §07, Q7 in §08 and §12.

QUESTION 1 / 7

Which number?

const fs = require("node:fs");
const a = fs.openSync("/etc/hostname", "r"); // prints 20
console.log(a);
fs.closeSync(a);
const b = fs.openSync("/etc/passwd", "r");
console.log(b);

QUESTION 2 / 7

Where does the error message go?

./build.sh 2>&1 > build.log — the script prints an error to stderr.

QUESTION 3 / 7

dup and offsets

// letters.txt contains ABCDEFGHIJKLMNOP
a, _ := os.Open("letters.txt")
b, _ := os.Open("letters.txt")
fd, _ := syscall.Dup(int(a.Fd()))
c := os.NewFile(uintptr(fd), "c")
fmt.Print(read4(a), read4(b), read4(c)) // read4 reads 4 bytes

QUESTION 4 / 7

Counting fds

A Go HTTP server (no files open) currently has 1,000 clients connected over keep-alive connections. Roughly how many fds does it use?

QUESTION 5 / 7

The hanging child

r, w, _ := os.Pipe()
cmd := exec.Command("sort")
cmd.Stdin = r
cmd.Stdout = os.Stdout
cmd.Start()
w.Write([]byte("pear\napple\n"))
cmd.Wait() // ← what happens here?

QUESTION 6 / 7

Who raises the limit?

You start a Node 22 process from a shell where ulimit -Sn is 1024 and ulimit -Hn is 4096. What soft limit does the Node process see in /proc/self/limits?

QUESTION 7 / 7

The forgotten body

for _, url := range urls {
    resp, err := http.Get(url)
    if err != nil {
        continue
    }
    if resp.StatusCode != 200 {
        continue // ← skip failures
    }
    defer resp.Body.Close()
    process(resp.Body)
}

Answered 0 / 7

Foundations§14

Cheat sheet

🎟️

An fd is a ticketA small number standing for a file, socket, pipe or terminal the kernel holds. Read, write and close all go through it.

0️⃣

0, 1, 2stdin, stdout, stderr. The shell (or Docker, or systemd) decides where they point before your program starts.

➡️

Redirections run left to right> f 2>&1 = both to f. 2>&1 > f = only stdout to f. A pipe is wired before the rest.

🔖

fd → open file description → fileopen twice = separate offsets. dup and fork share one offset.

🔢

Lowest free numberA closed number is reused by the very next open. Never use an fd after closing it.

👶

Children inherit fdsThat's how pipes work. Go and Node set close-on-exec, so children get only 0–2 plus what you pass.

📭

EOF needs every writer goneA pipe reader waits until every write end, in every process, is closed.

🔌

One fd per connectionPlus the listener, the poller and the runtime's own. Count outbound pools too.

🧱

EMFILE = at the limitSoft limit is enforced, hard limit is the ceiling. Go and Node raise soft to hard themselves.

🚰

Leaks are rampsClose on every path: defer right after open, drain and close bodies, pipeline not .pipe.

I want to…	Use
See a process's fds	`ls -l /proc/<pid>/fd`, or `lsof -p <pid>` (also macOS)
Count them	`ls /proc/<pid>/fd \| wc -l`; metric `process_open_fds`
See its limit	`grep 'open files' /proc/<pid>/limits`; in a shell `ulimit -Sn` / `ulimit -Hn`
Find who holds a connection or port	`ss -tanp \| grep :5432`, `lsof -i :5432`
Set a service's limit	systemd `LimitNOFILE=65536`; `docker run --ulimit nofile=65536:65536`
Capture all output of a command	`cmd > all.log 2>&1` (bash: `cmd &> all.log`)
Feed a child process and finish	Close your write end: `w.Close()` / `child.stdin.end()`
Close a file you wrote, safely	`Sync`, then check the error from `Close`
Survive running out of fds	Back off in the accept loop; cap connections below the limit
Restart without refusing connections	Pass the listening socket to the new process (fd 3), or use systemd socket activation

The 30-second version

"Everything is a file"

What you'll learn

stdin, stdout, stderr and the shell

Why two output streams?

The shell rewires fds before your program starts

Three tables: fd → open file → inode

The same thing in real code

The fork case, in your shell

Why this matters in practice

close() frees the number, not the thing

Shared offsets race

Lowest free number

Inheritance: fork, exec, pipes and close-on-exec

How ls | grep go is wired

The rule that causes hangs: EOF needs every write end closed

Close-on-exec: don't leak fds into strangers

Seeing them: /proc, lsof and ss

The specimen: a tiny Go and a tiny Node server

/proc/<pid>/fd: the raw table

lsof: the same table, explained

ss -tanp: sockets, from the network side

More places to look

Sockets are fds: listeners, connections, pollers

Fill one up

Count both directions

Limits: ulimit -n and “too many open files”

Soft and hard

Hitting it, for real

Who raises the limit for you

Where the limit comes from in production

systemd services

Docker and Kubernetes

Your terminal

Leaks: how fds escape, and how to catch them

Where leaks come from

Early returns and error paths

HTTP response bodies

Stuck goroutines and callbacks

Reproduced: the unclosed response body

Catching a leak

Closing correctly

The habit: close right after a successful open

For writes, check what Close says

Streams: use pipeline, not .pipe

Sockets: close, half-close, and closing on shutdown

Staff-level: fd reuse bugs and accept loops under EMFILE

Use-after-close: the wrong file, with no error

When accept runs out of fds

Go: write your own loop carefully

Node: watch for resets

Cap connections below the limit

Staff-level: handing fds to other processes, and SIGPIPE

Zero-downtime restarts

systemd socket activation

Passing an fd to a process that's already running: SCM_RIGHTS

When the reader goes away: SIGPIPE and EPIPE

Pitfalls gallery

Closing on the happy path only

defer inside a long loop

Leaving HTTP response bodies open

A new client or pool per request

2>&1 in the wrong place

Keeping a pipe's write end open

Raising ulimit to make a leak go away

Retrying accept() in a tight loop

Holding on to raw fd numbers

Ignoring the error from Close after writing

Testing with a different limit than production

Check yourself

Cheat sheet

How `ls | grep go` is wired

`/proc/<pid>/fd`: the raw table

`lsof`: the same table, explained

`ss -tanp`: sockets, from the network side

Streams: use `pipeline`, not `.pipe`