• Book: The Complete Guide to Go Programming
• Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
• My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
• Me: xgabriel.com | GitHub

You wrote a function that walks a directory and reads every
file. It opened cleanly in review. It passed tests on a fixture
folder with three files. Then a customer pointed it at a folder
with 20,000 files, and the logs filled with:

open /data/batch/img-08431.png: too many open files

The code never closed anything early. It deferred every close.
And that is exactly the problem.

defer runs at function return, not end of iteration

defer schedules a call to run when the surrounding function
returns. Not when the current block ends. Not when the loop
iteration ends. When the function returns.

Most of the time that distinction does not matter, because most
deferred calls live in short functions that return quickly. Put
a defer inside a for loop, though, and every iteration adds
one more deferred call to a stack that does not unwind until the
whole loop is done and the function exits.

Here is the version that ships:

func processFiles(paths []string) error {
    for _, p := range paths {
        f, err := os.Open(p)
        if err != nil {
            return err
        }
        defer f.Close() // runs at RETURN, not here

        if err := handle(f); err != nil {
            return err
        }
    }
    return nil
}

Read it the way the language reads it. On the first iteration,
os.Open returns a file handle and you defer its close. On the
second iteration, another handle, another deferred close. By the
time the loop has touched 20,000 files, you are holding 20,000
open descriptors, and not one of them closes until
processFiles returns.

Your process has a file-descriptor limit. On Linux the soft
default is often 1024. You blow through it somewhere around the
1024th file, and the error you get back says nothing about
defer. It says too many open files, which sends people
hunting for a leaked socket or a missing Close call that does
not exist. The close is there. It is just queued behind the
entire loop.

Why this passes review and tests

The line defer f.Close() reads as "close this file when I am
done with it." For a one-shot function that opens a single file,
that reading is correct. The function does one thing, the defer
fires on the way out, the handle is released.

The ambiguity hides in the words "when I am done." In a loop,
you are done with each file at the end of its iteration. The
language is done with the file when the function returns. Those
are the same moment for a single file and very different moments
for ten thousand.

Tests miss it because fixtures are small. Three files, even
three hundred, stay under the descriptor limit, so the deferred
closes pile up harmlessly and all fire at the end. The leak only
becomes a crash once the input crosses the limit, which is
usually in production with real data.

go vet does not flag it either. The code is valid Go. The
deferred close is legal, the loop is legal, the descriptor
exhaustion is a runtime condition the compiler cannot see.

Fix 1: extract the body into its own function

The cleanest fix gives the defer a smaller function to scope
to. Pull the per-iteration work into a helper. Now each call
returns after one file, and the deferred close fires at the end
of each iteration instead of the end of the loop.

func processFiles(paths []string) error {
    for _, p := range paths {
        if err := processOne(p); err != nil {
            return err
        }
    }
    return nil
}

func processOne(p string) error {
    f, err := os.Open(p)
    if err != nil {
        return err
    }
    defer f.Close() // fires when processOne returns
    return handle(f)
}

processOne opens one file, does the work, and returns. The
defer f.Close() runs on that return, which happens once per
file. You never hold more than one descriptor at a time. The
helper also reads better: the unit of work has a name and a
clear boundary.

This is the fix I reach for first. It keeps the close right next
to the open, which is where you want it for readability, and it
makes the scoping correct without any extra ceremony.

Fix 2: a per-iteration closure

When extracting a named function feels heavy, or the body
captures a lot of loop-local state, wrap the iteration in an
inline function literal and call it on the spot. The deferred
call scopes to the closure, so it runs at the end of each
iteration.

func processFiles(paths []string) error {
    for _, p := range paths {
        err := func() error {
            f, err := os.Open(p)
            if err != nil {
                return err
            }
            defer f.Close() // scopes to this closure
            return handle(f)
        }()
        if err != nil {
            return err
        }
    }
    return nil
}

The immediately-invoked function literal is its own scope, so
its defer fires when the literal returns, once per iteration.
The cost is a slightly busier loop body and the easy-to-forget
trailing () that actually calls it. Forget the parentheses and
you have defined a function and never run it, which the compiler
catches as an unused value.

Reach for this when the body is short and tightly coupled to
loop variables. Reach for the named helper when the body is long
enough to deserve a name.

When defer in a loop is actually fine

The rule is not "never defer inside a loop." It is "do not defer
a resource release inside a loop that runs many times in one
function call."

A defer inside a loop is harmless when the resource it releases
is meant to live for the whole function anyway, or when the loop
runs a small, bounded number of times. Acquiring a handful of
locks you intend to hold until the function exits, for example,
is fine, that is the deferred unlock doing exactly its job.

The danger is the combination of three things: a defer, a loop
that can run an unbounded number of times, and a resource with a
hard limit behind it. File descriptors, database connections,
locks held longer than you think, anything the OS or a pool caps.
When all three line up, the deferred releases stack until the cap
runs out.

If you are unsure, ask one question. How many times can this loop
run in the worst case, and what is the limit on the thing I am
deferring? If the loop can run more times than the limit allows,
move the defer into a smaller scope.

The grep for Monday

Open your codebase and search for defer on a line that sits
inside a for. Most editors will let you eyeball the indentation.
For each hit, check what is being deferred and how many times the
loop can run.

grep -rn 'defer' --include='*.go' .

That lists every defer in the tree, not just the ones in
loops, so you still have to eyeball which sit inside a for. It
is a first-pass smell test, not a precise filter, but it puts
every candidate in front of you in a couple of minutes. For
each one that releases a capped resource in an unbounded loop,
apply Fix 1 or Fix 2. You will likely find at least one handle
leak that has not crashed yet only because nobody has fed it a
large enough input.

defer is one of Go's best ideas for keeping cleanup next to
setup. The loop is where its function-scoped timing turns from a
convenience into a trap. Once you have seen the timing once, you
read every defer in a loop differently.

If this was useful

Most of Go's sharp edges are like this one: a feature behaving
exactly as specified, meeting a second assumption you made
without noticing. The Complete Guide to Go Programming walks
through defer, the runtime mechanics behind it, and the rest of
the language quirks that survive code review. Hexagonal
Architecture in Go shows how to shape services so resource
ownership is obvious enough that this kind of leak has nowhere to
hide.