me · demo · go · cv · notes

Graceful shutdown in Go: the five places it actually breaks

A Go service that handles SIGTERM in main is not done. Shutdown is a distributed problem disguised as a signal handler, and it usually breaks in one of five places.

1. The signal arrives before the server starts

If signal.Notify is registered after ListenAndServe, a fast deploy can kill the process during startup. Register the context first, pass it into the server, and let startup observe cancellation.

2. In-flight requests are not drained

http.Server.Shutdown(ctx) stops accepting new connections and waits for active ones. The bug is the context: a 5-second timeout on a 30-second report endpoint turns a graceful deploy into a 500. Set the drain deadline from the slowest route, or return 503 with Retry-After instead of truncating.

3. The database closes before the handlers finish

defer db.Close() runs when main returns, but handlers may still be using the pool. Close the pool only after Shutdown returns, and give background workers their own wait group. The same applies to Redis, queues, and anything with a connection.

4. Background goroutines have no ownership

A worker started with go func() and no cancellation is a leak with a schedule. Give every long-lived goroutine a context and a WaitGroup; on shutdown, cancel first, then wait with a timeout, then log what refused to stop.

5. The test never exercises shutdown

If no test sends SIGTERM, shutdown is untested code. A useful test starts the server on a random port, sends a request, triggers cancellation, and asserts the request completes and the listener is closed. In the Go template here, that behavior is covered by the handler/server tests and the make test target; the coverage report is checked in rather than quoted from memory.

A small checklist

None of this is exotic. It is the difference between a service that restarts cleanly under load and one that drops requests during every deploy.