Brief outages during deployment: let requests finish

Check when the new version starts receiving traffic

In Kubernetes, a readiness probe determines whether a container is ready to receive traffic. Configure it so the new version passes only after the initialisation needed to serve requests. A startup probe can delay the other probes during a slow start. A running process alone does not give you enough evidence to send clients to it.

For RollingUpdate, review maxUnavailable, which limits unavailable Pods, and maxSurge, which allows extra Pods. Running both versions requires capacity. These settings do not by themselves ensure that requests already accepted will finish. In your deployment timeline, track when the new version becomes ready and when the old one shuts down.

Separate the routing change from process exit

Kubernetes EndpointSlices distinguish ready, serving and terminating conditions. Normal routing generally skips terminating endpoints, with an exception when all endpoints are terminating. Do you also use an ingress or an external load balancer? Check its behaviour when a backend shuts down. A change to one condition does not prove that every layer has stopped sending new requests.

During shutdown, let responses in progress finish before closing the resources they need. Record when shutdown starts, the number of active requests and when they complete. If the process keeps accepting new work, identify which layer sends it. Extending an arbitrary delay will not explain that difference.

Give the application its own shutdown procedure

In Node.js on Linux, installing a SIGTERM listener removes the default exit behaviour for that signal. The application must therefore also ensure that shutdown completes. For an HTTP/1 server in Node.js 19 and later, server.close() stops accepting new connections and closes idle ones. Use its callback to check that closure has completed; calling the method alone is insufficient.

The server.closeAllConnections() method forcibly closes active HTTP connections too, and does not itself close connections upgraded to WebSocket. For long responses, WebSocket connections and background work, define appropriate procedures with a finite deadline. Keep the database pool open while finishing work still needs it. Logs should distinguish completion from interruption after the deadline expires.

Test the full time budget under load

In Kubernetes, preStop runs before the termination signal is sent and consumes the same terminationGracePeriodSeconds budget as container shutdown. The default grace period is 30 seconds; exhausting it can lead to SIGKILL. Check any custom stop signal too. Account for the hook, finishing requests and closing resources within the same budget.

In a separate environment, run continuous traffic through the same path clients use and deploy a new version while it runs. Include a longer response and a slower startup. Measure errors, interrupted connections and shutdown duration. Then test the same procedure when rolling back to the previous version. Compare the deployment command's success with the outcome of client requests.

• Check that the new version is ready before it receives traffic.
• Check routing behaviour when a backend shuts down.
• Track requests finishing and resources closing.
• Test deployment and rollback under continuous load.