From dc19819f467c9d796fe8087778c18e8d5e011610 Mon Sep 17 00:00:00 2001 From: Maxim Ivanov Date: Thu, 6 Nov 2025 12:43:25 +0000 Subject: [PATCH] Prevent snowballing connection creation on context cancellation. Change deadline delay for default DeadlineContextWatcherHander to prevent snowballing effect of establishing connections in excess of configured connection pool maximum size in pathological case with many context cancellations. Previously, with default deadline of 0, on context cancellation pgx was immediately marking connection as closed, even though pgConn.asyncClose still needs time to complete and PG need time to act on the cancellation request. Connection pooler (pgx pool or stdlib sql) checks `IsClosed()` state on Release, recognizes it as closed and immediately removes it from the pool, allowing new connection to be established. If new connection establishment time is less than full end-to-end query cancellation time, including time a server needs to act on cancellation request, and there are many context cancellations events happening per second, then number of connections seen by a PostgreSQL server start accumulate, exceeding configured total pool size. It eventually reaches `max_connections` limit on a server, with the majority of them being cancelled. No new connection can be established, bring app to a halt. New default deadline delay of 50ms is chosen to be larger than pessimistic time for a full query cancellation completion. With the new deadline pgx keeps connection in the pool after cancellation for up to 50ms, thus effectively throttling how fast cancelled queries can be replaced with a new connection. Because new connection creation is now throttled, it can't overrun query cancellations, preventing the snowball effect. It also has nice side effect of increasing a chance of connection reuse if what would be cancelled query completes in less than 50ms. --- pgconn/config.go | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/pgconn/config.go b/pgconn/config.go index 3cd5d8773..5f448a3ea 100644 --- a/pgconn/config.go +++ b/pgconn/config.go @@ -294,7 +294,10 @@ func ParseConfigWithOptions(connString string, options ParseConfigOptions) (*Con return pgproto3.NewFrontend(r, w) }, BuildContextWatcherHandler: func(pgConn *PgConn) ctxwatch.Handler { - return &DeadlineContextWatcherHandler{Conn: pgConn.conn} + return &DeadlineContextWatcherHandler{ + Conn: pgConn.conn, + DeadlineDelay: 50 * time.Millisecond, + } }, OnPgError: func(_ *PgConn, pgErr *PgError) bool { // we want to automatically close any fatal errors