Repository navigation
Conversation
A free socket with an idle timeout is normally destroyed by its timer. When the event loop is blocked long enough for that timeout to expire, the timer cannot run, and `addRequest()` can reuse the socket even though the server may already have closed the connection. This can cause the request to fail with `ECONNRESET`. Record when a socket enters the free pool and, before reuse, remove and destroy a free socket whose idle timeout has expired according to the timers' clock. Refs: nodejs#54293 Assisted-by: a closed-source coding agent Signed-off-by: dennev <dennev13@gmail.com>
|
Review requested:
|
|
Welcome to Node.js, and thank you for your first contribution! Before review, please take a moment to read:
Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #67292 +/- ##
=========================================
Coverage 90.69% 90.70%
=========================================
Files 797 802 +5
Lines 278787 282134 +3347
Branches 53610 54364 +754
=========================================
+ Hits 252849 255900 +3051
- Misses 16310 16368 +58
- Partials 9628 9866 +238
🚀 New features to boost your workflow:
|
| socket = this.scheduling === 'fifo' ? | ||
| freeSockets.shift() : | ||
| freeSockets.pop(); | ||
| if (!isFreeSocketExpired(socket, now)) |
There was a problem hiding this comment.
| if (!isFreeSocketExpired(socket, now)) | |
| if (!socket.destroyed && !isFreeSocketExpired(socket, now)) |
Since this loop now walks the pool anyway, could it also skip sockets that are already destroyed? This will fix another bug
In LIFO mode a request made from setImmediate in that window still pops it and fails with econnreset
There was a problem hiding this comment.
Thanks, good catch. Applied in a1d6973, together with a test case for it.
I reproduced it on v26.12.1: with two sockets in the free pool and the default lifo scheduling, a request made right after the most recently freed socket is destroyed fails with ECONNRESET (socket hang up).
I ran the updated test on a release build on Linux, and it passed. Agent-run validation: the new case fails without the change; with it, the regression test, the HTTP/HTTPS tests and the full make -j12 test passed.
A destroyed socket can stay in the agent's free pool until its 'close' event. `addRequest()` only dropped destroyed sockets from the head of the pool, so a request made before that event could still take one, for example from the tail of the pool with the default `lifo` scheduling, and fail with `ECONNRESET`. Skip destroyed sockets wherever they are in the pool, and remove them from the agent at once, as is done for expired sockets. Assisted-by: a closed-source coding agent Signed-off-by: dennev <dennev13@gmail.com>
When the agent takes a socket from the free pool, it removes expired sockets until it finds one that has not expired. No case had a pool that holds both kinds, so nothing checked that the request then reuses the unexpired socket and that only the expired one is removed. Assisted-by: a closed-source coding agent Signed-off-by: dennev <dennev13@gmail.com>
|
I pushed one more commit, test only (13a482f). It adds a case, for I ran the updated test on a release build on Linux, and it passed. Agent-run validation: the new case fails without the first commit of this PR and passes with it; the regression test, the HTTP/HTTPS tests and the full |
| // after keepSocketAlive(). The socket's idle timer was last started then or | ||
| // earlier (at its last read or write), so this does not expire a socket before | ||
| // that timer is due. | ||
| function isFreeSocketExpired(socket, now) { |
There was a problem hiding this comment.
Instead of tracking a separate kFreeSince timestamp, could we read the socket's idle timer directly? It already records when it was last armed or refreshed:
const { kTimeout } = require('internal/timers');
function isFreeSocketExpired(socket, now) {
const timer = socket[kTimeout];
return timer != null && timer._idleTimeout > 0 &&
now - timer._idleStart >= timer._idleTimeout;
}This removes the getLibuvNow() call and the extra property write from the 'free' handler, so the only new cost on the reuse path is the single clock read in addRequest().
It's also closer to the timer it's emulating. _idleStart is refreshed on every read and write (Socket.prototype._unrefTimer). When keepSocketAlive() doesn't call setTimeout() because socket.timeout already matches, the timer keeps counting from the last I/O, which is close to when the server's keep-alive timer started. That covers the "blocked in 'data'/'end' before the socket enters the pool" case listed as not covered. When setTimeout() is called, _idleStart is reset at that moment, which matches kFreeSince today.
The _idleTimeout > 0 check also handles a cleared timer. setStreamTimeout(0) now keeps the cleared Timeout in this[kTimeout] for reuse, and unenroll() sets its _idleTimeout to -1.
There was a problem hiding this comment.
Thanks, applied in 9ac5f02. isFreeSocketExpired() now reads socket[kTimeout], and the 'free' handler no longer reads the clock or writes kFreeSince. I also reworded the doc sentence to "a free socket whose timeout has passed", since the time spent in the pool is no longer what is measured.
One difference from your snippet: I kept the socket.timeout > 0 check. A timer that has fired keeps its _idleTimeout when it is cleared afterwards, because unenroll() returns early for a destroyed timer (lib/timers.js). Without the check, a socket with no timeout whose request timeout had fired while the response waited unread would count as expired and not be reused. The new testFiredTimeoutCleared case covers this; testRestartedTimeout covers a pooled socket whose timeout is set again, which the previous check expired from the time it entered the pool.
One trade-off of reading the timer: a stream from a custom createConnection() that keeps its own timeout without this timer is no longer expired by the clock; with the previous check it was, if it exposed timeout.
One caveat on the coverage: the "blocked in 'data'/'end'" case is still not covered in the client's own flow. When the request or the agent has a timeout, responseKeepAlive() clears the socket's timer with socket.setTimeout(0, req.timeoutCb) (lib/internal/http/client.js). With an agent timeout, keepSocketAlive() then sees socket.timeout === 0 and calls setTimeout() again as the socket enters the pool, which resets _idleStart there. Agent-run check: with a 300 ms block in the 'end' handler of a request through new Agent({ keepAlive: true, timeout: 100 }), the next request still reuses the socket with this change, so I kept that case under "Not covered" in the description.
Agent-run validation: testRestartedTimeout fails without this commit, testFiredTimeoutCleared fails without the socket.timeout > 0 check, and both pass with it; the regression test passed 200 of 200 runs, the HTTP/HTTPS and agent-related tests and the full make -j4 test passed, and the regression test also passed on a debug build, where its fast API call assertion runs.
…6171) The keep-alive timer of an idle socket cannot fire while the event loop is blocked. When the next request is dispatched from the poll phase (an I/O callback, a server request handler, or a continuation after awaiting I/O), the idle socket validation added for GHSA-35p6-xmwp-9g52 completes in the check phase of the same loop iteration, before that timer fires and before a FIN received during the block is read. The request is then written to a socket that the server may already have closed, and fails with ECONNRESET. Record when the keep-alive timer is armed, and destroy the socket with an InformationalError instead of reusing it once its keep-alive timeout has passed. Nothing is running on the socket, so the pending requests are sent on a new connection. The check does not depend on kSocketUsed, so it also covers the experimental Milo parser, which never sets that symbol. Refs: nodejs/node#67292 Claude-Session: https://claude.ai/code/session_01Sfj4nEWQ4ovycNDjbvXcRs Signed-off-by: Matteo Collina <hello@matteocollina.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Instead of recording when a socket enters the free pool, check the socket's idle timer, which records on the timers' clock when it was last started or refreshed. This removes the clock read and the property write from the 'free' handler, and a pooled socket whose timeout is set again is now expired by its restarted timer. With an agent timeout, the client clears the socket's timeout when it keeps the socket alive, and the agent restarts the timer as the socket enters the pool, so in that flow the expiry is measured from the same moment as before. socket.timeout is still checked, because a timer that fired before it was cleared keeps its _idleTimeout. Assisted-by: a closed-source coding agent Signed-off-by: dennev <dennev13@gmail.com>
When the event loop is blocked long enough for a free socket's idle timeout to expire, its timeout callback cannot run. The next request can then reuse that socket from
agent.freeSocketseven though the server may already have closed the connection, causing the request to fail withECONNRESET.This change addresses the timeout-expiry case discussed in #54293. In
addRequest(), the agent reads the timers' clock (timersBinding.getLibuvNow()) once and checks each free candidate against its idle timer (socket[kTimeout]), which records on the same clock when it was last started or refreshed. Expired candidates are removed and destroyed until an unexpired socket is found or the pool is empty. A socket without a timeout (socket.timeoutis 0), or a custom stream that does not have this timer, is never expired this way.Following review, a second commit also keeps
addRequest()from handing out a free socket that is already destroyed, wherever it is in the pool. A destroyed candidate is removed from the agent at once, like an expired one. Destroyed sockets at the head of the pool are still dropped by the existing loop, as before, and stay counted inagent.totalSocketCountuntil their'close'event. A third commit adds a test for a pool that holds an expired and an unexpired socket, and a fourth, following a review suggestion, reads the expiry from the socket's idle timer instead of a timestamp recorded when the socket entered the pool.The clock call preserves the binding object as its receiver, as required by the fast API call. The regression test also exercises an optimized
addRequest().Not covered by this change:
timeout(http: Agent with keepAlive and no timeout ignores the server's Keep-Alive timeout hint #66734, http: respect Keep-Alive timeout hint when agent has no explicit timeout #66779)'free'handler to a queued request'data'or'end'handler: with an agent timeout, the client clears the socket's timeout when it keeps the socket alive, and the agent restarts the idle timer as the socket enters the pool, so time spent blocked before that point is not countedTesting: I also ran a reproduction based on #54293, with
server.keepAliveTimeout = 5000and a 6-second block between two requests throughhttp.globalAgent. It fails withECONNRESETon v26.12.1 and succeeds with the first commit, using a new socket.Agent-run validation confirmed that the regression test fails without this change and passes with it. It covers LIFO and FIFO expiry, reuse before expiry, unchanged reuse without an idle timeout,
maxSocketsaccounting after a block, a pool that holds an expired and an unexpired socket (LIFO and FIFO), a pooled socket whose timeout is set again, a request timeout that fired and was cleared, a destroyed socket at the tail of the pool with LIFO, and optimizedaddRequest().Agent-run validation at the current head (Linux, release build): the regression test, the HTTP/HTTPS and agent-related tests and the full
make -j4 testpassed, and the regression test also passed on a debug build, where its fast API call assertion runs. In agent runs of the same cases, the mixed-pool case fails without the first commit, the destroyed-socket case fails without the second, and the restarted-timeout case fails without the fourth. An agent-run reproduction based on #54293 also fails withECONNRESETon v26.13.1 (official Docker image, Linux).AI disclosure: Claude Code (Claude Opus 5.5) was used to investigate the cause, write and review the code, test and documentation changes, and run the agent-run validation described above. Before opening this pull request, Claude and Codex (gpt-6.1-sol) reviewed the first commit, its commit message and the description as first posted. The second and third commits were prepared the same way with Claude Code (Claude Fable 5.2), and the fourth with Claude Code (Claude Opus 5.5). Claude reviewed the later commits and their messages before I pushed them, and drafts of my comments and of this updated description before I posted them; Codex also reviewed the third and fourth commits.
Refs: #54293