Skip to content

Conversation

@dwisiswant0
Copy link
Member

Changes:

  • sizedpool:
    feat(sizedpool): add `Discard` method to release semaphore slot
    
    Add `(*SizedPool[T]).Discard` method that releases the semaphore
    slot without returning the item to the underlying
    [sync.Pool]. This enables callers to properly
    release resources when intentionally discarding
    items (e.g., oversized buffers) instead of
    returning them to the pool.
    
    Also, refactor `(*SizedPool[T]).Put()` to use
    `(*SizedPool[T]).Discard()` internally.
    
  • httputil:
    fix(httputil): semaphore leak causing deadlock in buffer pool
    
    When `putBuffer()` discarded buffers (either
    oversized or when the large buffer channel was
    full), the sizedpool semaphore slot acquired by
    `getBuffer()` was never released. This caused
    semaphore exhaustion and eventual deadlock under
    sustained load with large response bodies.
    
    Call `bufPool.Discard()` in both discard paths to
    properly release the semaphore slot.
    

Proof

patch:

$ go test -race -run ^TestSemaphoreLeakDeadlock$ ./http
ok  	github.com/projectdiscovery/utils/http	3.451s

main:

$ git cherry-pick fcf5221
[main 8a60f1f] test(httputil): add TestSemaphoreLeakDeadlock
 Date: Fri Dec 19 20:47:49 2025 +0700
 1 file changed, 68 insertions(+)
$ go test -race -run ^TestSemaphoreLeakDeadlock$ ./http
--- FAIL: TestSemaphoreLeakDeadlock (1.24s)
    respChain_test.go:1267: iteration 1, response 0
    respChain_test.go:1267: iteration 1, response 1
    respChain_test.go:1267: iteration 2, response 0
    respChain_test.go:1267: iteration 2, response 1
    respChain_test.go:1267: iteration 3, response 0
    respChain_test.go:1267: iteration 3, response 1
    respChain_test.go:1267: iteration 4, response 0
    respChain_test.go:1267: iteration 4, response 1
    respChain_test.go:1267: iteration 5, response 0
    respChain_test.go:1267: iteration 5, response 1
    respChain_test.go:1267: iteration 6, response 0
    respChain_test.go:1267: iteration 6, response 1
    respChain_test.go:1267: iteration 7, response 0
    respChain_test.go:1267: iteration 7, response 1
    respChain_test.go:1267: iteration 8, response 0
    respChain_test.go:1267: iteration 8, response 1
    respChain_test.go:1267: iteration 9, response 0
    respChain_test.go:1267: iteration 9, response 1
    respChain_test.go:1267: iteration 10, response 0
    respChain_test.go:1267: iteration 10, response 1
    respChain_test.go:1267: iteration 11, response 0
    respChain_test.go:1267: iteration 11, response 1
    respChain_test.go:1291: Deadlock detected at iteration 11
FAIL
FAIL	github.com/projectdiscovery/utils/http	1.274s
FAIL

dwisiswant0 and others added 3 commits December 19, 2025 20:48
Co-authored-by: Egor <ezzer17@users.noreply.github.com>
Signed-off-by: Dwi Siswanto <git@dw1.io>
Add `(*SizedPool[T]).Discard` method that releases the semaphore
slot without returning the item to the underlying
[sync.Pool]. This enables callers to properly
release resources when intentionally discarding
items (e.g., oversized buffers) instead of
returning them to the pool.

Also, refactor `(*SizedPool[T]).Put()` to use
`(*SizedPool[T]).Discard()` internally.

Signed-off-by: Dwi Siswanto <git@dw1.io>
When `putBuffer()` discarded buffers (either
oversized or when the large buffer channel was
full), the sizedpool semaphore slot acquired by
`getBuffer()` was never released. This caused
semaphore exhaustion and eventual deadlock under
sustained load with large response bodies.

Call `bufPool.Discard()` in both discard paths to
properly release the semaphore slot.

Fixes #714.

Signed-off-by: Dwi Siswanto <git@dw1.io>
@Mzack9999 Mzack9999 merged commit c69ecd0 into main Dec 19, 2025
7 checks passed
@Mzack9999 Mzack9999 deleted the dwisiswant0/fix/respchain-deadlock branch December 19, 2025 14:40
@Mzack9999 Mzack9999 added the Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. label Dec 19, 2025
@Mzack9999 Mzack9999 linked an issue Dec 19, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deadlock in sizedpool

3 participants