Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 53 additions & 1 deletion deps/undici/src/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,8 @@ Installing undici as a module allows you to use a newer version than what's bund

## Quick Start

### Basic Request

```js
import { request } from 'undici'

Expand All @@ -184,6 +186,50 @@ for await (const data of body) { console.log('data', data) }
console.log('trailers', trailers)
```

### Using Cache Interceptor

Undici provides a powerful HTTP caching interceptor that follows HTTP caching best practices. Here's how to use it:

```js
import { fetch, Agent, interceptors, cacheStores } from 'undici';

// Create a client with cache interceptor
const client = new Agent().compose(interceptors.cache({
// Optional: Configure cache store (defaults to MemoryCacheStore)
store: new cacheStores.MemoryCacheStore({
maxSize: 100 * 1024 * 1024, // 100MB
maxCount: 1000,
maxEntrySize: 5 * 1024 * 1024 // 5MB
}),

// Optional: Specify which HTTP methods to cache (default: ['GET', 'HEAD'])
methods: ['GET', 'HEAD']
}));

// Set the global dispatcher to use our caching client
setGlobalDispatcher(client);

// Now all fetch requests will use the cache
async function getData() {
const response = await fetch('https://api.example.com/data');
// The server should set appropriate Cache-Control headers in the response
// which the cache will respect based on the cache policy
return response.json();
}

// First request - fetches from origin
const data1 = await getData();

// Second request - served from cache if within max-age
const data2 = await getData();
```

#### Key Features:
- **Automatic Caching**: Respects `Cache-Control` and `Expires` headers
- **Validation**: Supports `ETag` and `Last-Modified` validation
- **Storage Options**: In-memory or persistent SQLite storage
- **Flexible**: Configure cache size, TTL, and more

## Global Installation

Undici provides an `install()` function to add all WHATWG fetch classes to `globalThis`, making them available globally:
Expand Down Expand Up @@ -472,7 +518,7 @@ Note that consuming the response body is _mandatory_ for `request`:
```js
// Do
const { body, headers } = await request(url);
await res.body.dump(); // force consumption of body
await body.dump(); // force consumption of body

// Do not
const { headers } = await request(url);
Expand All @@ -487,6 +533,12 @@ const { headers } = await request(url);

The [Fetch Standard](https://fetch.spec.whatwg.org) requires implementations to exclude certain headers from requests and responses. In browser environments, some headers are forbidden so the user agent remains in full control over them. In Undici, these constraints are removed to give more control to the user.

#### Content-Encoding

* https://www.rfc-editor.org/rfc/rfc9110#field.content-encoding

Undici limits the number of `Content-Encoding` layers in a response to **5** to prevent resource exhaustion attacks. If a server responds with more than 5 content-encodings (e.g., `Content-Encoding: gzip, gzip, gzip, gzip, gzip, gzip`), the fetch will be rejected with an error. This limit matches the approach taken by [curl](https://curl.se/docs/CVE-2022-32206.html) and [urllib3](https://github.com/advisories/GHSA-gm62-xv2j-4rw9).

#### `undici.upgrade([url, options]): Promise`

Upgrade to a different protocol. See [MDN - HTTP - Protocol upgrade mechanism](https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism) for more details.
Expand Down
7 changes: 3 additions & 4 deletions deps/undici/src/build/wasm.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ const WASM_CC = process.env.WASM_CC || 'clang'
let WASM_CFLAGS = process.env.WASM_CFLAGS || '--sysroot=/usr/share/wasi-sysroot -target wasm32-unknown-wasi'
let WASM_LDFLAGS = process.env.WASM_LDFLAGS || ''
const WASM_LDLIBS = process.env.WASM_LDLIBS || ''
const WASM_OPT = process.env.WASM_OPT || './wasm-opt'
const WASM_OPT = process.env.WASM_OPT || 'wasm-opt'

// For compatibility with Node.js' `configure --shared-builtin-undici/undici-path ...`
const EXTERNAL_PATH = process.env.EXTERNAL_PATH
Expand Down Expand Up @@ -65,7 +65,7 @@ if (process.argv[2] === '--docker') {
-t ${WASM_BUILDER_CONTAINER} node build/wasm.js`
console.log(`> ${cmd}\n\n`)
execSync(cmd, { stdio: 'inherit' })
process.exit(0)
process.exit(0) // eslint-disable-line n/no-process-exit
}

const hasApk = (function () {
Expand All @@ -78,8 +78,7 @@ if (hasApk) {
// Gather information about the tools used for the build
const buildInfo = execSync('apk info -v').toString()
if (!buildInfo.includes('wasi-sdk')) {
console.log('Failed to generate build environment information')
process.exit(-1)
throw new Error('Failed to generate build environment information')
}
console.log(buildInfo)
}
Expand Down
3 changes: 2 additions & 1 deletion deps/undici/src/docs/docs/api/Client.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,11 @@ Returns: `Client`
* **maxResponseSize** `number | null` (optional) - Default: `-1` - The maximum length of response body in bytes. Set to `-1` to disable.
* **pipelining** `number | null` (optional) - Default: `1` - The amount of concurrent requests to be sent over the single TCP/TLS connection according to [RFC7230](https://tools.ietf.org/html/rfc7230#section-6.3.2). Carefully consider your workload and environment before enabling concurrent requests as pipelining may reduce performance if used incorrectly. Pipelining is sensitive to network stack settings as well as head of line blocking caused by e.g. long running requests. Set to `0` to disable keep-alive connections.
* **connect** `ConnectOptions | Function | null` (optional) - Default: `null`.
* **strictContentLength** `Boolean` (optional) - Default: `true` - Whether to treat request content length mismatches as errors. If true, an error is thrown when the request content-length header doesn't match the length of the request body.
* **strictContentLength** `Boolean` (optional) - Default: `true` - Whether to treat request content length mismatches as errors. If true, an error is thrown when the request content-length header doesn't match the length of the request body. **Security Warning:** Disabling this option can expose your application to HTTP Request Smuggling attacks, where mismatched content-length headers cause servers and proxies to interpret request boundaries differently. This can lead to cache poisoning, credential hijacking, and bypassing security controls. Only disable this in controlled environments where you fully trust the request source.
* **autoSelectFamily**: `boolean` (optional) - Default: depends on local Node version, on Node 18.13.0 and above is `false`. Enables a family autodetection algorithm that loosely implements section 5 of [RFC 8305](https://tools.ietf.org/html/rfc8305#section-5). See [here](https://nodejs.org/api/net.html#socketconnectoptions-connectlistener) for more details. This option is ignored if not supported by the current Node version.
* **autoSelectFamilyAttemptTimeout**: `number` - Default: depends on local Node version, on Node 18.13.0 and above is `250`. The amount of time in milliseconds to wait for a connection attempt to finish before trying the next address when using the `autoSelectFamily` option. See [here](https://nodejs.org/api/net.html#socketconnectoptions-connectlistener) for more details.
* **allowH2**: `boolean` - Default: `false`. Enables support for H2 if the server has assigned bigger priority to it through ALPN negotiation.
* **useH2c**: `boolean` - Default: `false`. Enforces h2c for non-https connections.
* **maxConcurrentStreams**: `number` - Default: `100`. Dictates the maximum number of concurrent streams for a single H2 session. It can be overridden by a SETTINGS remote frame.

> **Notes about HTTP/2**
Expand Down
57 changes: 57 additions & 0 deletions deps/undici/src/docs/docs/api/DiagnosticsChannel.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,3 +254,60 @@ diagnosticsChannel.channel('undici:websocket:pong').subscribe(({ payload, websoc
console.log(websocket) // the WebSocket instance
})
```

## `undici:proxy:connected`

This message is published after the `ProxyAgent` establishes a connection to the proxy server.

```js
import diagnosticsChannel from 'diagnostics_channel'

diagnosticsChannel.channel('undici:proxy:connected').subscribe(({ socket, connectParams }) => {
console.log(socket)
console.log(connectParams)
// const { origin, port, path, signal, headers, servername } = connectParams
})
```

## `undici:request:pending-requests`

This message is published when the deduplicate interceptor's pending request map changes. This is useful for monitoring and debugging request deduplication behavior.

The deduplicate interceptor automatically deduplicates concurrent requests for the same resource. When multiple identical requests are made while one is already in-flight, only one request is sent to the origin server, and all waiting handlers receive the same response.

```js
import diagnosticsChannel from 'diagnostics_channel'

diagnosticsChannel.channel('undici:request:pending-requests').subscribe(({ type, size, key }) => {
console.log(type) // 'added' or 'removed'
console.log(size) // current number of pending requests
console.log(key) // the deduplication key for this request
})
```

### Event Properties

- `type` (`string`): Either `'added'` when a new pending request is registered, or `'removed'` when a pending request completes (successfully or with an error).
- `size` (`number`): The current number of pending requests after the change.
- `key` (`string`): The deduplication key for the request, composed of the origin, method, path, and request headers.

### Example: Monitoring Request Deduplication

```js
import diagnosticsChannel from 'diagnostics_channel'

const channel = diagnosticsChannel.channel('undici:request:pending-requests')

channel.subscribe(({ type, size, key }) => {
if (type === 'added') {
console.log(`New pending request: ${key} (${size} total pending)`)
} else {
console.log(`Request completed: ${key} (${size} remaining)`)
}
})
```

This can be useful for:
- Verifying that request deduplication is working as expected
- Monitoring the number of concurrent in-flight requests
- Debugging deduplication behavior in production environments
86 changes: 86 additions & 0 deletions deps/undici/src/docs/docs/api/Dispatcher.md
Original file line number Diff line number Diff line change
Expand Up @@ -1043,6 +1043,7 @@ The `dns` interceptor enables you to cache DNS lookups for a given duration, per
- The function should return a single record from the records array.
- By default a simplified version of Round Robin is used.
- The `records` property can be mutated to store the state of the balancing algorithm.
- `storage: DNSStorage` - Custom storage for resolved DNS records

> The `Dispatcher#options` also gets extended with the options `dns.affinity`, `dns.dualStack`, `dns.lookup` and `dns.pick` which can be used to configure the interceptor at a request-per-request basis.

Expand All @@ -1057,6 +1058,14 @@ It represents a map of DNS IP addresses records for a single origin.
- `4.ips` - (`DNSInterceptorRecord[] | null`) The IPv4 addresses.
- `6.ips` - (`DNSInterceptorRecord[] | null`) The IPv6 addresses.

**DNSStorage**
It represents a storage object for resolved DNS records.
- `size` - (`number`) current size of the storage.
- `get` - (`(origin: string) => DNSInterceptorOriginRecords | null`) method to get the records for a given origin.
- `set` - (`(origin: string, records: DNSInterceptorOriginRecords | null, options: { ttl: number }) => void`) method to set the records for a given origin.
- `delete` - (`(origin: string) => void`) method to delete records for a given origin.
- `full` - (`() => boolean`) method to check if the storage is full, if returns `true`, DNS lookup will be skipped in this interceptor and new records will not be stored.

**Example - Basic DNS Interceptor**

```js
Expand All @@ -1073,6 +1082,45 @@ const response = await client.request({
})
```

**Example - DNS Interceptor and LRU cache as a storage**

```js
const { Client, interceptors } = require("undici");
const QuickLRU = require("quick-lru");
const { dns } = interceptors;

const lru = new QuickLRU({ maxSize: 100 });

const lruAdapter = {
get size() {
return lru.size;
},
get(origin) {
return lru.get(origin);
},
set(origin, records, { ttl }) {
lru.set(origin, records, { maxAge: ttl });
},
delete(origin) {
lru.delete(origin);
},
full() {
// For LRU cache, we can always store new records,
// old records will be evicted automatically
return false;
}
}

const client = new Agent().compose([
dns({ storage: lruAdapter })
])

const response = await client.request({
origin: `http://localhost:3030`,
...requestOpts
})
```

##### `responseError`

The `responseError` interceptor throws an error for responses with status code errors (>= 400).
Expand Down Expand Up @@ -1165,6 +1213,44 @@ The `cache` interceptor implements client-side response caching as described in
- `cacheByDefault` - The default expiration time to cache responses by if they don't have an explicit expiration and cannot have an heuristic expiry computed. If this isn't present, responses neither with an explicit expiration nor heuristically cacheable will not be cached. Default `undefined`.
- `type` - The [type of cache](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Caching#types_of_caches) for Undici to act as. Can be `shared` or `private`. Default `shared`. `private` implies privately cacheable responses will be cached and potentially shared with other users of your application.

##### `Deduplicate Interceptor`

The `deduplicate` interceptor deduplicates concurrent identical requests. When multiple identical requests are made while one is already in-flight, only one request is sent to the origin server, and all waiting handlers receive the same response. This reduces server load and improves performance.

**Options**

- `methods` - The [**safe** HTTP methods](https://www.rfc-editor.org/rfc/rfc9110#section-9.2.1) to deduplicate. Default `['GET']`.
- `skipHeaderNames` - Header names that, if present in a request, will cause the request to skip deduplication entirely. Useful for headers like `idempotency-key` where presence indicates unique processing. Header name matching is case-insensitive. Default `[]`.
- `excludeHeaderNames` - Header names to exclude from the deduplication key. Requests with different values for these headers will still be deduplicated together. Useful for headers like `x-request-id` that vary per request but shouldn't affect deduplication. Header name matching is case-insensitive. Default `[]`.

**Usage**

```js
const { Client, interceptors } = require("undici");
const { deduplicate, cache } = interceptors;

// Deduplicate only
const client = new Client("http://example.com").compose(
deduplicate()
);

// Deduplicate with caching
const clientWithCache = new Client("http://example.com").compose(
deduplicate(),
cache()
);
```

Requests are considered identical if they have the same:
- Origin
- HTTP method
- Path
- Request headers (excluding any headers specified in `excludeHeaderNames`)

All deduplicated requests receive the complete response including status code, headers, and body.

For observability, request deduplication events are published to the `undici:request:pending-requests` [diagnostic channel](/docs/docs/api/DiagnosticsChannel.md#undicirequestpending-requests).

## Instance Events

### Event: `'connect'`
Expand Down
2 changes: 1 addition & 1 deletion deps/undici/src/docs/docs/api/H2CClient.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ Returns: `H2CClient`
- **maxConcurrentStreams**: `number` - Default: `100`. Dictates the maximum number of concurrent streams for a single H2 session. It can be overridden by a SETTINGS remote frame.
- **pipelining** `number | null` (optional) - Default to `maxConcurrentStreams` - The amount of concurrent requests sent over a single HTTP/2 session in accordance with [RFC-7540](https://httpwg.org/specs/rfc7540.html#StreamsLayer) Stream specification. Streams can be closed up by remote server at any time.
- **connect** `ConnectOptions | null` (optional) - Default: `null`.
- **strictContentLength** `Boolean` (optional) - Default: `true` - Whether to treat request content length mismatches as errors. If true, an error is thrown when the request content-length header doesn't match the length of the request body.
- **strictContentLength** `Boolean` (optional) - Default: `true` - Whether to treat request content length mismatches as errors. If true, an error is thrown when the request content-length header doesn't match the length of the request body. **Security Warning:** Disabling this option can expose your application to HTTP Request Smuggling attacks, where mismatched content-length headers cause servers and proxies to interpret request boundaries differently. This can lead to cache poisoning, credential hijacking, and bypassing security controls. Only disable this in controlled environments where you fully trust the request source.
- **autoSelectFamily**: `boolean` (optional) - Default: depends on local Node version, on Node 18.13.0 and above is `false`. Enables a family autodetection algorithm that loosely implements section 5 of [RFC 8305](https://tools.ietf.org/html/rfc8305#section-5). See [here](https://nodejs.org/api/net.html#socketconnectoptions-connectlistener) for more details. This option is ignored if not supported by the current Node version.
- **autoSelectFamilyAttemptTimeout**: `number` - Default: depends on local Node version, on Node 18.13.0 and above is `250`. The amount of time in milliseconds to wait for a connection attempt to finish before trying the next address when using the `autoSelectFamily` option. See [here](https://nodejs.org/api/net.html#socketconnectoptions-connectlistener) for more details.

Expand Down
Loading
Loading