Proposal: Allow raw data type to be serialized as-is

[mxxk]

Summary

Would it make sense for structured-headers to support a raw data type which is meant to represent pre-serialized data? This would be somewhat similar to JSON.rawJSON() except for structured-headers (but with key differences; see the Security notes below).

Description

For example, suppose you are generating an HTTP signature using structured-headers directly. Doing so requires serializing the same Inner List value twice:

const signatureParameters = [
  ["@method", "@target-uri"],
  new Map([
    ["alg", "hmac-sha256"],
    ["created", Math.floor(new Date() / 1_000)],
  ]),
];
const serializedSignatureParameters = structuredHeaders.serializeInnerList(signatureParameters);
const signatureInput = structuredHeaders.serializeDictionary({
  sig: signatureParameters,
});

In the above code snippet, signatureParameters is first serialized as a standalone Inner List, and then serialized again as an Inner List enclosed in a Dictionary.

If structured-headers had a way to represent pre-serialized values, this double-serialization could be eliminated by explicitly passing a "raw" value:

const serializedSignatureParameters = structuredHeaders.serializeInnerList(signatureParameters);
const signatureInput = structuredHeaders.serializeDictionary({
  sig: new structuredHeaders.RawValue(serializedSignatureParameters),
});

(Of course, in this specific example, it is possible to simply prepend "sig=" to serializedSignatureParameters, but that would be side-stepping the serialization provided by structured-headers.)

What do you think?

Workaround

It is actually possible to do this today with a hacky workaround:

class RawValue extends structuredHeaders.Token {
  constructor(value) {
    // `Token` strictly validates its value, so pass a dummy string...
    super("a");
    // ...and then reassign the actual value.
    this.value = value;
  }
}

const serializedSignatureParameters = structuredHeaders.serializeInnerList(signatureParameters);
const signatureInput = structuredHeaders.serializeDictionary({
  sig: new RawValue(serializedSignatureParameters),
});

Additional information

The above code snippet was extracted from a larger example of producing HTTP signatures, which is included below for completeness:

import crypto from "node:crypto";
import * as structuredHeaders from "structured-headers";

const body = "Hello, world!";
const targetUri = new URL("https://example.com/foo?param=value&pet=dog");
const signatureComponents = {
  "@method": "GET",
  "@target-uri": targetUri.href,
};
const signatureParameters = [
  Object.keys(signatureComponents),
  new Map([
    ["alg", "hmac-sha256"],
    ["created", Math.floor(new Date() / 1_000)],
  ]),
];
const signatureLabel = "sig";

// NOTE: This is for illustration purposes only, and glosses over complexities
// of generating the signature base.
const signatureBase = Object.entries({
  ...signatureComponents,
  "@signature-params":
    structuredHeaders.serializeInnerList(signatureParameters),
})
  .map(([name, value]) => `${structuredHeaders.serializeItem(name)}: ${value}`)
  .join("\n");

const requestTarget = targetUri.href.slice(targetUri.origin.length);
const signature = crypto
  .createHmac("sha256", "secret")
  .update(signatureBase)
  .digest();
const headers = {
  Host: targetUri.host,
  "Signature-Input": structuredHeaders.serializeDictionary({
    [signatureLabel]: signatureParameters,
  }),
  Signature: structuredHeaders.serializeDictionary({
    [signatureLabel]: signature.buffer,
  }),
};

console.log(`${signatureComponents["@method"]} ${requestTarget} HTTP/1.1`);
for (const [header, value] of Object.entries(headers)) {
  console.log(`${header}: ${value}`);
}
console.log(`\n${body}`);

Security notes

JSON.rawJSON() was used as an inspiration for this request, but its implementation differs in some key ways:

1. JSON.rawJSON() does not support creation of objects and arrays.

   Presumably, this is to reduce the severity of security issues around untrusted input being passed to JSON.rawJSON(). However, without seeing discussion notes leading up this implementation decision, I can't say for sure.

   It may be useful for this proposed structuredHeaders.RawValue type to likewise prohibit Lists and Dictionaries, but allow representation of all other types. This is because Lists and Dictionaries can only appear at the top level, so neither type can be enclosed within a parent:

   const serializedDictionary = structuredHeaders.serializeDictionary({
     a: 1,
     b: 2,
   });
   structuredHeaders.serializeList([
     3,
     // INVALID! Lists and Dictionary types can only appear at the top level.
     new structuredHeaders.RawValue(serializedDictionary),
   ]);

2. JSON.rawJSON() rejects malformed JSON.

   Likewise, structuredHeaders.RawValue may reject malformed input.

Warning

For both of the above points, it would be required to parse/validate the raw string passed to structuredHeaders.RawValue, which would have a performance impact. Since this issue wasn't long enough already, 😄 I wrote up a potential alternative below:

I may be going down a rabbit hole here, but one idea to work around this trade-off is to limit structuredHeaders.RawValue instances to only originate from dedicated serialization functions. For example, these functions could be made available under structuredHeaders.raw to not pollute the top-level package namespace:

// structured-headers/src/raw.js

const rawValue = Symbol("rawValue");

class RawValue {
  // Allow extraction of raw value string. Not having this limits the utility of
  // `RawValue` .
  toString() {
    return this[rawValue];
  }
}

function createRawValue(value) {
  return Object.assign(new RawValue(), { [rawValue]: value });
}

export function serializeItem(input, params) {
  return createRawValue(structuredHeaders.serializeItem(input, params));
}

// NOTE: `serializeDictionary` and `serializeList` are intentionally omitted per (1) above.
//
// Same implementation for the rest:
//
// export function serializeInnerList ...
// export function serializeItem ...
// export function serializeInnerList ...
// export function serializeBareItem ...
// export function serializeInteger ...
// export function serializeDecimal ...
// export function serializeString ...
// export function serializeDisplayString ...
// export function serializeBoolean ...
// export function serializeByteSequence ...
// export function serializeToken ...
// export function serializeDate ...
// export function serializeParameters ...
// export function serializeKey ...

// structured-headers/src/index.js

...
export * as raw from './raw.js';