Streaming support

[noamr]

There are new proposals coming for allow streaming HTML into an existing document. See whatwg/html#11542 and whatwg/html#2142 (comment).

For this to work, it would need to come along with sanitation support.

To connect the dots, trusted-types would need some form of sanitizing an HTML stream on the go, probably using a TransformStream.

This can have the following shape:policy.createHTMLTransformStream(), where the source of the untrusted HTML would be piped to the result's writable and the readable would be piped to the element receiving the markup.

[otherdaniel]

Thought a bit about this. Creating a stream is quite unlike any other Trusted Types API, but writing to a stream seems to fit the existing model just fine: That's the boundary where a string gets accepted for inclusion into the DOM. So if the write operation for any stream created by any of the patch methods (patchSelf, patchBefore, etc.) would do a TT check, then this would seem to fit quite neatly with the existing design:s

• Keeps guarding the string-to-DOM boundary.
• Keeps operating on a single string value.
• Near 100% re-use of spec and (likely) implementation.
• The write method already supports any as parameter type.
• The only difference to existing APIs would likely be that a check failure would reject the promise, rather than throwing.

In more spec-y terms, the writers created by any WritableStream created by any of the patch methods would call get Trusted Type compliant string in their write method, and would return a rejected promise if the TT check has thrown.

Maybe one useful extension could be that the stream creator could set a TrustedTypesPolicy on the stream. That is, instead of reyling on the default policy as a catch-all for the entire page, a page developer could take an existing policy which accepts/rejects (or even re-writes) incoming strings, and apply that to any non-trusted values being written. That'd be a little different from what we do so far, but seems to fit well within the existing framework.

---

It'd be really nice if there was a non-TT version. You already mention sanitization. If there were "safe" and "unsafe" versions (with Sanitizer's definition of safe, like setHTML and setHTMLUnsafe), then the safe one wouldn't even need any TT checks at all. Even better, a page could set up a stream with whatever sanitization rules it wants, and then pass it around to libraries or semi-untrusted code and could still be sure the resulting DOM matches its expectations.

I see Trusted Types and Sanitizer a bit like "carrot and stick", in that TT blocks DOM operations and thus potentially gets in your way (the stick), while Sanitizer makes it really easy to modify the DOM (within certain security constraints, the carrot). For this new streaming API, it'd be really nice if it had a "carrot"-type API, too.

---

An alternative I've considered is that, in TT world, the stream in Sreaming HTML isn't really like a string value, because it can accept arbitrary values and arbitrarily many values. It's arguably more like a TT policy, which in TT world is the thing that can bless arbitrary input values. So one could try to treat streams as policies and restrict stream creation to a set of named streams; and/or have a callback for any created stream that a policy could then reject (or modify, by setting a sanitizer). But… while I think that train of thought has some merit, I haven't really managed to cast this into an API that I'd actually want to use; my attempts were all a little awkward. I guess sticking to the string-to-DOM boundary has more promise.

[noamr]

The sanitizer version is being worked on separately. We are indeed considering something equivalent to setHTML and setHTMLUnsafe. See WICG/declarative-partial-updates#42.

What I am struggling with regarding streaming and the string-to-DOM boundary, is that to perform effective sanitation of a string, the string would need to be buffered somewhere, and that buffering is not always controlled directly by whoever writes to the stream.

I am not sure how that exactly works for document.write(), as the following strings might be safe separately but not together?
<scr and ipt>do_something</ script>.

Please forgive me if these are noob questions in the sanitation world.

[annevk]

It doesn't really work for document.write(), though ideally we don't propagate that to new APIs.

[foolip]

I'm trying to make the API surface concrete enough to start writing spec text. The starting point in the explainer is a patchUnsafe() method that returns a WritableStream. Fitting TT into that, all the alternatives I can think of are:

• Let patchUnsafe() return a WritableStream with replaced internals that always passes chunks through TT. (May need something new in Streams to prevent tampering.)
• Treating the TT step as a TransformStream and let patchUnsafe() return a TransformStream already piped to a WritableStream. (May need something new in Stream to prevent them from being unlocked.)
• A new createHTMLStream() method on TrustedTypePolicy and let patchUnsafe() take a stream as an optional argument, which it would also return. A stream created from the default policy would be used by default if no argument is given.

For all options there's also the question of whether a plain WritableStream or TransformStream is OK here, or if a new TrustedHTMLStream is needed. Opinions on precise API shape welcome :)

Other than API shape my main question is the same as @noamr had, how to handle dangerous markup being written in small chunks.

[noamr]

I'm trying to make the API surface concrete enough to start writing spec text. The starting point in the explainer is a patchUnsafe() method that returns a WritableStream. Fitting TT into that, all the alternatives I can think of are:

• Let patchUnsafe() return a WritableStream with replaced internals that always passes chunks through TT. (May need something new in Streams to prevent tampering.)
• Treating the TT step as a TransformStream and let patchUnsafe() return a TransformStream already piped to a WritableStream. (May need something new in Stream to prevent them from being unlocked.)

• A new createHTMLStream() method on TrustedTypePolicy and let patchUnsafe() take a stream as an optional argument, which it would also return. A stream created from the default policy would be used by default if no argument is given.

For all options there's also the question of whether a plain WritableStream or TransformStream is OK here, or if a new TrustedHTMLStream is needed. Opinions on precise API shape welcome :)

Other than API shape my main question is the same as @noamr had, how to handle dangerous markup being written in small chunks.

The way I see it, the TT step should be a TransformStream (with a new createHTMLStream method or so if exists), and its writable part would be what's returned to the caller of patchUnsafe by the platform.

The implementer of the stream-enabled method should take care of buffering.

If there is a policy without that method, we can either:

• buffer everything and fall back to createHTML
• Pass each string separately to createHTML, but I'm not sure that's safe enough.

[lukewarlow]

If there is a policy without that method, we can either:

• buffer everything and fall back to CreateHTML

• Pass each string separately to createHTML, but l'm not sure that's safe enough.

Alternatively we could also just throw a type error like we would in most other cases.

[foolip]

The way I see it, the TT step should be a TransformStream (with a new createHTMLStream method or so if exists), and its writable part would be what's returned to the caller of patchUnsafe by the platform.

@noamr do you mean that patchUnsafe() would internally create this TransformStream and return it? If so, what is the createHTMLStream() method for? Or does one have to call createHTMLStream() and pass it to patchUnsafe()? Both API shapes feel a bit unusual, but maybe that's OK.

[noamr]

The way I see it, the TT step should be a TransformStream (with a new createHTMLStream method or so if exists), and its writable part would be what's returned to the caller of patchUnsafe by the platform.

@noamr do you mean that patchUnsafe() would internally create this TransformStream and return it? If so, what is the createHTMLStream() method for? Or does one have to call createHTMLStream() and pass it to patchUnsafe()? Both API shapes feel a bit unusual, but maybe that's OK.

It's how trusted types work. They protect against injecting sensitive HTML via script. The functions are called by the platform when the developer calls any HTML injection script (like setting innerHTML).

createHTMLStream would be called by the platform when patchUnsafe() is called by any script, and would inject the transform stream in the middle between the "unsafe" stream and the stream created by the patching internal as a security/custom-sanitation layer that does its own buffering.

[foolip]

I've never used Trusted Types, so let me double check my understanding. The most straightforward and typical way to use TT is a header like Content-Security-Policy: require-trusted-types-for 'script'; trusted-types my-policy and JS like this:

function sanitize(input) { /* good stuff */ }

// Early on
const policy = trustedTypes.createPolicy("my-policy", {
  createHTML(input) { return sanitize(input); }
});

// Later
element.innerHTML = policy.createHTML(input);

So the script both calls createHTML() and passes the returned TrustedHTML object to another API, the innerHTML setter.

If there's a default policy then createHTML() is automatically called, but the spec describes this as a last resort so I guess it's not how TT is typically used. The way it's described on MDN seems to confirm this.

So, what would be the equivalent to the typical "my-policy" case above? I think it would have to be:

const policy = trustedTypes.createPolicy("my-policy", {
  createHTMLStream() {
    return new TransformStream({
      transform(chunk, controller) {
        // TODO: some buffering so that chunks 
        controller.enqueue(sanitize(chunk));
      }
    });
  }
});

// Later
const trustedStream = policy.createHTMLStream(input);
const writable = element.patchUnsafe(trustedStream);

If relying on the default policy, the argument could be omitted, I suppose.

@noamr is this the shape of the integration you were thinking?

[foolip]

In #597 I have sketched out some idea of what the TT spec changes might look like.