Skip to content

Conversation

@PeterL328
Copy link
Contributor

@PeterL328 PeterL328 commented Dec 23, 2025

Commit Message: Add stream_to_metadata HTTP filter for stream parsing
Additional Description:
This PR introduces a new HTTP filter for extracting values from response flow streams and writes them into dynamic metadata. We start with supporting SSE (server-sent events) and JSON extraction from data events but keep interface generic so we can support other types of streams and formats later.
Primary use-case for this filter will be to extract token usage information from LLM responses and use in places like rate-limit or obersevability.
At Pinterest, we have an internal version of this filter that is more specific but have large part of the logic in this filter has been battle tested in our egress proxy.

Per-route configuration override and request stream parsing can be added in later PR.

Key Features

  • SSE support: Full SSE specification compliance, including CRLF/CR/LF line endings, comment handling, multiple data fields and chunked transfer.
  • JSON Path Selector: Path-based value extraction
  • Extensible API: Interface allows for future expansion to support other types of extractions such as regex, etc.
  • Multiple metadata targets: Able to extract values and write to multiple metadata namespaces.
  • Protection: Configured max_event_size (default: 8KB, max: 10MB) to prevent unbounded buffer growth.
  • First/Last match controls: stop_processing_on_match allows for picking first occurrence or last occurance.

Risk Level: Low (new filter)
Testing: Unit/Integration tests
Docs Changes: Added doc for stream_to_metadata
Release Notes: Added stream_to_metadata HTTP filter for extracting values from Server-Sent Events (SSE) streams and writing them to dynamic metadata. Useful for token-based rate limiting of LLM APIs.
Platform Specific Features: N/A

Fixes #29758

Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
@repokitteh-read-only
Copy link

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to (api/envoy/|docs/root/api-docs/).
envoyproxy/api-shepherds assignee is @adisuissa
CC @envoyproxy/api-watchers: FYI only for changes made to (api/envoy/|docs/root/api-docs/).

🐱

Caused by: #42762 was opened by PeterL328.

see: more, trace.

@PeterL328 PeterL328 marked this pull request as draft December 23, 2025 22:14
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
@PeterL328
Copy link
Contributor Author

Tagging @JuniorHsu and @KBaichoo for review 🙏
Seems we need at least 2 owners and 1 maintainer. Wonder if can add you two?

@PeterL328 PeterL328 marked this pull request as ready for review December 24, 2025 07:00
@JuniorHsu
Copy link
Contributor

Tagging @JuniorHsu and @KBaichoo for review 🙏 Seems we need at least 2 owners and 1 maintainer. Wonder if can add you two?

happy to help review and be one of the owner :)

Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
@PeterL328
Copy link
Contributor Author

/retest

Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
@PeterL328
Copy link
Contributor Author

@KBaichoo added you as maintainer for now but let me know if we should change that 😁

@PeterL328 PeterL328 changed the title Add stream_to_metadata HTTP filter for stream parsing http: add stream_to_metadata filter for stream parsing Dec 28, 2025
@PeterL328
Copy link
Contributor Author

/assign @KBaichoo

@repokitteh-read-only
Copy link

PeterL328 is not allowed to assign users.

🐱

Caused by: a #42762 (comment) was created by @PeterL328.

see: more, trace.

Copy link
Contributor

@JuniorHsu JuniorHsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i did a first pass but like we want api review at the moment as the patch is somewhat mature

Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Signed-off-by: Peter Leng <yleng@pinterest.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support parsing data from server-side events (SSE) and add to metadata

3 participants