Skip to content

Conversation

@Watson1978
Copy link
Contributor

@Watson1978 Watson1978 commented Jun 19, 2025

Which issue(s) this PR fixes:
Fixes #5005

What this PR does / why we need it:
This PR will fix RuntimeError with large zstd compressed data

Docs Changes:
Not needed because this feature has not been released yet.

Release Note:
Should be described with #4657

@Watson1978 Watson1978 changed the title in_forward: fix RuntimeError with large zstd compressed data compressable: fix RuntimeError with large zstd compressed data Jun 19, 2025
@Watson1978 Watson1978 marked this pull request as ready for review June 19, 2025 07:22
@Watson1978 Watson1978 requested review from daipom and kenhys June 19, 2025 07:22
@daipom
Copy link
Contributor

daipom commented Jun 19, 2025

Thanks for this fix!
This feature has not been released yet.
So glad we noticed this before the release!

@Watson1978
Could you please check if this should be fixed as well?

def io_decompress_zstd(input, output)
loop do
reader = Zstd::StreamReader.new(input)
# Zstd::StreamReader needs to specify the size of the buffer
v = reader.read(1024)
output.write(v)
# Zstd::StreamReader doesn't provide unused data, so we have to manually adjust the position
break if input.eof?
end
output
end

@daipom daipom added this to the v1.19.0 milestone Jun 19, 2025
Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
@Watson1978
Copy link
Contributor Author

Could you please check if this should be fixed as well?

Fixed io_decompress_zstd too. Thanks

Copy link
Contributor

@daipom daipom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@daipom daipom merged commit 6821e4b into fluent:master Jun 20, 2025
16 checks passed
@Watson1978 Watson1978 deleted the in_forward-zstd branch June 20, 2025 02:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

in_forward: RuntimeError with large zstd compressed data

2 participants