Support fragmented messages

[piegamesde]

I have the use case where I have to send a message of X bytes, and I know X in advance, but it is fairly large so I'd like to not have to have the entire message in memory beforehand. Instead, I'd like to lazily generate it in small chunks as they are being sent. (Of course there will need to be a check afterwards that exactly as many bytes as advertised were sent)

Would such a feature be feasible?

Similarly, but also less importantly, what about that feature for the reading direction?

[sdroege]

That sounds useful but I'm not sure how easy that would be to implement. I think it will also requires changes to tungstenite as that also operates in messages instead of plain bytes. If you want to take a look at this in more detail, please go ahead :)

From the application side, if you have the choice, you could split the huge message into smaller messages with a small header that allows you to reconstruct them again.

[piegamesde]

So apparently tungstenite already supports the feature. I have no idea about how to expose this in an async API though. For my use case, the length is already known beforehand which arguably makes it a bit easier, but the full feature seems to support messages of previously unknown length.

[sdroege]

Yeah I'm not sure either. This needs some design time :) Also it seems like tokio-tungstenite doesn't support this yet either.

[joshtriplett]

@piegamesde

So apparently tungstenite already supports the feature.

Could you elaborate on how tungstenite supports this? I'm not familiar with that support.