ClickHouse.Client
ClickHouse.Client copied to clipboard
Add overload for ClickHouseConnection.PostStreamAsync to support 'push stream'
I'm having performance issues when sending large amount of data 'computed in c#' to ClickHouse. Similar problem was described in #60 and #61.
Current version of ClickHouseConnection.PostStreamAsync expects Stream which means that objects has to be first serialized into temporary MemoryStream or FileStream. This causes performance problems when produced stream is large (excessive memory usage or IO delays in case of FileStream).
Having an overload like the one below will allow to avoid unnecessary copying and, for some formats like tsv or json, to send data without using intermediate temporary stream, similar to how nowadays dotnet core formatters work:
public async Task PostStreamAsync(
string sql,
string contentType,
Action<Stream, HttpContent, TransportContext> writer)
{
...
postMessage.Content = new PushStreamContent(writer);
postMessage.Content.Headers.ContentType = new MediaTypeHeaderValue(contentType);
...
}