databricks
Improve http client set up
Reuse http client across connections
Go http client is meant to be reused as it has its own connection pool. This would reduce the need to many https handshakes on connection creation. Be careful to not reuse the thrift client as that one is not thread safe.
Increase http client MaxIdleConnsPerHost
Our use case it pretty much all about connecting to the same host. So we should set MaxIdleConnsPerHost to a number way higher than default 2.
@andrefurlan-db the most common tuned value I've seen for MaxIdleConnsPerHost is 100. I definitely think it's a great idea to increase this from the default value of 2, but should this value also be configurable by users, such as from query params in the DSN?
Hi @zcking , until we have some real requests and use cases on why setting this is important to our users, I will prefer to not make it configurable. So, I am not disagreeing with you, I just want to wait and see if there is demand for such low consequence knob.