databricks-sql-python
databricks-sql-python copied to clipboard
[471] - Close underlying HTTP Client on closing `Connection`
(Addresses #471).
What type of PR is this?
- [x] Bug Fix
Description
Here, we resolve a few resource leaks with regards to TCP connections remaining open even after the Connection is closed:
- Currently, on session closure, the thrift backend invokes
self._transport.close()which simply releases a connection back to the underlying pool. It does not close the pool itself. This leads to the TCP connection not being closed properly and a danglingCLOSE_WAITstate as pointed out in the above issue. Thus, we rename_transport.close()to the more appropriate_transport.release_connection()and implement a new_transport.close()that actually closes the underlying pool. We invoke this method on SEA backend session closure as well. - Currently, the
close()method of theTelemetryClientdoes not close the unified HTTP client instantiated by it. We add code to close this as well.
How is this tested?
- [x] Unit tests
- [ ] E2E Tests
- [x] Manually
- using the same scripts mentioned in the description of #471 - the connection is now closed properly and there is no dangling
CLOSE_WAIT. - testing with an invalid token - before the addition of the
http_client.close()in theTelemetryClient, a similar resource leak as pointed out above was observed, that has now been mitigated.
- using the same scripts mentioned in the description of #471 - the connection is now closed properly and there is no dangling
- [ ] N/A
Related Tickets & Documents
#471