databricks-sql-python icon indicating copy to clipboard operation
databricks-sql-python copied to clipboard

[471] - Close underlying HTTP Client on closing `Connection`

Open Varun0157 opened this issue 5 months ago • 2 comments

(Addresses #471).

What type of PR is this?

  • [x] Bug Fix

Description

Here, we resolve a few resource leaks with regards to TCP connections remaining open even after the Connection is closed:

  • Currently, on session closure, the thrift backend invokes self._transport.close() which simply releases a connection back to the underlying pool. It does not close the pool itself. This leads to the TCP connection not being closed properly and a dangling CLOSE_WAIT state as pointed out in the above issue. Thus, we rename _transport.close() to the more appropriate _transport.release_connection() and implement a new _transport.close() that actually closes the underlying pool. We invoke this method on SEA backend session closure as well.
  • Currently, the close() method of the TelemetryClient does not close the unified HTTP client instantiated by it. We add code to close this as well.

How is this tested?

  • [x] Unit tests
  • [ ] E2E Tests
  • [x] Manually
    • using the same scripts mentioned in the description of #471 - the connection is now closed properly and there is no dangling CLOSE_WAIT.
    • testing with an invalid token - before the addition of the http_client.close() in the TelemetryClient, a similar resource leak as pointed out above was observed, that has now been mitigated.
  • [ ] N/A

Related Tickets & Documents

#471

Varun0157 avatar Aug 09 '25 18:08 Varun0157