crate-operator icon indicating copy to clipboard operation
crate-operator copied to clipboard

Using TCP keepalive on Azure

Open amotl opened this issue 5 years ago • 2 comments

Dear @MarkusH, @chaudum, @lukasbals and @Taliik,

regarding the infamous Azure TCP idle connection drop problem (https://github.com/crate/crate/issues/10779), I would like to give you a heads-up in the context of Kubernetes operators (https://github.com/Azure/AKS/issues/1877).

According to https://github.com/p7t/actus/issues/272, AKS clusters running Kubernetes v1.19+ are currently receiving a bugfix

The AKS control plane will always send RST for idle connections after 4min.

So, this will probably mitigate any problems in this regard for connections through kubernetes-asyncio.

With kind regards, Andreas.

P.S.: However, I don't know whether any connections through aiopg might still be affected.

amotl avatar Nov 19 '20 12:11 amotl

However, I don't know whether any connections through aiopg might still be affected.

On the psycopg2 side, this has been discussed within https://github.com/psycopg/psycopg2/issues/113 and https://github.com/psycopg/psycopg2/issues/561. On aiopg, https://github.com/aio-libs/aiopg/issues/491 might be related.

In order to mitigate that, efforts on psycopg2 coming from https://github.com/psycopg/psycopg2/pull/565 and https://github.com/psycopg/psycopg2/pull/974 apparently have been wrapped into psycopg2-pool, see also https://github.com/liberapay/postgres.py/issues/80.

So, while I want to apologize that I didn't look into how socket options can be set directly like https://github.com/crate/crate-python/pull/374 when using aiopg, I am inclined to believe that the pool_recycle option of aiopg.create_pool() might help us here:

pool_recycle (float) – number of seconds after which connection is recycled, helps to deal with stale connections in pool, default value is -1, means recycling logic is disabled.

amotl avatar Nov 19 '20 12:11 amotl

GH-548 addresses this. Thank you, @SStorm.

amotl avatar Sep 06 '23 15:09 amotl