"Connection dropped: socket connection broken" using gevent with low concurrency

[benbenqiang]

when i submit job using gevent with more than 100 pool size, i get the Connection dropped exception.

here is the example: def zk_scan(port): zk = KazooClient(hosts="127.0.0.1:2181") zk.start() print(zk.get_children("/")) zk.stop()

if name == 'main': from gevent.pool import Pool

pool = Pool(100)
pool.map(zk_scan, range(1, 65536))
pool.join()

when i use gevent with 10 pool size is ok, also using python thread pool with 1000 pool size is also going fine. but using gevent with 100 pool size will cause connection dropped

[norrs]

Quote from the zookeeper documentation ( https://zookeeper.apache.org/doc/r3.3.1/zookeeperAdmin.html ) :

Limits the number of concurrent connections (at the socket level) that a single client, identified by IP address, may make to a single member of the ZooKeeper ensemble. This is used to prevent certain classes of DoS attacks, including file descriptor exhaustion. The default is 10. Setting this to 0 entirely removes the limit on concurrent connections.

Are you sure you are not hitting maximum connections limit in zookeeper? This doesn't sound like a bug, but an configuration issue.

[Laurel-rao]

what configuration is it? I can't find it which change the zookeeper limit 10