optimize db connection pool

[siddontang]

Now, xcodis use different connection pool for different dbs, this is may be horrible if we want to support 1024 dbs or more.

We may need another better connection pool management. like

• every server has a connection pool not db.
• use a LIFO cache to store a connection for a db. if no associated db connection, we may use random and then do SELECT.

[kouhate]

Hi @siddontang, connection pool per server not db is needed. Then, do you think SELECT have so many overhead?

I guess its overhead come from the round trip between xcodis and ledisdb, so if we don't wait "ok" response or can send TMPSELECT like command with pipeline together, the overhead would be minimized.

[siddontang]

Hi @kouhate , this is the problem and you can see my commit message that I will optimize it later but have not found a better way now. :smile:

For redis, pipeline may be a good choice, but ledisdb does not support it. For ledisdb, I think may be we can append TMPSELECT db to every command like FROM MASTER.

[kouhate]

:+1: TMPSELECT db may be better.

[siddontang]

@kouhate , Now ledisdb supports XUSE command, like below:

XUSE db THEN set a 1

First we will select a db, then do the real command like before. Unlike TMPSELECT, this will change the connection db index.

[kouhate]

@siddontang

Maybe this command is mostly used as backend protocol between xcodis and ledisdb, so the syntax is not so important.

This should reduce redundant connection pool :-)

[siddontang]

@kouhate

I have optimized and refactored xcodis in develop branch. many changes:

• Use goredis to handle RESP, I don't like origin RESP and connection handling, :-), now the code seems simple.
• Optimize ledisdb db select using XSELECT db THEN command feature.
• Add ledisdb test.

[kouhate]

@siddontang

I got the following error when I tried to build latest ledisdb.

make
godep go install -tags 'linenoise leveldb rocksdb lua lmdb' ./...
# github.com/siddontang/ledisdb/server
server/client_resp.go:27: undefined: goredis.RespReader
godep: go exit status 2

This is not solved yet even after I got your goredis library.

go get github.com/siddontang/goredis

What do you think?

[siddontang]

run make clean, and rm -rf ./Godeps/_workspace/pkg please.

[kouhate]

Oops, works fine. I didn't know godep enough... Thanks :-)

[siddontang]

:smile:

I met the same problem before and make clean had no effect, so I removed Godeps pkg path and it worked.