Tendis icon indicating copy to clipboard operation
Tendis copied to clipboard

hgetall性能较慢有改进余地吗?

Open lieex opened this issue 5 years ago • 2 comments

Tendis的hgetall操作对比同为rocksdb后端的pika大约慢一倍。

使用vire-benchmark测试(https://github.com/vipshop/vire) Pika:3.4.0 Tendis:2.1.2

vire-benchmark -t hgetall,hset -n 1000000 -c 24 -h [HOST] -p [PORT]

服务端用fio存储。

pika: ====== HGETALL ====== 1000000 requests completed in 27.87 seconds 24 parallel clients 16 bytes payload keep alive: 1

Tendis: ====== HGETALL ====== 1000000 requests completed in 50.20 seconds 24 parallel clients 16 bytes payload keep alive: 1

同样环境下,hset比pika性能是更好的,get/set相差无几。hgetall则差别很大。

Tendis配置: bind 0.0.0.0 binlog-send-batch 256 binlog-send-bytes 16777216 binlog-using-defaultCF no binlogDelRange 1 binlogFileSecs 1200 binlogFileSizeMB 512 binlogRateLimitMB 512 checkkeytypeforsetcmd no chunkSize 16384 cluster-enabled no cluster-migration-barrier 1 cluster-migration-binlog-iters 10 cluster-migration-distance 10000 cluster-migration-rate-limit 32 cluster-migration-slots-num-per-task 10 cluster-node-timeout 15000 cluster-require-full-coverage yes cluster-single-node no cluster-slave-no-failover no cluster-slave-validity-factor 10 databases 16 delCntIndexMgr 10000 delJobCntIndexMgr 1 dir "/path/to/tendis6379/db" domain-enabled no dumpdir "/path/to/tendis6379/dump" executorThreadNum 24 executorWorkPoolSize 8 fullPushThreadnum 4 fullReceiveThreadnum 4 garbage-delete-size 30 garbageDeleteThreadnum 1 generalLog no incrPushThreadnum 4 keysDefaultLimit 100 kvStoreCount 10 lockWaitTimeOut 3600 logDir "/path/to/tendis6379/log" logLevel "notice" logRecycleThreadnum 4 masterauth "" maxBinlogKeepNum 200 maxClients 10000 migrate-gc-enabled yes migrate-snapshot-retry-num 1000 migrateReceiveThreadnum 4 migrateSenderThreadnum 4 minBinlogKeepSec 3600 netBatchSize 1048576 netBatchTimeoutSec 10 netIoThreadNum 0 noexpire no pauseTimeIndexMgr 10 pidFile "/path/to/tendis6379/tendisplus.pid" port 6379 proto-max-bulk-len 536870912 requirepass "" rocks.blockcache_strict_capacity_limit no rocks.blockcachemb 78848 rocks.compress_type "snappy" rocks.disable_wal no rocks.flush_log_at_trx_commit no rocks.level0_compress_enabled no rocks.level1_compress_enabled no rocks.wal_dir "" scanCntIndexMgr 1000 scanJobCntIndexMgr 1 slave-migrate-enabled no slaveBinlogKeepNum 1 slowlog "/path/to/tendis6379/slowlog" slowlog-file-enabled no slowlog-flush-interval 1000 slowlog-log-slower-than 100000 slowlog-max-len 128 storage "rocks" timeoutSecBinlogWaitRsp 30 truncateBinlogIntervalMs 1000 truncateBinlogNum 50000 version-increase yes

=================

pika配置: port : 9221 thread-num : 20 thread-pool-size : 24 sync-thread-num : 6 log-path : /path/to/pika9221/log db-path : /path/to/pika9221/db dump-path : /path/to/pika9221/dump pidfile : /path/to/pika9221/pika.pid db-sync-path : /path/to/pika9221/dbsync write-buffer-size : 268435456 arena-block-size : timeout : 60 requirepass : masterauth : userpass : userblacklist : instance-mode : classic databases : 8 default-slot-num : 1024 replication-num : 0 consensus-level : 0 dump-prefix : daemonize : yes dump-expire : 0 maxclients : 20000 target-file-size-base : 20971520 expire-logs-days : 7 expire-logs-nums : 50 root-connection-num : 2 slowlog-write-errorlog : no slowlog-log-slower-than : 10000 slowlog-max-len : 128 db-sync-speed : -1 slave-priority : 100 sync-window-size : 9000 max-conn-rbuf-size : 268435456 write-binlog : yes binlog-file-size : 104857600 max-cache-statistic-keys : 0 small-compaction-threshold : 5000 max-write-buffer-size : 10737418240 max-write-buffer-number : 2 max-client-response-size : 1073741824 compression : snappy max-background-flushes : 4 max-background-compactions : 4 max-cache-files : 5000 max-bytes-for-level-multiplier : 10

lieex avatar Dec 30 '20 06:12 lieex

Thanks for issues. We should check it later.

TendisDev avatar Dec 30 '20 12:12 TendisDev

It is suggested to use better formatting methods to replace stringstream. Because it's slow.

wolfkdy avatar Jan 13 '21 12:01 wolfkdy