hyperledger
unexpected results when running caliper benchmark on Ethereum network
Context
I am trying to test the performance of my smart contract in terms of latency and throughput under varied scenarios and control settings. These settings include the number of workers, TPS and txNumber/txDuration. I vary the number of workers and for each group of workers I modify the parameters of the round including TPS and txNumber/TxDuration. I am really confused about the definition and how dealing with "txNumber" and "txDuration" parameters. The results that I found when running the test do not correspond to the definition of those parameters. I can't find a logic or an explanation to these results.
As first attempt, I run 5 groups of workers (5,30,50,70,100). For each group of workers, I run three rounds. For each round, I modify the transaction count "txNumber" and the transaction rate "TPS" so that each worker executes a given number of transactions with a given rate (5TPS , 15TPS and 20 TPS). When running the test, I found that the latency increases when increasing the transaction rate and decreases when increasing the number of workers which I found not logic.
Then, I tried to modify the TPS and "txDuration" instead of modifying the "txNumber". The latency is then increasing when the number of workers increases. However, with 5 workers, the latency is still higher. As example:
for 5 workers I found: TPS= 5 , latency= 27.07 , throughput=1,0 TPS=15 , latency= 59,95 , throughput= 1,4 TPS=20 , latency = 75,25, throughput= 1,4
for 50 workers I found: TPS= 5 , latency= 6,74 , throughput=5,1 TPS=15 , latency= 7,25 , throughput= 6,6 TPS=20 , latency = 9,38, throughput= 6,9
for 70 workers I found: TPS= 5 , latency= 7,36, throughput=5,6 TPS=15 , latency= 10,10 , throughput= 6,4 TPS=20 , latency = 8,18, throughput= 8,6
for 100 workers I found: TPS= 5 , latency= 7,71, throughput=5,5 TPS=15 , latency= 8,23, throughput= 9,5 TPS=20 , latency = 9,04, throughput= 10,2
Another thing that I did not understand: I put the txDuration to 5 seconds and the TPS to 2. I expect that each worker will run 10 transactions. However, each worker runs just 4 transactions.
I need an explanation of how to configure my system in a way: for each group of workers, we have three rounds. the first round the worker must run 5 TPS and in the second round the worker runs 15 TPS and in the last round the worker runs 20TPS. WIth a good configuration, I expect that the latency increases when the number of workers increases independently of how many workers are running.
Are you using distributed workers? Bare in mind that these workers are node based and so are single threaded, meaning that you can become CPU bound very quickly.
Recommend the rocket chat channel for general questions - https://chat.hyperledger.org/channel/caliper
Are you using distributed workers? Bare in mind that these workers are node based and so are single threaded, meaning that you can become CPU bound very quickly.
Recommend the rocket chat channel for general questions - https://chat.hyperledger.org/channel/caliper
It is already the case when I reach 100 workers knowing that I use 9GO of RAM and 100GO of storage.
hi, have you checked the effect of "block confirmations" ? i get almost the same result, even if I change it from 12 to zero.
when I calculate the parameters myself using the logs , considering 12 block confirmations results are drastically different.
