a question about the performance of docker and hyperd

[jiangpengcheng]

I did a simple benchmark test on hyperd and docker using wrk and nginx, but found that hyperd is quite slower than docker, below is the detailed result:

hyperd

container: hyperctl run -td --name web --memory=1024 nginx

test command: wrk -t12 -c400 -d30s http://192.168.123.6 test result:

Running 30s test @ http://192.168.123.6
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    89.74ms   53.11ms   1.98s    98.29%
    Req/Sec   196.38    154.44     1.38k    61.88%
  37871 requests in 30.08s, 30.70MB read
  Socket errors: connect 0, read 0, write 0, timeout 351
Requests/sec:   1258.82
Transfer/sec:      1.02MB

docker

container: docker run -td --name web --memory=1024M nginx

test command: wrk -t12 -c400 -d30s http://172.17.0.2 test result:

Running 30s test @ http://172.17.0.2
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    78.93ms  157.39ms   2.00s    85.64%
    Req/Sec     1.98k     0.90k    8.81k    79.95%
  670482 requests in 30.09s, 543.48MB read
  Socket errors: connect 0, read 0, write 0, timeout 25
Requests/sec:  22281.63
Transfer/sec:     18.06MB

Docker and hyperd are on a same virtual machine with memory=2048M and vcpu=1.

Was I doing something wrong or the benchmark is not suitable? Cause the result seems incredible

[gnawux]

hyperd is intended to be deployed on physical machine by default, and it employs hypervisor to provide isolated containers.

For running in VMs, you could try to use xenpv driver with xen 4.9 environment, ref: https://blog.hyper.sh/performance_runv_gce.html .

[minzcmu]

@gnawux Interesting! I actually had the same performance issue when running hyperd on aws vm. So using runV with the xen pv mode will be better for that use case? Do we have permission to use the xen pv mode on aws vm?

BTW, have you guys investigated how this can be integrated with kubernetes?

[lbcheng888]

@gnawux deployed hyperd on physical machine end up with: hyperctl run -t --cpu 4 --memory 8192 centos7_go

wrk -t12 -c400 -d30s http://192.168.123.5
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.56ms    2.86ms 222.43ms   91.27%
    Req/Sec     5.09k   777.00    46.00k    97.56%
  1823029 requests in 30.10s, 226.01MB read
Requests/sec:  60569.22
Transfer/sec:      7.51MB

docker run -it centos7_go with 24cores 32G memory

 wrk -t12 -c400 -d30s http://172.17.0.3:1323
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.12ms    5.28ms 218.95ms   95.94%
    Req/Sec    11.85k     1.61k   20.74k    76.04%
  4254182 requests in 30.10s, 527.42MB read
Requests/sec: 141341.39
Transfer/sec:     17.52MB

compared with cc-time of docker run -it --cpus 4 --memory 8589934592 centos7_go

  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.92ms    6.34ms 232.35ms   91.85%
    Req/Sec     3.83k   410.75    15.51k    95.00%
  1373596 requests in 30.10s, 170.30MB read
Requests/sec:  45636.34
Transfer/sec:      5.66MB