livekit-cli icon indicating copy to clipboard operation
livekit-cli copied to clipboard
verified publisher icon livekit

No Latency Times for CLI Benchmark

Open alexbeattie42 opened this issue 3 years ago • 0 comments

Replication

  • livekit-cli version 0.7.1
  • macOS Monterey 12.3

Tested against livekit server with the 15.4 release using the following command

livekit-load-tester --url wss://<HOST> \
  --api-key <KEY> \
  --api-secret <SECRET> \
  --duration 1m \
  --publishers 3 \
  --subscribers 5

Result

Sub 0 | Track              | Kind  | Pkts  | Bitrate   | Latency | Dropped
      | 0A TR_Ly3y4PeTMwSw | audio | 1795  | 9.8kbps   |  -      | 0 (0%)
      | 0V TR_Nu25DAFUofuL | video | 12051 | 1.8mbps   |  -      | 0 (0%)
      | 1A TR_Jyf7uj2VrTRu | audio | 1765  | 9.8kbps   |  -      | 0 (0%)
      | 1V TR_DL22PuDNcTtK | video | 1425  | 152.0kbps |  -      | 0 (0%)
      | 2A TR_JdW49DAcempW | audio | 1766  | 9.8kbps   |  -      | 0 (0%)
      | 2V TR_ZZ3xVM9sU6mQ | video | 1430  | 153.2kbps |  -      | 0 (0%)

Sub 1 | Track              | Kind  | Pkts  | Bitrate   | Latency | Dropped
      | 0A TR_Ly3y4PeTMwSw | audio | 1795  | 9.8kbps   |  -      | 0 (0%)
      | 0V TR_Nu25DAFUofuL | video | 12051 | 1.8mbps   |  -      | 0 (0%)
      | 1A TR_Jyf7uj2VrTRu | audio | 1765  | 9.8kbps   |  -      | 0 (0%)
      | 1V TR_DL22PuDNcTtK | video | 1425  | 152.0kbps |  -      | 0 (0%)
      | 2A TR_JdW49DAcempW | audio | 1766  | 9.8kbps   |  -      | 0 (0%)
      | 2V TR_ZZ3xVM9sU6mQ | video | 1430  | 153.2kbps |  -      | 0 (0%)

Sub 2 | Track              | Kind  | Pkts  | Bitrate   | Latency | Dropped
      | 0A TR_Ly3y4PeTMwSw | audio | 1795  | 9.8kbps   |  -      | 0 (0%)
      | 0V TR_Nu25DAFUofuL | video | 12051 | 1.8mbps   |  -      | 0 (0%)
      | 1A TR_Jyf7uj2VrTRu | audio | 1765  | 9.8kbps   |  -      | 0 (0%)
      | 1V TR_DL22PuDNcTtK | video | 1425  | 152.0kbps |  -      | 0 (0%)
      | 2A TR_JdW49DAcempW | audio | 1766  | 9.8kbps   |  -      | 0 (0%)
      | 2V TR_ZZ3xVM9sU6mQ | video | 1430  | 153.2kbps |  -      | 0 (0%)

Sub 3 | Track              | Kind  | Pkts  | Bitrate   | Latency | Dropped
      | 0A TR_Ly3y4PeTMwSw | audio | 1775  | 9.8kbps   |  -      | 0 (0%)
      | 0V TR_Nu25DAFUofuL | video | 1409  | 152.1kbps |  -      | 0 (0%)
      | 1A TR_Jyf7uj2VrTRu | audio | 1766  | 9.8kbps   |  -      | 0 (0%)
      | 1V TR_DL22PuDNcTtK | video | 11930 | 1.8mbps   |  -      | 0 (0%)
      | 2A TR_JdW49DAcempW | audio | 1766  | 9.8kbps   |  -      | 0 (0%)
      | 2V TR_ZZ3xVM9sU6mQ | video | 1430  | 153.2kbps |  -      | 0 (0%)

Sub 4 | Track              | Kind  | Pkts  | Bitrate   | Latency | Dropped
      | 0A TR_Ly3y4PeTMwSw | audio | 1776  | 9.8kbps   |  -      | 0 (0%)
      | 0V TR_Nu25DAFUofuL | video | 1208  | 120.1kbps |  -      | 0 (0%)
      | 1A TR_Jyf7uj2VrTRu | audio | 1765  | 9.8kbps   |  -      | 0 (0%)
      | 1V TR_DL22PuDNcTtK | video | 11930 | 1.8mbps   |  -      | 0 (0%)
      | 2A TR_JdW49DAcempW | audio | 1766  | 9.8kbps   |  -      | 0 (0%)
      | 2V TR_ZZ3xVM9sU6mQ | video | 1430  | 153.2kbps |  -      | 0 (0%)

Summary | Tester | Tracks | Bitrate                | Latency | Total Dropped
        | Sub 0  | 6/6    | 2.1mbps                |  -      | 0 (0%)
        | Sub 1  | 6/6    | 2.1mbps                |  -      | 0 (0%)
        | Sub 2  | 6/6    | 2.1mbps                |  -      | 0 (0%)
        | Sub 3  | 6/6    | 2.1mbps                |  -      | 0 (0%)
        | Sub 4  | 6/6    | 2.1mbps                |  -      | 0 (0%)
        | Total  | 30/30  | 10.4mbps (2.1mbps avg) |  -      | 0 (0%)

Expected:

Latency times to be reported

alexbeattie42 avatar Mar 29 '22 16:03 alexbeattie42