benchmarks icon indicating copy to clipboard operation
benchmarks copied to clipboard

Better reflect real-world API with bigger payloads

Open abenhamdine opened this issue 8 years ago • 4 comments

Hello,

First, congrats for these nice & useful benchmarks !

I made a very simple benchmark here with fastify vs Express and found some intereresting data : see https://github.com/fastify/fastify/issues/178 for the background.

The important point is : size of payload matters : with a big JSON payload, json stringification becomes the bottleneck, and fastify matches Express performance only if a schema is applied in fastify route, see https://github.com/fastify/fastify/issues/178#issuecomment-328036894

I would be glad to send a PR to add a new route in the tests, in order to test a bigger payload, or you can directly copy the test data I used in my benchmark if you prefer !

abenhamdine avatar Sep 09 '17 13:09 abenhamdine

What a lovely welcome, thank you. It makes me happy to help. Response size matters.

If you add your tests like this,

  • benchmarks
    • ...
    • payloads
      • small
        • express
        • fastify
      • medium
        • express
        • fastify
      • big
        • express
        • fastify
      • huge
        • express
        • fastify

Then add list that'll help lot I think

cagataycali avatar Sep 09 '17 20:09 cagataycali

@abenhamdine I would suggest using more than 1 tool for benchmarking as well because during my prev experience some of them tend to show variance b/w themselves as well as their versions too.

agauniyal avatar Sep 11 '17 05:09 agauniyal

I add bigger json for express, when I add for every package this issue will close.

cagataycali avatar Sep 16 '17 23:09 cagataycali

@abenhamdine -Better late than never, right? In the next couple of days I will get some big payloads into the tests and make sure they get benchmarked properly. I will keep you posted.

aichholzer avatar Jun 12 '18 04:06 aichholzer