httpillage icon indicating copy to clipboard operation
httpillage copied to clipboard

Refactor to use Net::HTTP

Open forced-request opened this issue 10 years ago • 0 comments

A performance issue was discovered that seems to be related to the use of Mechanize.

Initial tests indicate that raw Net::HTTP GET queries are about 10x faster than leveraging Mechanize's get method. In these test cases the post request resulted in a server error causing significant latency. These tests should be taken with a grain of salt.

Tests with 5 threads and 100 requests on ruby 2.2.3p173 resulted in the following runtimes:

  • Net::HTTP.get - < 1 second
  • Mechanize Get - 6 seconds
  • Net::HTTP.post - 19 seconds
  • Mechanize Post - 23 seconds

Tests with 5 threads and 100 requests on jruby 9.0.1.0 (2.2.2) resulted in the following runtimes:

  • Net::HTTP.get - 2 seconds
  • Mechanize Get - 31 seconds
  • Net::HTTP.post - 98 seconds
  • Mechanize Post - 123 seconds

forced-request avatar Jan 23 '16 22:01 forced-request