crawly icon indicating copy to clipboard operation
crawly copied to clipboard

fix: request_options middleware overriding spider options

Open suzdalnitski opened this issue 3 years ago • 0 comments

Currently, any options set in the Crawly.Request will get overriden by the RequestOptions middleware. This PR fixes that to allow merging of both request options.

Crawly.Request.new(
  "http://example.com",
  headers,
  # the body option here will get overriden by the middleware from the config
  body: body
)
config :crawly,
  middlewares: [
    {Crawly.Middlewares.RequestOptions, proxy: {'https://proxy.com', 3128}}
  ],

suzdalnitski avatar Apr 17 '22 23:04 suzdalnitski