fiber icon indicating copy to clipboard operation
fiber copied to clipboard

🤗 [Question]: Potential memory leak?

Open damianham opened this issue 2 years ago • 3 comments

Question Description

how can I ensure all memory is freed after a request has been served. My API server has a memory leak that ultimately gets the process killed as the server runs out of memory. The memory leak is in fiber.Ctx.JSON

In all my API requests I return a JSON map from the handler, see the example snippet.

Code Snippet (optional)

package main

import "github.com/gofiber/fiber/v2"
import "log"

func main() {
  app := fiber.New()

  // An example to describe the question
// get the businesses for a user ID
	app.Get("/user/:id/businesses", func(c *fiber.Ctx) error {
		id := c.Params("id")

		businesses, err := services.GetBusinessesForUserID(id)

		if err != nil {
			return err
		}

		return c.JSON(fiber.Map{"result": businesses}) // memory leak here
	})
	

  log.Fatal(app.Listen(":3000"))
}

Checklist:

  • [X] I agree to follow Fiber's Code of Conduct.
  • [X] I have checked for existing issues that describe my questions prior to opening this one.
  • [X] I understand that improperly formatted questions may be closed without explanation.

damianham avatar May 17 '23 23:05 damianham

fasthttp and fiber are built to preserve and reuse the request and response capsules for the next loop using the concept of sync pools

this saves allocations and makes the application faster

how big is the json you want to write away? maybe you should stream the json to the response if it is too big ? https://github.com/gofiber/fiber/issues/1034#issuecomment-734487679

there is the possibility to make some settings in fasthttp which might reduce the pool or memory usage https://pkg.go.dev/github.com/valyala/fasthttp#Server check ReduceMemoryUsage, Concurrency

ReneWerner87 avatar May 19 '23 08:05 ReneWerner87

The issue isn't that I am writing large json data in a single request - the issue is that memory is not being freed over time so the RSS grows to 1.6GB over a period of 8-10 hours. The example I gave is just 1 route that causes a memory leak. I am going to try this and see what happens

` app.Get("/user/:id/businesses", func(c *fiber.Ctx) error { id := c.Params("id")

	businesses, err := services.GetBusinessesForUserID(id)

	if err != nil {
		return err
	}
	result := fiber.Map{"result": businesses}
	c.JSON(result)
	result = nil

	return nil
})`

damianham avatar May 19 '23 16:05 damianham

@damianham I'm repeatedly curling the server in the first code example, and I'm not able to reproduce the issue. Are you still able to reproduce the problem? Do you recall what version of fiber/go you were using at the time?

nickajacks1 avatar Dec 28 '23 23:12 nickajacks1