Reusing fasthttp client on multipart request got intermittent errors

[ghiyastfarisi]

I am experiencing intermittent errors, which the pattern seems like an alternating sequence error. The first request looks fine, but the second experiencing either [ERROR] request failed: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection or sometimes got [ERROR] request failed: write tcp 192.168.233.6:35692->localhost:3002: write: broken pipe

Then, the 3rd request fine, the 4th are error again. So, I run through simulating a loop for http call with the same payload and here are the result:

[loop 1][SUCCESS] len: 10979
[loop 2][ERROR] request failed: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[loop 3][SUCCESS] len: 10979
[loop 4][ERROR] request failed: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[loop 5][SUCCESS] len: 10979
[loop 6][ERROR] request failed: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[loop 7][SUCCESS] len: 10979
[loop 8][ERROR] request failed: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[loop 9][SUCCESS] len: 10979
[loop 10][ERROR] request failed: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection

then I try to use 2 different payload for reach call loop and here is the result:

[loop 1][SUCCESS] len: 10948
[loop 1][ERROR] request failed: write tcp 192.168.233.6:58234->localhost:3002: write: broken pipe
[loop 2][SUCCESS] len: 10948
[loop 2][ERROR] request failed: write tcp 192.168.233.6:58244->localhost:3002: write: broken pipe
[loop 3][SUCCESS] len: 10948
[loop 3][ERROR] request failed: write tcp 192.168.233.6:58260->localhost:3002: write: broken pipe
[loop 4][SUCCESS] len: 10948
[loop 4][ERROR] request failed: write tcp 192.168.233.6:37934->localhost:3002: write: broken pipe
[loop 5][SUCCESS] len: 10948
[loop 5][ERROR] request failed: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[loop 6][SUCCESS] len: 10948
[loop 6][ERROR] request failed: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[loop 7][SUCCESS] len: 10948
[loop 7][ERROR] request failed: write tcp 192.168.233.6:37952->localhost:3002: write: broken pipe
[loop 8][SUCCESS] len: 10948
[loop 8][ERROR] request failed: write tcp 192.168.233.6:37962->localhost:3002: write: broken pipe
[loop 9][SUCCESS] len: 10948
[loop 9][ERROR] request failed: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[loop 10][SUCCESS] len: 10948
[loop 10][ERROR] request failed: write tcp 192.168.233.6:37978->localhost:3002: write: broken pipe

here are given the code samples to run

func httpClientCall() {
	c := &fasthttp.Client{
		MaxConnsPerHost: 100,
		DialDualStack:   true,
	}

	img1 := loadFile("./file/img1")
	img2 := loadFile("./file/img2")

	for loop := range 10 {

		i := loop + 1

		r, err := withFasthttp(c, img1, "img1")
		if err != nil {
			fmt.Printf("[loop %d][ERROR] %s\n", i, err.Error())
		} else {
			fmt.Printf("[loop %d][SUCCESS] len: %d\n", i, len(r))
		}

		r2, err := withFasthttp(c, img2, "img2")
		if err != nil {
			fmt.Printf("[loop %d][ERROR] %s\n", i, err.Error())
		} else {
			fmt.Printf("[loop %d][SUCCESS] len: %d\n", i, len(r2))
		}
	}
}

func withFasthttp(c *fasthttp.Client, file string, fn string) (string, error) {
	// setup multipart
	var requestBody bytes.Buffer
	writer := multipart.NewWriter(&requestBody)
	decodedData, err := base64.StdEncoding.DecodeString(file)
	if err != nil {
		return "", fmt.Errorf("failed to decode base64 string: %w", err)
	}

	part, err := writer.CreateFormFile("file", fn)
	if err != nil {
		return "", fmt.Errorf("failed to create form file: %w", err)
	}

	if _, err := part.Write(decodedData); err != nil {
		return "", fmt.Errorf("failed to write data to part: %w", err)
	}

	if err := writer.Close(); err != nil {
		return "", fmt.Errorf("failed to close writer: %w", err)
	}

	// setup fasthttp req
	req := fasthttp.AcquireRequest()
	defer fasthttp.ReleaseRequest(req)

	req.Header.SetMethod(fasthttp.MethodPost)
	req.SetRequestURI("http://localhost:3002/upload-img")
	req.SetBody(requestBody.Bytes())
	req.Header.SetContentType(writer.FormDataContentType())

	resp := fasthttp.AcquireResponse()
	defer fasthttp.ReleaseResponse(resp)
	if err := c.Do(req, resp); err != nil {
		return "", fmt.Errorf("request failed: %w", err)
	}

	if resp.StatusCode() != fasthttp.StatusOK {
		return "", fmt.Errorf("unexpected status code: got %d, want %d", resp.StatusCode(), fasthttp.StatusOK)
	}

	result := string(resp.Body())

	return result, nil
}

func loadFile(path string) string {
	file, _ := os.ReadFile(path)

	return string(file)
}

I do not understand why its happened, is it not allowed to reuse single &fasthttp.Client{} instance as shown in the code, because when I implement a new initiation of &fasthttp.Client{} will solve the problem.

[erikdubbelboer]

fasthttp tries to reuse the connection multiple requests because your server doesn't send a connection:close header. But it seems like your server doesn't follow proper http protocol and doesn't allow connection reuse, and closed the connection.

It's probably best if you fix your server if possible.

I'm afraid fasthttp doesn't have an option to disable connection reuse as that is super important for performance. If your server really doesn't support connection reuse I would suggest just using net/http as you won't get any performance benefit from fasthttp.

[ghiyastfarisi]

fasthttp tries to reuse the connection multiple requests because your server doesn't send a connection:close header. But it seems like your server doesn't follow proper http protocol and doesn't allow connection reuse, and closed the connection.

It's probably best if you fix your server if possible.

I'm afraid fasthttp doesn't have an option to disable connection reuse as that is super important for performance. If your server really doesn't support connection reuse I would suggest just using net/http as you won't get any performance benefit from fasthttp.

I see, actually the server that I try to access is made in uvicorn within FastAPI in python, I am not sure if that framework has that problem. But, I will try to investigate more by checking through target service server implementation. The weird things is, the call using non multipart version is not affected with this issue.