go-json icon indicating copy to clipboard operation
go-json copied to clipboard

Decoding error message is misleading, when there is limit on the reader

Open openmohan opened this issue 3 years ago • 1 comments

Objective: When decoding a JSON, if the reader has a limit and if the data in the reader is exceeding the limit.

Original use case: Restricting the size of HTTP bodies using http.MaxBytesReader

Expected outcome: Throw error that the data is exceeding the limit provider on the reader

Outcome: The partial JSON is considered (cutting on the limit) and decoded on the interface.

Program to reproduce the issue:

package main

import (
	"bytes"
	"fmt"
	"io"

	json "github.com/goccy/go-json"
)

func main() {
	a := map[string]string{}

	jsonBody := `"region": "test", "title":"GreatSite.com","status": "suspended",`
	for i := 0; i < 20; i++ {
		jsonBody += jsonBody
	}
	jsonBody = "{" + jsonBody + `"region":"test"}`
	b := bytes.NewBuffer([]byte(jsonBody))
	r := io.LimitReader(b, 64000)
	decoder := json.NewDecoder(r)
	decoder.DisallowUnknownFields()
	err := decoder.Decode(&a)
	fmt.Println(err)
}

Output:

expected comma after object value

Try increasing the size of reader from 64000 to 99999999 and the error would be gone.

openmohan avatar Sep 23 '22 11:09 openmohan

I just came across this issue today. The error is inconsistent with the std lib.

package main

import (
	"bytes"
	"encoding/json"
	"errors"
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"

	goccyjson "github.com/goccy/go-json"
)

type Test struct {
	Test string `json:"test"`
}

func main() {
	w := httptest.NewRecorder()
	data := []byte(`{"test":"test"}`)
	maxErr := &http.MaxBytesError{}

	nop := io.NopCloser(bytes.NewBuffer(data))
	r := http.MaxBytesReader(w, nop, 1)
	var t Test
	err := json.NewDecoder(r).Decode(&t)
	fmt.Println(errors.As(err, &maxErr)) // true
	fmt.Println(err)                     // http: request body too large

	nop2 := io.NopCloser(bytes.NewBuffer(data))
	r2 := http.MaxBytesReader(w, nop2, 1)
	var t2 Test
	err = goccyjson.NewDecoder(r2).Decode(&t2)
	fmt.Println(errors.As(err, &maxErr)) // false
	fmt.Println(err)                     // EOF
}

Any workarounds @goccy ? Thanks!

phenpessoa avatar Oct 19 '22 23:10 phenpessoa