JSON.jl icon indicating copy to clipboard operation
JSON.jl copied to clipboard

Errors parsing big ints

Open samoconnor opened this issue 7 years ago • 1 comments

I've modified the LazyJSON tests so that they also apply the NST JSONTestSuite to JSON.jl and JSON2.jl: https://github.com/samoconnor/LazyJSON.jl/blob/master/test/runtests.jl#L360-L362

Below is a subset of failing tests for large ints (and one for a half-invalid UTF-16 surrogate pair). Note that the JSONTestSuite defines these tests as class "i" which means that parsers may choose to reject them as invalid. However, as it stands JSON.jl parses these incorrectly rather than rejecting them.

using JSON
using LazyJSON
using Test

for (name, jparse) in [("LazyJSON", x->LazyJSON.value(x; lazy=false)), ("JSON", x->JSON.parse(x))]

    @testset "$name" begin

    # See https://github.com/nst/JSONTestSuite

    @test jparse("[-123123123123123123123123123123]") ==
               Any[-123123123123123123123123123123] # i_number_too_big_neg_int.json

    @test jparse("[100000000000000000000]") ==
               Any[100000000000000000000] # i_number_too_big_pos_int.json

    @test jparse("[-237462374673276894279832749832423479823246327846]") ==
               Any[-237462374673276894279832749832423479823246327846] # i_number_very_big_negative_int.json

    @test jparse("[\"\\uD888\\u1234\"]") == Any["\ud888ሴ"] # i_string_1st_valid_surrogate_2nd_invalid.json

    end
end
Test Summary: | Pass  Total
LazyJSON      |    4      4

JSON: Test Failed at /Users/sam/julia/dev/LazyJSON/test/foo.jl:11
  Expression: jparse("[-123123123123123123123123123123]") == Any[@int128_str("-123123123123123123123123123123")]
   Evaluated: Any[6188572990942219341] == Any[-123123123123123123123123123123]

JSON: Test Failed at /Users/sam/julia/dev/LazyJSON/test/foo.jl:14
  Expression: jparse("[100000000000000000000]") == Any[@int128_str("100000000000000000000")]
   Evaluated: Any[7766279631452241920] == Any[100000000000000000000]

JSON: Test Failed at /Users/sam/julia/dev/LazyJSON/test/foo.jl:17
  Expression: jparse("[-237462374673276894279832749832423479823246327846]") == Any[@big_str("-237462374673276894279832749832423479823246327846")]
   Evaluated: Any[-182283650984083494] == Any[-237462374673276894279832749832423479823246327846]

JSON: Test Failed at /Users/sam/julia/dev/LazyJSON/test/foo.jl:20
  Expression: jparse("[\"\\uD888\\u1234\"]") == Any["\ud888ሴ"]
   Evaluated: Any["𥘴"] == Any["\ud888ሴ"]

Test Summary: | Fail  Total
JSON          |    4      4

samoconnor avatar Nov 05 '18 03:11 samoconnor

A couple of additional failures:

# i_string_incomplete_surrogates_escape_valid.json
  Expression: jparse("[\"\\uD800\\uD800\\n\"]") == Any["\ud800\ud800\n"]
   Evaluated: Any["ﰀ\n"] == Any["\ud800\ud800\n"]

# i_string_inverted_surrogates_U+1D11E.json
  Expression: jparse("[\"\\uDd1e\\uD834\"]") == Any["\udd1e\ud834"]
   Evaluated: Any["\U157434"] == Any["\udd1e\ud834"]

samoconnor avatar Nov 05 '18 03:11 samoconnor