jsoneditor icon indicating copy to clipboard operation
jsoneditor copied to clipboard

Values with large numbers getting changed

Open medigvijay opened this issue 10 years ago • 23 comments

The following sample {"abc":8143661439548533232}

is getting changed to following in the right panel

{"abc": 8143661439548533000}

medigvijay avatar Oct 15 '15 10:10 medigvijay

Thanks, this is a limitation in the Number format of JavaScript, which has a precision of about 16 digits.

Solving this is not so trivial but interesting. It would require wirting our own JSON parser which keeps numbers in some string format instead of parse them as regular numbers.

josdejong avatar Oct 16 '15 08:10 josdejong

Any chance of supporting the "transit" format? https://github.com/cognitect/transit-format It is a JSON encoding of extra meta-data, adding types where it's necessary, so that such truncation (and other too) do not happen.

Of course, there are libraries for many languages, including JS :) https://github.com/cognitect/transit-js

Because transit has much more "semantic types" than "raw JSON", it could offer more "cell editors", making it much more usable for end users.

aadrian avatar Nov 02 '15 08:11 aadrian

@aadrian this feature request has nothing to do with this issue, should be in it's own issue instead. Anyway, adding support for other data formats is not on the roadmap for JSONEditor, sorry.

josdejong avatar Nov 02 '15 19:11 josdejong

A short update on this topic: I've written a JavaScript a library which can handle large values without losing precision, see: https://github.com/josdejong/lossless-json I was planning on integrating this in the JSONEditor.

Please note that relying on JSON objects containing for example 64bit long numbers may be tricky, since for example web applications cannot handle long values and will truncate some digits.

josdejong avatar Nov 25 '16 08:11 josdejong

That's great news! I also need the ability to handle 64bits int. So when this feature will release?

crazier avatar Dec 22 '16 07:12 crazier

Yeah, still need to work that out. I'm focusing on a completely new version lately (see #337).

josdejong avatar Dec 25 '16 12:12 josdejong

@josdejong I've added BigInt support to json-source-map

epoberezkin avatar Aug 04 '19 19:08 epoberezkin

Cool :sunglasses:. I hadn't seen json-source-map before I think? Having big integers solves most practical use cases, thanks for sharing.

josdejong avatar Aug 04 '19 19:08 josdejong

You use it here it seems :)

epoberezkin avatar Aug 04 '19 19:08 epoberezkin

https://github.com/josdejong/jsoneditor/blob/develop/package.json#L31

epoberezkin avatar Aug 04 '19 19:08 epoberezkin

Ah :joy: you're right. The code using json-source-map was implemented by @meirotstein, I didn't recall that we started using this specific library then.

josdejong avatar Aug 05 '19 19:08 josdejong

@josdejong correct, this library is used by the selection api. A great library @epoberezkin !

meirotstein avatar Aug 05 '19 20:08 meirotstein

this project now use json-source-map or lossless-json?

i just want to upgrade jsoneditor version .... only use jsoneditor lib;

wilddylan avatar Aug 07 '19 08:08 wilddylan

upgrade jsoneditor to the latest version can fix the big int issue ?

wilddylan avatar Aug 07 '19 08:08 wilddylan

json-source-map is used to generate helpful error messages and JSON schema warnings, and to be able to click on the error message and go to the related line. The core of JSONEditor uses regular JSON and regular numbers though, we still need to replace this with a custom parser, see https://github.com/josdejong/jsoneditor/issues/231#issuecomment-262900954.

josdejong avatar Aug 07 '19 08:08 josdejong

json-source-map is used to generate helpful error messages and JSON schema warnings, and to be able to click on the error message and go to the related line. The core of JSONEditor uses regular JSON and regular numbers though, we still need to replace this with a custom parser, see #231 (comment).

what's the time (milestone) for fix this issue in jsoneditor project?

wilddylan avatar Aug 07 '19 08:08 wilddylan

Not anytime soon I expect.

josdejong avatar Aug 08 '19 19:08 josdejong

Not anytime soon I expect.

can you fix that now ? out project use this library... and we want to fix that;

wilddylan avatar Aug 09 '19 13:08 wilddylan

can you fix that now ?

I would love to but there are simply so much things that I would like to address "now". I simply have a limited amount of spare time that I can spend on open source projects.

If it's this important to you, you could implement it yourself and create a PR to get it merged into the project.

Alternatively, you could try a workaround like adding a pre-processing step where you replace all long values in your JSON with strings before opening it in the JSONEditor, and a post-processing step where you do the opposite.

josdejong avatar Aug 09 '19 14:08 josdejong

let me try; or use string

wilddylan avatar Aug 10 '19 15:08 wilddylan

I am having this same issue. The program is converting an id 10120124111101040103 to a number 10120124111101040000. wilddynlan commented to use a string, how do you do that? I tried enclosing the number in double quotes, but that just makes the value "\"10120124111101040103\"".

DannyL9143 avatar Jun 04 '20 12:06 DannyL9143

This is indeed still an open issue.

The trick is to load JSON containing the large value as a string, so instead of:

{
  "large value": 10120124111101040103 
}

You provide the following document:

{
  "large value": "10120124111101040103"
}

josdejong avatar Jun 04 '20 17:06 josdejong

FYI: the successor of this library, svelte-jsoneditor, now supports lossless-json or other JSON parsers, and https://jsoneditoronline.org/ now uses this parser to handle large values.

You can read more in the following article: https://jsoneditoronline.org/indepth/parse/why-does-json-parse-corrupt-large-numbers/

josdejong avatar Sep 30 '22 08:09 josdejong