Some floating point numbers are truncated significantly since version 7.2.0

[Arduinux]

Describe the bug
Some floating point numbers with two or more zeros after the decimal point only have four or three significant decimal digits.

Troubleshooter report
Here is the report generated by the ArduinoJson Troubleshooter:
[Paste the report here]

1. The program uses ArduinoJson 7
2. The issue happens at run time
3. The issue concerns serialization
4. Output is incomplete
5. JsonDocument::overflowed() returns false
6. The empty/missing value is neither an array, object, nor string
7. The output is neither a char*, char[N], MqttClient, nor PubSubClient

Environment
Here is the environment that I used:

• Microcontroller: ESP32
• Core/runtime: ESP32 core for Arduino v2.0.17 & v3.3.0]
• IDE: 2.3.6

Reproduction
Here is a small snippet that reproduces the issue.

#ifndef ARDUINO_ESP32_DEV
#  error - Board mismatch. "ESP32 Dev Module" required.
#endif /* ARDUINO_ESP32_DEV */

#define ARDUINOJSON_USE_DOUBLE 1
#include <ArduinoJson.h>

void setup() {
  Serial.begin(115200);

  delay(500);

  Serial.println();
  Serial.println("================================================================================");
  Serial.println();
  Serial.print("ArduinoJson-Version: ");
  Serial.println(ARDUINOJSON_VERSION);

  double doubleA = 0.001403982;
  double doubleB = 0.001043521;
  double doubleC = 0.033841712;
  double doubleD = 0.025639649;
  double doubleE = 0.000292723;

  JsonDocument testDouble;
  testDouble["testA"] = doubleA;
  testDouble["testB"] = doubleB;
  testDouble["testC"] = doubleC;
  testDouble["testD"] = doubleD;
  testDouble["testE"] = doubleE;

  String strTestDouble;
  serializeJson(testDouble, strTestDouble);
  Serial.println();
  Serial.println(strTestDouble);

  JsonDocument testJson;
  deserializeJson(testJson, strTestDouble);

  Serial.println();
  Serial.println(testJson["testA"].as<double>(), 9);
  Serial.println(testJson["testB"].as<double>(), 9);
  Serial.println(testJson["testC"].as<double>(), 9);
  Serial.println(testJson["testD"].as<double>(), 9);
  Serial.println(testJson["testE"].as<double>(), 9);

  String strTestJson;
  Serial.println();
  serializeJson(testJson, strTestJson);
  Serial.println(strTestJson);
}

Compiler output
No relevant output.

Program output
If relevant, include the repro program output.

Expected output:

e.g.:
{"testA":0.001403982,"testB":0.001043521,"testC":0.033841712,"testD":0.025639649,"testE":0.000292723}

0.001403982
0.001043521
0.033841712
0.025639649
0.000292723

{"testA":0.001403982,"testB":0.001043521,"testC":0.033841712,"testD":0.025639649,"testE":0.000292723}

Actual output:

ArduinoJson-Version: 7.4.2

{"testA":0.001403982,"testB":0.001043521,"testC":0.033841712,"testD":0.025639649,"testE":0.000292723}

0.001403982
0.001043521
0.033841712
0.025639649
0.000292723

{"testA":0.001404,"testB":0.001044,"testC":0.033841712,"testD":0.025639649,"testE":0.000293}
================================================================================

ArduinoJson-Version: 7.2.0

{"testA":0.001403982,"testB":0.001043521,"testC":0.033841712,"testD":0.025639649,"testE":0.000292723}

0.001403982
0.001043521
0.033841712
0.025639649
0.000292723

{"testA":0.001404,"testB":0.001044,"testC":0.033841712,"testD":0.025639649,"testE":0.000293}

================================================================================

ArduinoJson-Version: 7.1.0

{"testA":0.001403982,"testB":0.001043521,"testC":0.033841712,"testD":0.025639649,"testE":0.000292723}

0.001403982
0.001043521
0.033841712
0.025639649
0.000292723

{"testA":0.001403982,"testB":0.001043521,"testC":0.033841712,"testD":0.025639649,"testE":0.000292723}

[bblanchon]

Hi @Arduinux,

Thank you very much for reporting this bug ❤️

Indeed, since ArduinoJson 7.2.0, deserializeJson() optimizes floating-point values storage by using a double only when necessary. You just found a loophole in the algorithm ; I'll fix it as soon as possible.

Best regards, Benoit

[Arduinux]

Thank you!

Best regards, Arduinux

[bblanchon]

After some investigation, I realized that the issue is not that deserializeJson() stores a float instead of a double, but that serializeJson() doesn't produce enough decimal places when given a float with leading zeros (such as 0.000292723f).

I'll need a bit more time than I thought to fix this bug because the float-to-string conversion is a very tricky part.

[Arduinux]

I can imagine. I dealt with something like that many, many years ago.

Thank you very much!

Best regards, Arduinux

[bblanchon]

I'm still working on this issue.

Adapting the current algorithm to output the right number of significant digits was easy. Unfortunately, this change slightly decreased the accuracy, causing the last digits to sometimes be off by one.

As a consequence, I decided to take a new look at the problem, and rewrite the conversion algorithm from scratch. I'm currently reading a lot of material on the topic, looking for an algorithm that I can adapt to the constraints of low-end MCU boards.