printf icon indicating copy to clipboard operation
printf copied to clipboard

Siginificant-digits-to-precision transition made invalid by rounding

Open eyalroz opened this issue 4 years ago • 0 comments

If we sprintf(buffer,"%#.3g", 99.99);, we get 100.0 - with 4 significant digits - while we should be getting 100., with 3 significant digits.

This is because, initially, we find the power of 10 to be 2; then we translate significant digits to precision, setting the precision to 1. But after this happens, we perform rounding with precision 1, so that the power of 10 becomes 3...

eyalroz avatar Aug 14 '21 15:08 eyalroz