fend icon indicating copy to clipboard operation
fend copied to clipboard

Incorrect significant figure rounding

Open Markos-Th09 opened this issue 1 year ago • 5 comments

When rounding using significant figures, the number is truncated instead of being rounded. For example

12.9 to 2 sf

outputs 12 instead of 13

Markos-Th09 avatar Sep 04 '24 10:09 Markos-Th09

Thanks for the report, this would definitely be good to fix.

printfn avatar Sep 04 '24 10:09 printfn

also to <n> dp is also truncating but I don't what is the intended behavior of that

Markos-Th09 avatar Sep 04 '24 15:09 Markos-Th09

@printfn Do you intent to fix it by yourself in the near future ?

If not. I would be willing to fix it. However, after looking into the code for that, I came to the conclusion that making sf rounding work will take me a while to get it right.

What are thoughts about it ?

BoolPurist avatar Sep 16 '24 14:09 BoolPurist

@printfn also should regular rounding be preferred over to truncation for the to dp rounding mode? If so, should a truncation rounding mode be introduced?

Markos-Th09 avatar Sep 16 '24 14:09 Markos-Th09

I think both to dp and to sf should round correctly. Supporting optional truncation would be nice too, but less important than getting the rounding right.

printfn avatar Sep 28 '24 01:09 printfn

https://github.com/printfn/fend/pull/373

Safari77 avatar Dec 31 '25 20:12 Safari77