1.5 * 1/3 = 0.49999999999999995 instead of 0.5

[basnijholt]

I have just notices that the following happens:

a := decimal.RequireFromString("1.5")
b := decimal.RequireFromString("3")
one := decimal.NewFromInt(1)
bInv := one.Div(b)
fmt.Println(a, b, bInv, a.Mul(bInv), a.Div(b))

which prints

1.5 3 0.3333333333333333 0.49999999999999995

Is this a bug or is this to be expected?

[mwoss]

Hi! I would say it is not intended behavior, but I'm not sure what former authors of the library have in mind during implementations of the Mul method. Personally, I would also expect 0.5. I've also checked how other decimal libraries (Python, Java) behave, and all of them output 0.5. I will assume this is a bug. Thanks for reporting this, I will try to fix this!

[KerkDovan]

This seems to be not a bug, just a loss of infinitely repeating part. Other decimal libraries fight this problem by rounding after multiplication and division. Not perfectly, however: they fail on other numbers (Python).

P.S. Checked the same numbers for your library as well, and the error increases with the numbers.

P.P.S. Other decimal libraries (at least Python's) implement this specification.

[mwoss]

Yes, you are right @KerkDovan the product of div operation has a precision = 17, and the div factor has precision = 16. So if we round the result at the end to 16 places after a decimal point it should works as most people expect. Maybe it's not a bug, but either way it should be at least documented or the logic should be changed IMO.

[KerkDovan]

Precision of the inverted number (bInv) must be much more (twice seems to work) than the length of the multiplicand (a). Here are some tests: https://play.golang.org/p/BFN6BBzVAOz

Thus, the pattern should look like this:

aLen := int32(len(a.String()))
bInv := one.DivRound(b, 2*aLen)
c := a.Mul(bInv).Round(int32(decimal.DivisionPrecision))