bigdecimal
bigdecimal copied to clipboard
Inconsistent max precision affecting dump&load result
We've observed that BigDecimal treats number and string differently on max precision allocation.
Ruby 2.6.6
# precs returns [SIZET2NUM(p->Prec*VpBaseFig()), SIZET2NUM(p->MaxPrec*VpBaseFig())]
BigDecimal(123).precs
=> [9, 27]
BigDecimal('123').precs
=> [9, 18]
BigDecimal(123456789).precs
=> [9, 27]
BigDecimal('123456789').precs
=> [9, 18]
BigDecimal(123456789012).precs
=> [18, 27]
BigDecimal('123456789012').precs
=> [18, 27]
Ruby 3.1.2
# precs returns [SIZET2NUM(p->Prec*VpBaseFig()), SIZET2NUM(p->MaxPrec*VpBaseFig())]
BigDecimal(123).precs
=> [9, 9]
BigDecimal('123').precs
=> [9, 18]
BigDecimal(123456789).precs
=> [9, 9]
BigDecimal('123456789').precs
=> [9, 18]
BigDecimal(123456789012).precs
=> [18, 18]
BigDecimal('123456789012').precs
=> [18, 27]
The problem is also applied to dump & load, which is troublesome.
Especially when taking serialized result for comparison to check for duplicates or data changes.
Ruby 2.6.6
BigDecimal(123)._dump
=> "27:0.123e3"
BigDecimal._load("27:0.123e3")._dump
=> "18:0.123e3"
BigDecimal(123456)._dump
=> "27:0.123456e6"
BigDecimal._load("27:0.123456e6")._dump
=> "18:0.123456e6"
BigDecimal(123456789)._dump
=> "27:0.123456789e9"
BigDecimal._load("27:0.123456789e9")._dump
=> "27:0.123456789e9"
BigDecimal(123456789012)._dump
=> "27:0.123456789012e12"
BigDecimal._load("27:0.123456789012e12")._dump
=> "27:0.123456789012e12"
Ruby 3.1.2
BigDecimal(123)._dump
=> "9:0.123e3"
BigDecimal._load("9:0.123e3")._dump
=> "18:0.123e3"
BigDecimal(123456)._dump
=> "9:0.123456e6"
BigDecimal._load("9:0.123456e6")._dump
=> "18:0.123456e6"
BigDecimal(123456789)._dump
=> "9:0.123456789e9"
BigDecimal._load("9:0.123456789e9")._dump
=> "18:0.123456789e9"
BigDecimal(123456789012)._dump
=> "18:0.123456789012e12"
BigDecimal._load("18:0.123456789012e12")._dump
=> "18:0.123456789012e12"