concise-encoding icon indicating copy to clipboard operation
concise-encoding copied to clipboard

Strange behavior with float arrays in CTE

Open kengruven opened this issue 4 years ago • 3 comments

In float arrays, enctool sometimes tries to write numbers in base-16, but doesn't quite use valid syntax:

$ echo 'c1 |f16 1|' | ./enctool convert -df cte
c0
|f16x 1p+00|

The enctool parser correctly reports this as an error:

$ echo 'c1 |f16 1|' | ./enctool convert -df cte | ./enctool convert -df cte
c0
offset 10 (line 2, col 8): unexpected [p] while decoding hex float

My first thought was that "1" is an int, not a float, and "f16" might be expecting only a float (despite there being only one 'numeric type'?). But it fails the same way for some floats, too:

$ echo 'c1 |f16 1.0|' | ./enctool convert -df cte
c0
|f16x 1p+00|

It's not just forgetting the ".0" when writing base-16 values inside an array, though. There's something weirder going on. Here, it prints an invalid number, and also flips the sign:

$ echo 'c1 |f16 -1|' | ./enctool convert -df cte 
c0
|f16x 1p+00|

kengruven avatar Jun 13 '21 18:06 kengruven

There's a bug in the value printer somewhere. I had to put special code in for float16 values since it's not a native format, so something broke along the way I bet.

kstenerud avatar Jun 13 '21 18:06 kstenerud

Hmm, you're right that most of these issues are f16-specific. Even with f32/f64, though:

$ echo 'c1 |f32 -1|' | ./enctool convert -df cte 
c0
|f32x 1|

Perhaps that's a separate bug.

kengruven avatar Jun 13 '21 19:06 kengruven

Just to confirm, are |f32 1| and |i32 1.0| both considered valid?

kengruven avatar Jun 14 '21 19:06 kengruven