pinot icon indicating copy to clipboard operation
pinot copied to clipboard

Add read APIs for `BOOLEAN` and `TIMESTAMP`

Open Jackie-Jiang opened this issue 3 years ago • 2 comments

Currently BOOLEAN is supported using the INT data type, and TIMESTAMP is supported using the LONG data type. Even though the storage format are the same, the behavior should be different. E.g. reading BOOLEAN as STRING should give "true" or "false" instead of 1 or 0; reading DOUBLE as BOOLEAN should give doubleVal != 0 instead of (int) doubleVal. In order to achieve the expected behavior, we need to add read APIs for these 2 data types. It should be similar to how the BIG_DECIMAL is handled (storage format is BYTES). We want to add type support in:

  • Dictionary
  • ForwardIndexReader
  • BlockValSet
  • TransformFunction

Jackie-Jiang avatar Sep 16 '22 23:09 Jackie-Jiang

hey @Jackie-Jiang is someone working on this issue? I could pick this if no one is on it

ddcprg avatar Oct 18 '22 10:10 ddcprg

@ddcprg Thanks for volunteering, assigned to you

Jackie-Jiang avatar Oct 19 '22 23:10 Jackie-Jiang

@ddcprg are you still looking into this ? It looks like an interesting issue to tackle for a beginner. I'm willing to help out if needed ?

jonas-grgt avatar May 02 '23 17:05 jonas-grgt

@jonasgeiregat I was close to complete this but never did. Unfortunately, I've been away from this project for a while now, hope I can start contributing again in the future. I'm happy if @Jackie-Jiang assigns this to you

ddcprg avatar May 05 '23 10:05 ddcprg

@ddcprg if you were close it would be interesting to look into your approach.

jonas-grgt avatar May 05 '23 14:05 jonas-grgt