linkedin
PrimitiveGenericRecord interface
Nice! I think it definitely can improve deser performance further like PrimitiveFloatList. @radai-rosenblatt Here is the performance improvement brought by PrimitiveFloatList benchmarked by a Float Array with 800 elements -
- vanilla avro 1.4 Deser latency (ns) : 24603.5 Memory alloc (KB/exe) : 15.9
- avro-fastserde with PrimitiveFloatList Deser latency (ns) : 3926.4 Memory alloc (KB/exe) : 3.2
- avro-fastserde with GenericArray<Float> Deser latency (ns) : 6904.2 Memory alloc (KB/exe) : 15.8
@volauvent For primitive list, the big benefit is totally reasonable. But for the primitive type for each field, we may need to do another round of benchmark to understand the benefit accurately, and it should be related to how many primitive types the users are planning to store inside the record. Can we try different number of primitive fields and evaluate the benefit?
@FelixGV Also do we recommend the usage to everyone or just the ones who are heavily using primitive fields in the record types? Since I could see another coding overhead while using this new type, I would like to understand the proper use case.
@gaojieliu since we only have primitiveFloatList now, we can vary float array size and compare the performance diff between Primitive float and default box one. The above metric is for 800 FloatArray, are you saying reduce the array size to a small number to understand the real performance improvement brought by primitive float type, such as to 5?