libgdf icon indicating copy to clipboard operation
libgdf copied to clipboard

[WIP] Binary Operators

Open wmalpica opened this issue 7 years ago • 3 comments

Implemented so far is the first implementation concept using nested templating. It has been implemented with support for literals and 14 different binary operators.

We would like feedback on the API implemented, which is:

`` gdf_error gdf_scalar_operation(gdf_column* out, gdf_column* vax, gdf_scalar* vay, gdf_scalar* def, gdf_binary_operator ope);

gdf_error gdf_vector_operation(gdf_column* out, gdf_column* vax, gdf_column* vay, gdf_scalar* def, gdf_binary_operator ope);

enum gdf_binary_operator { GDF_ADD, GDF_SUB, GDF_MUL, GDF_DIV, GDF_TRUE_DIV, GDF_FLOOR_DIV, GDF_MOD, GDF_POW, //GDF_COMBINE, //GDF_COMBINE_FIRST, //GDF_ROUND, GDF_EQUAL, GDF_NOT_EQUAL, GDF_LESS, GDF_GREATER, GDF_LESS_EQUAL, GDF_GREATER_EQUAL, //GDF_PRODUCT, //GDF_DOT };

struct gdf_scalar { union gdf_data { std::int8_t si08; std::int16_t si16; std::int32_t si32; std::int64_t si64; float fp32; double fp64; };

gdf_data  data;
gdf_dtype type;

}; ``

We have already started on a second implementation using NVRTC which uses the same API.

Unit tests have not yet been implemented and will be implemented after an implementation method has been decided.

This current implementation right now only supports the second operand being able to be a scalar, which can be improved upon easily later.

In order to compile the binary operation functionality, use the cmake variable "BINARY_OPERATION_VERSION" and choose the binary version.

Example: cmake -DCMAKE_BUILD_TYPE=Release -DBINARY_OPERATION_VERSION:STRING=V1 ../../code/libgdf

This cmake switch was added right now because compilation time is very large due to all the functions generated by the templated code. For example compilation on a laptop with an 8-core i7 processor and 16Gb ram was: Time to compile: 77m10.996s Ram memory used: More than 13.5GB Release binary size: 69,931,016 bytes

wmalpica avatar Aug 13 '18 19:08 wmalpica

This has a conflict with master now

scopatz avatar Aug 14 '18 15:08 scopatz

what is the corresponding pygdf PR for the bindings update?

nsakharnykh avatar Sep 10 '18 20:09 nsakharnykh

Haven't had a chance to review yet, but please make sure that all the operators can handle empty inputs, i.e., just return GDF_SUCCESS if the input size is 0. Make sure there are unit tests for these as well. This has bitten us enough times elsewhere that I want to make sure it gets added from the start :)

jrhemstad avatar Sep 11 '18 20:09 jrhemstad