DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

add quant unit test

Open GuanhuaWang opened this issue 3 years ago • 0 comments

This PR adds unit test for ds_quantize kernel, In this example, we test over torch.float16 data format.

We tested in following two cases:

  1. INT8 quant/dequant over tensor in 1 group.

Here we form 1 single quantization group. We quantize input_tensor with data format of torch.float16 to torch.int8, and then convert back to torch.float16 tensor.

  1. INT4 quant/dequant over tensor partitioned into 16 groups.

Here we form 16 quantization groups. We quantize input_tensor with data format of torch.float16 to int4, then convert back to torch.float16 tensor.

TODOs:

  • [x] fix unit-tests compiling error of undefined symbol: curandSetPseudoRandomGeneratorSeed by adding -lcurand flag inside op_builder/quantizer.py
  • [x] create new folder for quantizer tests as tests/unit/ops/quantizer

The C++/CUDA APIs of the kernels are subject to refactoring, but it will lead to another PR.

CC: @awan-10 @cmikeh2

GuanhuaWang avatar Sep 09 '22 23:09 GuanhuaWang