DarkenStar

I followed the [https://github.com/llvm/torch-mlir/blob/main/docs/development.md](url) to build the project (in tree), but when I run unit test the command `cmake --build build --target check-torch-mlir` errors occured like `******************** ******************** Failed Tests...

Quantized model behave differently on multi GPU with device_map="auto" compared to single GPU

6

I quantized llma2-70b into Int8 format using [vllm expample](https://docs.vllm.com.cn/en/latest/features/quantization/gptqmodel.html). But I found that if I load the model with `device_map="auto"` on 2 GPUs, the output attention hidden states of second...