KissInt8 comments

Results 5 comments of


                                            KissInt8

函数注释无法识别指针的指针

我希望预期得到的注释能够识别出指针，并将其体现在**类型**中；或者至少**不在变量名**中出现`*`。如下： ```c /** * @description: * @param {int} a * @param {int *} b * @param {int **} c * @return {*} */ void func(int a, int *b, int **c)...

expandable_segments with PYTORCH_CUDA_ALLOC_CONF reduces VRAM

> Just wanted to confirm that running on A100, with the flag i can run bs=4, but without, it OOMs > > editing for clarity: Green has bsz=3. When i...

[Usage]: Does DeepSeek-R1 1.58-bit Dynamic Quant work on VLLM?

Unfortunately, I encountered the error `GGUF model with architecture deepseek2 is not supported yet.`, when I tried to deploy the DeepSeek R1 Q4_K_M model.

[Usage]: Does DeepSeek-R1 1.58-bit Dynamic Quant work on VLLM?

By the way, is there any way to run DeepSeekR1 with vllm on 8\*A100 or 8\*H100? - I haven't find the safetensors format of R1-Q4. - I have tried to...

[Bug]: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12

> it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH 2025.02.13, this method solved problem! Tks!