felixstander

Results 13 comments of felixstander

[BUG] Qwen72B-Chat-Int4部署vLLM_gptq，API调用，任务推理过慢时会出现 Aborted request

遇到相同的问题

RuntimeError: expected scalar type Half but found Float

使用transformers的trainer进行训练的时候，当不开fp16=True的时候，是可以train起来的，但是跑着跑着loss会变成0。我认为是在使用prepare model for int8 training的时候需要调整layer norm的数据类型到float32，为了训练的稳定性。而在调整layernorm到float32后，就会出现expected scalar type half but found float。（bloom里的LayerNorm叫layernorm而不是layer_norm）

RuntimeError: expected scalar type Half but found Float

顺便想问问作者是使用8卡A100 40G进行zero stage2的finetune吗?显存的占用是怎么样的呀？想问问8卡 v100 16G有可能跑起来吗

Example Finetune_opt_bnb_peft.ipynb runs in colab notebook but fails on my system with seemingly the same requirements installed: "expected scalar type Half but found Float"

any update on this issue? I also face the same problem

when to support chatglm2-6b?

mark

关于 InternLM 通用工具调用能力 的疑问

这个有官方的答复吗，能调用各种工具的能力很是刚需

[Feature Request]: Iterative query results for post search filtering

also facing the slow "where" filtering problem. I got around 50k data in one collection.

Is there any plan to support dynamic lora for qwen/chatglm models?

can support the Alibaba open-source Qwen model will be wonderful

Is there any plan to support dynamic lora for qwen/chatglm models?

> Hey @KrisWongz @felixstander, #103 should add support for Qwen. The base model appears to generate results consistent with the example on Huggingface Hub. Do you have an adapter I...

Is there any plan to support dynamic lora for qwen/chatglm models?

Does Lorax support Qwen-4bit-gptq version without the need of flash attention v2? As far as I can see, All models you support now are built on top of flash attention...