Kaiyan Zhang

看到有设置纯中文不翻译的，这个挺好的 但是当中文夹杂数字时，就显示翻译提示了，这个可以在中文判断时加入数字判断吗

现在有蛮大开源的中文生成式预训练模型了，缺少一个 benchmark 呀

[FSDP+QLoRA] ValueError: Expected a cuda device, but got: cpu

2

### System Info pip list ``` accelerate 0.29.3 bitsandbytes 0.43.1 datasets 2.14.6 huggingface-hub 0.20.3 llama-recipes 0.0.1 peft 0.10.0 safetensors 0.4.2 tokenizers 0.19.1 torch 2.1.2 transformers 4.40.0 cupy-cuda12x 12.1.0 nvidia-cuda-cupti-cu12 12.1.105...

FSDP + QDoRA Support

6

Hi the team, great work! QDoRA seems to be better than QLoRA, refer to [Efficient finetuning of Llama 3 with FSDP QDoRA](https://www.answer.ai/posts/2024-04-26-fsdp-qdora-llama3.html) I wonder whether there will be demo /...

Request for Scripts to Merge QDoRA Adapters with Base Model for vLLM Inference

4

Hello, I've successfully finetuned Llama-3 8B with QDoRA and am now looking to perform inference using vLLM. Could you provide guidance or scripts on how to merge the QDoRA adapters...

Request for Multi-Turn Chat Feature with Single Selection

1

Hi there! I’ve been enjoying using your extension, but I find it a bit cumbersome to re-select the paper item for each dialogue turn. Could we possibly implement a feature...

OpenAI Compatible API Support

1

Thanks for your wonderful project! Will your support OpenAI compatible API?

decoding parameters (e.g., temperature) for Gemma-2?

4

Hello, How should I set the decoding parameters (e.g., temperature) for Gemma-2? My result is about ~50.0, far from the benchmark of 76.

Hello, thanks to OpenRLHF for this great project. We're working on [MARTI](https://github.com/TsinghuaC3I/MARTI) based on OpenRLHF, a unified framework for LLM-based Multi-Agent Systems with centralized interaction & distributed policy reinforcement training....