Lei Zhang comments

Results 11 comments of


                                            Lei Zhang

Can it process images in pdf?

We really need this feature.🔥Does anyone know if there are any alternatives that can replace this project?

运行支持的128K上下文后报错

Can we use `transformer` directly to enable 128k context inference without deploying with vLLM?

[Question] What are the differences between two versions of pretrain datasets?

I also want to know the difference between the two of them, have you figure it out?

Aider benchmark, DeepSeek-6.7B-Instruct model hardly generates SEARCH/REPLACE blocks, leading to very low pass rates

@ytxmobile98 I think you need to set the `--max-model-len` to a larger number, like 8192. BTW, you may check the log file to locate the issues.

[Bug]: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12

> it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH Very helpful.

[Bug]: Error when use claude-3.5-sonnet through openai/ prefix.

> Hi [@Hambaobao](https://github.com/Hambaobao) , could you check if this works when using `litellm` directly? > > ``` > messages = [ > {"role": "system", "content": "Respond in pirate speak."}, >...

配置中转的模型接口只返回{}

same issue.

[Feature]: Return hidden states (in progress?)

Hi, I also have the same need. I hope to store the `hidden_states` during model inference so that I can conduct some interpretability research.

HBO Max 规则补充

没人提 PR 吗？那我提一个吧

[Bug]: max_input_tokens has no effect — input length still exceeds the limit on SWE-bench

Hi @enyst , thank you very much for your response, I'll see what I can do.