dash-infer

dash-infer copied to clipboard

x86平台前面都没报错，最后一步报错了。

4

983183947

Illegal instruction (core dumped)

3

我通过 pip install 安装dashinfer后，执行推理报错

983183947

CPU specifications for Qwen2-7B models

1

I would like to know what should it be the minimum requirement specifications for running 7B models. I have this configuration and only can run 1.5B with average throughput 8.1...

LiweiPE

汇编kernel是指x86和arm都有吗

1

zsbd

wly-115

Willamjie

有启动服务的功能吗？启动命令是什么？

1

如题

XufengXufengXufeng

When executing pip install dashinfer, torch, pandas, tabulate and so on are not installed automatically since the `install_requires` in python/setup.py are not managed correctly.

yejunjin

当prompt_token 超过模型支持的最大长度时, 程序就恢复不了, 一直返回报错信息

2

在dashinfer集成进fastchat过程中，~~当prompt token超过engine_max_length时~~ 当.generation_config.max_length < prompt token < .engine_config.engine_max_length，程序恢复不了。

yejunjin

Here is an example of config file at examples/python/model_config/config_qwen_v10_7b.json ```json { "model_name": "Qwen-7B-Chat", "model_type": "Qwen_v10", "model_path": "~/dashinfer_models/", "data_type": "float32", "device_type": "CPU", "device_ids": [ 0 ], "multinode_mode": false, "engine_config": { "engine_max_length":...

yejunjin

为啥我转换的时候输出是none。我是用的4b不是4chat，和这有关系吗？

4


txl0117