AGI-player
AGI-player
> ## Why are these changes needed? > [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) can greatly improve the inference speed of LLM. It would be helpful to support tensorRT-LLM in Fastchat. > > This commit...
> 请问你是如何使用llama 13b的config，然后随机初始化权重的，启动要设置什么参数？ 我也想尝试pretrain 参考的https://github.com/FlagAlpha/Llama2-Chinese/blob/main/train/pretrain/pretrain_clm.py 需要在LLaMA-Factory对应加载模型的地方进行修改 model = AutoModelForCausalLM.from_config(config) n_params = sum({p.data_ptr(): p.numel() for p in model.parameters()}.values()) logger.info(f"Training new model from scratch - Total size={n_params/2**20:.2f}M params")
> > > 请问你是如何使用llama 13b的config，然后随机初始化权重的，启动要设置什么参数？ 我也想尝试pretrain > > > > > > 参考的https://github.com/FlagAlpha/Llama2-Chinese/blob/main/train/pretrain/pretrain_clm.py 需要在LLaMA-Factory对应加载模型的地方进行修改 > > ``` > > model = AutoModelForCausalLM.from_config(config) > > n_params = sum({p.data_ptr(): p.numel() for p...
> @AGI-player Thanks for your feedback. This is a known issue and we will fix it soon. ok~
> > @AGI-player Thanks for your feedback. This is a known issue and we will fix it soon. > >  I got this error it should be caused by...
> > > > @AGI-player Thanks for your feedback. This is a known issue and we will fix it soon. > > > > > > > > > ...
> @AGI-player Thanks for your feedback. This is a known issue and we will fix it soon. @jershi425 is this issue fixed in the latest version?
> Please follow the issue template to share the full end to end reproduced steps. Thank you for cooperation. the trt engine was built with: trtllm-build --gemm_plugin float16 --max_batch_size=128 --max_input_len=8192...
> The `stop_words_list` is not supported well in 0.9.0, maybe you can try the latest main branch, we have refactored the `GenerationExecutor`, and the [stop_words](https://gitlab-master.nvidia.com/ftp/tekit/-/blob/main/tensorrt_llm/executor.py#L106) are supported. I update the...
> Hi @AGI-player , what's tensorrt_llm version? Could you please try the main branch? apt-get update && apt-get -y install python 3.10 python3-pip openmpi-bin libopenmpi-dev git git-lfs pip3 install tensorrt_llm...