taoxunqiang

RT. Thank you!

AttributeError: module 'paddleslim' has no attribute 'models'

1

安装信息如下： paddlepaddle-gpu 2.6.0.post117 paddleslim 2.6.0 遇到的问题如下： >>> import paddleslim as slim >>> >>> >>> model = slim.models.MobileNet() Traceback (most recent call last): File "", line 1, in AttributeError: module 'paddleslim'...

ImportError: cannot import name 'BatchPointInfer' from 'dds_cloudapi_sdk'

1

Thank you for your wonderful work. I installed the trex according to "Setup". The following error occurred when running "from dds_cloudapi_sdk import BatchRectInfer, BatchPointInfer, BatchEmbdInfer"： ImportError: cannot import name 'BatchPointInfer'...

RT Thank you!

Experiments for speculative_decoding

3

"We tested the speculative inference using the first 100 inputs from alpaca test dataset as prompts. When model=gpt2-xl, draft_model=gpt2". I want to test speedup for my own model and draft_model....

Thank you for wonderful job. I have a question. How to convert pth to onnx and tensorrt engine?