xiongjun19

如何转tensorrt engine

1

我想请问下目前框架支持直接转成tenorrt engine的方案吗？ 因为目前很多时候我们需要用offline 模式来比较不同模型和框架的性能指标

I'm wondering how to use this code to training for the WSC task in superglue. I'm getting very low accuracy running using the script in jiant scripts based on roberta-base.

pip install 有问题

5

NameError: name 'batch' is not defined File "/Users/xiongjun/.local/lib/python3.7/site-packages/paddle/__init__.py", line 35, in batch = batch.batch NameError: name 'batch' is not defined

I'm wondering how to do inference with the saved int8 trt engine file. Dose the process of inference is just the same as normal.

some details about ctc crf model

1

Hi Dear！ I'm a little confused about the code in the ctc crf model, I have four questions: 1. why 1024 * 5 is needed for the last linear layer?...

ctc decoding

2

I have few questions about ctc decoding: Is there any fast ctc decoding interface, which device does it runs ? GPU or cpu, I think it's hard to do ctc...

Dose mlc-llm support parallelism like multi-gpu, multi-node ?

3

Dose mlc-llm support parallelism like multi-gpu, multi-node ?

Import Error, _transducer can not be imported.

1

follow the readme, I got the error about importing error, suggest that there may be a circle import. finally I find a soulution I have to manually copy the _transducer.cpython-xxxx.so...

loss value and decode library?

10

thanks very much for your great project! I have two questions to ask: 1. how big is the the transducer loss for a well performed model? or the model is...