chen lian comments

Results 7 comments of


                                            chen lian

How should I speed up T5 original exported saved_model by using TRT?

@nvpohanh could you please look at this?

How should I speed up T5 original exported saved_model by using TRT?

> I think TRT should have good out-of-box performance for T5 now. can you try to export it to onnx and check the throughput? @zerollzeng thanks for your answer. Dose...

How should I speed up T5 original exported saved_model by using TRT?

Thanks you. > I think it's more convenient to export to onnx Yes, I think so. But unfortunately, I need a saved_model to deploy service. I tried > Or using...

How should I speed up T5 original exported saved_model by using TRT?

Thanks for your suggestion. I have some questions to discuss with you. Can I get your other contact details? email, work IM or something else? It would be nice if...

Is it possible to use tensorrt to speed up original tensorflow t5 exported saved_model?

> I no longer work in TF sorry, my fault.

Is it possible to use tensorrt to speed up original tensorflow t5 exported saved_model?

never mind, i found solution.

Is it possible to use tensorrt to speed up original tensorflow t5 exported saved_model?

The original saved_model tooks 300ms when batch_size=32 and sen_length=128, it's too long for deploy. So I wanted to speed up t5 by using tf-trt. But when I convert saved_model using...