Michael

性能问题

1

是否有和tensorrt8.0和fasttransformer4.0做过对比？这写都有一些更新，比如int8推理，比如优化了BS。 另外，有没有实际推理优化的数据，特别是针对大模型，几十亿，上百亿甚至更大的做过试验

支持大模型的推理吗

1

目前支持大模型吗？比如30亿（enc dec 均36层) 或者上百亿的，transformer模型，有一些推理优化的性能指标吗？

meteor issue

1

The meteor don't have the chinese word data, and the meteor mainpage don't have too. whether it's not proper to evaluate chinese nlg task using meteor ？

whether t5 can be used like transformer

1

I want to distill a big model to t5 model, and the big model use bpe not sentencepiece, So the tokenizer should be load the bpe codes and the vocabs...

I want to train my t5 model from scratch and with bpe tokenizer, is there a example ?

May i use triton-inference-server to run T5 model

36

I can see the T5 triton backed related file in https://github.com/NVIDIA/FasterTransformer/tree/dev/v5.0_beta_2021.11_tag/src/fastertransformer/triton_backend/t5 But i also find the https://github.com/triton-inference-server/fastertransformer_backend just support gpt I wonder whether T5 is supported by triton inference server....

目前框架支持transformer模型的优化吗？如果支持，使用的是那一套框架的代码，具体步骤有吗？thx

1，请教下triton pytorch model上线的话，是否支持自定义算子，因为有些推理有话的需求在里面。我看triton官方是支持的 https://github.com/triton-inference-server/server/blob/main/docs/custom_operations.md 2，上线模型有T5的case吗？

universal transformer

1

Could you make a ReadMe more specifically about universal translation? And is it recommended to continue to do some research about universal translation? please give me some advice, thx.

高版本的torch和transformers  在predict和train py文件做如下修改即可： #from torch._six import container_abcs, string_classes, int_classes TORCH_MAJOR=0 if TORCH_MAJOR == 0: import collections.abc as container_abcs int_classes = int string_classes = str else: from torch._six import container_abcs