NeMo icon indicating copy to clipboard operation
NeMo copied to clipboard

TRT-LLM Export Code Cleanup

Open oyilmaz-nvidia opened this issue 1 year ago • 4 comments

What does this PR do ?

Cleaning all of the unused functions, classes, etc, after the TRT-LLM API integration.

oyilmaz-nvidia avatar May 21 '24 21:05 oyilmaz-nvidia

get_tensor_parallel_group is not used anymore.

I think this one too:

jiemingz avatar May 22 '24 17:05 jiemingz

I believe cpu_map_location is duplicated with this

Also it seems to me cpu_map_location and gpu_map_location should be in trt_llm/nemo/nemo.py since thats where they're only used

jiemingz avatar May 22 '24 17:05 jiemingz

This function is only used by build and refit so it can be removed, eventually though we'll have to add it back as well when build and refit are moved onto the new API

jiemingz avatar May 22 '24 17:05 jiemingz

Thanks @jiemingz for your review. Updating the code per your comments.

oyilmaz-nvidia avatar May 22 '24 18:05 oyilmaz-nvidia