TensorRT-LLM
TensorRT-LLM copied to clipboard
Fixed rslora scaling in lora_manager
Addressing issue mentioned in https://github.com/NVIDIA/TensorRT-LLM/issues/1668
When weights were trained using rslora scaling they should be scaled differently. Code initially was always normalizing by rank regardless of "use_rslora" flag in huggingface adapter_config.json file.
Scaling has also been updated in examples/hf_lora_convert.py
Could you share a model trained by rslora?
Hi @TheCodeWrangler , thanks for your contributing. We've merged your contribution into code base and will add you into contributor list.