Adi Renduchintala comments

Repositories
Issues
Comments

Results 19 comments of


                                            Adi Renduchintala

Inference discrepancies after merging weights into a LoRA model

@mark-myzhao Thanks we will dig further. In the mean time can you try training lora weights using https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/language_modeling/tuning/megatron_gpt_finetuning.py

PEFT training of an LLM as an embedding model

jenkins

PEFT training of an LLM as an embedding model

jenkins

PEFT training of an LLM as an embedding model

jenkins

PEFT training of an LLM as an embedding model

jenkins

PEFT training of an LLM as an embedding model

closed in favor of #8622

Add LoRA support to all linear layers

jenkins

Add LoRA support to all linear layers

jenkins

Adi Renduchintala

add alpha scaling to lora

add alpha scaling to lora

Inference discrepancies after merging weights into a LoRA model

PEFT training of an LLM as an embedding model

PEFT training of an LLM as an embedding model

PEFT training of an LLM as an embedding model

PEFT training of an LLM as an embedding model

PEFT training of an LLM as an embedding model

Add LoRA support to all linear layers

Add LoRA support to all linear layers