Prakyath Kantharaju

Hello, Great theme, and thank you for making it open source. I have the same issue, I am not able to solve the issue. I hosting the website on netlify....

@RdoubleA I added a Dora-based update to my fork. If you approve, I can submit a pull request. Link to the Dora update line here: https://github.com/Prakyathkantharaju/torchtune/blob/aefb8cbb02712177d690ca65cbac480fcb8ac429/torchtune/modules/peft/lora.py#L137

Hello @ebsmothers , Yes, that is correct. The method presented in the paper and that I wrote is slightly different, I based my code on this repo: https://github.com/rasbt/dora-from-scratch/blob/main/Using-LinearDoRA.ipynb I updated...

Hi @joecummings , Thank you for taking the time to review and comment on the pull request. I wanted to highlight a few advantages of using ClearML over Wandb and...

Hello @kartikayk , I think that's a great suggestion. I agree that including every logging system may be excessive, and it would be better to mention it in the readme...

Hello @kartikayk, Done, Its in my main branch now, I have also synced the fork so its up to date now.

> Thanks for opening the PR! I am still not sure about the correctness of the implementation though. Can you run forward on the same input tensor and confirm you...

Hello and thank you for your response. I apologize for not updating you on this issue for a while. I am currently working on comparing the performance of the Dora...

Hello everyone, I apologize for the delayed response, and I appreciate your review of my changes. I have addressed the comments made by @ebsmothers and updated the structure of how...

@calvinpelletier Thank you for reviewing. You are right, I have to implement `torchtune/module/peft/peft_utils.py::get_merged_lora_ckpt` However, for dora merging, since it involves a weight normalization ([link](https://github.com/Prakyathkantharaju/torchtune/blob/7b4b8a4dbae45f5b1b436fe902c6ec5006e3e28c/torchtune/modules/peft/lora.py#L138C23-L138C29)) I need the linear layer weight...