Xinlin Zhuang

Need help for installing dolma

2

Hi, I encountered some problems when running ``` pip install dolma ``` At first, the error message prompts me to install Rust. After I install Rust and set in the...

Would you please support LoRA training?

1

Dear authors, Would you please support LoRA training? :) Since I have only 3090 GPUs.

No optimizer.bin in Step 2

3

Hi， When I run "Step 2: Building the gradient datastore" FileNotFoundError: [Errno 2] No such file or directory: '../out/llama2-7b-p0.05-lora-seed3/checkpoint-1688/optimizer.bin' I check the folder "llama2-7b-p0.05-lora-seed3" generate from Step 1, only files...

Dear authors, I tried to implement the rule on page 57 of your Dolma paper 'Remove documents with more than half of their line not ending in...'. And I modified...

Dear authors, I was trying to reimplement the Dolma-Web described in your paper. However, in the Step 2, using the dolma toolkit, I found Gopher implementation in this repo something...

[Feature] Please support Llama3.2 and Qwen2.5

5

### Motivation https://huggingface.co/collections/meta-llama/llama-32-66f448ffc8c32f949b04c8cf https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e ### Related resources _No response_ ### Additional context _No response_