BitNet icon indicating copy to clipboard operation
BitNet copied to clipboard
Published 8 months ago • verified publisher icon microsoft

FineTune the 1.58b

Open abal1000x opened this issue 9 months ago • 0 comments

I want to continue pretraining the 1.58b 2B model to add more on my language. Or finetune for specific knowledge.

Are there any base code i could start with to train for 1.58b. I've read the paper, and its used the unusual method to measure the gradient of the ternary parameter.

abal1000x avatar May 07 '25 06:05 abal1000x