Medusa icon indicating copy to clipboard operation
Medusa copied to clipboard

which dataset should i use when training medusa heads with llama2 7b

Open tu2022 opened this issue 1 year ago • 0 comments

I want to train a llama2 7b model, which is finedtuned by my own dataset. I don't know which dataset should i use while training the medusa heads? The dataset (this is a small dataset) i used in finetuning or an general big dataset?Is the training dataset different,when training the medusa heads in medusa 1 and medusa 2?

tu2022 avatar Apr 03 '24 07:04 tu2022