Medusa
Medusa copied to clipboard
which dataset should i use when training medusa heads with llama2 7b
I want to train a llama2 7b model, which is finedtuned by my own dataset. I don't know which dataset should i use while training the medusa heads? The dataset (this is a small dataset) i used in finetuning or an general big dataset?Is the training dataset different,when training the medusa heads in medusa 1 and medusa 2?