della icon indicating copy to clipboard operation
della copied to clipboard

how to merge the base model and the fine-tuned model in mergekit?

Open zepingyu0512 opened this issue 9 months ago • 0 comments

models:

  • model: models/vicuna-7b-v1.5-16k parameters: weight: 1.0
  • model: models/vicuna_7b_A parameters: weight: 1.0 base_model: models/vicuna-7b-v1.5-16k merge_method: della parameters: normalize: true int8_mask: true density: 0.7 lambda: 1.1 epsilon: 0.2 dtype: float16

I hope to merge the base model (e.g. models/vicuna-7b-v1.5-16k) and the fine-tuned model (e.g. models/vicuna_7b_A), using mergekit.

Is this setting correct? Do I need to remove the lines "-model: models/vicuna-7b-v1.5-16k" in models?

Thanks

zepingyu0512 avatar Apr 18 '25 20:04 zepingyu0512