della
della copied to clipboard
how to merge the base model and the fine-tuned model in mergekit?
models:
- model: models/vicuna-7b-v1.5-16k parameters: weight: 1.0
- model: models/vicuna_7b_A parameters: weight: 1.0 base_model: models/vicuna-7b-v1.5-16k merge_method: della parameters: normalize: true int8_mask: true density: 0.7 lambda: 1.1 epsilon: 0.2 dtype: float16
I hope to merge the base model (e.g. models/vicuna-7b-v1.5-16k) and the fine-tuned model (e.g. models/vicuna_7b_A), using mergekit.
Is this setting correct? Do I need to remove the lines "-model: models/vicuna-7b-v1.5-16k" in models?
Thanks