bigcode-project
Is it possible to fine-tune the model to obtain better results in Swift?
I would like to train the model specifically on swift documents that are not the the classical chat format ( instruction/input/output). Can I use the script in starcoder/finetune/finetune.py? In which form I should put the data?
The fine-tuning script, i.e. finetune.py is designed to fine-tune Starcoder to map an input text to an output text . If you have a dataset which follows that template (or if you can modify a dataset in order to have that format), you can use the provided code to perform your fine-tuning without any further issue.
If you want a similar data preparation to the pre-training you can use this dataset template which packs sequences of constant length.
