Gojko Čutura

icon location Lausanne, Switzerland icon location Graduate Student at EPFL | Research Fellow at Rice University

Results 2 issues of Gojko Čutura

Effect of FIM on StarCoder pre-training

2 comment

Hi! Curious to know some more details about FIM and its effect on the pre-trained model. Here's a paragraph from the SantaCoder paper: > FIM for cheap We observe a...

Formatting FIM data

1 comment

Hi, I want to finetune my model on FIM-only data. If I use this repo for FIM data formatting, seems like it could frequently happen that a single chunk (i.e....