Questions about system prompt

[ningshaoling]

I observe that the inference script contains an embedded system prompt. Is it also included in training? I wonder how much the performance will be affected if changing the system prompt.

[ningding97]

I observe that the inference script contains an embedded system prompt. Is it also included in training? I wonder how much the performance will be affected if changing the system prompt.

Embedding system prompts is a common practice that many models would use. However, we didn't use this method since we wanted to make the model more "customizable" with the system prompt. We find system prompts could exert a considerable impact on performance, especially when it comes to details.

[timothylimyl]

@ningding97 how can your model learn to use system prompt to steer responses?

the dataset has no system prompt data to teach it