How to continue pre-training Bloom?

[ShinoharaHare]

Hi I'm trying to continue pre-training the bloom-560m on my own dataset on a single GPU. I modified this script to fit my case. However, i cannot figure out how to load the checkpoint.

Is there any guide for what i'm doing?

[lwmlyy]

Hi, did you find any solution on this?

[noob-ctrl]

@ShinoharaHare Hi, have you solved this problem？