lwmlyy

https://scholar.google.com/citations?user=YyoelDMAAAAJ&hl=en

Shanghai University of Finance and Economics Shanghai NLP, WSD, machine learning, deep learning

Results 4 comments of


                                            lwmlyy

[BUG]: Fail to load huggingface pretraining when use shardinit

any update? @YuliangLiu0306 @FrankLeeeee

[BUG]: Fail to load huggingface pretraining when use shardinit

@ShinoharaHare Thx for the reply. I'll give it a try. By the way, did you compare the accelerate performance between this strategy and Megatron?

MultiSpider dataset availability

@longxudou Any progress?

How to continue pre-training Bloom?

Hi, did you find any solution on this?