Glenn Matlin comments

Repositories
Issues
Comments

Results 2 comments of


                                            Glenn Matlin

Question: update preprocessing scripts to use HuggingFace datasets for pretraining?

@stas00 @adammoody How hard would it be to bring in the changes from bigscience-workshop/Megatron-DeepSpeed#48 to this repo? Allowing DeepSpeed examples to use HuggingFace would be great for people engaged in...

Question: update preprocessing scripts to use HuggingFace datasets for pretraining?

Thank you for the blazing fast reply @stas00 @conglongli Apologies for the confusion — I am specifically curious about training the example BERT model from DeepSpeedExamples (DSE) with the HuggingFace...