elliottnv

Results 7 comments of


                                            elliottnv

[BUG] wrong loss scaling when context parallel is on

@xrennvidia Can you please help to answer this question? Thank you!

[BUG] `finish_embedding_wgrad_compute` appears after grad all-reduce

@sanandaraj5597 Can you share some comments? Thank you!

[BUG] context manager syntax bug in transformer_block.py

@sudhakarsingh27 Can you please take a look at the proposed changes? Thank you!

[QUESTION] Sample idx, bin files in public domain for trying out pretrain_gpt.py?

@ashors1 Can you please help with this question? Thank you!

Batch_input and elapsed time per iteration slow down during model training

Can you try running without deepspeed? Thanks

[BUG] pipeline_paralle is not available when pp_size > 2

Please add instructions how to reproduce.

[BUG] init_method not declared

Hi, this is in legacy so it has a low priority. Can you please provide a fix and we will try to merge it?