Alan Fang comments

Results 49 comments of


                                            Alan Fang

About FSDP，deepspeech.

fsdp has a bug: sharding_strategy needs “ShardingStrategy” not “str”

> > fsdp 有一个错误：sharding_strategy 需要“ShardingStrategy”而不是“str” > > 你解决了吗？ @fclearner yes，try this:https://github.com/wenet-e2e/wenet/blob/24375c51c8ccbf1dac0724f5734a6eae4ac9c428/wenet/utils/train_utils.py#L410

pip install 遗漏了一些包

加init.py就能解决吗，我这边没遇到这个问题

pip install 遗漏了一些包

了解了，后面补一个

pip install 遗漏了一些包

> 那请问这个问题怎么解决呀？你还有碰到类似的问题吗，最新的代码我已经加了__init__.py了

Model Checkpointing + FSDP causes Cuda OOM

Is this problem solved? I have just met the same one, maybe its related to the ckpt-save part

loss value and acc seems wrong when after setting gradient_accumulate_step>1 in Deepspeed training

try this: model_engine.backward(loss) if (step + 1) % model_engine.gradient_accumulation_steps() == 0: model_engine.step() model_engine.zero_grad()

loss value and acc seems wrong when after setting gradient_accumulate_step>1 in Deepspeed training

> try this: model_engine.backward(loss) > > if (step + 1) % model_engine.gradient_accumulation_steps() == 0: model_engine.step() model_engine.zero_grad() sorry, remove gradient_accumulation_steps is enough

Fine-tuning best practices for qwen2.5-72b-instruct and qwen2-vl-72b-instruct.

大佬，请教个问题，72B-bf16的理论显存不该是72*2字节，也就是144GB吗，为啥你这里只用了70GB，是上了int4?还是有我不了解的处理机制在里面。