flybird11111
flybird11111
May I take a look at the traceback？
Thank you for your concern. We will address this issue as soon as possible.
> Or is it possible to enable it on `HybridParallelPlugin` in a torch-like way (described in the [document](https://colossalai.org/docs/features/gradient_accumulation_with_booster/#gradient-accumulation-on-geminiplugin))? However, unlike `GeminiPlugin`, it seems there is no `enable_gradient_accumulation` for `HybridParallelPlugin`. It's...
> Or is it possible to enable it on `HybridParallelPlugin` in a torch-like way (described in the [document](https://colossalai.org/docs/features/gradient_accumulation_with_booster/#gradient-accumulation-on-geminiplugin))? However, unlike `GeminiPlugin`, it seems there is no `enable_gradient_accumulation` for `HybridParallelPlugin`. It's...
Thank you for your suggestion, it's beneficial for the development of ColossalAI.
您是用的哪个策略啊？用gemini auto策略试试呢？
您的模型是多大的呢？
Thank you, we will fix it soon.
hi，mistral模型已经支持了，可以关注一下这个pr：https://github.com/hpcaitech/ColossalAI/pull/5103
抱歉，shardformer已经支持了mistral，coati还未支持，我跟相关同事反馈一下。