hangchen426926

**Bug report:** I trained a deep learing model with Horovod KerasEstimator and wanted to optimize it,such as quantization, pruning, etc. Horovod saves the model as metadata by default, but I...

[BUG]: Colossalai-OpenMoE-8b : loss value is very large and cannot converge

4

### 🐛 Describe the bug I am currently running the Colossalai/examples/language/openmoe project with the following experimental setup: datasets: load_dataset("yizhongw/self_instruct/data/finetuning/self_instruct_221203", "super_natural_instructions"), model: openmoe-8b A100_num_GPU：8 Epoch：3 BatchSize：4 LR：0.00001 zero_stage：1 precision：bf16 Boost plugin：ep_zero...