hihyun
hihyun
https://github.com/hpcaitech/ColossalAI/blob/6e51d296f07c0ad34d7f85cf9a70d4ceee15ede7/colossalai/zero/sharded_optim/low_level_optim.py#L29 There is updates about Zero-DP in Colossal AI. Zero 1 -> Low_level_optimizer Zero 2 -> Low_level_optimizer (partitioned gradient =True) Zero 3 -> Sharded optimizer v2, sharded modelV2