wailord comments

Repositories
Issues
Comments

Results 3 comments of


                                            wailord

[ROOT optimizer] Is there a DDP version of ROOT?

Thanks for your interest! It's not implemented yet, but the distributed logic is similar to [Moonlight](https://arxiv.org/abs/2502.16982).

[ROOT optimizer] only support 2d parameters?

Hi, Similar to Muon, ROOT applies to tensors with ≥2 dimensions, while 1-D parameters (like bias and norm) are still optimzed by AdamW.

[ROOT optimizer] only support 2d parameters?

In the toy train script, we only have 2D and 1D parameters, so the current assertion works fine. For models with higher-dimensional parameters (like CNNs), reshaping is necessary. We'll update...