Automodel

Automodel copied to clipboard

ci: Update transformers upperbound to v5 rc0

2

thomasdhc

**Is your feature request related to a problem? Please describe.** A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] **Describe the solution you'd...

hemildesai

Hi Team, Do you have any plans to support DoRA? Paper: https://arxiv.org/abs/2402.09353 peft: https://github.com/huggingface/peft/pull/1474 Thank you

SJmdy

feat: Add TE rope fusion for custom MoE models

3

Can give up to 10% speedup Closes #606 Qwen - https://wandb.ai/Nemo-automodel/qwen3_moe-cp-pp-grid Moonlight - https://wandb.ai/Nemo-automodel/moonlight-automodel-cp-pp-grid

hemildesai

feat: vlm multiturn chat support and dataset

6

HuiyingLi

ci: lowerbound liger to v0.6.4

3

Lowerbounding Liger-Kernel to ensure support for TiledMLP is added https://github.com/linkedin/Liger-Kernel/pull/935 TODO: confirm convergence with FSDP2.

akoumpa

**Is your feature request related to a problem? Please describe.** leverage compiler techniques to support automatic model sharding. **Describe the solution you'd like** A clear and concise description of what...

akoumpa

**Is your feature request related to a problem? Please describe.** In multi-hop QA, a query would have multiple positive docs. We should use all of them for biencoder training. Currently...

shan-nvidia

microsoft/Phi-3-mini-4k-instruct fails with exceptions

2

**Describe the bug** Phi-3-mini-4k-instruct does not run with Automodel. I've tried two cases: 1. using `nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained` as the model `_target_` in the config. In this case I get the error...

torsli

Related to https://github.com/NVIDIA-NeMo/Automodel/pull/798, while attempting to run the `examples/llm_pretrain/pretrain.py` example with the `examples/llm_pretrain/megatron_pretrain_gpt2.yaml` configuration file, I ran into a compilation error when building C++ helpers. I was running on Databricks,...

jrbourbeau