SkanderBS2024

ERROR: Could not find a version that satisfies the requirement triton==2.1.0 (from versions: none) "MAMBA"

4

**Describe the bug** Hello, i tried to build an mage from the docker file 'examples/mamba' and after pulling the image there was an error while installing packages. I tried to...

(Pre training mamba with train.sh) Error : GPT2BPETokenizer : assert args.vocab_file is not None

6

Hello, after pre-processing the dataset with a BPE tokenizer, when i launch the 'train.sh' script for mamba i do get this error. In the script it's mentionned that i have...

Distributed Mamba Training

7

**How to customise the train.sh for a distributed Mamba Training ?** Hello, As i've seen in the megatron modules, there isn't a pre-defined bash script to pre-train a mamba model...

**Is your feature request related to a problem? Please describe.** When fine-tuning for a mamba2 hybrid model we can convert it to a .nemo format but we cannot convert back...

" ValueError: max() arg is an empty sequence " while converting mamba 2 hybrid checkpoint to nemo

2

**Describe the bug** As described in the title, after finishing all of the installs and building nemo and megatron-lm from source, assuming that the model has been trained with megatron-lm....