btrude

@tanner-ducharme It was something like 6s /iteration when I did this originally on an RTX 6000 with a batch size of 1 and it takes thousands of steps to achieve...

@mackamann I would recommend telling them that you are doing independent research or similar though I also had professional uses to discuss with them at the time so that was...

@mackamann At a glance, yes, this looks like the right idea but if you have a lot of data you should put the vqvae on a gpu otherwise this could...

@mackamann I recall having to fix a few legitimate bugs in order to get the 5b models training, but just calling `tensor.to("cuda")` should be enough to past that particular issue

@mackamann Removing ddp should be pretty easy and will definitely give you back some memory if you only have 1 GPU. Also, everything should be on the GPU. I find...

Nice, that's very cool that you got it working - everything is a lot easier with 80gb of ram 😅

I never got improved results from finetuning the upsamplers. In 2023 I would recommend some sort of seq2seq model to a different model (soundstream/encodec/etc) instead if you want to upsample...

@albertfgu Thank you for your work on mamba. I was wondering if you could elaborate on the architecture of the S4+mamba block a bit more? I am trying to reproduce...

> > I am using pytorch's FSDP > > what is your `auto_wrap_policy`? if you don't mind sharing it @binxuan This worked for me: ```py from mamba_ssm.modules.mamba_simple import Block from...

> Thanks for the help. Now I've got this error: > > ``` > Found no NVIDIA driver on your system. Please check that you > have an NVIDIA GPU...