Timothy O'Hear comments

Results 8 comments of


                                            Timothy O'Hear

Perceiver IO: Video Autoencoding notebook - reconstruction of full video doesn't display properly or is broken

Hi, thank you looking into it :-) . The video at the bottom now correctly displays but only shows the first segment (the same 16 frames as above). Changing the...

Out of memory (OOM) ResourceExhaustedError for 1558m model

> It's not RAM, its GPU. > > 1558 or 1.5B model will run only on >=P100 GPU with 16GB VRAM. FYI: running on a Colab Pro P100 I was...

GPT-J evaluation with multiple GPUs crashes

I can reproduce the error with GPT-J. This also happens with Salesforce/codegen-16B-nl and EleutherAI/gpt-neox-20b. In all cases the error is RuntimeError: Tensors must be contiguous. The problem doesn't occur with...

Saving a checkpoint when training with NVMe offloading?

> Yes, we do plan to add checkpointing for NVMe. In reality, you are the first user to my knowledge with this request. Can you please explain a bit your...

How do I continue training with PEFT?

When trying to resume training on LLama2 lora I was running into `RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn` This was solved...

RecursionError: maximum recursion depth exceeded

Forcing the unk_token fixed this for me (v4.30.1): `tokenizer = tokenizer_class.from_pretrained(model_name_or_path, unk_token="")`

Mixtral nf4 performance 2x slower than expected

Note that I'm also seeing this on TGI so it's not a Lorax-specific issue

Quantization takes a very long time

And thank you for eeqt, I've been wishing for high-speed 8-bit inference for quite some time :-)