Aleksey Smolenchuk comments

Results 88 comments of


                                            Aleksey Smolenchuk

AttributeError: type object 'Dataset' has no attribute 'from_list'

I just released v2 where I rewrote the whole thing from scratch. Give it a shot!

Slow generation speed: around 10 minutes / loading forever on rtx3090 with 64gb ram....

Doesn't seem normal. I'll try to repro this.

Slow generation speed: around 10 minutes / loading forever on rtx3090 with 64gb ram....

I've reworked this thing from scratch! Give it a shot?

LLAMA 13B HF: RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

Just ran into this one as well.

LLAMA 13B HF: RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

https://github.com/huggingface/transformers/pull/21955 is referencing this error. I think you're right -- it's related.

Add --int8-threshold argument

No benchmarks, but I can confirm it's working well

Add --int8-threshold argument

I've been using this PR for a while :) Any reasons not to merge?

Is it possible to do inference (i.e. run generate.py) without a CUDA GPU?

I think you might need deepspeed or accelerate for that.

Is it possible to "unload" the PEFT LoRA weights after mutating the base model with PeftModel.from_pretrained ?

This is now fixed in the multi adapter loading! Closing.

Licensing question regarding linking

Sorry to bring back this old thread, but why are there [COPYING.BSD2](https://github.com/espeak-ng/espeak-ng/blob/master/COPYING.BSD2) and [COPYING.APACHE](https://github.com/espeak-ng/espeak-ng/blob/master/COPYING.APACHE) files in the source?