Hyeongmin Moon

icon indicating an email [email protected]

icon company EstSoft / Perso.ai / VideoTranslator icon location Seoul, Gangnam gu, South Korea icon location AI Engineer, South Korea, POSTECH

Results 3 issues of Hyeongmin Moon

single gpu 6.7b lora CUDA OOM with A6000 48G

4 comment

I am trying to run DeepSpeed-Chat Example with single gpu, Nvidia A6000 48G. I could run all 3 steps well using 1.3b example. But when I run `single_gpu/run_6.7b_lora.sh`, I got...

question
deespeed chat

Fixed typo for --only_optimize_lora

I found a little of typo on some documetations&codes `--only_optimizer_lora` for `--only_optimize_lora`. So I fixed all the files. Thank you.

What is the best way to multi model serving on local storage?

1 comment

I executed Kserve successfully with my custom pytorch model(custom torchserve image) and local storage(PVC). And I am trying to run multi model serving with same above environments, but I found...