gokarbido

Results 5 comments of gokarbido

Which custom model you are using? tflite or yolo tensortrt model On Fri, 19 Jan, 2024, 10:08 pm alec-groff, ***@***.***> wrote: > Any updates on this? I'm having the same...

Have you quantised your model? If not follow this: https://www.tensorflow.org/lite/performance/post_training_quantization

@LIUKAI0815 this issue pops up when your mpi cuda built is set: False, you have to enable gpu aware in openmpi config. for linux: conda install -c conda-forge mpi4py openmpi...

currently running chatrtx with local Llama2-13b and Mistral-7b models in linux/Ubuntu 24.04. system specs: Nvidia RTX 4070ti SUPER 16 GB, 64 GB RAM and Intel® Core™ i7-14700K × 28. I...