geraldstanje1

Results 26 comments of geraldstanje1

use tensorrt inference bert, speed slow than onnxruntime

hi, can sentence-transformers e.g. https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 already used with tensorRT-LLM? my goal is to compile a sentence-transformers/all-MiniLM-L6-v2 model without quantization using tensorRT-LLM and serve with triton... are there any docs how...

Every 5-7 days Docker throws the following error and the engine + container all become unresponsive: "running engine: engine linux/virtualization-framework failed to run: running VM: com.docker.virtualization: process terminated unexpectedly: accepting AF_VSOCK connection: AF_VSOCK listener closed"

any update?

error with logs will not be displayed

@sbueringer where to call log.SetLogger(...) - in controller-runtime?

error with logs will not be displayed

@sbueringer do you mean there is an issue in the controller-runtime code in sigs.k8s.io/controller-runtime/pkg?

error with logs will not be displayed

but the error comes from the library - do you see?

[NEED HELP] migration from tensorrt8 to tensorrt10

@lix19937 can you also run tensorrt engine with golang?

When will Gemma 3 be supported?

hi any updated on gemma3 support?

When will Gemma 3 be supported?

hi @karljang and @juney-nvidia any update?

When will Gemma 3 be supported?

@karljang so for that case we need to use triton with vllm backend?

After upgrading to 4.33.0 (160616), now crashes with "process terminated unexpectedly: accepting AF_VSOCK connection: AF_VSOCK listener closed"

any idea?