ndeep27

CPU Memory occupied by TF Serving even though serving is on GPU

4

### System information - TensorFlow Serving v2.11 installed from source ### Describe the problem We are running TensorFlow Serving on GPU as a separate process which is interacting with the...

Why TF Serving using one CUDA Compute Stream

1

Trying to understand why TF uses one CUDA compute stream? Is there a metric which shows if ops are waiting to be scheduled on that one compute stream? I want...

CUDA Graphs support for Tensorflow Serving

2

Does TF Serving support CUDA graphs?

XXX failure of TensorRT X.Y when running XXX on GPU XXX

1

## Description Getting below error when running a TensorRT converted model. 2024-07-08 17:23:06.193422: E external/org_tensorflow/tensorflow/compiler/tf2tensorrt/utils/trt_logger.cc:87] DefaultLogger 7: [shapeMachine.cpp::executeContinuation::864] Error Code 7: Internal Error (IShuffleLayer (Unnamed Layer* 190) [Shuffle]: reshaping failed...

Error in TensorRT conversion of e5-large model

1

I am trying to convert the open source e5-large TF model to TensorRT. This is the model - https://huggingface.co/intfloat/multilingual-e5-large. I am getting below error [libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/stubs/stringpiece.cc:50] size too big:...

TRITON with Pytorch CPU only build not working

15

**Description** Triton Server with Pytorch Backend build not working for CPU_ONLY. It is expecting libraries like libcudart.so even though the build was for CPU. Below is how we invoke the...