client icon indicating copy to clipboard operation
client copied to clipboard
verified publisher icon triton-inference-server

feat: Client-side input shape/element validation

Open yinggeh opened this issue 1 year ago • 2 comments

What does the PR do?

Add client input size check to make sure input shape byte size matches input data byte size.

Checklist

  • [x] PR title reflects the change and is of format <commit_type>: <Title>
  • [x] Changes are described in the pull request.
  • [x] Related issues are referenced.
  • [x] Populated github labels field
  • [x] Added test plan and verified test passes.
  • [ ] Verified that the PR passes existing CI.
  • [x] Verified copyright is correct on all changed files.
  • [ ] Added succinct git squash message before merging ref.
  • [ ] All template sections are filled out.
  • [ ] Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type box here and add the label to the github PR.

  • [x] feat

Related PRs:

https://github.com/triton-inference-server/server/pull/7427

Where should the reviewer start?

src/c++/library/common.cc src/python/library/tritonclient/grpc/_infer_input.py src/python/library/tritonclient/http/_infer_input.py

Test plan:

n/a

  • CI Pipeline ID: 17202351

Caveats:

Shared memory byte size checks for string inputs is not implemented.

Background

Stop malformed input request at client side before sending to the server.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

Relates to https://github.com/triton-inference-server/server/issues/7171

yinggeh avatar Jul 09 '24 01:07 yinggeh

There is a known issue with TensorRT (Jira DLIS-6805 ) which causes TRT tests to fail again at client (CI job 102924904). There is no way to know the platform of inference model at the client side. Should we wait until @pskiran1 finish his change first? CC @tanmayv25 @GuanLuo @rmccorm4

yinggeh avatar Jul 26 '24 22:07 yinggeh

There is a known issue with TensorRT (Jira DLIS-6805 ) which causes TRT tests to fail again at client (CI job 102924904). There is no way to know the platform of inference model at the client side. Should we wait until @pskiran1 finish his change first? CC @tanmayv25 @GuanLuo @rmccorm4

@yinggeh, I just merged DLIS-6805 changes, could you please try with the latest code?

pskiran1 avatar Jul 27 '24 16:07 pskiran1