kserve
Unable to run inference service
I have followed the official documentation and deployed the serverless deployment, but after running the first inference service example. I am unable to get desired result that has shown in the page. Please find attached image here with
I followed this link for running the inference service https://kserve.github.io/website/master/get_started/first_isvc/#5-perform-inference
Classification: Confidential Hi Filippe, Please find attached image.
From: Filippe Spolti @.> Sent: Monday, April 1, 2024 7:40 PM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
Hi @VikasAbhishekhttps://github.com/VikasAbhishek, what the command below returns?
kubectl get isvc
Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2029817867, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUQVDG6GF26CPMYY6I3Y3FTDDAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRZHAYTOOBWG4. You are receiving this because you were mentioned.Message ID: @.@.>>
::DISCLAIMER::
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
Classification: Confidential @.***:/home/ubuntu# kubectl get isvc -n kserve-test NAME URL READY PREV LATEST PREVROLLEDOUTREVISION LATESTREADYREVISION AGE sklearn-iris http://sklearn-iris.kserve-test.svc.cluster.localhttp://sklearn-iris.kserve-test.svc.cluster.local/ True 100 sklearn-iris-predictor-00003 5d6h
From: Filippe Spolti @.> Sent: Tuesday, April 2, 2024 7:07 PM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
there is nothing attached, just pasting the text here should be enough,.
Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2032068195, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUXHNDP4WMWROT6GBW3Y3KYBLAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZSGA3DQMJZGU. You are receiving this because you were mentioned.Message ID: @.@.>>
::DISCLAIMER::
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
@VikasAbhishek Seems like you have not configured a domain or DNS. If you try to infer from outside the cluster it won't work
Classification: Confidential Hi Sivanantham, Can you please guide me how to configure DNS and in which services should I make the changes. If you have any example file for the same, please share with me.
Thank you, Vikas Ghunawat Meena
From: Sivanantham @.> Sent: Wednesday, April 3, 2024 12:34 AM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
@VikasAbhishekhttps://github.com/VikasAbhishek Seems like you have not configured a domain or DNS. If you try to infer from outside the cluster it won't work
Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2032844184, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUQNHV5IZQ5LZM6WCC3Y3L6JTAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZSHA2DIMJYGQ. You are receiving this because you were mentioned.Message ID: @.@.>>
::DISCLAIMER::
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
Here is the docs from knative for configuring domain. If you are just doing it for testing purposes, then you can just use this command below and you can use the kubectl port-forward to access the cluster without the need to configuring DNS.
kubectl patch cm config-domain --patch '{"data":{"example.com":""}}' -n knative-serving
If you want to configure DNS follow this doc
Classification: Confidential @.***:/home/ubuntu# kubectl logs load-testf4ps9-qznqs -n kserve-test Requests [total, rate, throughput] 30000, 500.02, 0.00 Duration [total, attack, wait] 59.999s, 59.998s, 663.663µs Latencies [min, mean, 50, 90, 95, 99, max] 427.534µs, 790.51µs, 748.113µs, 955.089µs, 1.052ms, 1.593ms, 26.421ms Bytes In [total, mean] 0, 0.00 Bytes Out [total, mean] 2460000, 82.00 Success [ratio] 0.00% Status Codes [code:count] 404:30000 Error Set: 404 Not Found
I am getting these logs while running the performance test.
From: Sivanantham @.> Sent: Thursday, April 4, 2024 12:22 PM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
Here is the docs from knative for configuring domainhttps://knative.dev/docs/serving/using-a-custom-domain/. If you are just doing it for testing purposes, then you can just use this command below and you can use the kubectl port-forward to access the cluster without the need to configuring DNS.
kubectl patch cm config-domain --patch '{"data":{"example.com":""}}' -n knative-serving
If you want to configure DNS follow this dochttps://knative.dev/docs/install/yaml-install/serving/install-serving-with-yaml/#configure-dns
Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2036326725, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUUXD2RMECSLURDQVV3Y3T2BBAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZWGMZDMNZSGU. You are receiving this because you were mentioned.Message ID: @.@.>>
::DISCLAIMER::
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
Classification: Confidential @.***:/home/ubuntu# curl -v -H "Host: ${SERVICE_HOSTNAME}" -H "Content-Type: application/json" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/sklearn-iris:predicthttp://$%7bINGRESS_HOST%7d:$%7bINGRESS_PORT%7d/v1/models/sklearn-iris:predict -d @./iris-input.json
- Trying 127.0.0.1:8080...
- TCP_NODELAY set
- Connected to localhost (127.0.0.1) port 8080 (#0)
POST /v1/models/sklearn-iris:predict HTTP/1.1 Host: sklearn-iris.kserve-test.example.com User-Agent: curl/7.68.0 Accept: / Content-Type: application/json Content-Length: 76
- upload completely sent off: 76 out of 76 bytes
- Mark bundle as not supporting multiuse < HTTP/1.1 404 Not Found < date: Thu, 04 Apr 2024 13:15:37 GMT < server: istio-envoy < x-envoy-upstream-service-time: 3 < content-length: 0 <
- Connection #0 to host localhost left intact
I am still getting the same issue.
From: Sivanantham @.> Sent: Thursday, April 4, 2024 12:22 PM To: kserve/website @.> Cc: Vikas Ghunawat Meena @.>; Mention @.> Subject: Re: [kserve/website] Unable to run inference service (Issue #355)
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
Here is the docs from knative for configuring domainhttps://knative.dev/docs/serving/using-a-custom-domain/. If you are just doing it for testing purposes, then you can just use this command below and you can use the kubectl port-forward to access the cluster without the need to configuring DNS.
kubectl patch cm config-domain --patch '{"data":{"example.com":""}}' -n knative-serving
If you want to configure DNS follow this dochttps://knative.dev/docs/install/yaml-install/serving/install-serving-with-yaml/#configure-dns
Reply to this email directly, view it on GitHubhttps://github.com/kserve/website/issues/355#issuecomment-2036326725, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BHL3AUUXD2RMECSLURDQVV3Y3T2BBAVCNFSM6AAAAABFRTVHB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZWGMZDMNZSGU. You are receiving this because you were mentioned.Message ID: @.@.>>
::DISCLAIMER::
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
@VikasAbhishek
- Ensure the KServe inference service (sklearn-iris) is correctly deployed and accessible within the Kubernetes cluster. The command kubectl get isvc -n kserve-test should list the service as READY.
- Ensure the ingress setup and network policies allow external traffic to the service. The persistent 404 errors could indicate network misconfiguration or an incorrect URL.
- lacking a proper DNS configuration could prevent external access. Using
kubectl patch cm config-domain --patch '{"data":{"example.com":""}}' -n knative-servingcan set a temporary domain for testing purposes.
