gptscript
gptscript copied to clipboard
gptscript-ai
Request too large for gpt-4o
Hi
I sent a .gpt script the following message to my mode gpt-4o
"use kubectl to get the pods in the all namespaces if there is a pod name contains the word 'poly' get its container image name and version?"
But I got the following error, I am not sure why I exceed the limit in a small question like this ?? please advice?
Error executing GPT script: 2024/05/29 07:34:45 error, status code: 429, message: Request too large for gpt-4o in organization org-gLnL4YPUmJUw1RonfNCRnuiS on tokens per min (TPM): Limit 30000, Requested 30872. The input or output tokens must be reduced in order to run successfully. Visit https://platform.openai.com/account/rate-limits to learn more.
Hi @wmashal, I have a couple ideas for why this request may have been so many tokens. Part of the token count in the request includes the descriptions of all the tools that you give the LLM the ability to use. Did your script include a lot of tools in it? Also, if you were executing this in the context of a chat (a GPTScript with chat: true in it), the previous chat messages are included in each request, which can increase the token usage as well.
@g-linville I have few build in tools tools and one custom tool using llamaindex to read docs (RAG)
sys.read, sys.write, sys.exec?, custom-tool
Name: custom-tool Description: Get info from test docs Args: question: The question to ask docs.
#!/bin/bash
llamaindex-cli rag --files "EXTRA_COMMANDS/**/*.md" && llamaindex-cli rag --question "${question}"
About chat: true I am not sure I did not configure any explicit anything like this. what the default ? How I can see the prompt we sent , is there any tracing or debug mechanism ?
Thanks
Yeah if you did not explicitly put chat: true in your script, then you don't need to worry about that. The default is false.
That tool's definition is pretty short, so it probably isn't adding very many tokens. Can you run it with gptscript --debug and see if you get any helpful information from that?
@g-linville I hope this debug help
root@k8s-poly-gpt-5cbcfc89d9-cvmzg:/app# gptscript --debug-messages kube.gpt
15:56:56 started [main]
15:56:57 sent [main]
15:56:57 messages [request={"model":"gpt-4o","messages":[{"role":"system","content":"\nYou are task oriented system.\nYou receive input from a user, process the input from the given instructions, and then output the result.\nYour objective is to provide consistent and correct results.\nYou do not need to explain the steps taken, only provide the result to the given instructions.\nYou are referred to as a tool.\nYou don't move to the next step until you have a result.\n\nI have a kubectl available get the pods in the all namespaces if there is a pod name contains the word 'poly' get its container image name and version?"}],"temperature":0,"tools":[{"type":"function","function":{"name":"read","description":"Reads the contents of a file","parameters":{"properties":{"filename":{"description":"The name of the file to read","type":"string"}},"type":"object"}}},{"type":"function","function":{"name":"write","description":"Write the contents to a file","parameters":{"properties":{"content":{"description":"The content to write","type":"string"},"filename":{"description":"The name of the file to write to","type":"string"}},"type":"object"}}},{"type":"function","function":{"name":"exec","description":"Execute a command and get the output of the command","parameters":{"properties":{"command":{"description":"The command to run including all applicable arguments","type":"string"},"directory":{"description":"The directory to use as the current working directory of the command. The current directory \".\" will be used if no argument is passed","type":"string"}},"type":"object"}}}]}]
15:56:57 messages [response={"role":"assistant","content":[{"toolCall":{"index":0,"id":"call_ZJ1Y8LVgLiHO9DKQqsHabBac","function":{"name":"exec","arguments":"{\"command\":\"kubectl get pods --all-namespaces\"}"}}}],"usage":{"promptTokens":262,"completionTokens":21,"totalTokens":283}}]
15:56:57 started [sys.exec(2)] [input={"command":"kubectl get pods --all-namespaces"}]
15:56:57 sent [sys.exec(2)]
15:56:57 messages [request={"command":["sys.exec"],"input":"{\"command\":\"kubectl get pods --all-namespaces\"}"}]
15:56:58 messages [response={"err":null,"output":"NAMESPACE NAME READY STATUS RESTARTS AGE\ndefault not-running-pod-7b66d698b5-qx5l8 0/1 ImagePullBackOff 0 7d3h\nk8s-poly-gpt k8s-poly-gpt-5cbcfc89d9-cvmzg 1/1 Running 0 3h57m\nkube-system coredns-6d4b75cb6d-c7lq7 1/1 Running 1 (7d3h ago) 7d4h\nkube-system coredns-6d4b75cb6d-js6mb 1/1 Running 1 (7d3h ago) 7d4h\nkube-system etcd-kind-control-plane 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kindnet-cl8hc 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kube-apiserver-kind-control-plane 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kube-controller-manager-kind-control-plane 1/1 Running 16 (62m ago) 7d4h\nkube-system kube-proxy-5vhbj 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kube-scheduler-kind-control-plane 1/1 Running 12 (62m ago) 7d4h\nlocal-path-storage local-path-provisioner-9cd9bd544-f9cvp 1/1 Running 2 (7d3h ago) 7d4h\npoly-bot poly-bot-deployment-6d5b56bb48-tlgqn 1/1 Running 0 4d5h\n"}]
15:56:58 ended [sys.exec(2)] [output=NAMESPACE NAME READY STATUS RESTART...]
15:56:58 continue [main]
15:56:58 sent [main]
15:56:58 messages [request={"model":"gpt-4o","messages":[{"role":"system","content":"\nYou are task oriented system.\nYou receive input from a user, process the input from the given instructions, and then output the result.\nYour objective is to provide consistent and correct results.\nYou do not need to explain the steps taken, only provide the result to the given instructions.\nYou are referred to as a tool.\nYou don't move to the next step until you have a result.\n\nI have a kubectl available get the pods in the all namespaces if there is a pod name contains the word 'poly' get its container image name and version?"},{"role":"assistant","content":"","tool_calls":[{"id":"call_ZJ1Y8LVgLiHO9DKQqsHabBac","type":"function","function":{"name":"exec","arguments":"{\"command\":\"kubectl get pods --all-namespaces\"}"}}]},{"role":"tool","content":"NAMESPACE NAME READY STATUS RESTARTS AGE\ndefault not-running-pod-7b66d698b5-qx5l8 0/1 ImagePullBackOff 0 7d3h\nk8s-poly-gpt k8s-poly-gpt-5cbcfc89d9-cvmzg 1/1 Running 0 3h57m\nkube-system coredns-6d4b75cb6d-c7lq7 1/1 Running 1 (7d3h ago) 7d4h\nkube-system coredns-6d4b75cb6d-js6mb 1/1 Running 1 (7d3h ago) 7d4h\nkube-system etcd-kind-control-plane 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kindnet-cl8hc 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kube-apiserver-kind-control-plane 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kube-controller-manager-kind-control-plane 1/1 Running 16 (62m ago) 7d4h\nkube-system kube-proxy-5vhbj 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kube-scheduler-kind-control-plane 1/1 Running 12 (62m ago) 7d4h\nlocal-path-storage local-path-provisioner-9cd9bd544-f9cvp 1/1 Running 2 (7d3h ago) 7d4h\npoly-bot poly-bot-deployment-6d5b56bb48-tlgqn 1/1 Running 0 4d5h\n","name":"exec","tool_call_id":"call_ZJ1Y8LVgLiHO9DKQqsHabBac"}],"temperature":0,"tools":[{"type":"function","function":{"name":"read","description":"Reads the contents of a file","parameters":{"properties":{"filename":{"description":"The name of the file to read","type":"string"}},"type":"object"}}},{"type":"function","function":{"name":"write","description":"Write the contents to a file","parameters":{"properties":{"content":{"description":"The content to write","type":"string"},"filename":{"description":"The name of the file to write to","type":"string"}},"type":"object"}}},{"type":"function","function":{"name":"exec","description":"Execute a command and get the output of the command","parameters":{"properties":{"command":{"description":"The command to run including all applicable arguments","type":"string"},"directory":{"description":"The directory to use as the current working directory of the command. The current directory \".\" will be used if no argument is passed","type":"string"}},"type":"object"}}}]}]
content [1] content | Waiting for model response...
content [1] content | <tool call> exec -> {"command": "kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'"}
content [1] content | <tool call> exec -> {"command": "kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'"}
15:57:01 messages [response={"role":"assistant","content":[{"toolCall":{"index":0,"id":"call_hSj0iZBsFUbpOX538Ko3B7yt","function":{"name":"exec","arguments":"{\"command\": \"kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'\"}"}}},{"toolCall":{"index":1,"id":"call_MFL9d9HEa67LccrzXyEcVUhd","function":{"name":"exec","arguments":"{\"command\": \"kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'\"}"}}}],"usage":{"promptTokens":752,"completionTokens":119,"totalTokens":871}}]
15:57:01 started [sys.exec(3)] [input={"command": "kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'"}]
15:57:01 started [sys.exec(4)] [input={"command": "kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'"}]
15:57:01 sent [sys.exec(3)]
15:57:01 messages [request={"command":["sys.exec"],"input":"{\"command\": \"kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'\"}"}]
15:57:01 sent [sys.exec(4)]
15:57:01 messages [request={"command":["sys.exec"],"input":"{\"command\": \"kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'\"}"}]
15:57:01 messages [response={"err":null,"output":"wmashal/poly-bot-server:v1.0"}]
15:57:01 ended [sys.exec(4)] [output=wmashal/poly-bot-server:v1.0]
15:57:01 messages [response={"err":null,"output":"wmashal/k8s-poly-gpt-server:v1.0.0"}]
15:57:01 ended [sys.exec(3)] [output=wmashal/k8s-poly-gpt-server:v1.0.0]
15:57:01 continue [main]
15:57:02 sent [main]
15:57:02 messages [request={"model":"gpt-4o","messages":[{"role":"system","content":"\nYou are task oriented system.\nYou receive input from a user, process the input from the given instructions, and then output the result.\nYour objective is to provide consistent and correct results.\nYou do not need to explain the steps taken, only provide the result to the given instructions.\nYou are referred to as a tool.\nYou don't move to the next step until you have a result.\n\nI have a kubectl available get the pods in the all namespaces if there is a pod name contains the word 'poly' get its container image name and version?"},{"role":"assistant","content":"","tool_calls":[{"id":"call_ZJ1Y8LVgLiHO9DKQqsHabBac","type":"function","function":{"name":"exec","arguments":"{\"command\":\"kubectl get pods --all-namespaces\"}"}}]},{"role":"tool","content":"NAMESPACE NAME READY STATUS RESTARTS AGE\ndefault not-running-pod-7b66d698b5-qx5l8 0/1 ImagePullBackOff 0 7d3h\nk8s-poly-gpt k8s-poly-gpt-5cbcfc89d9-cvmzg 1/1 Running 0 3h57m\nkube-system coredns-6d4b75cb6d-c7lq7 1/1 Running 1 (7d3h ago) 7d4h\nkube-system coredns-6d4b75cb6d-js6mb 1/1 Running 1 (7d3h ago) 7d4h\nkube-system etcd-kind-control-plane 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kindnet-cl8hc 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kube-apiserver-kind-control-plane 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kube-controller-manager-kind-control-plane 1/1 Running 16 (62m ago) 7d4h\nkube-system kube-proxy-5vhbj 1/1 Running 1 (7d3h ago) 7d4h\nkube-system kube-scheduler-kind-control-plane 1/1 Running 12 (62m ago) 7d4h\nlocal-path-storage local-path-provisioner-9cd9bd544-f9cvp 1/1 Running 2 (7d3h ago) 7d4h\npoly-bot poly-bot-deployment-6d5b56bb48-tlgqn 1/1 Running 0 4d5h\n","name":"exec","tool_call_id":"call_ZJ1Y8LVgLiHO9DKQqsHabBac"},{"role":"assistant","content":"","tool_calls":[{"id":"call_hSj0iZBsFUbpOX538Ko3B7yt","type":"function","function":{"name":"exec","arguments":"{\"command\": \"kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'\"}"}},{"id":"call_MFL9d9HEa67LccrzXyEcVUhd","type":"function","function":{"name":"exec","arguments":"{\"command\": \"kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'\"}"}}]},{"role":"tool","content":"wmashal/k8s-poly-gpt-server:v1.0.0","name":"exec","tool_call_id":"call_hSj0iZBsFUbpOX538Ko3B7yt"},{"role":"tool","content":"wmashal/poly-bot-server:v1.0","name":"exec","tool_call_id":"call_MFL9d9HEa67LccrzXyEcVUhd"}],"temperature":0,"tools":[{"type":"function","function":{"name":"read","description":"Reads the contents of a file","parameters":{"properties":{"filename":{"description":"The name of the file to read","type":"string"}},"type":"object"}}},{"type":"function","function":{"name":"write","description":"Write the contents to a file","parameters":{"properties":{"content":{"description":"The content to write","type":"string"},"filename":{"description":"The name of the file to write to","type":"string"}},"type":"object"}}},{"type":"function","function":{"name":"exec","description":"Execute a command and get the output of the command","parameters":{"properties":{"command":{"description":"The command to run including all applicable arguments","type":"string"},"directory":{"description":"The directory to use as the current working directory of the command. The current directory \".\" will be used if no argument is passed","type":"string"}},"type":"object"}}}]}]
content [1] content | Waiting for model response...
content [1] content | - k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0
content [1] content | - poly-bot-deployment-6d5b56bb48-tlgqn: wmashal/poly-bot-server:v1.0
15:57:03 messages [response={"role":"assistant","content":[{"text":"- k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0\n- poly-bot-deployment-6d5b56bb48-tlgqn: wmashal/poly-bot-server:v1.0"}],"usage":{"promptTokens":912,"completionTokens":65,"totalTokens":977}}]
15:57:03 ended [main] [output=- k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0\n- poly-bot-deployment-6d5b56bb48...]
15:57:03 usage [total=1848] [prompt=1664] [completion=184]
OUTPUT:
- k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0
- poly-bot-deployment-6d5b56bb48-tlgqn: wmashal/poly-bot-server:v1.0
The funny thing when I send the same question using my UI I got the error
Same script in the pod shell where my ui deployed
root@k8s-poly-gpt-5cbcfc89d9-cvmzg:/app# gptscript kube.gpt
16:03:34 started [main]
16:03:35 sent [main]
16:03:35 started [sys.exec(2)] [input={"command":"kubectl get pods --all-namespaces"}]
16:03:35 sent [sys.exec(2)]
16:03:36 ended [sys.exec(2)] [output=NAMESPACE NAME READY STATUS RESTART...]
16:03:36 continue [main]
16:03:36 sent [main]
content [1] content | Waiting for model response...
content [1] content | <tool call> exec -> {"command": "kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'"}
content [1] content | <tool call> exec -> {"command": "kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'"}
16:03:40 started [sys.exec(3)] [input={"command": "kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'"}]
16:03:40 sent [sys.exec(3)]
16:03:40 started [sys.exec(4)] [input={"command": "kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'"}]
16:03:40 sent [sys.exec(4)]
16:03:40 ended [sys.exec(4)] [output=wmashal/k8s-poly-gpt-server:v1.0.0]
16:03:40 ended [sys.exec(3)] [output=wmashal/poly-bot-server:v1.0]
16:03:40 continue [main]
16:03:41 sent [main]
content [1] content | Waiting for model response...
content [1] content | - k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0
content [1] content | - poly-bot-deployment-6d5b56bb48-tlgqn: wmashal/poly-bot-server:v1.0
16:03:43 ended [main] [output=- k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0\n- poly-bot-deployment-6d5b56bb48...]
16:03:43 usage [total=1848] [prompt=1664] [completion=184]
OUTPUT:
- k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0
- poly-bot-deployment-6d5b56bb48-tlgqn: wmashal/poly-bot-server:v1.0
gptscript --debug kube.gpt Result
root@k8s-poly-gpt-5cbcfc89d9-cvmzg:/app# gptscript --debug kube.gpt
{
"level": "debug",
"logger": "-ai/gptscript/pkg/loader",
"msg": "opened kube.gpt",
"time": "2024-05-29T16:09:33Z"
}
{
"level": "debug",
"logger": "-ai/gptscript/pkg/loader",
"msg": "loaded program kube.gpt source took 14.76275ms",
"time": "2024-05-29T16:09:33Z"
}
{
"input": "",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "Run started",
"program": {
"name": "kube.gpt",
"entryToolId": "kube.gpt:",
"toolSet": {
"kube.gpt:": {
"modelName": "gpt-4o",
"internalPrompt": null,
"tools": [
"sys.read",
"sys.write",
"sys.exec?"
],
"instructions": "I have a kubectl available get the pods in the all namespaces if there is a pod name contains the word 'poly' get its container image name and version?",
"id": "kube.gpt:",
"toolMapping": {
"sys.exec?": "sys.exec",
"sys.read": "sys.read",
"sys.write": "sys.write"
},
"localTools": {
"": "kube.gpt:"
},
"source": {
"location": "kube.gpt",
"lineNo": 1
},
"workingDir": "."
},
"sys.exec": {
"name": "sys.exec",
"description": "Execute a command and get the output of the command",
"modelName": "gpt-4o",
"internalPrompt": null,
"arguments": {
"properties": {
"command": {
"description": "The command to run including all applicable arguments",
"type": "string"
},
"directory": {
"description": "The directory to use as the current working directory of the command. The current directory \".\" will be used if no argument is passed",
"type": "string"
}
},
"type": "object"
},
"instructions": "#!sys.exec",
"id": "sys.exec",
"source": {}
},
"sys.read": {
"name": "sys.read",
"description": "Reads the contents of a file",
"modelName": "gpt-4o",
"internalPrompt": null,
"arguments": {
"properties": {
"filename": {
"description": "The name of the file to read",
"type": "string"
}
},
"type": "object"
},
"instructions": "#!sys.read",
"id": "sys.read",
"source": {}
},
"sys.write": {
"name": "sys.write",
"description": "Write the contents to a file",
"modelName": "gpt-4o",
"internalPrompt": null,
"arguments": {
"properties": {
"content": {
"description": "The content to write",
"type": "string"
},
"filename": {
"description": "The name of the file to write to",
"type": "string"
}
},
"type": "object"
},
"instructions": "#!sys.write",
"id": "sys.write",
"source": {}
}
}
},
"runID": "1716998974",
"time": "2024-05-29T16:09:33Z",
"type": "runStart"
}
{
"id": "1716998975",
"input": "",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "started [main]",
"parentID": "",
"time": "2024-05-29T16:09:33Z",
"toolID": "kube.gpt:",
"type": "callStart"
}
{
"id": "1716998975",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "sent [main]",
"parentID": "",
"time": "2024-05-29T16:09:34Z",
"toolID": "kube.gpt:",
"type": "callChat"
}
{
"completionID": "1716998976",
"id": "1716998975",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "",
"request": {
"model": "gpt-4o",
"messages": [
{
"role": "system",
"content": "\nYou are task oriented system.\nYou receive input from a user, process the input from the given instructions, and then output the result.\nYour objective is to provide consistent and correct results.\nYou do not need to explain the steps taken, only provide the result to the given instructions.\nYou are referred to as a tool.\nYou don't move to the next step until you have a result.\n\nI have a kubectl available get the pods in the all namespaces if there is a pod name contains the word 'poly' get its container image name and version?"
}
],
"temperature": 0,
"tools": [
{
"type": "function",
"function": {
"name": "read",
"description": "Reads the contents of a file",
"parameters": {
"properties": {
"filename": {
"description": "The name of the file to read",
"type": "string"
}
},
"type": "object"
}
}
},
{
"type": "function",
"function": {
"name": "write",
"description": "Write the contents to a file",
"parameters": {
"properties": {
"content": {
"description": "The content to write",
"type": "string"
},
"filename": {
"description": "The name of the file to write to",
"type": "string"
}
},
"type": "object"
}
}
},
{
"type": "function",
"function": {
"name": "exec",
"description": "Execute a command and get the output of the command",
"parameters": {
"properties": {
"command": {
"description": "The command to run including all applicable arguments",
"type": "string"
},
"directory": {
"description": "The directory to use as the current working directory of the command. The current directory \".\" will be used if no argument is passed",
"type": "string"
}
},
"type": "object"
}
}
}
]
},
"time": "2024-05-29T16:09:34Z",
"toolID": "kube.gpt:",
"type": "callChat"
}
{
"cached": true,
"completionID": "1716998976",
"id": "1716998975",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "",
"response": {
"role": "assistant",
"content": [
{
"toolCall": {
"index": 0,
"id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac",
"function": {
"name": "exec",
"arguments": "{\"command\":\"kubectl get pods --all-namespaces\"}"
}
}
}
],
"usage": {
"promptTokens": 262,
"completionTokens": 21,
"totalTokens": 283
}
},
"time": "2024-05-29T16:09:34Z",
"toolID": "kube.gpt:",
"type": "callChat"
}
{
"id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac",
"input": "{\"command\":\"kubectl get pods --all-namespaces\"}",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "started [sys.exec(2)]",
"parentID": "1716998975",
"time": "2024-05-29T16:09:34Z",
"toolID": "sys.exec",
"type": "callStart"
}
{
"level": "debug",
"logger": "-ai/gptscript/pkg/builtin",
"msg": "Running kubectl get pods --all-namespaces in .",
"time": "2024-05-29T16:09:34Z"
}
{
"id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "sent [sys.exec(2)]",
"parentID": "1716998975",
"time": "2024-05-29T16:09:34Z",
"toolID": "sys.exec",
"type": "callChat"
}
{
"completionID": "1716998977",
"id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "1716998975",
"request": {
"command": [
"sys.exec"
],
"input": "{\"command\":\"kubectl get pods --all-namespaces\"}"
},
"time": "2024-05-29T16:09:34Z",
"toolID": "sys.exec",
"type": "callChat"
}
{
"cached": false,
"completionID": "1716998977",
"id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "1716998975",
"response": {
"err": null,
"output": "NAMESPACE NAME READY STATUS RESTARTS AGE\ndefault not-running-pod-7b66d698b5-qx5l8 0/1 ImagePullBackOff 0 7d3h\nk8s-poly-gpt k8s-poly-gpt-5cbcfc89d9-cvmzg 1/1 Running 0 4h9m\nkube-system coredns-6d4b75cb6d-c7lq7 1/1 Running 1 (7d4h ago) 7d4h\nkube-system coredns-6d4b75cb6d-js6mb 1/1 Running 1 (7d4h ago) 7d4h\nkube-system etcd-kind-control-plane 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kindnet-cl8hc 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-apiserver-kind-control-plane 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-controller-manager-kind-control-plane 1/1 Running 16 (75m ago) 7d4h\nkube-system kube-proxy-5vhbj 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-scheduler-kind-control-plane 1/1 Running 12 (75m ago) 7d4h\nlocal-path-storage local-path-provisioner-9cd9bd544-f9cvp 1/1 Running 2 (7d4h ago) 7d4h\npoly-bot poly-bot-deployment-6d5b56bb48-tlgqn 1/1 Running 0 4d5h\n"
},
"time": "2024-05-29T16:09:34Z",
"toolID": "sys.exec",
"type": "callChat"
}
{
"id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "ended [sys.exec(2)]",
"output": "NAMESPACE NAME READY STATUS RESTARTS AGE\ndefault not-running-pod-7b66d698b5-qx5l8 0/1 ImagePullBackOff 0 7d3h\nk8s-poly-gpt k8s-poly-gpt-5cbcfc89d9-cvmzg 1/1 Running 0 4h9m\nkube-system coredns-6d4b75cb6d-c7lq7 1/1 Running 1 (7d4h ago) 7d4h\nkube-system coredns-6d4b75cb6d-js6mb 1/1 Running 1 (7d4h ago) 7d4h\nkube-system etcd-kind-control-plane 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kindnet-cl8hc 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-apiserver-kind-control-plane 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-controller-manager-kind-control-plane 1/1 Running 16 (75m ago) 7d4h\nkube-system kube-proxy-5vhbj 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-scheduler-kind-control-plane 1/1 Running 12 (75m ago) 7d4h\nlocal-path-storage local-path-provisioner-9cd9bd544-f9cvp 1/1 Running 2 (7d4h ago) 7d4h\npoly-bot poly-bot-deployment-6d5b56bb48-tlgqn 1/1 Running 0 4d5h\n",
"parentID": "1716998975",
"time": "2024-05-29T16:09:34Z",
"toolID": "sys.exec",
"type": "callFinish"
}
{
"id": "1716998975",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "continue [main]",
"parentID": "",
"time": "2024-05-29T16:09:34Z",
"toolID": "kube.gpt:",
"toolResults": 1,
"type": "callContinue"
}
{
"id": "1716998975",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "sent [main]",
"parentID": "",
"time": "2024-05-29T16:09:34Z",
"toolID": "kube.gpt:",
"type": "callChat"
}
{
"completionID": "1716998978",
"id": "1716998975",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "",
"request": {
"model": "gpt-4o",
"messages": [
{
"role": "system",
"content": "\nYou are task oriented system.\nYou receive input from a user, process the input from the given instructions, and then output the result.\nYour objective is to provide consistent and correct results.\nYou do not need to explain the steps taken, only provide the result to the given instructions.\nYou are referred to as a tool.\nYou don't move to the next step until you have a result.\n\nI have a kubectl available get the pods in the all namespaces if there is a pod name contains the word 'poly' get its container image name and version?"
},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac",
"type": "function",
"function": {
"name": "exec",
"arguments": "{\"command\":\"kubectl get pods --all-namespaces\"}"
}
}
]
},
{
"role": "tool",
"content": "NAMESPACE NAME READY STATUS RESTARTS AGE\ndefault not-running-pod-7b66d698b5-qx5l8 0/1 ImagePullBackOff 0 7d3h\nk8s-poly-gpt k8s-poly-gpt-5cbcfc89d9-cvmzg 1/1 Running 0 4h9m\nkube-system coredns-6d4b75cb6d-c7lq7 1/1 Running 1 (7d4h ago) 7d4h\nkube-system coredns-6d4b75cb6d-js6mb 1/1 Running 1 (7d4h ago) 7d4h\nkube-system etcd-kind-control-plane 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kindnet-cl8hc 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-apiserver-kind-control-plane 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-controller-manager-kind-control-plane 1/1 Running 16 (75m ago) 7d4h\nkube-system kube-proxy-5vhbj 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-scheduler-kind-control-plane 1/1 Running 12 (75m ago) 7d4h\nlocal-path-storage local-path-provisioner-9cd9bd544-f9cvp 1/1 Running 2 (7d4h ago) 7d4h\npoly-bot poly-bot-deployment-6d5b56bb48-tlgqn 1/1 Running 0 4d5h\n",
"name": "exec",
"tool_call_id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac"
}
],
"temperature": 0,
"tools": [
{
"type": "function",
"function": {
"name": "read",
"description": "Reads the contents of a file",
"parameters": {
"properties": {
"filename": {
"description": "The name of the file to read",
"type": "string"
}
},
"type": "object"
}
}
},
{
"type": "function",
"function": {
"name": "write",
"description": "Write the contents to a file",
"parameters": {
"properties": {
"content": {
"description": "The content to write",
"type": "string"
},
"filename": {
"description": "The name of the file to write to",
"type": "string"
}
},
"type": "object"
}
}
},
{
"type": "function",
"function": {
"name": "exec",
"description": "Execute a command and get the output of the command",
"parameters": {
"properties": {
"command": {
"description": "The command to run including all applicable arguments",
"type": "string"
},
"directory": {
"description": "The directory to use as the current working directory of the command. The current directory \".\" will be used if no argument is passed",
"type": "string"
}
},
"type": "object"
}
}
}
]
},
"time": "2024-05-29T16:09:34Z",
"toolID": "kube.gpt:",
"type": "callChat"
}
{
"id": "call_cHKeiyBhgnUdSZGjABrZwLfa",
"input": "{\"command\": \"kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'\"}",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "started [sys.exec(3)]",
"parentID": "1716998975",
"time": "2024-05-29T16:09:34Z",
"toolID": "sys.exec",
"type": "callStart"
}
{
"id": "call_cHKeiyBhgnUdSZGjABrZwLfa",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "sent [sys.exec(3)]",
"parentID": "1716998975",
"time": "2024-05-29T16:09:34Z",
"toolID": "sys.exec",
"type": "callChat"
}
{
"completionID": "1716998979",
"id": "call_cHKeiyBhgnUdSZGjABrZwLfa",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "1716998975",
"request": {
"command": [
"sys.exec"
],
"input": "{\"command\": \"kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'\"}"
},
"time": "2024-05-29T16:09:34Z",
"toolID": "sys.exec",
"type": "callChat"
}
{
"level": "debug",
"logger": "-ai/gptscript/pkg/builtin",
"msg": "Running kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}' in .",
"time": "2024-05-29T16:09:34Z"
}
{
"cached": true,
"completionID": "1716998978",
"id": "1716998975",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "",
"response": {
"role": "assistant",
"content": [
{
"toolCall": {
"index": 0,
"id": "call_cHKeiyBhgnUdSZGjABrZwLfa",
"function": {
"name": "exec",
"arguments": "{\"command\": \"kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'\"}"
}
}
},
{
"toolCall": {
"index": 1,
"id": "call_QM3m8Cboz4PSIpcpq3jGVHrY",
"function": {
"name": "exec",
"arguments": "{\"command\": \"kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'\"}"
}
}
}
],
"usage": {
"promptTokens": 752,
"completionTokens": 119,
"totalTokens": 871
}
},
"time": "2024-05-29T16:09:34Z",
"toolID": "kube.gpt:",
"type": "callChat"
}
{
"id": "call_QM3m8Cboz4PSIpcpq3jGVHrY",
"input": "{\"command\": \"kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'\"}",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "started [sys.exec(4)]",
"parentID": "1716998975",
"time": "2024-05-29T16:09:35Z",
"toolID": "sys.exec",
"type": "callStart"
}
{
"level": "debug",
"logger": "-ai/gptscript/pkg/builtin",
"msg": "Running kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}' in .",
"time": "2024-05-29T16:09:35Z"
}
{
"id": "call_QM3m8Cboz4PSIpcpq3jGVHrY",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "sent [sys.exec(4)]",
"parentID": "1716998975",
"time": "2024-05-29T16:09:35Z",
"toolID": "sys.exec",
"type": "callChat"
}
{
"completionID": "1716998980",
"id": "call_QM3m8Cboz4PSIpcpq3jGVHrY",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "1716998975",
"request": {
"command": [
"sys.exec"
],
"input": "{\"command\": \"kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'\"}"
},
"time": "2024-05-29T16:09:35Z",
"toolID": "sys.exec",
"type": "callChat"
}
{
"cached": false,
"completionID": "1716998979",
"id": "call_cHKeiyBhgnUdSZGjABrZwLfa",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "1716998975",
"response": {
"err": null,
"output": "wmashal/k8s-poly-gpt-server:v1.0.0"
},
"time": "2024-05-29T16:09:35Z",
"toolID": "sys.exec",
"type": "callChat"
}
{
"id": "call_cHKeiyBhgnUdSZGjABrZwLfa",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "ended [sys.exec(3)]",
"output": "wmashal/k8s-poly-gpt-server:v1.0.0",
"parentID": "1716998975",
"time": "2024-05-29T16:09:35Z",
"toolID": "sys.exec",
"type": "callFinish"
}
{
"cached": false,
"completionID": "1716998980",
"id": "call_QM3m8Cboz4PSIpcpq3jGVHrY",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "1716998975",
"response": {
"err": null,
"output": "wmashal/poly-bot-server:v1.0"
},
"time": "2024-05-29T16:09:35Z",
"toolID": "sys.exec",
"type": "callChat"
}
{
"id": "call_QM3m8Cboz4PSIpcpq3jGVHrY",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "ended [sys.exec(4)]",
"output": "wmashal/poly-bot-server:v1.0",
"parentID": "1716998975",
"time": "2024-05-29T16:09:35Z",
"toolID": "sys.exec",
"type": "callFinish"
}
{
"id": "1716998975",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "continue [main]",
"parentID": "",
"time": "2024-05-29T16:09:35Z",
"toolID": "kube.gpt:",
"toolResults": 2,
"type": "callContinue"
}
{
"id": "1716998975",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "sent [main]",
"parentID": "",
"time": "2024-05-29T16:09:35Z",
"toolID": "kube.gpt:",
"type": "callChat"
}
{
"completionID": "1716998981",
"id": "1716998975",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "",
"request": {
"model": "gpt-4o",
"messages": [
{
"role": "system",
"content": "\nYou are task oriented system.\nYou receive input from a user, process the input from the given instructions, and then output the result.\nYour objective is to provide consistent and correct results.\nYou do not need to explain the steps taken, only provide the result to the given instructions.\nYou are referred to as a tool.\nYou don't move to the next step until you have a result.\n\nI have a kubectl available get the pods in the all namespaces if there is a pod name contains the word 'poly' get its container image name and version?"
},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac",
"type": "function",
"function": {
"name": "exec",
"arguments": "{\"command\":\"kubectl get pods --all-namespaces\"}"
}
}
]
},
{
"role": "tool",
"content": "NAMESPACE NAME READY STATUS RESTARTS AGE\ndefault not-running-pod-7b66d698b5-qx5l8 0/1 ImagePullBackOff 0 7d3h\nk8s-poly-gpt k8s-poly-gpt-5cbcfc89d9-cvmzg 1/1 Running 0 4h9m\nkube-system coredns-6d4b75cb6d-c7lq7 1/1 Running 1 (7d4h ago) 7d4h\nkube-system coredns-6d4b75cb6d-js6mb 1/1 Running 1 (7d4h ago) 7d4h\nkube-system etcd-kind-control-plane 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kindnet-cl8hc 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-apiserver-kind-control-plane 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-controller-manager-kind-control-plane 1/1 Running 16 (75m ago) 7d4h\nkube-system kube-proxy-5vhbj 1/1 Running 1 (7d4h ago) 7d4h\nkube-system kube-scheduler-kind-control-plane 1/1 Running 12 (75m ago) 7d4h\nlocal-path-storage local-path-provisioner-9cd9bd544-f9cvp 1/1 Running 2 (7d4h ago) 7d4h\npoly-bot poly-bot-deployment-6d5b56bb48-tlgqn 1/1 Running 0 4d5h\n",
"name": "exec",
"tool_call_id": "call_ZJ1Y8LVgLiHO9DKQqsHabBac"
},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_cHKeiyBhgnUdSZGjABrZwLfa",
"type": "function",
"function": {
"name": "exec",
"arguments": "{\"command\": \"kubectl get pod k8s-poly-gpt-5cbcfc89d9-cvmzg -n k8s-poly-gpt -o jsonpath='{.spec.containers[*].image}'\"}"
}
},
{
"id": "call_QM3m8Cboz4PSIpcpq3jGVHrY",
"type": "function",
"function": {
"name": "exec",
"arguments": "{\"command\": \"kubectl get pod poly-bot-deployment-6d5b56bb48-tlgqn -n poly-bot -o jsonpath='{.spec.containers[*].image}'\"}"
}
}
]
},
{
"role": "tool",
"content": "wmashal/k8s-poly-gpt-server:v1.0.0",
"name": "exec",
"tool_call_id": "call_cHKeiyBhgnUdSZGjABrZwLfa"
},
{
"role": "tool",
"content": "wmashal/poly-bot-server:v1.0",
"name": "exec",
"tool_call_id": "call_QM3m8Cboz4PSIpcpq3jGVHrY"
}
],
"temperature": 0,
"tools": [
{
"type": "function",
"function": {
"name": "read",
"description": "Reads the contents of a file",
"parameters": {
"properties": {
"filename": {
"description": "The name of the file to read",
"type": "string"
}
},
"type": "object"
}
}
},
{
"type": "function",
"function": {
"name": "write",
"description": "Write the contents to a file",
"parameters": {
"properties": {
"content": {
"description": "The content to write",
"type": "string"
},
"filename": {
"description": "The name of the file to write to",
"type": "string"
}
},
"type": "object"
}
}
},
{
"type": "function",
"function": {
"name": "exec",
"description": "Execute a command and get the output of the command",
"parameters": {
"properties": {
"command": {
"description": "The command to run including all applicable arguments",
"type": "string"
},
"directory": {
"description": "The directory to use as the current working directory of the command. The current directory \".\" will be used if no argument is passed",
"type": "string"
}
},
"type": "object"
}
}
}
]
},
"time": "2024-05-29T16:09:35Z",
"toolID": "kube.gpt:",
"type": "callChat"
}
{
"cached": true,
"completionID": "1716998981",
"id": "1716998975",
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "debug",
"parentID": "",
"response": {
"role": "assistant",
"content": [
{
"text": "- k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0\n- poly-bot-deployment-6d5b56bb48-tlgqn: wmashal/poly-bot-server:v1.0"
}
],
"usage": {
"promptTokens": 912,
"completionTokens": 65,
"totalTokens": 977
}
},
"time": "2024-05-29T16:09:35Z",
"toolID": "kube.gpt:",
"type": "callChat"
}
{
"id": "1716998975",
"level": "info",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "ended [main]",
"output": "- k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0\n- poly-bot-deployment-6d5b56bb48-tlgqn: wmashal/poly-bot-server:v1.0",
"parentID": "",
"time": "2024-05-29T16:09:35Z",
"toolID": "kube.gpt:",
"type": "callFinish"
}
{
"err": null,
"level": "debug",
"logger": "-ai/gptscript/pkg/monitor",
"msg": "Run stopped",
"output": "- k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0\n- poly-bot-deployment-6d5b56bb48-tlgqn: wmashal/poly-bot-server:v1.0",
"runID": "1716998974",
"time": "2024-05-29T16:09:35Z",
"type": "runFinish"
}
OUTPUT:
- k8s-poly-gpt-5cbcfc89d9-cvmzg: wmashal/k8s-poly-gpt-server:v1.0.0
- poly-bot-deployment-6d5b56bb48-tlgqn: wmashal/poly-bot-server:v1.0
Interesting, thanks for the details. So it looks like it runs successfully when you run the script directly, but fails when you run it through your UI? Does your UI introduce additional tool calls perhaps?
Also, is it possible for you to get your rate limit changed? I believe gpt-4o can handle a context window of 128k tokens, so I'm surprised that 30k tokens per minute would be your rate limit.
My code execute the .gpt file in a simple way, but dynamic
https://github.com/wmashal/k8s-poly-gpt/blob/c551a101719d647a2b47f5ae891a71c7878bd205/server/app.py#L48
Yeah, as long as the script that it runs there is the same as what you are running with the CLI, then there shouldn't be any difference. Did you look into why your rate limit may be so low?
This Funny,
For testing I charge 10$ , so I am in Tier 1 with 30000 TPM, to have bigger I need Tier 2 | $50 paid and 7+ days since first successful payment
