devika
devika copied to clipboard
stitionai
GPT 429 request to large
Describe the bug Devika was able to make a ~ 100 requests spaced over an hour or so to GTP4 and 3.5. Then I started to get this error:
Error code: 429 - {'error': {'message': 'Request too large for gpt-4-turbo-preview in organization org-... on tokens per min (TPM): Limit 300000, Requested 469373. The input or output tokens must be reduced in order to run successfully. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
To Reproduce Steps to reproduce the behavior:
- Continue to ask Devika to update features on the app in progress
Expected behavior Not to run into rate limit issues with GPT
Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: Mac 11.7.10 (Big Sur)
- Browser Firefox
Additional context
- Plenty of tokens left in my openai api budget.
- restarting the servers didn't help.
- same issue even after creating a new project
It seems it's sending all code to GPT instead of only relevent segments..
24.04.04 20:25:51: root: INFO : SOCKET server-message MESSAGE: {'messages': {'from_devika': False, 'message': 'hello', 'timestamp': '2024-04-04 20:25:51'}}
24.04.04 20:25:51: root: INFO : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 1570, 'timestamp': '2024-04-04 17:01:20'}, {'internal_monologue': "Okay, first things first, I need to ensure the user has all the preliminary setup done. Meson and SDL2 can be tricky newbies. Hope they find installing SDL2 straightforward. Then, onto the fun part of coding! It’s key to make the instructions clear and digestible. Hmm, I'll need to double-check the Meson setup steps to keep things smooth.", 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 2586, 'timestamp': '2024-04-04 17:01:45'}, {'internal_monologue': 'Writing code...', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': 'vim meson.build', 'output': "project('SDLStarterKit', 'cpp', version : '0.1.0')\n\nsdl2_dep = dependency('sdl2')\n\nexecutable('sdlstarterkit', 'src/main.cpp', dependencies : sdl2_dep)\n", 'title': 'Editing meson.build'}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 0, 'timestamp': '2024-04-04 17:02:21'}, {'internal_monologue': 'Writing code...', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': 'vim src/main.cpp', 'output': '#include <SDL.h>\n#include <iostream>\n\nint main() {\n if (SDL_Init(SDL_INIT_VIDEO) != 0) {\n std::cout << "SDL_Init Error: " << SDL_GetError() << std::endl;\n return 1;\n }\n\n SDL_Window *window = SDL_CreateWindow("SDL Starter Kit", 100, 100, 640, 480, SDL_WINDOW_SHOWN);\n if (window == nullptr) {\n std::cout << "SDL_CreateWindow Error: " << SDL_GetError() << std::endl;\n SDL_Quit();\n return 1;\n }\n\n SDL_Renderer *renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC);\n if (renderer == nullptr) {\n SDL_DestroyWindow(window);\n std::cout << "SDL_CreateRenderer Error: " << SDL_GetError() << std::endl;\n SDL_Quit();\n return 1;\n }\n\n SDL_SetRenderDrawColor(renderer, 255, 0, 0, 255);\n\n bool running = true;\n SDL_Event event;\n while (running) {\n while (SDL_PollEvent(&event)) {\n if (event.type == SDL_QUIT) {\n running = false;\n }\n }\n SDL_RenderClear(renderer);\n SDL_RenderPresent(renderer);\n }\n\n SDL_DestroyRenderer(renderer);\n SDL_DestroyWindow(window);\n SDL_Quit();\n\n return 0;\n}\n', 'title': 'Editing src/main.cpp'}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 0, 'timestamp': '2024-04-04 17:02:23'}, {'internal_monologue': 'Agent has completed the task.', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': 'vim README.md', 'output': '# SDL Starter Kit\n\nThis project is a simple SDL GUI application developed in C++ and built with Meson. It demonstrates creating a window and handling basic event loop.\n\n## Prerequisites\n\nMake sure you have the following installed:\n- A C++ compiler (GCC or Clang)\n- Meson build system\n- Ninja build system\n- SDL2 library\n\nOn Debian/Ubuntu systems, you can install them with:\nsudo apt-get install g++ meson ninja-build libsdl2-dev\n\n## Building and Running\n\nTo compile the project, navigate to the project directory and run:\nmeson setup builddir\ncd builddir\nninja\n\nTo run the compiled application:\n./sdlstarterkit', 'title': 'Editing README.md'}, 'step': None, 'message': None, 'completed': True, 'agent_is_active': True, 'token_usage': 11124, 'timestamp': '2024-04-04 17:02:25'}, {'internal_monologue': 'Agent has completed the task.', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': 'vim src/main.cpp', 'output': '#include <SDL.h>\n#include <iostream>\n\nint main() {\n if (SDL_Init(SDL_INIT_VIDEO) != 0) {\n std::cout << "SDL_Init Error: " << SDL_GetError() << std::endl;\n return 1;\n }\n\n SDL_Window *window = SDL_CreateWindow("SDL Starter Kit", 100, 100, 640, 480, SDL_WINDOW_SHOWN);\n if (window == nullptr) {\n std::cout << "SDL_CreateWindow Error: " << SDL_GetError() << std::endl;\n SDL_Quit();\n return 1;\n }\n\n SDL_Renderer *renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC);\n if (renderer == nullptr) {\n SDL_DestroyWindow(window);\n std::cout << "SDL_CreateRenderer Error: " << SDL_GetError() << std::endl;\n SDL_Quit();\n return 1;\n }\n\n // Set the background color to dark gray.\n SDL_SetRenderDrawColor(renderer, 169, 169, 169, 255); // Dark Gray color\n\n bool running = true;\n SDL_Event event;\n while (running) {\n while (SDL_PollEvent(&event)) {\n if (event.type == SDL_QUIT) {\n running = false;\n } else if (event.type == SDL_KEYDOWN) {\n // Handling key press event\n std::cout << "Key Pressed: " << SDL_GetKeyName(event.key.keysym.sym) << std::endl;\n }\n }\n SDL_RenderClear(renderer);\n SDL_RenderPresent(renderer);\n }\n\n SDL_DestroyRenderer(renderer);\n SDL_DestroyWindow(window);\n SDL_Quit();\n\n return 0;\n}', 'title': 'Editing src/main.cpp'}, 'step': None, 'message': None, 'completed': True, 'agent_is_active': True, 'token_usage': 11891, 'timestamp': '2024-04-04 17:04:21'}, {'internal_monologue': 'Agent has completed the task.', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': 'vim src/main.cpp', 'output': '#include <SDL.h>\n#include <iostream>\n#include <string>\n\n// Function to initialize TTF and create a texture from a text\nSDL_Texture* renderText(SDL_Renderer* renderer, const std::string& fontPath, int fontSize, const std::string& message, SDL_Color color) {\n TTF_Font* font = TTF_OpenFont(fontPath.c_str(), fontSize);\n if (font == nullptr) {\n std::cerr << "TTF_OpenFont Error: " << TTF_GetError() << std::endl;\n return nullptr;\n }\n\n SDL_Surface* surface = TTF_RenderText_Solid(font, message.c_str(), color);\n if (surface == nullptr) {\n TTF_CloseFont(font);\n std::cerr << "TTF_RenderText_Solid Error: " << TTF_GetError() << std::endl;\n return nullptr;\n }\n\n SDL_Texture* texture = SDL_CreateTextureFromSurface(renderer, surface);\n if (texture == nullptr) {\n std::cerr << "SDL_CreateTextureFromSurface Error: " << SDL_GetError() << std::endl;\n }\n\n SDL_FreeSurface(surface);\n TTF_CloseFont(font);\n\n return texture;\n}\n\nint main() {\n if (SDL_Init(SDL_INIT_VIDEO) != 0) {\n std::cerr << "SDL_Init Error: " << SDL_GetError() << std::endl;\n return 1;\n }\n\n if (TTF_Init() == -1) {\n std::cerr << "TTF_Init Error: " << TTF_GetError() << std::endl;\n SDL_Quit();\n return 1;\n }\n\n SDL_Window *window = SDL_CreateWindow("SDL Starter Kit", 100, 100, 640, 480, SDL_WINDOW_SHOWN);\n if (window == nullptr) {\n std::cerr << "SDL_CreateWindow Error: " << SDL_GetError() << std::endl;\n TTF_Quit();\n SDL_Quit();\n return 1;\n }\n\n SDL_Renderer *renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC);\n if (renderer == nullptr) {\n SDL_DestroyWindow(window);\n std::cerr << "SDL_CreateRenderer Error: " << SDL_GetError() << std::endl;\n TTF_Quit();\n SDL_Quit();\n return 1;\n }\n\n SDL_SetRenderDrawColor(renderer, 169, 169, 169, 255); // Dark Gray background\n \n SDL_Color textColor = {255, 255, 255, 255}; // White color\n const std::string fontPath = "path/to/your/font.ttf";\n int fontSize = 24;\n SDL_Texture* messageTexture = nullptr;\n \n bool running = true;\n SDL_Event event;\n while (running) {\n while (SDL_PollEvent(&event)) {\n if (event.type == SDL_QUIT) {\n running = false;\n } else if (event.type == SDL_KEYDOWN) {\n // Display the key name inside the window GUI\n std::string keyName = SDL_GetKeyName(event.key.keysym.sym);\n messageTexture = renderText(renderer, fontPath, fontSize, keyName, textColor);\n }\n }\n SDL_RenderClear(renderer);\n\n if (messageTexture != nullptr) {\n int texW = 0;\n int texH = 0;\n SDL_QueryTexture(messageTexture, NULL, NULL, &texW, &texH);\n SDL_Rect dstRect = {0, 0, texW, texH};\n SDL_RenderCopy(renderer, messageTexture, NULL, &dstRect);\n }\n\n SDL_RenderPresent(renderer);\n }\n\n if (messageTexture != nullptr) {\n SDL_DestroyTexture(messageTexture);\n }\n SDL_DestroyRenderer(renderer);\n SDL_DestroyWindow(window);\n TTF_Quit();\n SDL_Quit();\n\n return 0;\n}', 'title': 'Editing src/main.cpp'}, 'step': None, 'message': None, 'completed': True, 'agent_is_active': True, 'token_usage': 11299, 'timestamp': '2024-04-04 17:06:08'}, {'internal_monologue': 'Agent has completed the task.', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': 'vim data/projects/sdl-app/src/main.cpp', 'output': '#include <SDL.h>\n#include <SDL_ttf.h>\n#include <iostream>\n#include <string>\n\nconst int WINDOW_WIDTH = 640;\nconst int WINDOW_HEIGHT = 480;\n\n// Initializes SDL and SDL_ttf, creates a window and renderer.\nbool init(SDL_Window **window, SDL_Renderer **renderer) {\n if (SDL_Init(SDL_INIT_VIDEO) != 0) {\n std::cerr << "SDL_Init Error: " << SDL_GetError() << std::endl;\n return false;\n }\n\n if (TTF_Init() == -1) {\n std::cerr << "TTF_Init Error: " << TTF_GetError() << std::endl;\n SDL_Quit();\n return false;\n }\n\n *window = SDL_CreateWindow("SDL Starter Kit", SDL_WINDOWPOS_CENTERED,\n SDL_WINDOWPOS_CENTERED, WINDOW_WIDTH, WINDOW_HEIGHT, SDL_WINDOW_SHOWN);\n if (*window == nullptr) {\n std::cerr << "SDL_CreateWindow Error: " << SDL_GetError() << std::endl;\n TTF_Quit();\n SDL_Quit();\n return false;\n }\n\n *renderer = SDL_CreateRenderer(*window, -1,\n SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC);\n if (*renderer == nullptr) {\n SDL_DestroyWindow(*window);\n std::cerr << "SDL_CreateRenderer Error: " << SDL_GetError() << std::endl;\n TTF_Quit();\n SDL_Quit();\n return false;\n }\n\n return true;\n}\n\n// Loads the font and renders the text to a texture.\nSDL_Texture* renderText(SDL_Renderer *renderer, const std::string &message, const std::string &fontPath, int fontSize, SDL_Color textColor) {\n TTF_Font *font = TTF_OpenFont(fontPath.c_str(), fontSize);\n if (font == nullptr) {\n std::cerr << "TTF_OpenFont Error: " << TTF_GetError() << std::endl;\n return nullptr;\n }\n\n SDL_Surface *surface = TTF_RenderText_Solid(font, message.c_str(), textColor);\n if (surface == nullptr) {\n TTF_CloseFont(font);\n std::cerr << "TTF_RenderText_Solid Error: " << TTF_GetError() << std::endl;\n return nullptr;\n }\n\n SDL_Texture *texture = SDL_CreateTextureFromSurface(renderer, surface);\n SDL_FreeSurface(surface);\n TTF_CloseFont(font);\n\n if (texture == nullptr) {\n std::cerr << "SDL_CreateTextureFromSurface Error: " << SDL_GetError() << std::endl;\n }\n\n return texture;\n}\n\nint main() {\n SDL_Window *window = nullptr;\n SDL_Renderer *renderer = nullptr;\n if (!init(&window, &renderer)) {\n return 1; // Initialization failed\n }\n\n bool running = true;\n SDL_Event event;\n SDL_Texture *messageTexture = nullptr;\n SDL_Color textColor = {255, 255, 255, 255}; // White\n std::string fontPath = "path/to/your/font.ttf";\n\n while (running) {\n while (SDL_PollEvent(&event)) {\n if (event.type == SDL_QUIT) {\n running = false;\n } else if (event.type == SDL_KEYDOWN) {\n std::string keyName = SDL_GetKeyName(event.key.keysym.sym);\n if (messageTexture != nullptr) {\n SDL_DestroyTexture(messageTexture); // Free previous texture\n }\n messageTexture = renderText(renderer, keyName, fontPath, 24, textColor);\n }\n }\n\n SDL_SetRenderDrawColor(renderer, 169, 169, 169, 255); // Dark Gray\n SDL_RenderClear(renderer);\n\n if (messageTexture != nullptr) {\n int texW = 0;\n int texH = 0;\n SDL_QueryTexture(messageTexture, NULL, NULL, &texW, &texH);\n SDL_Rect dstRect = { (WINDOW_WIDTH - texW) / 2, (WINDOW_HEIGHT - texH) / 2, texW, texH };\n SDL_RenderCopy(renderer, messageTexture, NULL, &dstRect);\n }\n\n SDL_RenderPresent(renderer);\n }\n\n if (messageTexture != nullptr) {\n SDL_DestroyTexture(messageTexture);\n }\n SDL_DestroyRenderer(renderer);\n SDL_DestroyWindow(window);\n TTF_Quit();\n SDL_Quit();\n\n return 0;\n}', 'title': 'Editing data/projects/sdl-app/src/main.cpp'}, 'step': None, 'message': None, 'completed': True, 'agent_is_active': True, 'token_usage': 15410, 'timestamp': '2024-04-04 17:07:25'}, {'internal_monologue': 'Writing code...', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': 'vim meson.build', 'output': "project('SDLStarterKit', 'cpp', version : '0.1.0')\n\nsdl2_dep = dependency('sdl2')\nsdl2_ttf_dep = dependency('SDL2_ttf')\n\nexecutable('sdlstarterkit', 'src/main.cpp', dependencies : [sdl2_dep, sdl2_ttf_dep])\n", 'title': 'Editing meson.build'}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 0, 'timestamp': '2024-04-04 17:09:03'}, {'internal_monologue': 'Writing code...', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': 'vim src/main.cpp', 'output': '#include <SDL.h>\n#include <SDL_ttf.h>\n#include <iostream>\n#include <string>\n\nconst int WINDOW_WIDTH = 640;\nconst int WINDOW_HEIGHT = 480;\n\n// Initializes SDL and SDL_ttf, creates a window and renderer.\nbool init(SDL_Window **window, SDL_Renderer **renderer) {\n if (SDL_Init(SDL_INIT_VIDEO) != 0) {\n std::cerr << "SDL_Init Error: " << SDL_GetError() << std::endl;\n return false;\n }\n\n if (TTF_Init() == -1) {\n std::cerr << "TTF_Init Error: " << TTF_GetError() << std::endl;\n SDL_Quit();\n return false;\n }\n\n *window = SDL_CreateWindow("SDL Starter Kit", SDL_WINDOWPOS_CENTERED,\n SDL_WINDOWPOS_CENTERED, WINDOW_WIDTH, WINDOW_HEIGHT, SDL_WINDOW_SHOWN);\n if (*window == nullptr) {\n std::cerr << "SDL_CreateWindow Error: " << SDL_GetError() << std::endl;\n TTF_Quit();\n SDL_Quit();\n return false;\n }\n\n *renderer = SDL_CreateRenderer(*window, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC);\n if (*renderer == nullptr) {\n SDL_DestroyWindow(*window);\n std::cerr << "SDL_CreateRenderer Error: " << SDL_GetError() << std::endl;\n TTF_Quit();\n SDL_Quit();\n return false;\n }\n\n return true;\n}\n\n// Loads the font and renders the text to a texture.\nSDL_Texture* renderText(SDL_Renderer *renderer, const std::string &message, const std::string &fontPath, int fontSize, SDL_Color textColor) {\n TTF_Font *font = TTF_OpenFont(fontPath.c_str(), fontSize);\n if (font == nullptr) {\n std::cerr << "TTF_OpenFont Error: " << TTF_GetError() << std::endl;\n return nullptr;\n }\n\n SDL_Surface *surface = TTF_RenderText_Solid(font, message.c_str(), textColor);\n if (surface == nullptr) {\n TTF_CloseFont(font);\n std::cerr << "TTF_RenderText_Solid Error: " << TTF_GetError() << std::endl;\n return nullptr;\n }\n\n SDL_Texture *texture = SDL_CreateTextureFromSurface(renderer, surface);\n SDL_FreeSurface(surface);\n TTF_CloseFont(font);\n\n if (texture == nullptr) {\n std::cerr << "SDL_CreateTextureFromSurface Error: " << SDL_GetError() << std::endl;\n }\n\n return texture;\n}\n\nint main() {\n SDL_Window *window = nullptr;\n SDL_Renderer *renderer = nullptr;\n if (!init(&window, &renderer)) {\n return 1; // Initialization failed\n }\n\n bool running = true;\n SDL_Event event;\n SDL_Texture *messageTexture = nullptr;\n SDL_Color textColor = {255, 255, 255, 255}; // White\n std::string fontPath = "path/to/your/font.ttf";\n\n while (running) {\n while (SDL_PollEvent(&event)) {\n if (event.type == SDL_QUIT) {\n running = false;\n } else if (event.type == SDL_KEYDOWN) {\n std::string keyName = SDL_GetKeyName(event.key.keysym.sym);\n if (messageTexture != nullptr) {\n SDL_DestroyTexture(messageTexture); // Free previous texture\n }\n messageTexture = renderText(renderer, keyName, fontPath, 24, textColor);\n }\n }\n\n SDL_SetRenderDrawColor(renderer, 169, 169, 169, 255); // Dark Gray\n SDL_RenderClear(renderer);\n\n if (messageTexture != nullptr) {\n int texW = 0;\n int texH = 0;\n SDL_QueryTexture(messageTexture, NULL, NULL, &texW, &texH);\n SDL_Rect dstRect = { (WINDOW_WIDTH - texW) / 2, (WINDOW_HEIGHT - texH) / 2, texW, texH };\n SDL_RenderCopy(renderer, messageTexture, NULL, &dstRect);\n }\n\n SDL_RenderPresent(renderer);\n }\n\n if (messageTexture != nullptr) {\n SDL_DestroyTexture(messageTexture);\n }\n SDL_DestroyRenderer(renderer);\n SDL_DestroyWindow(window);\n TTF_Quit();\n SDL_Quit();\n\n return 0;\n}\n', 'title': 'Editing src/main.cpp'}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 0, 'timestamp': '2024-04-04 17:09:04'}, {'internal_monologue': 'Agent has completed the task.', 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': 'vim README.md', 'output': '# SDL Starter Kit\n\nThis project is a simple SDL GUI application developed in C++ and built with Meson. It demonstrates creating a window, initializing SDL2_ttf, capturing key press events, and displaying the pressed key as text inside the window.\n\n## Prerequisites\n\nMake sure you have the following installed:\n- A C++ compiler (GCC or Clang)\n- Meson build system\n- Ninja build system\n- SDL2 library\n- SDL2_ttf library\n\n## Building and Running\n\nTo compile the project, navigate to the project directory and run:\nmeson setup builddir\ncd builddir\nninja\n\nThis will compile the project. To run the compiled application:\n./sdlstarterkit\n\nEnsure you replace `"path/to/your/font.ttf"` in the `src/main.cpp` with the actual path to your TTF font file for the text rendering to work.\n', 'title': 'Editing README.md'}, 'step': None, 'message': None, 'completed': True, 'agent_is_active': True, 'token_usage': 4182676, 'timestamp': '2024-04-04 17:09:05'}]
24.04.04 20:25:51: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 4187274}
Model: gpt-4-0125-preview, Enum: OPENAI
24.04.04 20:25:53: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 4185037}
==============================
('Hello! How can I assist you further with the project today?', 'answer')
24.04.04 20:25:53: root: INFO : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': 'Hello! How can I assist you further with the project today?', 'timestamp': '2024-04-04 20:25:53'}}
action :: answer
24.04.04 20:25:54: root: INFO : SOCKET tokens MESSAGE: {'token_usage': 5379642}
Model: gpt-4-0125-preview, Enum: OPENAI
Exception in thread Thread-5 (<lambda>):
Traceback (most recent call last):
File "/usr/local/Cellar/[email protected]/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
self.run()
File "/usr/local/Cellar/[email protected]/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1010, in run
self._target(*self._args, **self._kwargs)
File "/Users/chris/devika/devika.py", line 92, in <lambda>
thread = Thread(target=lambda: agent.subsequent_execute(message, project_name))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/devika/src/agents/agent.py", line 199, in subsequent_execute
response = self.answer.execute(
^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/devika/src/agents/answer/answer.py", line 45, in execute
response = self.llm.inference(prompt, project_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/devika/src/llm/llm.py", line 96, in inference
response = model.inference(self.model_id, prompt).strip()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/devika/src/llm/openai_client.py", line 13, in inference
chat_completion = self.client.chat.completions.create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/devika/venv/lib/python3.12/site-packages/openai/_utils/_utils.py", line 275, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/devika/venv/lib/python3.12/site-packages/openai/resources/chat/completions.py", line 667, in create
return self._post(
^^^^^^^^^^^
File "/Users/chris/devika/venv/lib/python3.12/site-packages/openai/_base_client.py", line 1213, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/devika/venv/lib/python3.12/site-packages/openai/_base_client.py", line 902, in request
return self._request(
^^^^^^^^^^^^^^
File "/Users/chris/devika/venv/lib/python3.12/site-packages/openai/_base_client.py", line 978, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/devika/venv/lib/python3.12/site-packages/openai/_base_client.py", line 1026, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "/Users/chris/devika/venv/lib/python3.12/site-packages/openai/_base_client.py", line 978, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/devika/venv/lib/python3.12/site-packages/openai/_base_client.py", line 1026, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "/Users/chris/devika/venv/lib/python3.12/site-packages/openai/_base_client.py", line 993, in _request
raise self._make_status_error_from_response(err.response) from None
openai.RateLimitError: Error code: 429 - {'error': {'message': 'Request too large for gpt-4-turbo-preview in organization org-np1jnWymgWaJHoYSrfZPX4yk on tokens per min (TPM): Limit 300000, Requested 469214. The input or output tokens must be reduced in order to run successfully. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
Was about to post an issue for this, I believe it's because right now it tries to send the entire codebase in a single prompt using read_code. Ideally it should split it into multiple requests or only focus on the relevant chunks.
Almost like some models, even API ones have a context limit and you can't just cram everything in them at once.
I'm seeing the same messages for Claude and Mistral models, obviously.
I found a workaround. So when I get this error, i simply went and created a new project, then asked something, once the project structure is created, I simply copied the previous content to the new project and the asked to continue from there by providing the last instruction before the error and it worked for me. Maybe even creating a new folder manually by the same name of the new project will also work I believe. I haven't tried that. This way you don't need to ask something at the beginning to initialize. 😉
