CAI Stucked While Using Gemini Models

[trinoxx452]

https://github.com/user-attachments/assets/e818bd8d-caff-4df6-a0a0-c0d8bbc5a2e8

While using the /model gemini/gemini-2.5-pro endpoint to solve a simple XSS challenge, the model stops responding after 2-3 interactions. This appears to be caused by a 503 error from the Gemini API, with the following error message: "error": { "code": 503, "message": "The model is overloaded. Please try again later.", "status": "UNAVAILABLE" }

It seems that the current implementation lacks a retry mechanism to handle such errors. Please if add retry mechanisms for such error also that would be great help for us.

[O3-FRENZY]

same facing the same issue with gemini api

[liamsimfty]

where you put gemini API?

[SoyGema]

Hello @trinoxx452 . Thanks for opening this issue. CAI actually implements some retry mechanisms but I definetly agree that this is a rational and useful request, as 503 error is not contemplated.

We are indeed listening and hopefully we will implement more verbosity soon. Thanks for your patience, apologies for the inconvenience and please don´t hesitate to ping us in discord for further discussions!

[SoyGema]

where you put gemini API?

Hello @liamsimfty trying to help here. As far as I understand you can put the API in the .env file , or echo -e 'OPENAI_API_KEY="sk-1234"\nGEMINI_API_KEY="gemini-api-key-here"\nOLLAMA=""\nPROMPT_TOOLKIT_NO_CPR=1\nCAI_STREAM=false' > .env

LMK if this worked for you

[SoyGema]

@liamsimfty FYI https://github.com/aliasrobotics/cai/issues/265 Thanks @SyrusKyury . LMK if that helped

[aliasrobotics-support]

Hi trinox, 03, and liam and Since we haven’t heard back from you, we assume the fix provided by Syrus was helpful. Thank you for using CAI!