OpenInterpreter
llm.model is defaulting to openai/hugging face
Describe the bug
config.yaml file: llm.model: "huggingface/codellama/CodeLlama-34b-Instruct-hf" Model: openai/huggingface/codellama/CodeLlama-34b-Instruct-hf
This is from config.yaml file:
OPEN INTERPRETER CONFIGURATION FILE
llm.model: "huggingface/codellama/CodeLlama-34b-Instruct-hf" llm.temperature: 0.4
Be sure to remove the "#" before the following settings to use them.
custom_instructions: "" # This will be appended to the system message
This is what interpreter generated:
Interpreter Info
Vision: False
Model: openai/huggingface/codellama/CodeLlama-34b-Instruct-hf
Function calling: None
Context window: 3000
Max tokens: 400
Reproduce
set llm.model in config.yaml file. Update API_Key or generate info.
Expected behavior
I thought setting the model as "huggingface/codellama/CodeLlama-34b-Instruct-hf" would set that as the model.
Screenshots
No response
Open Interpreter version
Open-interpreter Version: cmd:Interpreter, pkg: 0.2.0
Python version
3.10.11
Operating System name and version
Windows 11
Additional context
No response
Hey @RisingVoicesBk What command are you doing to run Open Interpreter? What tool are you using to serve the model? (Jan, Ollama, etc)
At the time I was running --profiles which also doesn't work well for me. I can't get to the profiles directory. But when I did use the command, this was part of the response.
Vision: False Model: openai/huggingface/codellama/CodeLlama-34b-Instruct-hf Function calling: None Context window: 3000 Max tokens: 400
I was not running the model on my local computer I was running it through Huggingface API Inference. It gives you free hosted access to certain models with limited use for testing purposes. This was the model I was trying to run: huggingface/codellama/CodeLlama-34b-Instruct-hf
I am chatted with the OI Help / Search feature which was interesting and it offered a corrected version which I am including below.
llm_model: "codellama/CodeLlama-34b-Instruct-hf" llm_temperature: 0.4 auto_run: False # If True, code will run without asking for confirmation verbose: False # If True, will print detailed logs llm_api_base: "https://api-inference.huggingface.co" # The URL where an OpenAI-compatible server is running to handle LLM API requests llm_max_output: 3000 # The maximum characters of code output visible to the LLM llm_context_window: 3000 # The maximum characters of code context visible to the LLM llm_max_tokens: 400 # The maximum number of tokens to send to the LLM
--profiles shouldn't work unless you are running the git version
We add openai to the model name to use the openai api format, this will change next release.
Seems like only openai, azure and ollama works without it adding openai to the model name. Fix for next release
Newby here, sorry. What's the git version? I am running version 0.2.0 now. Can I access that? Should I? Or should I just maintain the script for multiple profiles somewhere else and manually sub them into config.yaml? I can ask some of this on the OI website. The search function is actually excellent so kudos to whoever did that work. Thank you. It is super helpful.
Sorry, I think my original issue was about openai/ which seems to be intentional for now and soon to change, but ultimately, I have been wanting to setup profiles for easier testing and learning.
Really glad to hear you had so much success on the website!
If you want to use OI as it currently exists in this repo ( pre-release or 'the git version') run this command: pip install --upgrade --force-reinstall git+https://github.com/KillianLucas/open-interpreter.git
Seems like the change is still in a PR, i need to modify some parts before merging it...
@RisingVoicesBk I would wait for the update coming soon and upgrade then, the git version is changing a lot and might break