Enhancement: Using Groq

[bakongi]

Hi everyone:

To use Groq, just add in models.py:

url = 'https://api.groq.com/openai/v1/chat/completions'
groq = dict(type=GPTAPI,
            model_type='gemma2-9b-it',
            key=os.environ.get('GROQ_API_KEY', 'YOUR GROQ API KEY'),
            openai_api_base=url,
            temperature=0,
            max_new_tokens=4096,
            repetition_penalty=1.02
            )

[snehpushp]

I tried this, for me it threw an error (because of GPTAPI). It was: NotImplementedError(f'Model type {model_type} is not supported')

For a quick fix, I changed the following code in openai.py file (from where this error is occurring) - From if model_type.lower().startswith('gpt'): to if model_type.lower().startswith('gpt') or model_type.lower().startswith('llama'):

I am using llama-3.1-8b-versatile.

[bakongi]

I tried this, for me it threw an error (because of GPTAPI). It was: NotImplementedError(f'Model type {model_type} is not supported')

For a quick fix, I changed the following code in openai.py file (from where this error is occurring) - From if model_type.lower().startswith('gpt'): to if model_type.lower().startswith('gpt') or model_type.lower().startswith('llama'):

I am using llama-3.1-8b-versatile.

You should modify lagent\llms\openai.py file in lagent package:

['gpt', 'gemma', 'llama', 'mixtral']

        # Model-specific processing
        data = {}
        if any(model_type.lower().startswith(prefix) for prefix in ['gpt', 'gemma', 'llama', 'mixtral']):
            if 'top_k' in gen_params:

[MyraBaba]

@bakongi

when use groq with above changes I have below error:

ERROR:root:Exception in sync_generator_wrapper: 'detail' Traceback (most recent call last): File "/home/bc/Projects/ODS/MindSearch/mindsearch/app.py", line 69, in sync_generator_wrapper for response in agent.stream_chat(inputs): File "/home/bc/Projects/ODS/MindSearch/mindsearch/agent/mindsearch_agent.py", line 237, in stream_chat yield from self._process_code(agent_return, inner_history, File "/home/bc/Projects/ODS/MindSearch/mindsearch/agent/mindsearch_agent.py", line 284, in _process_code reference, references_url = self._generate_reference( File "/home/bc/Projects/ODS/MindSearch/mindsearch/agent/mindsearch_agent.py", line 316, in _generate_reference actions = agent_return.nodes[node_name]['detail'].actions KeyError: 'detail'