guidance RateLimitError

When running the first example of the tutorial notebook, I get a RateLimit error:

import guidance

# we will use GPT-3 for most of the examples in this tutorial
guidance.llm = guidance.llms.OpenAI("text-davinci-003", api_key="XXXX")
program = guidance('''The best thing about the beach is {{~gen 'best' temperature=0.7 max_tokens=7}}''')
program()

Exception Traceback (most recent call last) Cell In[23], line 4 1 # we can use the {{gen}} command to generate text from the language model 2 # note that we used a ~ at the start of the command tag to remove the whitespace before it (just like in Handlebars) 3 program = guidance('''The best thing about the beach is {{~gen 'best' temperature=0.7 max_tokens=7}}''') ----> 4 program()

File ~/.python/current/lib/python3.10/site-packages/guidance/_program.py:234, in Program.call(self, **kwargs) 232 return self._stream_run(loop, new_program) 233 else: --> 234 loop.run_until_complete(new_program.execute()) 236 return new_program

File ~/.local/lib/python3.10/site-packages/nest_asyncio.py:90, in _patch_loop..run_until_complete(self, future) 87 if not f.done(): 88 raise RuntimeError( 89 'Event loop stopped before Future completed.') ---> 90 return f.result()

File ~/.python/current/lib/python3.10/asyncio/futures.py:201, in Future.result(self) 199 self.__log_traceback = False 200 if self._exception is not None: --> 201 raise self._exception 202 return self._result ... --> 531 raise Exception(f"Too many (more than {self.llm.max_retries}) OpenAI API RateLimitError's in a row!") 533 if stream: 534 return self.llm.stream_then_save(out, key, stop_regex, n)

Exception: Too many (more than 5) OpenAI API RateLimitError's in a row!

May 30 '23 13:05 bernardocaldas

Same problem

May 30 '23 18:05 0dminnimda

same here

May 31 '23 13:05 chiragshah285

@bernardocaldas - are you running in an OpenAI free trial account? It has an RPM (requests per minute) of 3. and n=3 is exactly that limit. You could try dropping it from 3 -> 2.

May 31 '23 14:05 chrisfentiman

No, I'm on a paid account

Jun 01 '23 09:06 bernardocaldas

And setting max_calls_per_min=2 on the llm doesn't solve it, unfortunately

Jun 01 '23 09:06 bernardocaldas

Passing llm=guidance.llms.OpenAI("gpt-3.5-turbo") when you are initialising guidance object worked for me. ex. prompt = guidance('''PROMPT''', llm =guidance.llms.OpenAI("gpt-3.5-turbo"))