Return multiple choices

[kstathou]

Describe the solution you'd like Thanks a lot for making this library, it's been super helpful! I am working on a text classification task and I wanted to get multiple parsed responses from a single client.chat.completions.create call using OpenAI's n parameter.

The following would only parse the first completion I believe:

client.chat.completions.create_with_completion(
        model="gpt-3.5-turbo-0613",
        response_model=SinglePrediction,
        messages=[
            {
                "role": "user",
                "content": f"Classify the following text: {data}",
            },
        ],
        n=3,
    )

How would you recommend going on about it? Thank you!

[jxnl]

I think we can create a overload that does this thanks for the new sdk

[jxnl]

Will experiment with overloads.

[jxnl]

https://github.com/jxnl/instructor/pull/578

[NicolaiLolansen]

#578

This was closed without merge, is this functionality working? It's a really nice feature to have if you want to extract multiple elements from a larger chunk of text, like menu items.

EDIT:

Reading https://python.useinstructor.com/examples/classification/#defining-the-structures_1 it seems like this can be solved by constructing your Pydantic models into a list of items:

class ItemType(str, Enum):
    MAIN_COURSE = "main_course"
    APPETIZER = "appetizer"
    DESSERT = "dessert"
    DRINK = "drink"

class MenuItem(BaseModel):
    name: str  # Name of the menu item
    item_type: ItemType  # Type of menu item (e.g., main course, appetizer, etc.)
    description: str  # Description of the menu item
    price: float  # Price of the menu item

class Menu(BaseModel):
    items: List[MenuItem]
   

In my limited testing, this works!

[josepablog]

Also interested in this feature

[NicolaiLolansen]

Also interested in this feature

@josepablog Did you look at the solution I posted? It seems to work very well for me.

[josepablog]

ohhh.. I didn't understand it at first.

I'll think about it, I think I may be able to solve my issue with prompting: I'm outputting a list to begin with, and I don't think a list of lists. I am just trying to make sure that the list is long enough.

thanks for getting back to me!

[thomasnormal]

Pydantic also supports minimum list lengths: https://github.com/annotated-types/annotated-types?tab=readme-ov-file#minlen-maxlen-len

from annotated_types import Len
from pydantic import BaseModel

class Foo(BaseModel):
  my_list: Annotated[list[str], Len(min_length=5, max_length=5)]

[thomasnormal]

To be clear, this feature would still be incredibly useful to have. When you force the model to output a certain list length, you get answers that are "influences" by each other. Lots of state of the art LLM based methods require you to create 1000 answers independently, or more, and you can save a lot of money by using n=... rather than sending individual requests.