genkit icon indicating copy to clipboard operation
genkit copied to clipboard

[JS] dotprompt `.generate` and `.render` methods don't allow custom config options

Open pavelgj opened this issue 1 year ago • 1 comments

Describe the bug

this code does not compile:

    const resp = await dotpromptPrompt.render({
      model: 'vertexai/gemini-1.5-flash',
      input: {
        foo: 'bar',
      },
      config: {
        location: 'us-central1',
      },
    });

Error:

Object literal may only specify known properties, and 'customOption' does not exist in type '{ version?: string | undefined; temperature?: number | undefined; maxOutputTokens?: number | undefined; topK?: number | undefined; topP?: number | undefined; stopSequences?: string[] | undefined; }'.ts(2353)

generate.d.ts(261, 5): The expected type comes from property 'config' which is declared here on type 'PromptGenerateOptions<any>'

Most Vertex AI models have support passing location as a config option.

Known workaround

    const req = await dotpromptPrompt.render({
      model: 'vertexai/gemini-1.5-flash',
      input: {
        foo: 'bar',
      },
    });
    req.config = {
      location:  'us-central1',
    };
    const resp = await generate(req);

pavelgj avatar Aug 07 '24 01:08 pavelgj

I looked into this a bit. It's not possible to identify the correct custom options type if we accept string model arguments, so it should be any in that case. But we can make it check the custom options type strictly if a fully qualified model is also passed in to render or generate.

image

maxl0rd avatar Aug 14 '24 18:08 maxl0rd