Make is easier to select either openai or azure openai, ideally heuristically

[codefromthecrypt]

We currently use spring-specific ways to reduce ambiguity between which llm to use. However, there's a routine issue with OpenAI as often the same app needs to use Azure OpenAI. Right now, they are in different auto-configuration and have no way to collaborate with each other besides custom (albeit defined in spring) properties.

I would like very much for a spring-ai application to work with the same ENV vars that apps in python etc use, and employ those to make life even easier.

Expected Behavior

Specifically, if AZURE_OPENAI_API_KEY is non empty, don't export a bean for azure, and visa versa for openai.. make it conditional on the one required ENV variable. We can also keep the other conditions, just I would like this.

Even if we don't support official env variables, nor support a heuristic, I would like to make it much easier to choose one with a single property.

Current Behavior

You'll get something like this, or worse depending on the approach.

Parameter 0 of constructor in example.Main$VersionAgent required a single bean, but 2 were found:
        - azureOpenAiChatModel: defined by method 'azureOpenAiChatModel' in class path resource [org/springframework/ai/model/azure/openai/autoconfigure/AzureOpenAiChatAutoConfiguration.class]
        - openAiChatModel: defined by method 'openAiChatModel' in class path resource [org/springframework/ai/model/openai/autoconfigure/OpenAiChatAutoConfiguration.class]

Context

Right now, it is very distracting to work around this, but below works.

    public static void main(String[] args) {
        // For convenience, we allow users to choose either OpenAI or Azure OpenAI using official
        // SDK variables mapped in application.yaml. However, as of spring-ai 1.0.0-M7, you cannot
        // choose one or the other either heuristically (enabling only when azure's api key is set)
        // or by a single property (e.g. "spring.ai.model"). Instead, you have to look for which
        // autoconfiguration are conditional on which property and set them all.
        String azureApiKey = System.getenv("AZURE_OPENAI_API_KEY");
        String chatModel = azureApiKey != null && !azureApiKey.trim().isEmpty()
                ? SpringAIModels.AZURE_OPENAI
                : SpringAIModels.OPENAI;
        new SpringApplicationBuilder(Main.class)
                .properties(
                        SpringAIModelProperties.AUDIO_TRANSCRIPTION_MODEL + "=" + chatModel,
                        SpringAIModelProperties.CHAT_MODEL + "=" + chatModel,
                        SpringAIModelProperties.EMBEDDING_MODEL + "=" + chatModel,
                        SpringAIModelProperties.IMAGE_MODEL + "=" + chatModel
                ).run(args);
    }

Ideally, we can fold azure and openai into the same auto-configuration similar to how official SDKs work. this should make it a lot easier to coordinate things like this. However, using a single property in addition to the fine-grained ones would also work. e.g. MODEL_PREFIX= chatModel

[markpollack]

I will look into this as part of a larger issue for creating many instances of many chat models. That said, I'm not that keen to ensure we use the same environment variables as that in other ecosystems.

[codefromthecrypt]

That said, I'm not that keen to ensure we use the same environment variables as that in other ecosystems.

ok mainly if it is at least the same cardinality and shape will be helpful. For example mechanically rewriting openai ENV to spring's conventions. Thanks for considering @markpollack!

[markpollack]

There likely isn't a single standard in python land for all the env-vars that can be translated to spring's property format. It isn't a path I really want to pursue.

Maybe i'm just tired, but can you explain a bit more clearly what behavior you are looking for

Specifically, if AZURE_OPENAI_API_KEY is non empty, don't export a bean for azure,

this seems counter intuitive to me.

There are projects such as https://github.com/pivotal-cf/java-cfenv/tree/main/java-cfenv-jdbc that do some conversion in a perhaps more reusable manner.

[codefromthecrypt]

I need only one line to swap azure for normal openai in python and the AZURE_OPENAI_API_KEY is reliable as it is canonical.

    openai_client = AsyncAzureOpenAI() if os.getenv("AZURE_OPENAI_API_KEY") else None
    model = OpenAIProvider(openai_client=openai_client, use_responses=False).get_model(model_name)

I would be easy to heuristically configure azure openai vs normal openai depending on which ENV are defined. I would like to have both azure and openai in the same dependency as sometimes the same app needs to switch between one or the other depending on dev vs prod. Ideal case is auto-configuration using canonical ENV variables defined by openai.

If not keen, please close the issue, I mainly raised this to give a chance for it to be addressed.