semantic-kernel icon indicating copy to clipboard operation
semantic-kernel copied to clipboard
verified publisher icon microsoft

.Net: Bug: OpenAI Assistant throws Invalid URI: The Uri string is too long.

Open randysimplist opened this issue 9 months ago • 1 comments

Describe the bug The Semantic Kernel C# library isn't properly handling images for OpenAI assistants. The issue lies in how the AssistantMessageFactory creates the underlying image content which can exceed the 65k limit of System.Uri.

To Reproduce Steps to reproduce the behavior:

  1. Setup an OpenAI assistant agent
  2. Setup a group chat (one agent is fine)
  3. Add a ChatMessageContent with ImageContent item(s)
  4. Observe the System.UriFormatException: Invalid URI: The Uri string is too long.

Expected behavior I would expect Semantic Kernel to use the appropriate OpenAI overloads to avoid this error

Screenshots N/A

Platform

  • Language: C#
  • Source: Microsoft.SemanticKernel, v1.48.0
  • AI model: OpenAI 4o (2024-11-20)
  • IDE: Visual Studio
  • OS: Windows

Additional context There was a similar issue reported here for the OpenAI library where the user was able to work around the problem by supplying a list of bytes. https://github.com/openai/openai-dotnet/issues/138

It's worth noting that the ChatCompletions API doesn't suffer from this issue. Appears to only occur when using an OpenAI Assistant.

System.UriFormatException: Invalid URI: The Uri string is too long. at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind, UriCreationOptions& creationOptions) at Microsoft.SemanticKernel.Agents.OpenAI.Internal.AssistantMessageFactory.GetMessageContents(ChatMessageContent message)+MoveNext() at OpenAI.Assistants.AssistantClient.CreateMessageAsync(String threadId, MessageRole role, IEnumerable1 content, MessageCreationOptions options, CancellationToken cancellationToken) at Microsoft.SemanticKernel.Agents.OpenAI.Internal.AssistantThreadActions.CreateMessageAsync(AssistantClient client, String threadId, ChatMessageContent message, CancellationToken cancellationToken) at Microsoft.SemanticKernel.Agents.OpenAI.OpenAIAssistantChannel.ReceiveAsync(IEnumerable1 history, CancellationToken cancellationToken) at Microsoft.SemanticKernel.Agents.AgentChat.GetOrCreateChannelAsync(Agent agent, CancellationToken cancellationToken) at Microsoft.SemanticKernel.Agents.AgentChat.InvokeStreamingAgentAsync(Agent agent, CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.Agents.AgentChat.InvokeStreamingAgentAsync(Agent agent, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult() at Microsoft.SemanticKernel.Agents.AgentGroupChat.InvokeStreamingAsync(Agent agent, CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.Agents.AgentGroupChat.InvokeStreamingAsync(Agent agent, CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.Agents.AgentGroupChat.InvokeStreamingAsync(Agent agent, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult() at Microsoft.SemanticKernel.Agents.AgentGroupChat.InvokeStreamingAsync(CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.Agents.AgentGroupChat.InvokeStreamingAsync(CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.Agents.AgentGroupChat.InvokeStreamingAsync(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()

randysimplist avatar May 04 '25 14:05 randysimplist

Facing same issue here! SK version v1.51.0

I'm wondering if loading the image with FileReferenceContent produces equivalent interpretation from the model or not

gsubiran avatar May 19 '25 04:05 gsubiran