.Net Bug: StreamingAnnotationContent isn't populated for AI Agent Service Internet Tool

[davidames]

Describe the bug When using the Internet Search Tool with the AI Agent Service, citations are returned via the StreamingAnnotationContent class. This class is mostly empty and does not contain any useful information such as the citation URL.

To Reproduce Steps to reproduce the behavior:

1. Create an AI in the AI Agent Service, with a Grounding with Bing tool.
2. As a question that invokes the Grounding with Bing tool
3. Inspect the StreamingAnnotationContent that is returned in the response

  [Fact]
    [Experimental("OPENAI001")]
    public async Task InternetSearchFlattened()
    {
        var foundryConfig = GetService<AiFoundryConfig>();
        var client = AzureAIAgent.CreateAzureAIClient(foundryConfig.ConnectionString, new DefaultAzureCredential());
        var agentsClient = client.GetAgentsClient();
        var getAgentResponse =  await agentsClient.GetAgentAsync(AgentId);

        AzureAIAgent  agent = new(getAgentResponse.Value, agentsClient);
        AgentThread thread = new AzureAIAgentThread(agentsClient);
        
        await foreach (var responseUpdate in agent.InvokeStreamingAsync("What are today's news headlines", thread))
        {
            Logger.Debug("Streaming chunk: {Chunk}", responseUpdate.Message.Content);
            
            if (responseUpdate.Message.Items.Any())
            {
                foreach (var item in responseUpdate.Message.Items)
                {
                    if (item is StreamingAnnotationContent annotationContent)
                    {
                        Logger.Information("Citation Content {InnerContent}:", annotationContent.InnerContent);
                    }
                }
            }

            
        }
    }
    

Expected behavior There should be way to get the citation URL and page title.

Screenshots

Platform

• Language: C#
• Source: Microsoft.SemanticKernel 1.47
• AI model: gpt4 2024-08-06
• IDE: Rider
• OS: Windows

Additional context Add any other context about the problem here.

[markharrison]

I have a similar / maybe connected issue - if my Agent is using Azure AI Search for its Knowledge ... the returned AnnotationContent is missing UrlCitation information