microsoft
Integration of Semantic Kernel and AzureBotFramework SDK
The big Picture in my Mind
GenAI Applications should be capable of executing every workload to succeed in the market. Currently I see GenAI Applications more as something that works in a passive way (retrieving data / answering questions), but the final idea is they should be able to execute critical workload aswell to really impact the industry and change the market.
The Problem
When extracting Parameters from Prompts to Function Calling they can run into hallucinations. To ensure to execute the workload a human check would be necessary. This check should be to 100% deterministic and not involve a LLM to ensure the workload is executed as desired.
The Solution i see
The collaboration between Semantic Kernel and the BotFramework could be key here to succeed in the market. The overall idea is to use Planners to run the corresponding function's and then when a critical function is called you use the Hooks that are available on Semantic Kernel to start a Dialog in BotFramework SDK to get into the "deterministic Mode" to ensure the workload is executed with the correct parameters.
This is probably already possible when we start a Dialog of the BotFramework in a SKFunction, but here we run into the issue. The Semantic Kernel Function expects a result but the "Turn Based" approach of the BotFramework will end the Function without an immediately answer.
You can program around this with stopping the planner, starting a Dialog and then pick the Conversation Up, but since the Stateless Approach of BotFramework and the "keep the whole history" approach in LLM's it is pretty hard to achieve.
What i would like to see
I would like to see a Concept to execute high critical workloads with LLM's with avoiding Hallucinations to 100% here. I think Semantic Kernel and Azure Bot Framework are capable of achieving this together !
Because it could enable the current flow :
- User requires critical Workload --> Planner is getting Parameters (Hallucinations possible) --> Bot SDK is starting Dialog to validate Parameters --> User Approves --> Critical Workload is executed
The Final conclusion
I see great potential as synergy between the Products to overcome the Problem that currently hinders to execute critical workloads with LLM's. A concept or example of this would really help the industry to speed up on the Investment in AI Probably !
I came here to see what's happening in this space and glad to see that I am not the only one wanting this! 🙏
I came here to see what's happening in this space and glad to see that I am not the only one wanting this! 🙏
I want this too.
Hope can do something like this with SK: https://github.com/microsoft/semantic-kernel/discussions/4853
If SK alone cannot, then seamless integration of SK with Microsoft Bot Framework might be the answer.
