BotSharp Conversation Message Exception

Hello again,

Following the steps of the documentation, I have configured my LlamaSharp model here:

"LlamaSharp": { "Interactive": true, "ModelDir": "D:/C#/Llms", "DefaultModel": "llama-2-7b-chat.Q4_K_M.gguf", "MaxContextLength": 1024, "NumberOfGpuLayer": 20 },

and in the providers section too:

"LlmProviders": [ { "Provider": "llama-sharp", "Models": [ { "Name": "llama-2-7b-chat.Q4_K_M.gguf", "Type": "chat" } ] } ],

As you can tell, I am only adding Llamasharp, no other provider.

When running the app, my plugins folder remains empty, and when trying to send a message to start the conversation I get an exception:

System.ArgumentNullException: Value cannot be null. (Parameter 'path2') at System.ArgumentNullException.Throw(String paramName) at System.IO.Path.Combine(String path1, String path2) at BotSharp.Plugins.LLamaSharp.LlamaAiModel.LoadModel(String model) at BotSharp.Plugin.LLamaSharp.Providers.ChatCompletionProvider.GetChatCompletions(Agent agent, List1 conversations) at BotSharp.Core.Routing.RoutingService.InvokeAgent(String agentId, List1 dialogs, Func2 onFunctionExecuting) at BotSharp.Core.Routing.Handlers.RouteToAgentRoutingHandler.Handle(IRoutingService routing, FunctionCallFromLlm inst, RoleDialogModel message, Func2 onFunctionExecuting) at BotSharp.Core.Routing.RoutingService.InstructDirect(Agent agent, RoleDialogModel message) at BotSharp.Core.Conversations.Services.ConversationService.SendMessage(String agentId, RoleDialogModel message, PostbackMessageModel replyMessage, Func2 onMessageReceived, Func2 onFunctionExecuting, Func2 onFunctionExecuted) at BotSharp.OpenAPI.Controllers.ConversationController.SendMessage(String agentId, String conversationId, NewMessageModel input) at lambda_method6(Closure, Object) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask1 actionResultValueTask) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

Seems that it is not finding the provider, am I right? I have added the plugin "BotSharp.Plugin.LLamaSharp", so I have no idea what I could be missing.

Aug 12 '24 12:08 gkapellmann

Now trying from home is a different error, but same code:

System.NullReferenceException: Object reference not set to an instance of an object. at BotSharp.Core.Routing.RoutingService.InvokeAgent(String agentId, List1 dialogs, Func2 onFunctionExecuting) at BotSharp.Core.Routing.Handlers.RouteToAgentRoutingHandler.Handle(IRoutingService routing, FunctionCallFromLlm inst, RoleDialogModel message, Func2 onFunctionExecuting) at BotSharp.Core.Routing.RoutingService.InstructDirect(Agent agent, RoleDialogModel message) at BotSharp.Core.Conversations.Services.ConversationService.SendMessage(String agentId, RoleDialogModel message, PostbackMessageModel replyMessage, Func2 onMessageReceived, Func2 onFunctionExecuting, Func2 onFunctionExecuted) at BotSharp.OpenAPI.Controllers.ConversationController.SendMessage(String agentId, String conversationId, NewMessageModel input) at lambda_method6(Closure, Object) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

This is very weird.

I have added the Router parameter, but is empty in the end, is that correct?

Aug 12 '24 22:08 gkapellmann

Ok, figured it out, I had to add the llmConfig to the Agent structure in the settings. It is now like this:

"Agent": { "DataDir": "agents", "TemplateFormat": "liquid", //"HostAgentId": "01e2fc5c-2c89-4ec7-8470-7688608b496c", //"EnableTranslator": false "llmConfig": { "Provider": "llama-sharp", "Model": "llama-2-7b-chat.Q8_0.gguf", //"is_inherit": true, "max_recursion_depth": 3 } },

But I dont really understand this settings.

Are these settings general for all Agents?
What is the function of HostAgentId?
If I dont fill llmConfig, is there a way to leave a predetermined model? Or is this that way?

Thank you in advance!

Aug 12 '24 22:08 gkapellmann

The HostAgentId is the Routing agent entry when you're using the BotSharp-UI.

Aug 13 '24 10:08 Oceania2018

OK! Thanks for clarifying that @Oceania2018

So if I am not using the UI, but just by API calls, there is really no need of a Hosting agent, I guess??

But then, the "Agent" structure setting, those are the generic settings for all agents?
What is the default llmConfig value if it is not specified?

Aug 13 '24 11:08 gkapellmann