[Feature request] Integration of local LLMs

[lightningRalf]

Hello, with the integration of other models like gpt3.5 the first step was made by introducing the ability to make the net wider. This was shortly discussed in #337.

So I wanted to ask if there is a chance to widen the net further and introduce more models. There is already an open source project that provides an API for this, but I dont know if you want to add dependencies. Just wanted to share, for better understanding what I mean.

Thanks in advance.

[evchaki]

@lightningRalf check this out - https://github.com/microsoft/semantic-kernel/blob/main/samples/dotnet/kernel-syntax-examples/Example16_CustomLLM.cs it should get you started on how to do this.

[lightningRalf]

Dear @evchaki,

Thanks for your quick response! As I continue exploring this repository, I realize my initial question might have been a bit premature.

For context, I'm focusing on the Python codebase.

Could you please help me confirm the following understandings?

• The core functionality is already part of the kernel, right?
• This specific kernel feature won't be ported to Python, correct?
• The example you provided is a starting point for building this on my own. Is this meant to be part of the codebase eventually, or should it stay as a standalone project?
• Is it accurate that this feature is only available for text completion and not within ChatCompletion, Embeddings, or ImageGeneration? Or is it possible to integrate it into those areas, but the example didn't demonstrate that?

Thanks again for your help, and I'm looking forward to your insights.

Also found another resource for interacting with other models from huggingface via API

[dluc]

@awharrison-28 could you help with some examples? We have some code in the repo which I think would be useful at least to start, while we improve the framework

[awharrison-28]

@lightningRalf check out https://github.com/microsoft/semantic-kernel/pull/658 - introduces huggingface models (local) for use with SK. There are a number of examples added in this PR under tests/end-to-end.

There are still a number of edge cases and scenarios not covered in this PR, but it's a start to a set of features that will quickly grow.