Support for llama.cpp server function calling

[vijaysaayi]

Does bitnet-b1.58-2B-4T model support tool calling ?

If so could you suggest how we can use this feature with bitnet.cpp ?

• https://github.com/ggml-org/llama.cpp/blob/master/docs/function-calling.md

[sd983527]

Please refer to this pr https://github.com/microsoft/BitNet/pull/204 which has been merged.

[vijaysaayi]

Thanks for sharing the PR.

The llama.cpp fork seems to be older. It does not support tools

• https://github.com/Eddie-Wang1120/llama.cpp/blob/20f1789dfb4e535d64ba2f523c64929e7891f428/examples/server/utils.hpp#L394

Could you share any tentative timeline when this would be supported with bitnet ?

[santiac89]

I was looking for the same