bug: Server mode - missing tool call id

[SauceChord]

Jan version

0.5.13

Describe the Bug

Jan Server does not include a tool call id for called tools. This causes an issue with using langchain/langgraph which requires a tool call id. Included is a python script that tests OpenAI and Jan server tool calls (using qwen2.5-coder-7b-instruct model for Jan)

Expected: tool call id for tool calls.

Steps to Reproduce

• Install Jan
• Download Qwen2.5 Coder 7B Instruct Q4 from the Hub
• Activate Qwen2.5 Coder 7B Instruct Q4
• Start Local API Server
  • Note: It says it's an OpenAI compatible local HTTP server
• Install python
• pip install openai to install library dependencies
• Set your OPENAI_API_KEY environment variable to your OpenAI Api Key
• Add code to test.py

from openai import OpenAI

def main():
    jan = OpenAI(base_url="http://localhost:1337/v1")
    oai = OpenAI()
    TEST(oai, "gpt-4o-mini")
    TEST(jan, "qwen2.5-coder-7b-instruct")

def TEST(client, m):
    try:
        history = []
        history.append({"role": "system", "content": "You are a helpful assistant."})
        history.append({"role": "user", "content": "Add 1 and 2"})
        response = client.chat.completions.create(
            model=m,
            messages=history,
            tools=[
                schema_add(),
            ],
            tool_choice="auto"
        )
        for tool in response.choices[0].message.tool_calls:
            if tool.id is None:
                raise Exception(f"{m}: tool call id is None for {tool.function.name}")        
        print(f"Pass: {m}")
    except Exception as e:
        print(f"Fail: {e}")

# A function that we can use as a tool
def add(a: float, b: float) -> float:
    print(f"{a} + {b} = {a + b}")
    return a + b

# A schema that lets the LLM know that we can use the add function as a tool
def schema_add() -> dict:
    return {
            "type": "function",
            "function": {
                "name": "add",
                "description": "Add two numbers",
                "parameters": {
                    "type": "object", 
                    "properties": {
                        "a": {"type": "number"},
                        "b": {"type": "number"}
                    },
                    "required": ["a", "b"]
                },
            },
        }

if __name__ == "__main__":
    main()

• python test.py

Screenshots / Logs

Pass: gpt-4o-mini
Fail: qwen2.5-coder-7b-instruct: tool call id is None for add

What is your OS?

• [ ] MacOS
• [X] Windows
• [ ] Linux

[louis-jan]

cc @nguyenhoangthuan99 I think it's a known issue from cortex.cpp.