It seems that LLM struggle to comprehend JSON schemas that include Java-type definitions?

[luckygqx]

for example

@Bean
public ChatClient chatClient(OllamaChatModel ollamaChatModel) {
    return ChatClient.builder(ollamaChatModel)
            .defaultAdvisors(new MessageChatMemoryAdvisor(new InMemoryChatMemory()))
            .build();
}

@Component
public class MatchPlanTool {
    @Tool(name = "matchPlan", returnDirect = true, description = "Analyze the user's travel plan, which includes departure location, destination, trip start time(in the user's timezone), and number of travelers (including adults and children).")
    public String matchPlan(Request request) {
        return request.desc();
    }

    @Data
    @JsonClassDescription("Analyze the user's travel plan service API request parameter")
    public static class Request {

        @JsonPropertyDescription("departure location")
        private String departure;

        @JsonPropertyDescription("destination")
        private String destination;

        @JsonPropertyDescription("trip start time")
        private String tripStartTime;

        @JsonPropertyDescription("number of adults")
        private Integer adultCount;

        @JsonPropertyDescription("number of children")
        private Integer childrenCount;

        public String desc() {
            return String.format("user need %s time，from %s  to %s，include adults %s，chlidren%s", tripStartTime, departure, destination, adultCount, childrenCount);
        }
    }
}

  @GetMapping(value = "chat")
  public String chat(@RequestParam String sessionId, @RequestParam String message) {
      return chatClient.prompt()
              .user(message)
              .advisors(i -> i.param(AbstractChatMemoryAdvisor.CHAT_MEMORY_CONVERSATION_ID_KEY, sessionId))
              .tools(matchPlanTool)
              .call()
              .content();
  }

This is the JSON schema generated by Spring AI before calling LLM

{
  "$schema" : "https://json-schema.org/draft/2020-12/schema",
  "type" : "object",
  "properties" : {
    "request" : {
      "type" : "object",
      "properties" : {
        "adults" : {
          "type" : "integer",
          "format" : "int32",
          "description" : "number of adults"
        },
        "children" : {
          "type" : "integer",
          "format" : "int32",
          "description" : "number of children"
        },
        "departure" : {
          "type" : "string",
          "description" : "departure location"
        },
        "destination" : {
          "type" : "string",
          "description" : "destination"
        },
        "tripStartTime" : {
          "type" : "string",
          "description" : "trip start time"
        }
      },
      "required" : [ "adults", "children", "departure", "destination", "tripStartTime" ],
      "description" : "Analyze the user's travel plan service API request parameter"
    }
  },
  "required" : [ "request" ],
  "additionalProperties" : false
}

It seems that LLM struggle to comprehend JSON schemas that include Java-type definitions? LLM response is this

{"request":{"departureLocation":"beijing","destination":"shanghai","numAdults":2,"numChildren":1,"tripStartTime":"2023-12-25T10:00:00+08:00"}}

look at this, field name is not match, Causing JSON deserialization failure。

Did I make a mistake somewhere？

[PierpaoloGuenzani]

U are using LLM to create the JSON structure. U should serialize your class, send it to the LLM and ask for the content in that specific format.

The most elegante solution is to use record and fluent API as such: https://docs.spring.io/spring-ai/reference/api/structured-output-converter.html#_bean_output_converter

[luckygqx]

U are using LLM to create the JSON structure. U should serialize your class, send it to the LLM and ask for the content in that specific format.

The most elegante solution is to use record and fluent API as such: https://docs.spring.io/spring-ai/reference/api/structured-output-converter.html#_bean_output_converter

thanks