AI Chat

[rolfsmeds]

Description

A collection of Flow APIs and component features that make it easy to implement an AI chat in your application.

Tier

Free

License

Apache 2.0

Motivation

Background

AI chat features are becoming commonplace in all kinds of applications. Adding such features to a Vaadin application should be as simple as possible.

Problem

Although Vaadin provides the MessageList and MessageInput components, a lot of boilerplate is required to connect them with an LLM. Additionally, the MessageInput component does not provide a way to upload files, which are often desired as part of the input to an LLM.

Solution

Introduce an orchestrator API that connects the existing MessageList, MessageInput and Upload components with an LLM provider (from a set of predefined provider classes for popular models).

The API could be something like this:

public class MyView extends Div {
    public MyView(ChatClient.Builder chatClientBuilder) {
        var messageList = new MessageList();
        var messageInput = new MessageInput();
        var upload = new Upload();
        // Wrap the input inside an Upload to include it in the drop area:
        upload.setDropAreaContent(messageInput);

        var llmProvider = new SpringAiLLMProvider(chatClientBuilder.build());

        var orchestrator = AiOrchestrator.builder(llmProvider, "You are a helpful assistant.")
            .withMessageList(messageList)
            .withInput(messageInput)
            .withFileReceiver(upload)
            .build();

        add(messageList, upload);
    }
}

Notes

This modular approach was chosen in favor of a fully integrated AiChat component, because it allows developers to freely define the layout to fit their needs, and even opt for other alternatives to the aforementioned Vaadin components.

Requirements

MessageList enhancements

• [ ] Way of showing the attachments that were sent with a prompt in MessageList.
  • MessageListItem.setAttachments(List<Attachment>) or similar API

Upload component enhancements

• [ ] A slot in the Upload component for wrapping the MessageInput (or any custom component for entering the prompt) into it, so that the entire prompt area functions as a file drop target.
  • Upload.setDropAreaContent(Component)
• [ ] A compact rendering mode for the Upload component's file list, that renders attachments as thumbnails.
  • UploadVariant.THUMBNAILS or similar theme variant
• [ ] A mode in Upload for rendering the file list above instead of below.
  • Upload.setFileListPosition(FileListPosition.ABOVE) or similar API
• [ ] A theme variant for Upload with a compact icon-only upload button.
  • UploadVariant.COMPACT_UPLOAD_BUTTON or similar theme variant

Orchestrators and providers

• [ ] AI chat orchestrator class, that connects the MessageList, MessageInput and Upload components with each other and an LLM provider.
• [ ] LLM provider interface and a set of predefined implementations for popular LLM vendors (Spring and LangChain4J).

Miscellaneous

• [ ] Documentation (TBD where in vaadin.com/docs it should be)
• [ ] Feature flag (initial versions will be shipped behind a Preview feature flag)

Nice-to-haves

• [ ] Default builder that constructs a complete AI chat layout based on provided LLMProvider and orchestrator.

Risks, limitations and breaking changes

Risks

None identified.

Limitations

Missing features within the scope of this PRD: see Out of scope below.

Breaking changes

Changes to the DOM structure of the aforementioned existing UI components may cause minor breaking changes to custom styling.

Out of scope

• Message-specific actions
• Selecting a message for context
• Message editing (and version history)
• Rich prompt input (e.g. @mentions)
• Voice input
• Support for custom additions to prompt UI (e.g. configuration options)

Materials

Sketch of what the AI chat features could look like in an application UI (not a final visual design):

Metrics

No response

Pre-implementation checklist

• [ ] Estimated (estimate entered into Estimate custom field)
• [ ] Product Manager sign-off
• [ ] Engineering Manager sign-off

Pre-release checklist

• [ ] Documented (link to documentation provided in sub-issue or comment)
• [ ] UX/DX tests conducted and blockers addressed
• [ ] Approved for release by Product Manager

Security review

Peer reviewed

[knoobie]

Just a random note: Make sure the component can work with a "nullable" Upload - not all Chats require or allow people to upload documents.

[rolfsmeds]

Just a random note: Make sure the component can work with a "nullable" Upload - not all Chats require or allow people to upload documents.

That is the idea with this modular API (instead of a monolithic component) – you can choose which components to involve and connect to the orchestrator. If you don't want upload functionality, you just don't provide an upload component to the orchestrator.

[TatuLund]

Related (adding components to message list): https://github.com/vaadin/web-components/issues/10278 and https://github.com/vaadin/flow-components/issues/7885

Note, there are couple of scenarios for this

• Getting content embedded in AI response shown. AI can include HTML or SVG in response to be shown in HTML component (remember to sanitize with Jsoup) or JSON data to be parsed and shown e.g. Chart
• Or getting output via MCP integrated

[TatuLund]

Related (Lazy loading, as AI generated content can become long chat): https://github.com/vaadin/flow-components/issues/6640

[samie]

Here are some popular “Chat UI” examples for comparison and benchmarking purposes.

• GitHub Copilot
• MS Copilot
• Claude
• Perplexity
• HubSpot
• Mermaid
• Google AI Studio
• 11Labs
• artlist.io

Functionality can be summarized as follows:

#	Section/Component	Description	Examples
1	Core Input Area	Main prompt and input controls	Text input, Send button, File upload, Voice input
2	Mode & Model Controls	Switch AI model or operation mode	Model selector (GPT/Gemini/Claude), Text-to-Image, Search mode
3	Output Configuration	Adjust output format or style	Image ratio, # of images, tone/style presets, temperature
4	Context and Navigation	Manage chats, history, and workspaces	Conversation list, rename/delete chat, project picker
5	System Actions	App-level controls and settings	Profile menu, integrations, theme, language
6	Output Area	Displays results and related actions	Copy, download, regenerate, edit & retry
7	Special Modules	Domain-specific tools	Diagram templates, audio playback, fact check, code runner

[ruizrube]

Dear Vaadiners, Have you heard about AG-UI? (https://docs.ag-ui.com/sdk/js/core/overview#@ag-ui/core)

Regards