mtmd_init_from_file: error: Failed to load CLIP model to some multimodal model

[lihaofd]

Name and Version

ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4060 Laptop GPU, compute capability 8.9, VMM: yes version: 6907 (bea04522f) built with MSVC 19.44.35219.0 for x64

Operating systems

Windows

GGML backends

HIP, CUDA

Hardware

4060 or radeon 8060S

Models

No response

Problem description & steps to reproduce

llama-server -m obsidian-q6.gguf --mmproj mmproj-obsidian-f16.gguf https://huggingface.co/NousResearch/Obsidian-3B-V0.5-GGUF

clip_init: failed to load model 'mmproj-obsidian-f16.gguf': load_hparams: unknown projector type:

mtmd_init_from_file: error: Failed to load CLIP model from mmproj-obsidian-f16.gguf

srv load_model: failed to load multimodal model, 'mmproj-obsidian-f16.gguf' srv operator(): operator(): cleaning up before exit... main: exiting due to model loading error

I have go through https://github.com/ggml-org/llama.cpp?tab=readme-ov-file#multimodal Looks below 4 models failed with same issue https://huggingface.co/advanced-stack/bakllava-mistral-v1-gguf/ https://huggingface.co/NousResearch/Obsidian-3B-V0.5-GGUF/tree/main https://huggingface.co/nakodanei/ShareGPT4V-13B_GGUF/ https://huggingface.co/cmp-nct/Yi-VL-6B-GGUF

First Bad Commit

No response

Relevant log output

clip_init: failed to load model 'mmproj-obsidian-f16.gguf': load_hparams: unknown projector type:

mtmd_init_from_file: error: Failed to load CLIP model from mmproj-obsidian-f16.gguf

srv    load_model: failed to load multimodal model, 'mmproj-obsidian-f16.gguf'
srv    operator(): operator(): cleaning up before exit...
main: exiting due to model loading error