Multimodal prompting?

Open danielpuglisi opened this issue 2 years ago • 0 comments

Hello there! Thanks for providing this Gem!

I'm new to llama.cpp and was wondering how I might use the following llava-cli command to use with the Ruby bindings:

./llava-cli -m /Users/pulleasy/.cache/lm-studio/models/AI-Engine/BakLLaVA1-MistralLLaVA-7B-GGUF/BakLLaVA1-MistralLLaVA-7B.q5_K_M.gguf --mmproj /Users/pulleasy/.cache/lm-studio/models/AI-Engine/BakLLaVA1-MistralLLaVA-7B-GGUF/BakLLaVA1-clip-mmproj-model-f16.gguf --image ~/Downloads/test2.jpeg --temp 0.1

Any pointers where to inject the mmproj and image parameters would be highly appreciated 🙏

Dec 11 '23 14:12 danielpuglisi